#go #argo #argo_cd #cd #ci_cd #cicd #continuous_delivery #continuous_deployment #devops #docker #gitops #helm #ksonnet #kubernetes #pipeline

https://github.com/argoproj/argo-cd

#java #airbyte #connectors #data #data_analysis #data_ingestion #data_integration #data_science #data_transfers #elt #etl #incremental_updates #integration #open_source #pipeline #pipelines #replications

https://github.com/airbytehq/airbyte

#python #cleandata #data_engineering #data_profilers #data_profiling #data_quality #data_science #data_unit_tests #datacleaner #datacleaning #dataquality #dataunittest #eda #exploratory_analysis #exploratory_data_analysis #exploratorydataanalysis #mlops #pipeline #pipeline_debt #pipeline_testing #pipeline_tests

https://github.com/great-expectations/great_expectations

#python #data_parallelism #deep_learning #distributed_training #hpc #large_scale #model_parallelism #pipeline_parallelism

https://github.com/hpcaitech/ColossalAI

#java #data #data_engineering #data_orchestration #data_orchestrator #data_pipeline #dataflow #elt #etl #kestra #orchestration #pipeline #scheduler #workflow #workflow_automation #workflow_engine

https://github.com/kestra-io/kestra

#python #computer_vision #convolutional_networks #embedding_vectors #embeddings #feature_extraction #feature_vector #image_processing #image_retrieval #machine_learning #milvus #pipeline #towhee #transformer #unstructured_data #video_processing #vision_transformer #vit

https://github.com/towhee-io/towhee

#rust #data #pipeline #sql

https://github.com/PRQL/prql

#jupyter_notebook #ai #aihub #argo #automl #gpt #inference #kubeflow #kubernetes #llmops #mlops #notebook #pipeline #pytorch #spark #vgpu #workflow

https://github.com/tencentmusic/cube-studio

#python #billion_parameters #compression #data_parallelism #deep_learning #gpu #inference #machine_learning #mixture_of_experts #model_parallelism #pipeline_parallelism #pytorch #trillion_parameters #zero

DeepSpeed is a powerful tool for training and using large artificial intelligence models quickly and efficiently. It allows you to train models with billions or even trillions of parameters, which is much faster and cheaper than other methods. With DeepSpeed, you can achieve significant speedups, reduce costs, and improve the performance of your models. For example, it can train ChatGPT-like models 15 times faster than current state-of-the-art systems. This makes it easier to work with large language models without needing massive resources, making AI more accessible and efficient for everyone.

https://github.com/microsoft/DeepSpeed

#cplusplus #android #audio_processing #c_plus_plus #calculator #computer_vision #deep_learning #framework #graph_based #graph_framework #inference #machine_learning #mediapipe #mobile_development #perception #pipeline_framework #stream_processing #video_processing

MediaPipe is a tool that helps you add smart machine learning features to your apps and devices. It works on mobile, web, desktop, and other devices. You can use pre-made solutions for tasks like vision, text, and audio processing, or customize the models to fit your needs. MediaPipe also offers tools like Model Maker and Studio to help you create and test your solutions easily. This makes it easier to delight your customers with innovative features without needing deep machine learning expertise.

https://github.com/google-ai-edge/mediapipe