#python #data_drift #data_science #data_validation #deep_learning #html_report #jupyter_notebook #machine_learning #ml #mlops #model_monitoring #model_validation #pandas_dataframe #pytorch

https://github.com/deepchecks/deepchecks

#python #ai #awesome #data_science #machine_learning #machine_learning_engineering #ml #mle #mlops

https://github.com/kelvins/awesome-mlops

#python #big_data #data_engineering #data_quality #data_science #feature_store #features #machine_learning #ml #mlops

https://github.com/feast-dev/feast

#python #ai #automl #data_science #deep_learning #devops_tools #hacktoberfest #llm #llmops #machine_learning #metadata_tracking #ml #mlops #pipelines #production_ready #pytorch #tensorflow #workflow #zenml

https://github.com/zenml-io/zenml

#go #data_science #deep_learning #distributed_training #hyperparameter_optimization #hyperparameter_search #hyperparameter_tuning #kubernetes #machine_learning #ml_infrastructure #ml_platform #mlops #pytorch #tensorflow

https://github.com/determined-ai/determined

#python #ai #ai_alignment #ai_safety #ai_test #ai_testing #artificial_intelligence #cicd #explainable_ai #llmops #machine_learning #machine_learning_testing #ml #ml_safety #ml_test #ml_testing #ml_validation #mlops #model_testing #model_validation #quality_assurance

https://github.com/Giskard-AI/giskard

#python #ai #data #data_structures #database #long_term_memory #machine_learning #ml #mlops #mongodb #pytorch #scikit_learn #sklearn #torch #transformers #vector_search

https://github.com/SuperDuperDB/superduperdb

#jupyter_notebook #ai #aihub #argo #automl #gpt #inference #kubeflow #kubernetes #llmops #mlops #notebook #pipeline #pytorch #spark #vgpu #workflow

https://github.com/tencentmusic/cube-studio

#go #approximate_nearest_neighbor_search #generative_search #grpc #hnsw #hybrid_search #image_search #information_retrieval #mlops #nearest_neighbor_search #neural_search #recommender_system #search_engine #semantic_search #semantic_search_engine #similarity_search #vector_database #vector_search #vector_search_engine #vectors #weaviate

Weaviate is a powerful, open-source vector database that uses machine learning to make your data searchable. It's fast, scalable, and flexible, allowing you to vectorize your data at import or upload your own vectors. Weaviate supports various modules for integrating with popular AI services like OpenAI, Cohere, and Hugging Face. It's designed for production use with features like scaling, replication, and security. You can use Weaviate for tasks beyond search, such as recommendations, summarization, and integration with neural search frameworks. It offers APIs in GraphQL, REST, and gRPC and has client libraries for several programming languages. This makes it easy to build applications like chatbots, recommendation systems, and image search tools quickly and efficiently. Joining the Weaviate community provides access to tutorials, demos, blogs, and forums to help you get started and stay updated.

https://github.com/weaviate/weaviate

#javascript #annotation #annotation_tool #annotations #boundingbox #computer_vision #data_labeling #dataset #datasets #deep_learning #image_annotation #image_classification #image_labeling #image_labelling_tool #label_studio #labeling #labeling_tool #mlops #semantic_segmentation #text_annotation #yolo

Label Studio is a free, open-source tool that helps you label different types of data like images, audio, text, videos, and more. It has a simple and user-friendly interface that makes it easy to prepare or improve your data for machine learning models. You can customize it to fit your needs and export labeled data in various formats. It supports multi-user labeling, multiple projects, and integration with machine learning models for pre-labeling and active learning. You can install it locally using Docker, pip, or other methods, or deploy it in cloud services like Heroku or Google Cloud Platform. This tool streamlines your data labeling process and helps you create more accurate ML models.

https://github.com/HumanSignal/label-studio

#python #analytics #dagster #data_engineering #data_integration #data_orchestrator #data_pipelines #data_science #etl #metadata #mlops #orchestration #python #scheduler #workflow #workflow_automation

Dagster is a tool that helps you manage and automate your data workflows. You can define your data assets, like tables or machine learning models, using Python functions. Dagster then runs these functions at the right time and keeps your data up-to-date. It offers features like integrated lineage and observability, making it easier to track and manage your data. This tool is useful for every stage of data development, from local testing to production, and it integrates well with other popular data tools. Using Dagster, you can build reusable components, spot data quality issues early, and scale your data pipelines efficiently. This makes your work more productive and helps maintain control over complex data systems.

https://github.com/dagster-io/dagster

#jupyter_notebook #aws #data_science #deep_learning #examples #inference #jupyter_notebook #machine_learning #mlops #reinforcement_learning #sagemaker #training

SageMaker-Core is a new Python SDK for Amazon SageMaker that makes it easier to work with machine learning resources. It provides an object-oriented interface, which means you can manage resources like training jobs, models, and endpoints more intuitively. The SDK simplifies code by allowing resource chaining, eliminating the need to manually specify parameters. It also includes features like auto code completion, comprehensive documentation, and type hints, making it faster and less error-prone to write code. This helps developers customize their ML workloads more efficiently and streamline their development process.

https://github.com/aws/amazon-sagemaker-examples

#python #amd #cuda #gpt #inference #inferentia #llama #llm #llm_serving #llmops #mlops #model_serving #pytorch #rocm #tpu #trainium #transformer #xpu

vLLM is a library that makes it easy, fast, and cheap to use large language models (LLMs). It is designed to be fast with features like efficient memory management, continuous batching, and optimized CUDA kernels. vLLM supports many popular models and can run on various hardware including NVIDIA GPUs, AMD CPUs and GPUs, and more. It also offers seamless integration with Hugging Face models and supports different decoding algorithms. This makes it flexible and easy to use for anyone needing to serve LLMs, whether for research or other applications. You can install vLLM easily with `pip install vllm` and find detailed documentation on their website.

https://github.com/vllm-project/vllm

#python #airflow #apache #apache_airflow #automation #dag #data_engineering #data_integration #data_orchestrator #data_pipelines #data_science #elt #etl #machine_learning #mlops #orchestration #python #scheduler #workflow #workflow_engine #workflow_orchestration

Apache Airflow is a tool that helps you manage and automate workflows. You can write your workflows as code, making them easier to maintain, version, test, and collaborate on. Airflow lets you schedule tasks and monitor their progress through a user-friendly interface. It supports dynamic pipeline generation, is highly extensible, and scalable, allowing you to define your own operators and executors.

Using Airflow benefits you by making your workflows more organized, efficient, and reliable. It simplifies the process of managing complex tasks and provides clear visualizations of your workflow's performance, helping you identify and troubleshoot issues quickly. This makes it easier to manage data processing and other automated tasks effectively.

https://github.com/apache/airflow

#python #ai #cv #data_analytics #data_wrangling #embeddings #llm #llm_eval #machine_learning #mlops #multimodal

DataChain is a powerful tool for managing and processing large amounts of data, especially useful for artificial intelligence tasks. It helps you organize unstructured data from various sources like cloud storage or local files into structured datasets. You can process this data efficiently using Python, without needing SQL or Spark, and even use local AI models or APIs to enrich your data. Key benefits include parallel processing, out-of-memory computing, and optimized vector searches, making it faster and more efficient. Additionally, DataChain integrates well with popular libraries like PyTorch and TensorFlow, allowing you to easily export data for further analysis or training models. This makes it easier to handle complex data tasks and improves your overall workflow.

https://github.com/iterative/datachain

#python #cloud_native #cncf #deep_learning #docker #fastapi #framework #generative_ai #grpc #jaeger #kubernetes #llmops #machine_learning #microservice #mlops #multimodal #neural_search #opentelemetry #orchestration #pipeline #prometheus

Jina-serve is a tool that helps you build and deploy AI services easily. It supports major machine learning frameworks and allows you to scale your services from local development to production quickly. You can use it to create AI services that communicate via gRPC, HTTP, and WebSockets. It has features like built-in Docker integration, one-click cloud deployment, and support for Kubernetes and Docker Compose, making it easy to manage and scale your AI applications. This makes it simpler for you to focus on the core logic of your AI projects without worrying about the technical details of deployment and scaling.

https://github.com/jina-ai/serve