#python #amd #cuda #gpt #inference #inferentia #llama #llm #llm_serving #llmops #mlops #model_serving #pytorch #rocm #tpu #trainium #transformer #xpu

vLLM is a library that makes it easy, fast, and cheap to use large language models (LLMs). It is designed to be fast with features like efficient memory management, continuous batching, and optimized CUDA kernels. vLLM supports many popular models and can run on various hardware including NVIDIA GPUs, AMD CPUs and GPUs, and more. It also offers seamless integration with Hugging Face models and supports different decoding algorithms. This makes it flexible and easy to use for anyone needing to serve LLMs, whether for research or other applications. You can install vLLM easily with `pip install vllm` and find detailed documentation on their website.

https://github.com/vllm-project/vllm

#python #airflow #apache #apache_airflow #automation #dag #data_engineering #data_integration #data_orchestrator #data_pipelines #data_science #elt #etl #machine_learning #mlops #orchestration #python #scheduler #workflow #workflow_engine #workflow_orchestration

Apache Airflow is a tool that helps you manage and automate workflows. You can write your workflows as code, making them easier to maintain, version, test, and collaborate on. Airflow lets you schedule tasks and monitor their progress through a user-friendly interface. It supports dynamic pipeline generation, is highly extensible, and scalable, allowing you to define your own operators and executors.

Using Airflow benefits you by making your workflows more organized, efficient, and reliable. It simplifies the process of managing complex tasks and provides clear visualizations of your workflow's performance, helping you identify and troubleshoot issues quickly. This makes it easier to manage data processing and other automated tasks effectively.

https://github.com/apache/airflow

#python #ai #cv #data_analytics #data_wrangling #embeddings #llm #llm_eval #machine_learning #mlops #multimodal

DataChain is a powerful tool for managing and processing large amounts of data, especially useful for artificial intelligence tasks. It helps you organize unstructured data from various sources like cloud storage or local files into structured datasets. You can process this data efficiently using Python, without needing SQL or Spark, and even use local AI models or APIs to enrich your data. Key benefits include parallel processing, out-of-memory computing, and optimized vector searches, making it faster and more efficient. Additionally, DataChain integrates well with popular libraries like PyTorch and TensorFlow, allowing you to easily export data for further analysis or training models. This makes it easier to handle complex data tasks and improves your overall workflow.

https://github.com/iterative/datachain

#python #cloud_native #cncf #deep_learning #docker #fastapi #framework #generative_ai #grpc #jaeger #kubernetes #llmops #machine_learning #microservice #mlops #multimodal #neural_search #opentelemetry #orchestration #pipeline #prometheus

Jina-serve is a tool that helps you build and deploy AI services easily. It supports major machine learning frameworks and allows you to scale your services from local development to production quickly. You can use it to create AI services that communicate via gRPC, HTTP, and WebSockets. It has features like built-in Docker integration, one-click cloud deployment, and support for Kubernetes and Docker Compose, making it easy to manage and scale your AI applications. This makes it simpler for you to focus on the core logic of your AI projects without worrying about the technical details of deployment and scaling.

https://github.com/jina-ai/serve

#cplusplus #cublas #cuda #cudnn #gpu #mlops #networking #nvml #remote_access

SCUDA is a tool that lets you use GPUs from other computers over the internet. This means you can run programs that need powerful GPUs on your local machine, even if it doesn't have one. Here’s how it helps: You can test and develop applications using remote GPUs, train machine learning models from your laptop, perform complex data processing tasks, and even fine-tune pre-trained models without needing a powerful GPU locally. This makes it easier to work with GPUs without having to physically have one, saving time and resources.

https://github.com/kevmo314/scuda

#other #awesome #awesome_list #data_mining #deep_learning #explainability #interpretability #large_scale_machine_learning #large_scale_ml #machine_learning #machine_learning_operations #ml_operations #ml_ops #mlops #privacy_preserving #privacy_preserving_machine_learning #privacy_preserving_ml #production_machine_learning #production_ml #responsible_ai

This repository provides a comprehensive list of open-source libraries and tools for deploying, monitoring, versioning, scaling, and securing machine learning models in production. Here are the key benefits The repository includes a wide range of tools categorized into sections such as adversarial robustness, agentic workflow, AutoML, computation load distribution, data labelling and synthesis, data pipelines, data storage optimization, data stream processing, deployment and serving, evaluation and monitoring, explainability and fairness, feature stores, and more.

- **Production Readiness** The repository is actively maintained and contributed to by a community of developers, ensuring that the tools are up-to-date and well-supported.

- **Ease of Use** Tools for optimized computation, model storage optimization, and neural search and retrieval help in improving the performance and efficiency of machine learning models.

- **Privacy and Security**: Libraries focused on privacy and security, such as federated learning and homomorphic encryption, ensure that sensitive data is protected during model training and deployment.

Using this repository, you can streamline your machine learning workflows, improve model performance, and ensure robustness and security in your production environments.

https://github.com/EthicalML/awesome-production-machine-learning

#other #ai #data_science #devops #engineering #federated_learning #machine_learning #ml #mlops #software_engineering

This resource is a comprehensive guide to Machine Learning Operations (MLOps), providing a wide range of tools, articles, courses, and communities to help you manage and deploy machine learning models effectively.

**Key Benefits** Access to numerous books, articles, courses, and talks on MLOps, machine learning, and data science.
- **Community Support** Detailed guides on workflow management, feature stores, model deployment, testing, monitoring, and maintenance.
- **Infrastructure Tools** Resources on model governance, ethics, and responsible AI practices.

Using these resources, you can improve your skills in designing, training, and running machine learning models efficiently, ensuring they are reliable, scalable, and maintainable in production environments.

https://github.com/visenger/awesome-mlops

#python #cleandata #data_engineering #data_profilers #data_profiling #data_quality #data_science #data_unit_tests #datacleaner #datacleaning #dataquality #dataunittest #eda #exploratory_analysis #exploratory_data_analysis #exploratorydataanalysis #mlops #pipeline #pipeline_debt #pipeline_testing #pipeline_tests

GX Core is a powerful tool for ensuring data quality. It allows you to write simple tests, called "Expectations," to check if your data meets certain standards. This helps teams work together more effectively and keeps everyone informed about the data's quality. You can automatically generate reports, making it easy to share results and preserve your organization's knowledge about its data. To get started, you just need to install GX Core in a Python virtual environment and follow some simple steps. This makes managing data quality much simpler and more efficient.

https://github.com/great-expectations/great_expectations

#rust #ai #ai_engineering #anthropic #artificial_intelligence #deep_learning #genai #generative_ai #gpt #large_language_models #llama #llm #llmops #llms #machine_learning #ml #ml_engineering #mlops #openai #python #rust

TensorZero is a free, open-source tool that helps you build and improve large language model (LLM) applications by using real-world data and feedback. It gives you one simple API to connect with all major LLM providers, collects data from your app’s use, and lets you easily test and improve prompts, models, and strategies. You can see how your LLMs perform, compare different options, and make them smarter, faster, and cheaper over time—all while keeping your data private and under your control. This means you get better results with less effort and cost, and your apps keep improving as you use them[1][2][3].

https://github.com/tensorzero/tensorzero

#python #agent #agentic_ai #llm #mlops #reinforcement_learning

Agent Lightning is a tool that helps improve AI agents using reinforcement learning. It allows you to train your agents without making big changes to their code, which is very convenient. You can use it with many different frameworks like LangChain or OpenAI Agent SDK. It also supports various training methods, including reinforcement learning and automatic prompt optimization. This means you can make your agents better at their tasks without a lot of extra work.

https://github.com/microsoft/agent-lightning

#python #agents #gcp #gemini #genai_agents #generative_ai #llmops #mlops #observability

You can quickly create and deploy AI agents using the Agent Starter Pack, a Python package with ready-made templates and full infrastructure on Google Cloud. It handles everything except your agent’s logic, including deployment, monitoring, security, and CI/CD pipelines. You can start a project in just one minute, customize agents for tasks like document search or real-time chat, and extend them as needed. This saves you time and effort by providing production-ready tools and integration with Google Cloud services, letting you focus on building smart AI agents without worrying about backend setup or deployment details.

https://github.com/GoogleCloudPlatform/agent-starter-pack