#python #deep_learning #pre_trained #model #awesome #nlp #vision #paddlehub #ai_models

https://github.com/PaddlePaddle/PaddleHub

#scala #ai #apache_spark #azure #big_data #cognitive_services #data_science #databricks #deep_learning #http #lightgbm #machine_learning #microsoft #ml #model_deployment #onnx #opencv #pyspark #spark #synapse

https://github.com/microsoft/SynapseML

#cplusplus #deployment #model_converter #ncnn #onnxruntime #openvino #pplnn #sdk #tensorrt

https://github.com/open-mmlab/mmdeploy

#python #data_parallelism #deep_learning #distributed_training #hpc #large_scale #model_parallelism #pipeline_parallelism

https://github.com/hpcaitech/ColossalAI

#python #data_analysis #data_drift #data_science #deep_learning #jupyter_notebook #machine_learning #machinelearning #ml #mlops #model_monitoring #monitoring #performance_monitoring #visualization

https://github.com/NannyML/nannyml

#python #caffe #computer_vision #coreml #edgetpu #keras #mediapipe #model #model_zoo #models #onnx #openvino #pretrained_models #pytorch #tensorflow #tensorflow_lite #tensorflowjs #tf_trt #tfjs #tflite #tflite_models

https://github.com/PINTO0309/PINTO_model_zoo

#jupyter_notebook #ai #artificial_intelligence #image_generation #img2img #latent_diffusion #machine_learning #model_training #stable_diffusion #txt2img

https://github.com/JoePenna/Dreambooth-Stable-Diffusion

#python #data_drift #data_science #hacktoberfest #html_report #jupyter_notebook #machine_learning #machine_learning_operations #mlops #model_monitoring #pandas_dataframe #production_machine_learning

https://github.com/evidentlyai/evidently

#python #data_drift #data_science #data_validation #deep_learning #html_report #jupyter_notebook #machine_learning #ml #mlops #model_monitoring #model_validation #pandas_dataframe #pytorch

https://github.com/deepchecks/deepchecks

#python #ai #ai_alignment #ai_safety #ai_test #ai_testing #artificial_intelligence #cicd #explainable_ai #llmops #machine_learning #machine_learning_testing #ml #ml_safety #ml_test #ml_testing #ml_validation #mlops #model_testing #model_validation #quality_assurance

https://github.com/Giskard-AI/giskard

#python #ai #control #decision_making #distributed_computing #machine_learning #marl #model_based_reinforcement_learning #multi_agent_reinforcement_learning #pytorch #reinforcement_learning #rl #robotics #torch

https://github.com/pytorch/rl

#python #large_language_models #model_para #transformers

Megatron-LM and Megatron-Core are powerful tools for training large language models (LLMs) on NVIDIA GPUs. Megatron-Core offers GPU-optimized techniques and system-level optimizations, allowing you to train custom transformers efficiently. It supports advanced parallelism strategies, activation checkpointing, and distributed optimization to reduce memory usage and improve training speed. You can use Megatron-Core with other frameworks like NVIDIA NeMo for end-to-end solutions or integrate its components into your preferred training framework. This setup enables scalable training of models with hundreds of billions of parameters, making it beneficial for researchers and developers aiming to advance LLM technology.

https://github.com/NVIDIA/Megatron-LM

#python #billion_parameters #compression #data_parallelism #deep_learning #gpu #inference #machine_learning #mixture_of_experts #model_parallelism #pipeline_parallelism #pytorch #trillion_parameters #zero

DeepSpeed is a powerful tool for training and using large artificial intelligence models quickly and efficiently. It allows you to train models with billions or even trillions of parameters, which is much faster and cheaper than other methods. With DeepSpeed, you can achieve significant speedups, reduce costs, and improve the performance of your models. For example, it can train ChatGPT-like models 15 times faster than current state-of-the-art systems. This makes it easier to work with large language models without needing massive resources, making AI more accessible and efficient for everyone.

https://github.com/microsoft/DeepSpeed

#python #bert #deep_learning #flax #hacktoberfest #jax #language_model #language_models #machine_learning #model_hub #natural_language_processing #nlp #nlp_library #pretrained_models #python #pytorch #pytorch_transformers #seq2seq #speech_recognition #tensorflow #transformer

The Hugging Face Transformers library provides thousands of pretrained models for various tasks like text, image, and audio processing. These models can be used for tasks such as text classification, image detection, speech recognition, and more. The library supports popular deep learning frameworks like JAX, PyTorch, and TensorFlow, making it easy to switch between them.

The benefit to the user is that you can quickly download and use these pretrained models with just a few lines of code, saving time and computational resources. You can also fine-tune these models on your own datasets and share them with the community. Additionally, the library offers a simple `pipeline` API for immediate use on different inputs, making it user-friendly for both researchers and practitioners. This helps in reducing compute costs and carbon footprint while enabling high-performance results across various machine learning tasks.

https://github.com/huggingface/transformers

#python #amd #cuda #gpt #inference #inferentia #llama #llm #llm_serving #llmops #mlops #model_serving #pytorch #rocm #tpu #trainium #transformer #xpu

vLLM is a library that makes it easy, fast, and cheap to use large language models (LLMs). It is designed to be fast with features like efficient memory management, continuous batching, and optimized CUDA kernels. vLLM supports many popular models and can run on various hardware including NVIDIA GPUs, AMD CPUs and GPUs, and more. It also offers seamless integration with Hugging Face models and supports different decoding algorithms. This makes it flexible and easy to use for anyone needing to serve LLMs, whether for research or other applications. You can install vLLM easily with `pip install vllm` and find detailed documentation on their website.

https://github.com/vllm-project/vllm

#java #ai_catalog #data_catalog #datalake #federated_query #lakehouse #metadata #metalake #model_catalog #opendatacatalog #skycomputing #stratosphere

Apache Gravitino is a powerful tool for managing metadata across different sources and regions. It's available under the Apache 2.0 license, which means you can use it freely for any purpose, including commercial projects. You can modify and distribute the software as needed. This flexibility allows businesses to integrate Gravitino into their systems without worrying about royalties or strict usage restrictions. The benefit to users is that they can easily manage complex data environments while having full control over how they use and customize the software.

https://github.com/apache/gravitino

#python #ai #big_model #data_parallelism #deep_learning #distributed_computing #foundation_models #heterogeneous_training #hpc #inference #large_scale #model_parallelism #pipeline_parallelism

Colossal-AI is a powerful tool that helps make large AI models faster, cheaper, and easier to use. It uses special techniques like parallelism to speed up training on big models without needing expensive hardware. This means users can train complex AI models even on regular computers or laptops, saving time and money. Colossal-AI also supports various applications across industries like medicine, video generation, and chatbots, making it very versatile for developers.

https://github.com/hpcaitech/ColossalAI

#python #anthropic #api #claude #llm #model_context_protocol #python #server

FastMCP is a tool that helps developers build servers for AI applications using the Model Context Protocol (MCP). It makes it easy to create tools, expose data, and define interaction patterns for AI models. With FastMCP, you can focus on building great tools without worrying about complex protocol details. It's fast, simple, and uses Pythonic code, making it easy for developers to integrate AI with various data sources and tools. This simplifies AI development and makes it more efficient.

https://github.com/jlowin/fastmcp

#python #agents #ai #ai_agents #llm #llms #mcp #model_context_protocol #python

The Model Context Protocol (MCP) is a standard way for AI agents to connect with different tools and data sources, making it much easier to build powerful AI applications without writing custom code for each integration[2][5]. The mcp-agent framework uses MCP to let you quickly create agents that can do things like read files, fetch web pages, or manage emails, and you can combine these agents in flexible ways to handle complex tasks. This means you can focus on what you want your AI to do, while mcp-agent takes care of connecting to the right tools and managing the workflow, saving you time and effort[3][5].

https://github.com/lastmile-ai/mcp-agent