#python #bert #chatgpt #chatgpt_api #chatgpt_python #chatgpt3 #gpt_2 #gpt_3 #gpt_3_prompts #gpt_neo #gpt3_library #large_language_models #openai #prompt_engineering #prompt_toolkit #prompt_tuning #prompting #prompts #transformers

https://github.com/promptslab/Promptify

#python #attention_mechanism #deep_learning #gpt #gpt_2 #gpt_3 #language_model #linear_attention #lstm #pytorch #rnn #rwkv #transformer #transformers

https://github.com/BlinkDL/RWKV-LM

#python #deepspeed_library #gpt_3 #language_model #transformers

https://github.com/EleutherAI/gpt-neox

#python #chatgpt #clip #deep_learning #gpt #hacktoberfest #hnsw #information_retrieval #knn #large_language_models #machine_learning #machinelearning #multi_modal #natural_language_processing #search_engine #semantic_search #tensor_search #transformers #vector_search #vision_language #visual_search

https://github.com/marqo-ai/marqo

#python #graphcore #habana #inference #intel #onnx #onnxruntime #optimization #pytorch #quantization #tflite #training #transformers

https://github.com/huggingface/optimum

#python #embeddings #information_retrieval #language_model #large_language_models #llm #machine_learning #nearest_neighbor_search #neural_search #nlp #search #search_engine #semantic_search #sentence_embeddings #similarity_search #transformers #txtai #vector_database #vector_search #vector_search_engine

https://github.com/neuml/txtai

#python #ai #data #data_structures #database #long_term_memory #machine_learning #ml #mlops #mongodb #pytorch #scikit_learn #sklearn #torch #transformers #vector_search

https://github.com/SuperDuperDB/superduperdb

#jupyter_notebook #ai #azure #chatgpt #dall_e #generative_ai #generativeai #gpt #language_model #llms #openai #prompt_engineering #semantic_search #transformers

This course teaches you how to build Generative AI applications with 21 comprehensive lessons from Microsoft Cloud Advocates. You'll learn about Generative AI, Large Language Models (LLMs), prompt engineering, and how to build various applications like text generation, chat apps, and image generation using Python and TypeScript. The course includes videos, written lessons, code samples, and additional learning resources. You can start anywhere and even join a Discord server for support and networking with other learners. This helps you gain practical skills in building and deploying Generative AI applications responsibly and effectively.

https://github.com/microsoft/generative-ai-for-beginners

#python #large_language_models #model_para #transformers

Megatron-LM and Megatron-Core are powerful tools for training large language models (LLMs) on NVIDIA GPUs. Megatron-Core offers GPU-optimized techniques and system-level optimizations, allowing you to train custom transformers efficiently. It supports advanced parallelism strategies, activation checkpointing, and distributed optimization to reduce memory usage and improve training speed. You can use Megatron-Core with other frameworks like NVIDIA NeMo for end-to-end solutions or integrate its components into your preferred training framework. This setup enables scalable training of models with hundreds of billions of parameters, making it beneficial for researchers and developers aiming to advance LLM technology.

https://github.com/NVIDIA/Megatron-LM

#python #chinese #clip #computer_vision #contrastive_loss #coreml_models #deep_learning #image_text_retrieval #multi_modal #multi_modal_learning #nlp #pretrained_models #pytorch #transformers #vision_and_language_pre_training #vision_language

This project is about a Chinese version of the CLIP (Contrastive Language-Image Pretraining) model, trained on a large dataset of Chinese text and images. Here’s what you need to know This model helps you quickly perform tasks like calculating text and image features, cross-modal retrieval (finding images based on text or vice versa), and zero-shot image classification (classifying images without any labeled examples).
- **Ease of Use** The model has been tested on various datasets and shows strong performance in zero-shot image classification and cross-modal retrieval tasks.
- **Resources**: The project includes pre-trained models, training and testing codes, and detailed tutorials on how to use the model for different tasks.

Overall, this project makes it easy to work with Chinese text and images using advanced AI techniques, saving you time and effort.

https://github.com/OFA-Sys/Chinese-CLIP

#swift #inference #ios #macos #pretrained_models #speech_recognition #swift #transformers #visionos #watchos #whisper

WhisperKit is a tool that helps your Apple devices recognize speech from audio files or live recordings using OpenAI's Whisper model. It works locally on your device, which means it doesn't need internet connection once set up. To use it, you can add WhisperKit to your Swift project easily through the Swift Package Manager or install a command-line version using Homebrew. This tool is beneficial because it allows you to transcribe audio quickly and efficiently right on your device, making it useful for various applications like voice assistants or transcription services.

https://github.com/argmaxinc/WhisperKit

#python #asr #audio #audio_processing #deep_learning #huggingface #language_model #pytorch #speaker_diarization #speaker_recognition #speaker_verification #speech_enhancement #speech_processing #speech_recognition #speech_separation #speech_to_text #speech_toolkit #speechrecognition #spoken_language_understanding #transformers #voice_recognition

SpeechBrain is an open-source toolkit that helps you quickly develop Conversational AI technologies, such as speech assistants, chatbots, and language models. It uses PyTorch and offers many pre-trained models and tutorials to make it easy to get started. You can train models for various tasks like speech recognition, speaker recognition, and text processing with just a few lines of code. SpeechBrain also supports GPU training, dynamic batching, and integration with HuggingFace models, making it powerful and efficient. This toolkit is beneficial because it simplifies the development process, provides extensive documentation and tutorials, and is highly customizable, making it ideal for research, prototyping, and educational purposes.

https://github.com/speechbrain/speechbrain

#python #agent #ai #chatglm #fine_tuning #gpt #instruction_tuning #language_model #large_language_models #llama #llama3 #llm #lora #mistral #moe #peft #qlora #quantization #qwen #rlhf #transformers

LLaMA Factory is a tool that makes it easy to fine-tune large language models. It supports many different models like LLaMA, ChatGLM, and Qwen, among others. You can use various training methods such as full-tuning, freeze-tuning, LoRA, and QLoRA, which are efficient and save GPU memory. The tool also includes advanced algorithms and practical tricks to improve performance.

Using LLaMA Factory, you can train models up to 3.7 times faster with better results compared to other methods. It provides a user-friendly interface through Colab, PAI-DSW, or local machines, and even offers a web UI for easier management. The benefit to you is that it simplifies the process of fine-tuning large language models, making it faster and more efficient, which can be very useful for research and development projects.

https://github.com/hiyouga/LLaMA-Factory

#python #artificial_intelligence #attention_mechanism #deep_learning #transformers

The `x-transformers` library offers a versatile and feature-rich implementation of transformer models, allowing users to easily build and customize various types of transformers. Here are the key benefits You can create full encoder/decoder models, decoder-only (GPT-like) models, encoder-only (BERT-like) models, and even image classification and image-to-caption models.
- **Experimental Features** You can customize layers with various normalization techniques (e.g., RMSNorm, ScaleNorm), attention variants (e.g., Talking-Heads, One Write-Head), and other enhancements like residual attention and gated feedforward networks.
- **Efficiency** The library provides simple wrappers for autoregressive models, continuous embeddings, and other specialized tasks, making it easier to set up and train complex models.

Overall, `x-transformers` simplifies the process of building advanced transformer models while offering a wide range of customization options to improve performance and efficiency.

https://github.com/lucidrains/x-transformers