#python #bert #chatgpt #chatgpt_api #chatgpt_python #chatgpt3 #gpt_2 #gpt_3 #gpt_3_prompts #gpt_neo #gpt3_library #large_language_models #openai #prompt_engineering #prompt_toolkit #prompt_tuning #prompting #prompts #transformers

https://github.com/promptslab/Promptify

#python #attention_mechanism #deep_learning #gpt #gpt_2 #gpt_3 #language_model #linear_attention #lstm #pytorch #rnn #rwkv #transformer #transformers

https://github.com/BlinkDL/RWKV-LM

#python #deepspeed_library #gpt_3 #language_model #transformers

https://github.com/EleutherAI/gpt-neox

#python #chatgpt #clip #deep_learning #gpt #hacktoberfest #hnsw #information_retrieval #knn #large_language_models #machine_learning #machinelearning #multi_modal #natural_language_processing #search_engine #semantic_search #tensor_search #transformers #vector_search #vision_language #visual_search

https://github.com/marqo-ai/marqo

#python #graphcore #habana #inference #intel #onnx #onnxruntime #optimization #pytorch #quantization #tflite #training #transformers

https://github.com/huggingface/optimum

#python #embeddings #information_retrieval #language_model #large_language_models #llm #machine_learning #nearest_neighbor_search #neural_search #nlp #search #search_engine #semantic_search #sentence_embeddings #similarity_search #transformers #txtai #vector_database #vector_search #vector_search_engine

https://github.com/neuml/txtai

#python #ai #data #data_structures #database #long_term_memory #machine_learning #ml #mlops #mongodb #pytorch #scikit_learn #sklearn #torch #transformers #vector_search

https://github.com/SuperDuperDB/superduperdb

#jupyter_notebook #ai #azure #chatgpt #dall_e #generative_ai #generativeai #gpt #language_model #llms #openai #prompt_engineering #semantic_search #transformers

This course teaches you how to build Generative AI applications with 21 comprehensive lessons from Microsoft Cloud Advocates. You'll learn about Generative AI, Large Language Models (LLMs), prompt engineering, and how to build various applications like text generation, chat apps, and image generation using Python and TypeScript. The course includes videos, written lessons, code samples, and additional learning resources. You can start anywhere and even join a Discord server for support and networking with other learners. This helps you gain practical skills in building and deploying Generative AI applications responsibly and effectively.

https://github.com/microsoft/generative-ai-for-beginners

#python #large_language_models #model_para #transformers

Megatron-LM and Megatron-Core are powerful tools for training large language models (LLMs) on NVIDIA GPUs. Megatron-Core offers GPU-optimized techniques and system-level optimizations, allowing you to train custom transformers efficiently. It supports advanced parallelism strategies, activation checkpointing, and distributed optimization to reduce memory usage and improve training speed. You can use Megatron-Core with other frameworks like NVIDIA NeMo for end-to-end solutions or integrate its components into your preferred training framework. This setup enables scalable training of models with hundreds of billions of parameters, making it beneficial for researchers and developers aiming to advance LLM technology.

https://github.com/NVIDIA/Megatron-LM

#python #chinese #clip #computer_vision #contrastive_loss #coreml_models #deep_learning #image_text_retrieval #multi_modal #multi_modal_learning #nlp #pretrained_models #pytorch #transformers #vision_and_language_pre_training #vision_language

This project is about a Chinese version of the CLIP (Contrastive Language-Image Pretraining) model, trained on a large dataset of Chinese text and images. Here’s what you need to know This model helps you quickly perform tasks like calculating text and image features, cross-modal retrieval (finding images based on text or vice versa), and zero-shot image classification (classifying images without any labeled examples).
- **Ease of Use** The model has been tested on various datasets and shows strong performance in zero-shot image classification and cross-modal retrieval tasks.
- **Resources**: The project includes pre-trained models, training and testing codes, and detailed tutorials on how to use the model for different tasks.

Overall, this project makes it easy to work with Chinese text and images using advanced AI techniques, saving you time and effort.

https://github.com/OFA-Sys/Chinese-CLIP

#swift #inference #ios #macos #pretrained_models #speech_recognition #swift #transformers #visionos #watchos #whisper

WhisperKit is a tool that helps your Apple devices recognize speech from audio files or live recordings using OpenAI's Whisper model. It works locally on your device, which means it doesn't need internet connection once set up. To use it, you can add WhisperKit to your Swift project easily through the Swift Package Manager or install a command-line version using Homebrew. This tool is beneficial because it allows you to transcribe audio quickly and efficiently right on your device, making it useful for various applications like voice assistants or transcription services.

https://github.com/argmaxinc/WhisperKit

#python #asr #audio #audio_processing #deep_learning #huggingface #language_model #pytorch #speaker_diarization #speaker_recognition #speaker_verification #speech_enhancement #speech_processing #speech_recognition #speech_separation #speech_to_text #speech_toolkit #speechrecognition #spoken_language_understanding #transformers #voice_recognition

SpeechBrain is an open-source toolkit that helps you quickly develop Conversational AI technologies, such as speech assistants, chatbots, and language models. It uses PyTorch and offers many pre-trained models and tutorials to make it easy to get started. You can train models for various tasks like speech recognition, speaker recognition, and text processing with just a few lines of code. SpeechBrain also supports GPU training, dynamic batching, and integration with HuggingFace models, making it powerful and efficient. This toolkit is beneficial because it simplifies the development process, provides extensive documentation and tutorials, and is highly customizable, making it ideal for research, prototyping, and educational purposes.

https://github.com/speechbrain/speechbrain

#python #agent #ai #chatglm #fine_tuning #gpt #instruction_tuning #language_model #large_language_models #llama #llama3 #llm #lora #mistral #moe #peft #qlora #quantization #qwen #rlhf #transformers

LLaMA Factory is a tool that makes it easy to fine-tune large language models. It supports many different models like LLaMA, ChatGLM, and Qwen, among others. You can use various training methods such as full-tuning, freeze-tuning, LoRA, and QLoRA, which are efficient and save GPU memory. The tool also includes advanced algorithms and practical tricks to improve performance.

Using LLaMA Factory, you can train models up to 3.7 times faster with better results compared to other methods. It provides a user-friendly interface through Colab, PAI-DSW, or local machines, and even offers a web UI for easier management. The benefit to you is that it simplifies the process of fine-tuning large language models, making it faster and more efficient, which can be very useful for research and development projects.

https://github.com/hiyouga/LLaMA-Factory

#python #artificial_intelligence #attention_mechanism #deep_learning #transformers

The `x-transformers` library offers a versatile and feature-rich implementation of transformer models, allowing users to easily build and customize various types of transformers. Here are the key benefits You can create full encoder/decoder models, decoder-only (GPT-like) models, encoder-only (BERT-like) models, and even image classification and image-to-caption models.
- **Experimental Features** You can customize layers with various normalization techniques (e.g., RMSNorm, ScaleNorm), attention variants (e.g., Talking-Heads, One Write-Head), and other enhancements like residual attention and gated feedforward networks.
- **Efficiency** The library provides simple wrappers for autoregressive models, continuous embeddings, and other specialized tasks, making it easier to set up and train complex models.

Overall, `x-transformers` simplifies the process of building advanced transformer models while offering a wide range of customization options to improve performance and efficiency.

https://github.com/lucidrains/x-transformers

#python #artificial_intelligence #attention_mechanism #computer_vision #image_classification #transformers

This text describes a comprehensive implementation of Vision Transformers (ViT) in PyTorch, offering various models and techniques for image classification. Here’s the key information and benefits**
- The repository provides multiple ViT variants, including the original ViT, Simple ViT, NaViT, Deep ViT, CaiT, Token-to-Token ViT, CCT, Cross ViT, PiT, LeViT, CvT, Twins SVT, RegionViT, CrossFormer, ScalableViT, SepViT, MaxViT, NesT, MobileViT, XCiT, and others.
- Each variant introduces different architectural improvements such as efficient attention mechanisms, multi-scale processing, and innovative embedding techniques.
- The implementation includes pre-trained models and supports various tasks like masked image modeling, distillation, and self-supervised learning.

**Benefits** Users can choose from a wide range of ViT models tailored for different needs, such as efficiency, performance, or specific tasks.
- **Performance** Some models, like NaViT and ScalableViT, are designed to be more efficient in terms of computational resources and training time.
- **Ease of Use** The inclusion of various research ideas and techniques allows users to explore new approaches in vision transformer research.

Overall, this repository offers a powerful toolkit for anyone working with vision transformers, providing both practical solutions and cutting-edge research opportunities.

https://github.com/lucidrains/vit-pytorch

#python #auto_regressive_model #autoregressive_models #diffusion_models #generative_ai #generative_model #gpt #gpt_2 #image_generation #large_language_models #neurips #transformers #vision_transformer

VAR (Visual Autoregressive Modeling) is a new way to generate images that improves upon existing methods. It uses a "next-scale prediction" approach, which means it generates images from coarse to fine details, unlike the traditional method of predicting pixel by pixel. This makes VAR models better than diffusion models for the first time. You can try VAR on a demo website and generate images interactively, which is fun and easy. VAR also follows power-law scaling laws, making it efficient and scalable. The benefit to you is that you can create high-quality images quickly and easily, and even explore technical details through provided scripts and models.

https://github.com/FoundationVision/VAR

#python #anonymization #anonymization_service #data_anonymization #data_loss_prevention #data_masking #data_protection #data_scrubbing #de_identification #dlp #microsoft #pii #pii_anonymization #pii_anonymization_service #pii_detection #presidio #privacy #privacy_protection #python #text_anonymization #transformers

Presidio is a tool that helps protect sensitive information like names, credit card numbers, and addresses in text and images. It can quickly identify and hide this private data, making it safer to use. You can customize Presidio to fit your specific needs and use it in various ways, such as with Python, Docker, or Kubernetes. This helps organizations keep their data private and secure, which is important for protecting user information.

https://github.com/microsoft/presidio

#python #gpu #llm #pytorch #transformers

The `ipex-llm` library is a powerful tool for accelerating Large Language Models (LLMs) on Intel GPUs, NPUs, and CPUs. It integrates seamlessly with popular frameworks like HuggingFace transformers, LangChain, LlamaIndex, and more. Here are the key benefits `ipex-llm` optimizes LLM performance with advanced quantization techniques (FP8, FP6, FP4, INT4) and self-speculative decoding, leading to significant speedups.
- **Wide Model Support** It works on various Intel hardware such as Arc GPUs, Core Ultra NPUs, and CPUs, making it versatile for different setups.
- **Easy Integration** Detailed quickstart guides, code examples, and tutorials help users get started quickly.

Overall, `ipex-llm` enhances the performance and usability of LLMs on Intel hardware, making it a valuable tool for developers and researchers.

https://github.com/intel/ipex-llm

#mdx #deep_learning #hacktoberfest #nlp #transformers

The Hugging Face course teaches you how to use Transformers for natural language processing tasks. You'll learn about the Hugging Face ecosystem, including tools like Transformers, Datasets, Tokenizers, and Accelerate, as well as the Hugging Face Hub. This free course helps you understand how to fine-tune models and share your results. It's beneficial because it provides hands-on experience with popular AI libraries and allows you to build and showcase your own projects on the Hugging Face platform.

https://github.com/huggingface/course