#python #ai #big_model #data_parallelism #deep_learning #distributed_computing #foundation_models #heterogeneous_training #hpc #inference #large_scale #model_parallelism #pipeline_parallelism

Colossal-AI is a powerful tool that helps make large AI models faster, cheaper, and easier to use. It uses special techniques like parallelism to speed up training on big models without needing expensive hardware. This means users can train complex AI models even on regular computers or laptops, saving time and money. Colossal-AI also supports various applications across industries like medicine, video generation, and chatbots, making it very versatile for developers.

https://github.com/hpcaitech/ColossalAI

#go #bugtracker #decentralized_application #distributed_systems #git #gitdb

git-bug is a powerful, decentralized issue tracker that stores issues, comments, and users directly inside a Git repository as versioned objects, not just files. This means you can manage your issues offline, sync them later, and keep everything clean and organized within your existing Git workflow. It’s very fast, supports syncing with platforms like GitHub and GitLab, and offers multiple ways to interact, including command line, text user interface, or web browser. This tool helps you track and manage project issues efficiently without needing a separate server or database, making collaboration and version control seamless.

https://github.com/git-bug/git-bug

#rust #blockchain #distributed_ledger_technology #move #smart_contracts

Sui is a next-generation blockchain platform designed for very fast, low-cost transactions and high scalability, making it ideal for apps like gaming, DeFi, and NFTs. It uses a unique object-based data model and the Move programming language, which helps create secure, flexible smart contracts and allows many transactions to happen at the same time. This means you get instant transaction finality and a smooth user experience. Sui’s native token, SUI, is used for fees, staking, and governance, helping keep the network secure and decentralized. Overall, Sui offers a powerful, efficient foundation for building and using web3 applications.

https://github.com/MystenLabs/sui

#cplusplus #compaction #database #distributed_database #kvstore #nosql #rocksdb

ToplingDB is a faster and more advanced key-value database built on RocksDB, designed for better performance and flexibility. It supports easy configuration through JSON/YAML, has an embedded web server to monitor and change settings without restarting, and improves speed with features like faster transaction locks and concurrent IO. It also offers plugins for enhanced functions and cloud-native services like MySQL and Redis on ToplingDB. This means you get a powerful, efficient database that is easier to manage and scales well for large or distributed systems, saving you time and improving your application's speed and reliability.

https://github.com/topling/toplingdb

#rust #bigdata #cloud_native #distributed_systems #filesystem #minio #object_storage #oss #rust #s3

RustFS is a fast and safe distributed object storage system built with Rust, offering high performance and scalability for large data needs like AI and big data. It is compatible with S3, easy to use, and open source under the business-friendly Apache 2.0 license. Compared to others like MinIO, RustFS provides better memory safety, no risky data logging, and supports local cloud providers. You can quickly install it via a script or Docker, manage storage through a simple web console, and benefit from a strong community and detailed documentation. This makes RustFS a reliable, cost-effective choice for secure, scalable storage.

https://github.com/rustfs/rustfs

#javascript #distributed_companies #hacktoberfest #jobs_search #jobsearch #jobseeker #remote #remote_companies #remote_job #remote_work

This list shows hundreds of companies, mostly in tech, that let people work from home either part-time or full-time, with many offering jobs to people all over the world. The list includes big names like Microsoft, Amazon, and Shopify, as well as smaller companies, and covers many different types of work, from software and design to education and health. For anyone looking for a remote job, this is a helpful starting point because it saves time—instead of searching one by one, you can quickly see which companies are open to remote work and find links to their websites for more details or to apply. This makes it much easier to find a job that fits your skills and lets you work from anywhere.

https://github.com/remoteintech/remote-jobs

#java #distributed_systems #durable_execution #grpc #java #javascript #microservice_orchestration #orchestration_engine #orchestrator #reactjs #spring_boot #workflow_automation #workflow_engine #workflow_management #workflows

Conductor is an open-source tool that helps you manage and automate complex workflows involving many microservices and systems. It makes your workflows flexible, reliable, and scalable by handling retries, errors, and monitoring automatically. You can define workflows as code in JSON, use various task types, and manage workflows dynamically without tightly coupling services. It offers an easy-to-use web interface and supports multiple databases like Redis and MySQL. This helps you build, run, and monitor workflows efficiently, saving time and reducing errors in managing distributed applications. It also has SDKs for Java, Python, JavaScript, Go, and C# to integrate easily with your projects.

https://github.com/conductor-oss/conductor

#rust #artificial_intelligence #big_data #data_engineering #distributed_computing #machine_learning #multimodal #python #rust

Daft is a powerful, easy-to-use data engine that lets you process large-scale data using Python or SQL with high speed and efficiency. It supports complex data types like images and tensors, works well interactively for quick data exploration, and can scale to huge cloud clusters using Ray. Daft integrates smoothly with cloud storage and data catalogs, making it ideal for data engineering, analytics, and machine learning workflows. By using Daft, you can handle big, multimodal datasets faster and more flexibly, improving your ability to analyze and prepare data for AI models without complex setup or slowdowns.

https://github.com/Eventual-Inc/Daft

#go #blob_storage #cloud_drive #distributed_file_system #distributed_storage #distributed_systems #erasure_coding #fuse #hadoop_hdfs #hdfs #kubernetes #object_storage #posix #replication #s3 #s3_storage #seaweedfs #tiered_file_system

SeaweedFS is a fast, simple, and highly scalable distributed file system designed to store billions of files and serve them quickly, especially small files. It uses a master server to manage volumes on volume servers, which handle file data and metadata, enabling very fast file access with minimal disk reads. It supports features like replication, erasure coding, cloud integration for elastic storage, and compatibility with many metadata stores and APIs including Amazon S3. This means you get efficient, cost-effective storage with fast access, easy scaling, and flexible deployment options for large-scale file storage needs.

https://github.com/seaweedfs/seaweedfs

#java #awesome #backend #computer_science #distributed_systems #high_level_design #hld #interview #interview_questions #scalability #system_design

You can learn important system design concepts for free, covering topics like scalability, availability, CAP theorem, caching, databases, APIs, microservices, and distributed systems. This resource offers clear explanations, interview preparation guides, and practical design problems from easy to hard, helping you understand how to build reliable, scalable software systems. It also provides links to courses, books, newsletters, and videos to deepen your knowledge. Using these materials can improve your skills for system design interviews and real-world software architecture, making you more confident and effective in designing complex systems.

https://github.com/ashishps1/awesome-system-design-resources