Library
85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
EmbedAnything
AI repositoriesHighly Performant, Modular, Memory Safe and Production-ready Inference, Ingestion and Indexing built in Rust 🦀
git clone https://github.com/StarlightSearch/EmbedAnything.git -
awsome-distributed-ai
AI repositoriesBest practices, reference architectures, and examples for distributed AI training and inference on AWS.
git clone https://github.com/awslabs/awsome-distributed-ai.git -
Halfrost-Field
AI repositories✍🏻 Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field 冰霜之地:源码解析、系统设计与工程实践笔记
git clone https://github.com/halfrost/Halfrost-Field.git -
TensorSharp
AI repositoriesA native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability
git clone https://github.com/zhongkaifu/TensorSharp.git -
SwiftLM
AI repositories⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.
git clone https://github.com/SharpAI/SwiftLM.git -
jev
AI repositoriesjevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.
git clone https://github.com/feder-cr/jev.git -
OlliteRT
AI repositoriesTurn your Android phone into an OpenAI-compatible LLM inference server - Fully local, private and Open Source
git clone https://github.com/NightMean/OlliteRT.git -
LTX-2
AI repositoriesOfficial Python inference and LoRA trainer package for the LTX-2 audio–video generative model.
git clone https://github.com/Lightricks/LTX-2.git -
mlx-serve
AI repositoriesNative LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.
git clone https://github.com/ddalcu/mlx-serve.git -
ai-engineering-interview-questions
AI repositoriesYour Cheat Sheet for AI Engineering Interview – Questions and Answers.
git clone https://github.com/amitshekhariitbhu/ai-engineering-interview-questions.git -
airllm
AI repositoriesAirLLM 2.8T LLM inference with single 4GB GPU
git clone https://github.com/lyogavin/airllm.git -
ai-infra-book
AI repositories《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
git clone https://github.com/bojieli/ai-infra-book.git -
AI-Compass
AI repositories“AI-Compass”将为社区指引在 AI 技术海洋中航行的方向,无论你是初学者还是进阶开发者,都能在这里找到通往 AI 各大方向的路径。旨在帮助开发者系统性地了解 AI 的核心概念、主流技术、前沿趋势,并通过实践掌握从理论到落地的全过程。
git clone https://github.com/tingaicompass/AI-Compass.git -
oumi
AI repositoriesEasily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!
git clone https://github.com/oumi-ai/oumi.git -
nvidia_gpu_exporter
AI repositoriesNvidia GPU exporter for prometheus using nvidia-smi binary OR using NVML
git clone https://github.com/utkuozdemir/nvidia_gpu_exporter.git -
Atomic-Chat
AI repositoriesLocal AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V
git clone https://github.com/AtomicBot-ai/Atomic-Chat.git -
vllm-mlx
AI repositoriesHigh-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
git clone https://github.com/waybarrios/vllm-mlx.git -
reasoning-from-scratch
AI repositoriesImplement a reasoning LLM in PyTorch from scratch, step by step
git clone https://github.com/rasbt/reasoning-from-scratch.git -
wllama
AI repositoriesWebAssembly binding for llama.cpp - Enabling on-browser LLM inference
git clone https://github.com/ngxson/wllama.git -
laya
AI repositoriesRun Laya, the open-source Jev-compatible System-1 decision model, from Node.js / TypeScript via ONNX Runtime
git clone https://github.com/receptron/laya.git -
larql
AI repositoriesRun, serve, query and edit AI models — a Rust model runtime built on VINDEX3, an executable, inspectable model format.
git clone https://github.com/chrishayuk/larql.git -
AI-Engineer-Headquarters
AI repositoriesA collection of scientific methods, processes, algorithms, and systems to build stories & models.
git clone https://github.com/hemansnation/AI-Engineer-Headquarters.git -
janas
AI repositories37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
git clone https://github.com/prabanta-dev/janas.git -
tiny-llm
AI repositorieslearn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen
git clone https://github.com/skyzh/tiny-llm.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.