Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
llm-action
AI repositories本项目旨在分享大模型相关技术原理以及实战经验(大模型工程化、大模型应用落地)
git clone https://github.com/liguodongiot/llm-action.git -
TensorFold
AI repositoriesLLM Inference Engine for Metal, CUDA and Vulkan.
git clone https://github.com/ashhart/TensorFold.git -
PowerInfer
AI repositoriesHigh-speed Large Language Model Serving for Local Deployment
git clone https://github.com/Tiiny-AI/PowerInfer.git -
mistral.rs
AI repositoriesFast, flexible LLM inference
git clone https://github.com/EricLBuehler/mistral.rs.git -
litgpt
AI repositories20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
git clone https://github.com/Lightning-AI/litgpt.git -
lmdeploy
AI repositoriesLMDeploy is a toolkit for compressing, deploying, and serving LLMs.
git clone https://github.com/InternLM/lmdeploy.git -
gitleaks
AI repositoriesFind secrets with Gitleaks 🔑
git clone https://github.com/gitleaks/gitleaks.git -
zero-to-sglang
Docs & tutorialsOfficial SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
git clone https://github.com/datawhalechina/zero-to-sglang.git -
chatllm.cpp
AI repositoriesPure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
git clone https://github.com/foldl/chatllm.cpp.git -
GenerativeAIExamples
AI repositoriesGenerative AI reference workflows optimized for accelerated infrastructure and microservice architecture.
git clone https://github.com/NVIDIA/GenerativeAIExamples.git -
web-llm
AI repositoriesHigh-performance In-browser LLM Inference Engine
git clone https://github.com/mlc-ai/web-llm.git -
OlliteRT
AI repositoriesTurn your Android phone into an OpenAI-compatible LLM inference server - Fully local, private and Open Source
git clone https://github.com/NightMean/OlliteRT.git -
engine
AI repositories2bit quants, kernels and runtime for consumer devices.
git clone https://github.com/cactus-compute/engine.git -
pegainfer
AI repositoriesPure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
git clone https://github.com/pegainfer-project/pegainfer.git -
turbo-fieldfare
AI repositoriesGemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
git clone https://github.com/drumih/turbo-fieldfare.git -
picollm
AI repositoriesOn-device LLM Inference Powered by X-Bit Quantization
git clone https://github.com/Picovoice/picollm.git -
AI-Compass
AI repositories“AI-Compass”将为社区指引在 AI 技术海洋中航行的方向,无论你是初学者还是进阶开发者,都能在这里找到通往 AI 各大方向的路径。旨在帮助开发者系统性地了解 AI 的核心概念、主流技术、前沿趋势,并通过实践掌握从理论到落地的全过程。
git clone https://github.com/tingaicompass/AI-Compass.git -
ai-engineering-interview-questions
AI repositoriesYour Cheat Sheet for AI Engineering Interview – Questions and Answers.
git clone https://github.com/amitshekhariitbhu/ai-engineering-interview-questions.git -
ray
AI repositoriesRay is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
git clone https://github.com/ray-project/ray.git -
superduper
AI repositoriesSuperduper: End-to-end framework for building custom AI applications and agents.
git clone https://github.com/superduper-io/superduper.git -
prompt-poet
PromptsStreamlines and simplifies prompt design for both developers and non-technical users with a low code approach.
git clone https://github.com/character-ai/prompt-poet.git -
llmflows
PromptsLLMFlows - Simple, Explicit and Transparent LLM Apps
git clone https://github.com/stoyan-stoyanov/llmflows.git -
ai-infra-book
AI repositories《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
git clone https://github.com/bojieli/ai-infra-book.git -
openvino
AI repositoriesOpenVINO™ is an open source toolkit for optimizing and deploying AI inference
git clone https://github.com/openvinotoolkit/openvino.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.