Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
LMCache
AI repositoriesLMCache: Supercharge Your LLM with the Fastest KV Cache Layer
git clone https://github.com/LMCache/LMCache.git -
AI-CUDA-Engineer-Archive
Datasets & benchmarksHugging Face dataset by SakanaAI
SakanaAI/AI-CUDA-Engineer-Archive -
pegainfer
AI repositoriesPure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
git clone https://github.com/pegainfer-project/pegainfer.git -
vllm
AI repositoriesA high-throughput and memory-efficient inference and serving engine for LLMs
git clone https://github.com/vllm-project/vllm.git -
lmdeploy
AI repositoriesLMDeploy is a toolkit for compressing, deploying, and serving LLMs.
git clone https://github.com/InternLM/lmdeploy.git -
sglang
AI repositoriesSGLang is a high-performance serving framework for large language models and multimodal models.
git clone https://github.com/sgl-project/sglang.git -
skills
AI repositoriesAgent Skills for NVIDIA products — install into Claude Code, Codex, and other coding agents to run Physical AI, robotics, simulation, CUDA, and RAG workflows end to end.
git clone https://github.com/NVIDIA/skills.git -
Ternary-Bonsai-2-27B-gguf
Modelstext generation model by prism-ml · llama.cpp
prism-ml/Ternary-Bonsai-2-27B-gguf -
cuda-kernel-optimizer
SkillsEvidence-driven CUDA, CUTLASS, Triton and GPU workload optimization for ChatGPT · 使用 ChatGPT 驱动 GPU workload 性能优化
git clone https://github.com/troycheng/cuda-kernel-optimizer.git -
Breeze-TTS-2
Modelstext to speech model by BreezeBlue · transformers
BreezeBlue/Breeze-TTS-2 -
Edge0-145B-NVFP4
Modelstext generation model by Edge0 · edge0
Edge0/Edge0-145B-NVFP4 -
surogate
AI repositoriesTrain and serve LLMs at extreme speed and massive throughput.
git clone https://github.com/invergent-ai/surogate.git -
Bonsai-8B-gguf
Modelstext generation model by prism-ml · llama.cpp
prism-ml/Bonsai-8B-gguf -
Ternary-Bonsai-27B-gguf
Modelstext generation model by prism-ml · llama.cpp
prism-ml/Ternary-Bonsai-27B-gguf -
ds4-on-spark
Datasets & benchmarksEntrpi/ds4, a Blackwell CUDA perf fork of antirez/ds4 on NVIDIA DGX Spark: one-command install, ~3x upstream prefill, ~1.5x decode, DSpark, and full continuous batch support
git clone https://github.com/Entrpi/ds4-on-spark.git -
llm-algo-leetcode
Docs & tutorialsHands-on LLM algorithms and systems with PyTorch, Triton & CUDA.《大模型算法与系统教程》:模型架构与训练、后训练、推理加速、性能与算子优化,配套 Notebook 练习、答案解析、测试与实验。
git clone https://github.com/datawhalechina/llm-algo-leetcode.git -
Bonsai-27B-gguf
Modelstext generation model by prism-ml · llama.cpp
prism-ml/Bonsai-27B-gguf -
tt-metal
AI repositories:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
git clone https://github.com/tenstorrent/tt-metal.git -
image text to text model by huihui-ai · transformers
huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF -
cuda-mcp
MCP serversCUDA MCP: 20 real tools to read, check, lint, scaffold, build and run CUDA code.
@mrfentmen/cuda-mcp@1.0.0 -
TensorFold
AI repositoriesLLM Inference Engine for Metal, CUDA and Vulkan.
git clone https://github.com/ashhart/TensorFold.git -
gpustack
AI repositoriesA GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
git clone https://github.com/gpustack/gpustack.git -
inferssd
AI repositoriesRun models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
git clone https://github.com/wici-ai/inferssd.git -
raft
AI repositoriesRAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.
git clone https://github.com/NVIDIA/raft.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.