Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
LMCache
AI repositoriesLMCache: Supercharge Your LLM with the Fastest KV Cache Layer
git clone https://github.com/LMCache/LMCache.git -
pegainfer
AI repositoriesPure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
git clone https://github.com/pegainfer-project/pegainfer.git -
vllm
AI repositoriesA high-throughput and memory-efficient inference and serving engine for LLMs
git clone https://github.com/vllm-project/vllm.git -
lmdeploy
AI repositoriesLMDeploy is a toolkit for compressing, deploying, and serving LLMs.
git clone https://github.com/InternLM/lmdeploy.git -
sglang
AI repositoriesSGLang is a high-performance serving framework for large language models and multimodal models.
git clone https://github.com/sgl-project/sglang.git -
skills
AI repositoriesAgent Skills for NVIDIA products — install into Claude Code, Codex, and other coding agents to run Physical AI, robotics, simulation, CUDA, and RAG workflows end to end.
git clone https://github.com/NVIDIA/skills.git -
surogate
AI repositoriesTrain and serve LLMs at extreme speed and massive throughput.
git clone https://github.com/invergent-ai/surogate.git -
tt-metal
AI repositories:metal: TT-NN operator library, and TT-Metalium low level kernel programming model.
git clone https://github.com/tenstorrent/tt-metal.git -
TensorFold
AI repositoriesLLM Inference Engine for Metal, CUDA and Vulkan.
git clone https://github.com/ashhart/TensorFold.git -
gpustack
AI repositoriesA GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
git clone https://github.com/gpustack/gpustack.git -
inferssd
AI repositoriesRun models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
git clone https://github.com/wici-ai/inferssd.git -
raft
AI repositoriesRAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.
git clone https://github.com/NVIDIA/raft.git -
cuvs
AI repositoriescuVS - a library for vector search and clustering on the GPU
git clone https://github.com/NVIDIA/cuvs.git -
beta9
AI repositoriesUltrafast serverless GPU inference, sandboxes, and background jobs
git clone https://github.com/beam-cloud/beta9.git -
ramalama
AI repositoriesRamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.
git clone https://github.com/containers/ramalama.git -
uccl
AI repositoriesUCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)
git clone https://github.com/uccl-project/uccl.git -
starling
AI repositoriesLocal AI in .NET, with no Python sidecar and no API key: embeddings, entity extraction, image and video search and chat on ONNX Runtime (CPU/CUDA), through Microsoft.Extensions.AI.
git clone https://github.com/starlinghq/starling.git -
yzma
AI repositoriesGo with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.
git clone https://github.com/hybridgroup/yzma.git -
project-maya
AI repositoriesProject Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.
git clone https://github.com/mw00/project-maya.git -
TensorSharp
AI repositoriesA native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability
git clone https://github.com/zhongkaifu/TensorSharp.git -
PolyStrata
AI repositoriesRun large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.
git clone https://github.com/VecSzn/PolyStrata.git -
self-hosted-ai-stack
AI repositoriesDeploy a self-hosted AI stack with Docker Compose: local LLMs, chat, document parsing, embeddings, transcription, text-to-speech and MCP tools. Powered by Ollama, LiteLLM, AnythingLLM and more. Local-first with auto-configured authentication, lightweight stacks, optional HTTPS and NVIDIA GPU support. MIT licensed. Multi-arch: amd64, arm64.
git clone https://github.com/hwdsl2/self-hosted-ai-stack.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.