Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
nano-vllm
AI repositoriesNano vLLM
git clone https://github.com/GeeeekExplorer/nano-vllm.git -
vllm
AI repositoriesA high-throughput and memory-efficient inference and serving engine for LLMs
git clone https://github.com/vllm-project/vllm.git -
vllm-ascend
AI repositoriesCommunity maintained hardware plugin for vLLM on Huawei Ascend
git clone https://github.com/vllm-project/vllm-ascend.git -
vllm-metal
AI repositoriesCommunity maintained hardware plugin for vLLM on Apple Silicon
git clone https://github.com/vllm-project/vllm-metal.git -
tiny-llm
AI repositorieslearn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen
git clone https://github.com/skyzh/tiny-llm.git -
semantic-router
AI repositoriesAn open, programmable decision layer for models and compute.
git clone https://github.com/vllm-project/semantic-router.git -
LMCache
AI repositoriesLMCache: Supercharge Your LLM with the Fastest KV Cache Layer
git clone https://github.com/LMCache/LMCache.git -
vllm-mlx
AI repositoriesHigh-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
git clone https://github.com/waybarrios/vllm-mlx.git -
gpustack
AI repositoriesA GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
git clone https://github.com/gpustack/gpustack.git -
sparrow
AI repositoriesStructured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLM
git clone https://github.com/katanaml/sparrow.git -
Mooncake
AI repositoriesMooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
git clone https://github.com/kvcache-ai/Mooncake.git -
UltraRAG
AI repositoriesA Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines
git clone https://github.com/OpenBMB/UltraRAG.git -
llmman
AI repositoriesRun any agent on any model, models stored as OCI images
git clone https://github.com/llmmanorg/llmman.git -
Halfrost-Field
AI repositories✍🏻 Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field 冰霜之地:源码解析、系统设计与工程实践笔记
git clone https://github.com/halfrost/Halfrost-Field.git -
ome
AI repositoriesOpen Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton
git clone https://github.com/ome-projects/ome.git -
kvcached
AI repositoriesVirtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
git clone https://github.com/ovg-project/kvcached.git -
BricksLLM
AI repositories🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.
git clone https://github.com/bricks-cloud/BricksLLM.git -
llama-cookbook
AI repositoriesWelcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model family and using them on various provider services
git clone https://github.com/meta-llama/llama-cookbook.git -
TinyLLM
AI repositoriesSetup and run a local LLM and Chatbot using consumer grade hardware.
git clone https://github.com/jasonacox/TinyLLM.git -
smg
AI repositoriesEngine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
git clone https://github.com/smg-project/smg.git -
ramalama
AI repositoriesRamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.
git clone https://github.com/containers/ramalama.git -
inference
AI repositoriesOpen-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.
git clone https://github.com/xorbitsai/inference.git -
pegainfer
AI repositoriesPure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
git clone https://github.com/pegainfer-project/pegainfer.git -
curvine
AI repositoriesAI-native, cloud-native file system in Rust. Full POSIX on object storage with a multi-tier distributed cache, for AI training, inference, and agents. CNCF Sandbox project.
git clone https://github.com/CurvineIO/curvine.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.