InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. LMCache

    AI repositories

    LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

    Indexed GitHub, LMCache 12.4k ★Apache-2.0Updated 2 days ago

    git clone https://github.com/LMCache/LMCache.git
  2. C2C

    Agents & frameworks

    [ICLR'26] The official code implementation for "Cache-to-Cache: Direct Semantic Communication Between Large Language Models"

    Stale GitHub, thu-nics 757 ★Apache-2.0Updated 19 days ago

    git clone https://github.com/thu-nics/C2C.git
  3. ai-infra-book

    AI repositories

    《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验

    Stale GitHub, bojieli 6,489 ★Apache-2.0Updated 46 days ago

    git clone https://github.com/bojieli/ai-infra-book.git
  4. soul

    Agents & frameworks

    Soul: The Persistent Memory Layer for Multi-Agent Systems. High-performance session orchestration with SQLite-based KV-Cache for MCP. Give your AI agents a soul that never forgets.

    Stale GitHub, choihyunsus 68 ★Custom licence (see repository)Updated 8 days ago

    git clone https://github.com/choihyunsus/soul.git
  5. R-KV

    AI repositories

    [Neurips 2025] R-KV: Redundancy-aware KV Cache Compression for Reasoning Models

    Stale GitHub, Zefan-Cai 1,217 ★Updated 29 days ago

    git clone https://github.com/Zefan-Cai/R-KV.git
  6. zero-to-sglang

    Docs & tutorials

    Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

    Indexed GitHub, datawhalechina 2,148 ★Updated today

    git clone https://github.com/datawhalechina/zero-to-sglang.git
  7. Glyd

    Datasets & benchmarks

    Lossless AI compression: 33% less GPU memory, bit for bit.

    Stale GitHub, surya-koritala 321 ★Custom licence (see repository)Updated 14 days ago

    git clone https://github.com/surya-koritala/Glyd.git
  8. pegainfer

    AI repositories

    Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    Stale GitHub, pegainfer-project 729 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/pegainfer-project/pegainfer.git
  9. Flide

    Agents & frameworks

    🔥🔥 The extremely fast inference engine for decision model

    Stale GitHub, OmniJev 396 ★Apache-2.0Updated 5 days ago

    git clone https://github.com/OmniJev/Flide.git
  10. kvcached

    AI repositories

    Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

    Stale GitHub, ovg-project 2,033 ★Apache-2.0Updated 8 days ago

    git clone https://github.com/ovg-project/kvcached.git
  11. A curated list of strategies, tools, papers, and resources for reducing LLM token costs and improving efficiency in production.

    Indexed GitHub, pleasedodisturb 123 ★Custom licence (see repository)Updated yesterday

    git clone https://github.com/pleasedodisturb/awesome-llm-token-optimization.git
  12. uccl

    AI repositories

    UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

    Indexed GitHub, uccl-project 1,585 ★Apache-2.0Updated yesterday

    git clone https://github.com/uccl-project/uccl.git
  13. SwiftLM

    AI repositories

    ⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.

    Indexed GitHub, SharpAI 799 ★MITUpdated today

    git clone https://github.com/SharpAI/SwiftLM.git
  14. taOS

    Agents & frameworks

    Self-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC).

    Indexed GitHub, jaylfc 608 ★AGPL-3.0Updated yesterday

    git clone https://github.com/jaylfc/taOS.git
  15. mingxin-mcp-server

    MCP servers

    Measured AI-inference-storage benchmarks with citations, article search, KV-cache ROI estimation.

    Stale Official MCP Registry, xyz.mingxinstorage Updated 3 months ago 2 notes

    mingxin-mcp-server@1.0.0
  16. tokcalc MCP Server

    MCP servers

    LLM serving capacity planner: VRAM, KV cache, GPU topology, latency, and cost.

    Stale Official MCP Registry, io.github.stevecrates489-commits Updated 23 days ago 2 notes

    @tokcalc/mcp-server@0.3.4
  17. FitLLM

    MCP servers

    Will this LLM fit on your GPU, multi-GPU rig or Mac? Exact VRAM & KV-cache math. Read-only.

    Stale Official MCP Registry, run.fitllm Updated 2 months ago 1 note

  18. Does an LLM fit on your GPU? VRAM, KV cache and GPU count for any Hugging Face model.

    Stale Official MCP Registry, com.studiotvai Updated 37 days ago 1 note

  19. smg

    AI repositories

    Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

    Indexed GitHub, smg-project 723 ★Apache-2.0Updated today

    git clone https://github.com/smg-project/smg.git
  20. shepherd

    Tools & apps

    A runtime substrate that turns an agent's execution into a reversible, Git-like trace, so meta-agents can observe, fork, replay, and revert any run. Couples agent and environments in a copy-on-write fork ~5x faster than docker commit, with ~95% KV-cache reuse on replay. Framework built for meta-agents to supervise, optimize, and train other agents

    Stale GitHub, shepherd-agents 2,483 ★MITUpdated 2 months ago

    git clone https://github.com/shepherd-agents/shepherd.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.