InnerVault

LMCache

IndexedGitHub, LMCache

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

Open the source (opens github.com) https://github.com/lmcache/lmcache

What it is

AI repositories from LMCache.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/LMCache/LMCache.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. vllm

    AI repositories

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Indexed GitHub, vllm-project 97.0k ★Apache-2.0Updated today

    git clone https://github.com/vllm-project/vllm.git
  2. project-maya

    AI repositories

    Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

    Stale GitHub, mw00 2,482 ★MITUpdated 12 days ago

    git clone https://github.com/mw00/project-maya.git
  3. uccl

    AI repositories

    UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

    Indexed GitHub, uccl-project 1,585 ★Apache-2.0Updated yesterday

    git clone https://github.com/uccl-project/uccl.git
  4. pegainfer

    AI repositories

    Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    Stale GitHub, pegainfer-project 729 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/pegainfer-project/pegainfer.git
  5. halogen-flash-server

    AI repositories

    Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.

    Stale GitHub, peonist-ai 1,258 ★Custom licence (see repository)Updated 8 days ago

    git clone https://github.com/peonist-ai/halogen-flash-server.git
  6. lmdeploy

    AI repositories

    LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

    Stale GitHub, InternLM 8,136 ★Apache-2.0Updated 6 days ago

    git clone https://github.com/InternLM/lmdeploy.git