InnerVault

vllm

IndexedGitHub, vllm-project

A high-throughput and memory-efficient inference and serving engine for LLMs

Open the source (opens github.com) https://github.com/vllm-project/vllm

What it is

AI repositories from vllm-project.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/vllm-project/vllm.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. LMCache

    AI repositories

    LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

    Indexed GitHub, LMCache 12.4k ★Apache-2.0Updated 2 days ago

    git clone https://github.com/LMCache/LMCache.git
  2. sglang

    AI repositories

    SGLang is a high-performance serving framework for large language models and multimodal models.

    Indexed GitHub, sgl-project 38.9k ★Apache-2.0Updated today

    git clone https://github.com/sgl-project/sglang.git
  3. pegainfer

    AI repositories

    Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    Stale GitHub, pegainfer-project 729 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/pegainfer-project/pegainfer.git
  4. gpustack

    AI repositories

    A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

    Indexed GitHub, gpustack 6,039 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/gpustack/gpustack.git
  5. project-maya

    AI repositories

    Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

    Stale GitHub, mw00 2,482 ★MITUpdated 12 days ago

    git clone https://github.com/mw00/project-maya.git
  6. uccl

    AI repositories

    UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

    Indexed GitHub, uccl-project 1,585 ★Apache-2.0Updated yesterday

    git clone https://github.com/uccl-project/uccl.git