InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

  1. nano-vllm

    AI repositories

    Nano vLLM

    Stale GitHub, GeeeekExplorer 15.8k ★MITUpdated 6 months ago

    git clone https://github.com/GeeeekExplorer/nano-vllm.git
  2. vllm

    AI repositories

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Indexed GitHub, vllm-project 97.0k ★Apache-2.0Updated today

    git clone https://github.com/vllm-project/vllm.git
  3. vllm-ascend

    AI repositories

    Community maintained hardware plugin for vLLM on Huawei Ascend

    Indexed GitHub, vllm-project 3,084 ★Apache-2.0Updated today

    git clone https://github.com/vllm-project/vllm-ascend.git
  4. vllm-metal

    AI repositories

    Community maintained hardware plugin for vLLM on Apple Silicon

    Stale GitHub, vllm-project 1,987 ★Apache-2.0Updated 4 days ago

    git clone https://github.com/vllm-project/vllm-metal.git
  5. tiny-llm

    AI repositories

    learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen

    Stale GitHub, skyzh 5,680 ★Apache-2.0Updated 9 days ago

    git clone https://github.com/skyzh/tiny-llm.git
  6. semantic-router

    AI repositories

    An open, programmable decision layer for models and compute.

    Indexed GitHub, vllm-project 6,481 ★Apache-2.0Updated today

    git clone https://github.com/vllm-project/semantic-router.git
  7. LMCache

    AI repositories

    LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

    Indexed GitHub, LMCache 12.4k ★Apache-2.0Updated 2 days ago

    git clone https://github.com/LMCache/LMCache.git
  8. vllm-mlx

    AI repositories

    High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.

    Stale GitHub, waybarrios 1,646 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/waybarrios/vllm-mlx.git
  9. gpustack

    AI repositories

    A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

    Indexed GitHub, gpustack 6,039 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/gpustack/gpustack.git
  10. sparrow

    AI repositories

    Structured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLM

    Stale GitHub, katanaml 5,226 ★GPL-3.0Updated 39 days ago

    git clone https://github.com/katanaml/sparrow.git
  11. Mooncake

    AI repositories

    Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

    Indexed GitHub, kvcache-ai 7,047 ★Apache-2.0Updated today

    git clone https://github.com/kvcache-ai/Mooncake.git
  12. UltraRAG

    AI repositories

    A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines

    Indexed GitHub, OpenBMB 5,743 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/OpenBMB/UltraRAG.git
  13. llmman

    AI repositories

    Run any agent on any model, models stored as OCI images

    Indexed GitHub, llmmanorg 590 ★Apache-2.0Updated yesterday

    git clone https://github.com/llmmanorg/llmman.git
  14. Halfrost-Field

    AI repositories

    ✍🏻 Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field 冰霜之地:源码解析、系统设计与工程实践笔记

    Stale GitHub, halfrost 13.2k ★CC-BY-SA-4.0Updated 47 days ago

    git clone https://github.com/halfrost/Halfrost-Field.git
  15. ome

    AI repositories

    Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

    Stale GitHub, ome-projects 523 ★Apache-2.0Updated 13 days ago

    git clone https://github.com/ome-projects/ome.git
  16. kvcached

    AI repositories

    Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

    Stale GitHub, ovg-project 2,033 ★Apache-2.0Updated 8 days ago

    git clone https://github.com/ovg-project/kvcached.git
  17. BricksLLM

    AI repositories

    🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.

    Stale GitHub, bricks-cloud 1,230 ★MITUpdated 22 months ago

    git clone https://github.com/bricks-cloud/BricksLLM.git
  18. llama-cookbook

    AI repositories

    Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model family and using them on various provider services

    Stale GitHub, meta-llama 18.6k ★MITUpdated 6 months ago

    git clone https://github.com/meta-llama/llama-cookbook.git
  19. TinyLLM

    AI repositories

    Setup and run a local LLM and Chatbot using consumer grade hardware.

    Stale GitHub, jasonacox 353 ★MITUpdated 22 days ago

    git clone https://github.com/jasonacox/TinyLLM.git
  20. smg

    AI repositories

    Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

    Indexed GitHub, smg-project 723 ★Apache-2.0Updated today

    git clone https://github.com/smg-project/smg.git
  21. ramalama

    AI repositories

    RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

    Stale GitHub, containers 3,097 ★MITUpdated 16 days ago

    git clone https://github.com/containers/ramalama.git
  22. inference

    AI repositories

    Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.

    Indexed GitHub, xorbitsai 9,660 ★Apache-2.0Updated today

    git clone https://github.com/xorbitsai/inference.git
  23. pegainfer

    AI repositories

    Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    Stale GitHub, pegainfer-project 729 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/pegainfer-project/pegainfer.git
  24. curvine

    AI repositories

    AI-native, cloud-native file system in Rust. Full POSIX on object storage with a multi-tier distributed cache, for AI training, inference, and agents. CNCF Sandbox project.

    Indexed GitHub, CurvineIO 1,068 ★Apache-2.0Updated yesterday

    git clone https://github.com/CurvineIO/curvine.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.