InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. LMCache

    AI repositories

    LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

    Indexed GitHub, LMCache 12.4k ★Apache-2.0Updated 2 days ago

    git clone https://github.com/LMCache/LMCache.git
  2. pegainfer

    AI repositories

    Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    Stale GitHub, pegainfer-project 729 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/pegainfer-project/pegainfer.git
  3. vllm

    AI repositories

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Indexed GitHub, vllm-project 97.0k ★Apache-2.0Updated today

    git clone https://github.com/vllm-project/vllm.git
  4. lmdeploy

    AI repositories

    LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

    Stale GitHub, InternLM 8,136 ★Apache-2.0Updated 6 days ago

    git clone https://github.com/InternLM/lmdeploy.git
  5. sglang

    AI repositories

    SGLang is a high-performance serving framework for large language models and multimodal models.

    Indexed GitHub, sgl-project 38.9k ★Apache-2.0Updated today

    git clone https://github.com/sgl-project/sglang.git
  6. skills

    AI repositories

    Agent Skills for NVIDIA products — install into Claude Code, Codex, and other coding agents to run Physical AI, robotics, simulation, CUDA, and RAG workflows end to end.

    Indexed GitHub, NVIDIA 4,010 ★Apache-2.0Updated yesterday

    git clone https://github.com/NVIDIA/skills.git
  7. surogate

    AI repositories

    Train and serve LLMs at extreme speed and massive throughput.

    Stale GitHub, invergent-ai 885 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/invergent-ai/surogate.git
  8. tt-metal

    AI repositories

    :metal: TT-NN operator library, and TT-Metalium low level kernel programming model.

    Stale GitHub, tenstorrent 1,747 ★Apache-2.0Updated 8 days ago

    git clone https://github.com/tenstorrent/tt-metal.git
  9. TensorFold

    AI repositories

    LLM Inference Engine for Metal, CUDA and Vulkan.

    Indexed GitHub, ashhart 2,266 ★Apache-2.0Updated today

    git clone https://github.com/ashhart/TensorFold.git
  10. gpustack

    AI repositories

    A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

    Indexed GitHub, gpustack 6,039 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/gpustack/gpustack.git
  11. inferssd

    AI repositories

    Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

    Indexed GitHub, wici-ai 1,025 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/wici-ai/inferssd.git
  12. raft

    AI repositories

    RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.

    Stale GitHub, NVIDIA 1,051 ★Apache-2.0Updated 13 days ago

    git clone https://github.com/NVIDIA/raft.git
  13. cuvs

    AI repositories

    cuVS - a library for vector search and clustering on the GPU

    Indexed GitHub, NVIDIA 895 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/NVIDIA/cuvs.git
  14. beta9

    AI repositories

    Ultrafast serverless GPU inference, sandboxes, and background jobs

    Indexed GitHub, beam-cloud 1,852 ★AGPL-3.0Updated today

    git clone https://github.com/beam-cloud/beta9.git
  15. ramalama

    AI repositories

    RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

    Stale GitHub, containers 3,097 ★MITUpdated 16 days ago

    git clone https://github.com/containers/ramalama.git
  16. uccl

    AI repositories

    UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

    Indexed GitHub, uccl-project 1,585 ★Apache-2.0Updated yesterday

    git clone https://github.com/uccl-project/uccl.git
  17. starling

    AI repositories

    Local AI in .NET, with no Python sidecar and no API key: embeddings, entity extraction, image and video search and chat on ONNX Runtime (CPU/CUDA), through Microsoft.Extensions.AI.

    Indexed GitHub, starlinghq 102 ★Apache-2.0Updated today

    git clone https://github.com/starlinghq/starling.git
  18. yzma

    AI repositories

    Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.

    Stale GitHub, hybridgroup 652 ★Custom licence (see repository)Updated 32 days ago

    git clone https://github.com/hybridgroup/yzma.git
  19. project-maya

    AI repositories

    Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

    Stale GitHub, mw00 2,482 ★MITUpdated 12 days ago

    git clone https://github.com/mw00/project-maya.git
  20. TensorSharp

    AI repositories

    A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability

    Stale GitHub, zhongkaifu 675 ★BSD-3-ClauseUpdated 4 days ago

    git clone https://github.com/zhongkaifu/TensorSharp.git
  21. PolyStrata

    AI repositories

    Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

    Stale GitHub, VecSzn 722 ★MITUpdated 16 days ago

    git clone https://github.com/VecSzn/PolyStrata.git
  22. self-hosted-ai-stack

    AI repositories

    Deploy a self-hosted AI stack with Docker Compose: local LLMs, chat, document parsing, embeddings, transcription, text-to-speech and MCP tools. Powered by Ollama, LiteLLM, AnythingLLM and more. Local-first with auto-configured authentication, lightweight stacks, optional HTTPS and NVIDIA GPU support. MIT licensed. Multi-arch: amd64, arm64.

    Indexed GitHub, hwdsl2 179 ★MITUpdated yesterday

    git clone https://github.com/hwdsl2/self-hosted-ai-stack.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.