InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to connect AI to my tools

  1. llm-action

    AI repositories

    本项目旨在分享大模型相关技术原理以及实战经验(大模型工程化、大模型应用落地)

    Stale GitHub, liguodongiot 25.2k ★Apache-2.0Updated 33 days ago

    git clone https://github.com/liguodongiot/llm-action.git
  2. awesome-llm-services

    Docs & tutorials

    A list of self-hostable LLM services

    Indexed GitHub, av 281 ★MITUpdated today

    git clone https://github.com/av/awesome-llm-services.git
  3. vllm

    AI repositories

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Indexed GitHub, vllm-project 97.0k ★Apache-2.0Updated today

    git clone https://github.com/vllm-project/vllm.git
  4. vllm-ascend

    AI repositories

    Community maintained hardware plugin for vLLM on Huawei Ascend

    Indexed GitHub, vllm-project 3,084 ★Apache-2.0Updated today

    git clone https://github.com/vllm-project/vllm-ascend.git
  5. chitu

    AI repositories

    High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.

    Stale GitHub, thu-pacman 2,998 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/thu-pacman/chitu.git
  6. rtp-llm

    AI repositories

    RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

    Indexed GitHub, alibaba 1,400 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/alibaba/rtp-llm.git
  7. superduper

    AI repositories

    Superduper: End-to-end framework for building custom AI applications and agents.

    Stale GitHub, superduper-io 5,327 ★Apache-2.0Updated 14 months ago

    git clone https://github.com/superduper-io/superduper.git
  8. ai-infra-book

    AI repositories

    《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验

    Stale GitHub, bojieli 6,489 ★Apache-2.0Updated 46 days ago

    git clone https://github.com/bojieli/ai-infra-book.git
  9. ray

    AI repositories

    Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

    Indexed GitHub, ray-project 44.3k ★Apache-2.0Updated today

    git clone https://github.com/ray-project/ray.git
  10. gpustack

    AI repositories

    A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

    Indexed GitHub, gpustack 6,039 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/gpustack/gpustack.git
  11. OpenLLM

    AI repositories

    Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

    Stale GitHub, bentoml 12.6k ★Apache-2.0Updated 25 days ago

    git clone https://github.com/bentoml/OpenLLM.git
  12. kvcached

    AI repositories

    Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

    Stale GitHub, ovg-project 2,033 ★Apache-2.0Updated 8 days ago

    git clone https://github.com/ovg-project/kvcached.git
  13. Local LLM serving setup audit: a deliverable delivered per API call as an independent atom service.

    Indexed Official MCP Registry, org.aimlds Updated today 1 note

  14. BentoML

    AI repositories

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

    Stale GitHub, bentoml 8,935 ★Apache-2.0Updated 14 days ago

    git clone https://github.com/bentoml/BentoML.git
  15. Mooncake

    AI repositories

    Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

    Indexed GitHub, kvcache-ai 7,047 ★Apache-2.0Updated today

    git clone https://github.com/kvcache-ai/Mooncake.git
  16. sugarcane-ai

    Prompts

    npm like package ecosystem for Prompts 🤖

    Stale GitHub, sugarcane-ai 52 ★Apache-2.0Updated 21 months ago

    git clone https://github.com/sugarcane-ai/sugarcane-ai.git
  17. pegainfer

    AI repositories

    Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    Stale GitHub, pegainfer-project 729 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/pegainfer-project/pegainfer.git
  18. LLM API proxy. Same prompt twice costs one call.

    Indexed GitHub, Mattbusel 61 ★MITUpdated today

    git clone https://github.com/Mattbusel/tokio-prompt-orchestrator.git
  19. Flide

    Agents & frameworks

    🔥🔥 The extremely fast inference engine for decision model

    Stale GitHub, OmniJev 396 ★Apache-2.0Updated 5 days ago

    git clone https://github.com/OmniJev/Flide.git
  20. helix

    Agents & frameworks

    ♾️ Private Agent Fleet with Spec Coding. Each agent gets their own GPU-accelerated desktop. Run Claude, Codex, Gemini and open models on a full private AI Stack ♾️

    Indexed GitHub, helixml 831 ★Custom licence (see repository)Updated yesterday

    git clone https://github.com/helixml/helix.git
  21. aa-agentperf-local

    Datasets & benchmarks

    Benchmark local LLM serving by replaying real agent trajectories

    Stale GitHub, ArtificialAnalysis 105 ★Apache-2.0Updated 31 days ago

    git clone https://github.com/ArtificialAnalysis/aa-agentperf-local.git
  22. halogen-flash-server

    AI repositories

    Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.

    Stale GitHub, peonist-ai 1,258 ★Custom licence (see repository)Updated 8 days ago

    git clone https://github.com/peonist-ai/halogen-flash-server.git
  23. flama

    AI repositories

    The production framework for Predictive and Generative AI. Serve any model as an API in one line, with OpenAI/Anthropic/Ollama-compatible endpoints, a built-in chat UI, and native MCP.

    Indexed GitHub, vortico 307 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/vortico/flama.git
  24. ome

    AI repositories

    Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

    Stale GitHub, ome-projects 523 ★Apache-2.0Updated 13 days ago

    git clone https://github.com/ome-projects/ome.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.