Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
Looks like a goal we have a starting path for: I want to connect AI to my tools
-
llm-action
AI repositories本项目旨在分享大模型相关技术原理以及实战经验(大模型工程化、大模型应用落地)
git clone https://github.com/liguodongiot/llm-action.git -
vllm
AI repositoriesA high-throughput and memory-efficient inference and serving engine for LLMs
git clone https://github.com/vllm-project/vllm.git -
vllm-ascend
AI repositoriesCommunity maintained hardware plugin for vLLM on Huawei Ascend
git clone https://github.com/vllm-project/vllm-ascend.git -
chitu
AI repositoriesHigh-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.
git clone https://github.com/thu-pacman/chitu.git -
rtp-llm
AI repositoriesRTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
git clone https://github.com/alibaba/rtp-llm.git -
superduper
AI repositoriesSuperduper: End-to-end framework for building custom AI applications and agents.
git clone https://github.com/superduper-io/superduper.git -
ai-infra-book
AI repositories《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
git clone https://github.com/bojieli/ai-infra-book.git -
ray
AI repositoriesRay is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
git clone https://github.com/ray-project/ray.git -
gpustack
AI repositoriesA GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
git clone https://github.com/gpustack/gpustack.git -
OpenLLM
AI repositoriesRun any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
git clone https://github.com/bentoml/OpenLLM.git -
kvcached
AI repositoriesVirtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
git clone https://github.com/ovg-project/kvcached.git -
BentoML
AI repositoriesThe easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
git clone https://github.com/bentoml/BentoML.git -
Mooncake
AI repositoriesMooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
git clone https://github.com/kvcache-ai/Mooncake.git -
pegainfer
AI repositoriesPure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
git clone https://github.com/pegainfer-project/pegainfer.git -
halogen-flash-server
AI repositoriesLocal inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.
git clone https://github.com/peonist-ai/halogen-flash-server.git -
flama
AI repositoriesThe production framework for Predictive and Generative AI. Serve any model as an API in one line, with OpenAI/Anthropic/Ollama-compatible endpoints, a built-in chat UI, and native MCP.
git clone https://github.com/vortico/flama.git -
ome
AI repositoriesOpen Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton
git clone https://github.com/ome-projects/ome.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.