Library
85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
pegainfer
AI repositoriesPure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
git clone https://github.com/pegainfer-project/pegainfer.git -
gitleaks
AI repositoriesFind secrets with Gitleaks 🔑
git clone https://github.com/gitleaks/gitleaks.git -
litgpt
AI repositories20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
git clone https://github.com/Lightning-AI/litgpt.git -
rtp-llm
AI repositoriesRTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
git clone https://github.com/alibaba/rtp-llm.git -
kvcached
AI repositoriesVirtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
git clone https://github.com/ovg-project/kvcached.git -
sglang
AI repositoriesSGLang is a high-performance serving framework for large language models and multimodal models.
git clone https://github.com/sgl-project/sglang.git -
mistral.rs
AI repositoriesFast, flexible LLM inference
git clone https://github.com/EricLBuehler/mistral.rs.git -
lmdeploy
AI repositoriesLMDeploy is a toolkit for compressing, deploying, and serving LLMs.
git clone https://github.com/InternLM/lmdeploy.git -
reef
AI repositoriesInfrastructure for continually self‑improving agents
git clone https://github.com/Human-Agent-Society/reef.git -
pooled
AI repositoriesPeer-to-peer LLM inference in the browser: pool your devices to run big open models, for chat and coding agents.
git clone https://github.com/Nehanth/pooled.git -
semantic-router
AI repositoriesAn open, programmable decision layer for models and compute.
git clone https://github.com/vllm-project/semantic-router.git -
mojo-dllm
AI repositoriesLocal inference for diffusion language models (LLaDA, Dream), written in Mojo
git clone https://github.com/vladimirrott/mojo-dllm.git -
vllm-ascend
AI repositoriesCommunity maintained hardware plugin for vLLM on Huawei Ascend
git clone https://github.com/vllm-project/vllm-ascend.git -
Mooncake
AI repositoriesMooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
git clone https://github.com/kvcache-ai/Mooncake.git -
halogen-flash-server
AI repositoriesLocal inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.
git clone https://github.com/peonist-ai/halogen-flash-server.git -
ramalama
AI repositoriesRamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.
git clone https://github.com/containers/ramalama.git -
web-llm
AI repositoriesHigh-performance In-browser LLM Inference Engine
git clone https://github.com/mlc-ai/web-llm.git -
picollm
AI repositoriesOn-device LLM Inference Powered by X-Bit Quantization
git clone https://github.com/Picovoice/picollm.git -
runanywhere-sdks
AI repositoriesProduction ready toolkit to run AI locally
git clone https://github.com/RunanywhereAI/runanywhere-sdks.git -
beta9
AI repositoriesUltrafast serverless GPU inference, sandboxes, and background jobs
git clone https://github.com/beam-cloud/beta9.git -
agent-router
AI repositoriesManages Unified Access to Generative AI Services built on Envoy Gateway
git clone https://github.com/theagentrouter/agent-router.git -
mistral-inference
AI repositoriesOfficial inference library for Mistral models
git clone https://github.com/mistralai/mistral-inference.git -
ray
AI repositoriesRay is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
git clone https://github.com/ray-project/ray.git -
chatllm.cpp
AI repositoriesPure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
git clone https://github.com/foldl/chatllm.cpp.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.