InnerVault

Library

85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. pegainfer

    AI repositories

    Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    Stale GitHub, pegainfer-project 729 ★Apache-2.0Updated 19 days ago

    git clone https://github.com/pegainfer-project/pegainfer.git
  2. gitleaks

    AI repositories

    Find secrets with Gitleaks 🔑

    Stale GitHub, gitleaks 30.3k ★MITUpdated 25 days ago

    git clone https://github.com/gitleaks/gitleaks.git
  3. litgpt

    AI repositories

    20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

    Stale GitHub, Lightning-AI 13.7k ★Apache-2.0Updated 35 days ago

    git clone https://github.com/Lightning-AI/litgpt.git
  4. rtp-llm

    AI repositories

    RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

    Indexed GitHub, alibaba 1,400 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/alibaba/rtp-llm.git
  5. kvcached

    AI repositories

    Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

    Stale GitHub, ovg-project 2,033 ★Apache-2.0Updated 9 days ago

    git clone https://github.com/ovg-project/kvcached.git
  6. sglang

    AI repositories

    SGLang is a high-performance serving framework for large language models and multimodal models.

    Indexed GitHub, sgl-project 38.9k ★Apache-2.0Updated yesterday

    git clone https://github.com/sgl-project/sglang.git
  7. mistral.rs

    AI repositories

    Fast, flexible LLM inference

    Stale GitHub, EricLBuehler 7,821 ★MITUpdated 8 days ago

    git clone https://github.com/EricLBuehler/mistral.rs.git
  8. lmdeploy

    AI repositories

    LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

    Stale GitHub, InternLM 8,136 ★Apache-2.0Updated 7 days ago

    git clone https://github.com/InternLM/lmdeploy.git
  9. reef

    AI repositories

    Infrastructure for continually self‑improving agents

    Indexed GitHub, Human-Agent-Society 7,823 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/Human-Agent-Society/reef.git
  10. pooled

    AI repositories

    Peer-to-peer LLM inference in the browser: pool your devices to run big open models, for chat and coding agents.

    Stale GitHub, Nehanth 589 ★MITUpdated 32 days ago

    git clone https://github.com/Nehanth/pooled.git
  11. semantic-router

    AI repositories

    An open, programmable decision layer for models and compute.

    Indexed GitHub, vllm-project 6,481 ★Apache-2.0Updated yesterday

    git clone https://github.com/vllm-project/semantic-router.git
  12. mojo-dllm

    AI repositories

    Local inference for diffusion language models (LLaDA, Dream), written in Mojo

    Stale GitHub, vladimirrott 506 ★Apache-2.0Updated 7 days ago

    git clone https://github.com/vladimirrott/mojo-dllm.git
  13. vllm-ascend

    AI repositories

    Community maintained hardware plugin for vLLM on Huawei Ascend

    Indexed GitHub, vllm-project 3,084 ★Apache-2.0Updated yesterday

    git clone https://github.com/vllm-project/vllm-ascend.git
  14. Mooncake

    AI repositories

    Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

    Indexed GitHub, kvcache-ai 7,047 ★Apache-2.0Updated yesterday

    git clone https://github.com/kvcache-ai/Mooncake.git
  15. halogen-flash-server

    AI repositories

    Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.

    Stale GitHub, peonist-ai 1,258 ★Custom licence (see repository)Updated 9 days ago

    git clone https://github.com/peonist-ai/halogen-flash-server.git
  16. ramalama

    AI repositories

    RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

    Stale GitHub, containers 3,097 ★MITUpdated 17 days ago

    git clone https://github.com/containers/ramalama.git
  17. web-llm

    AI repositories

    High-performance In-browser LLM Inference Engine

    Stale GitHub, mlc-ai 19.3k ★Apache-2.0Updated 37 days ago

    git clone https://github.com/mlc-ai/web-llm.git
  18. picollm

    AI repositories

    On-device LLM Inference Powered by X-Bit Quantization

    Indexed GitHub, Picovoice 320 ★Apache-2.0Updated 5 days ago

    git clone https://github.com/Picovoice/picollm.git
  19. runanywhere-sdks

    AI repositories

    Production ready toolkit to run AI locally

    Stale GitHub, RunanywhereAI 10.3k ★Custom licence (see repository)Updated 13 days ago

    git clone https://github.com/RunanywhereAI/runanywhere-sdks.git
  20. beta9

    AI repositories

    Ultrafast serverless GPU inference, sandboxes, and background jobs

    Indexed GitHub, beam-cloud 1,852 ★AGPL-3.0Updated yesterday

    git clone https://github.com/beam-cloud/beta9.git
  21. agent-router

    AI repositories

    Manages Unified Access to Generative AI Services built on Envoy Gateway

    Stale GitHub, theagentrouter 2,353 ★Apache-2.0Updated 12 days ago

    git clone https://github.com/theagentrouter/agent-router.git
  22. mistral-inference

    AI repositories

    Official inference library for Mistral models

    Stale Archived GitHub, mistralai 10.8k ★Apache-2.0Updated 5 months ago 1 note

    git clone https://github.com/mistralai/mistral-inference.git
  23. ray

    AI repositories

    Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

    Indexed GitHub, ray-project 44.3k ★Apache-2.0Updated yesterday

    git clone https://github.com/ray-project/ray.git
  24. chatllm.cpp

    AI repositories

    Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)

    Stale GitHub, foldl 932 ★MITUpdated 30 days ago

    git clone https://github.com/foldl/chatllm.cpp.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.