InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to connect AI to my tools

  1. vllm

    AI repositories

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Indexed GitHub, vllm-project 97.0k ★Apache-2.0Updated today

    git clone https://github.com/vllm-project/vllm.git
  2. vllm-ascend

    AI repositories

    Community maintained hardware plugin for vLLM on Huawei Ascend

    Indexed GitHub, vllm-project 3,084 ★Apache-2.0Updated today

    git clone https://github.com/vllm-project/vllm-ascend.git
  3. llm-d

    AI repositories

    State of the art inference performance with modern accelerators on Kubernetes

    Indexed GitHub, llm-d 5,274 ★Apache-2.0Updated today

    git clone https://github.com/llm-d/llm-d.git
  4. chitu

    AI repositories

    High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.

    Stale GitHub, thu-pacman 2,998 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/thu-pacman/chitu.git
  5. rtp-llm

    AI repositories

    RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

    Indexed GitHub, alibaba 1,400 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/alibaba/rtp-llm.git
  6. Olares

    Agents & frameworks

    Open-Source Personal Cloud OS for Always-On Agents

    Indexed GitHub, beclab 5,362 ★AGPL-3.0Updated today

    git clone https://github.com/beclab/Olares.git
  7. openlake

    AI repositories

    OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

    Stale GitHub, openlake-project 2,851 ★Apache-2.0Updated 7 days ago

    git clone https://github.com/openlake-project/openlake.git
  8. LightLLM

    AI repositories

    LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

    Indexed GitHub, ModelTC 4,362 ★Apache-2.0Updated today

    git clone https://github.com/ModelTC/LightLLM.git
  9. PowerInfer

    AI repositories

    High-speed Large Language Model Serving for Local Deployment

    Stale GitHub, Tiiny-AI 9,828 ★MITUpdated 6 months ago

    git clone https://github.com/Tiiny-AI/PowerInfer.git
  10. VMware Private AI (NVIDIA): GPU inventory, vGPU profiles, utilization, model serving — 20 tools

    Stale Official MCP Registry, io.github.vmware-skills Updated 5 days ago 1 note

    vmware-privateai@2.5.3
  11. BentoML

    AI repositories

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

    Stale GitHub, bentoml 8,935 ★Apache-2.0Updated 14 days ago

    git clone https://github.com/bentoml/BentoML.git
  12. pegainfer

    AI repositories

    Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    Stale GitHub, pegainfer-project 729 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/pegainfer-project/pegainfer.git
  13. ome

    AI repositories

    Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

    Stale GitHub, ome-projects 523 ★Apache-2.0Updated 13 days ago

    git clone https://github.com/ome-projects/ome.git
  14. gpustack

    AI repositories

    A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

    Indexed GitHub, gpustack 6,039 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/gpustack/gpustack.git
  15. flama

    AI repositories

    The production framework for Predictive and Generative AI. Serve any model as an API in one line, with OpenAI/Anthropic/Ollama-compatible endpoints, a built-in chat UI, and native MCP.

    Indexed GitHub, vortico 307 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/vortico/flama.git
  16. ollaya

    MCP servers

    Jev-compatible API server for open decision models. Ollama for decision models: pull and serve 29 model families (Laya, pplx-decider, Clef, Winnow, coba...) locally or self-hosted behind /v1/systemone. The TypeSafe SDK works unchanged: set its base URL and pick an open model. CLI, desktop app, Docker, MCP server, public bench.

    Stale GitHub, ollaya-dev 1,436 ★Apache-2.0Updated 10 days ago

    git clone https://github.com/ollaya-dev/ollaya.git
  17. Persome

    MCP servers

    Local-first personal memory and model server for trusted MCP agents on macOS.

    Stale Official MCP Registry, io.github.Intuition-Lab Updated 4 months ago 1 note

    personal-model@0.3.2
  18. HARTOS

    Agents & frameworks

    An AI-native OS. Models run on your own hardware, nodes federate peer-to-peer with no broker, and the API is OpenAI-compatible. Boots, has its own Wayland compositor, and runs on 8GB. Apache 2.0.

    Indexed GitHub, hertz-ai 52 ★Apache-2.0Updated today

    git clone https://github.com/hertz-ai/HARTOS.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.