InnerVault

Library

85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. omlx

    AI repositories

    LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

    Indexed GitHub, jundot 23.7k ★Apache-2.0Updated yesterday

    git clone https://github.com/jundot/omlx.git
  2. wllama

    AI repositories

    WebAssembly binding for llama.cpp - Enabling on-browser LLM inference

    Stale GitHub, ngxson 1,363 ★MITUpdated 11 days ago

    git clone https://github.com/ngxson/wllama.git
  3. kvcached

    AI repositories

    Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

    Stale GitHub, ovg-project 2,033 ★Apache-2.0Updated 9 days ago

    git clone https://github.com/ovg-project/kvcached.git
  4. BentoML

    AI repositories

    The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

    Stale GitHub, bentoml 8,935 ★Apache-2.0Updated 15 days ago

    git clone https://github.com/bentoml/BentoML.git
  5. airllm

    AI repositories

    AirLLM 2.8T LLM inference with single 4GB GPU

    Indexed GitHub, lyogavin 42.5k ★Apache-2.0Updated yesterday

    git clone https://github.com/lyogavin/airllm.git
  6. AI-Engineer-Headquarters

    AI repositories

    A collection of scientific methods, processes, algorithms, and systems to build stories & models.

    Stale GitHub, hemansnation 3,776 ★Updated 20 days ago

    git clone https://github.com/hemansnation/AI-Engineer-Headquarters.git
  7. ecologits

    AI repositories

    🌱 EcoLogits tracks the energy consumption and environmental footprint of using generative AI models through APIs.

    Stale GitHub, mlco2 354 ★MPL-2.0Updated 22 days ago

    git clone https://github.com/mlco2/ecologits.git
  8. beta9

    AI repositories

    Ultrafast serverless GPU inference, sandboxes, and background jobs

    Indexed GitHub, beam-cloud 1,852 ★AGPL-3.0Updated yesterday

    git clone https://github.com/beam-cloud/beta9.git
  9. LLM-TPU

    AI repositories

    Run generative AI models in sophgo BM1684X/BM1688

    Stale GitHub, sophgo 321 ★Custom licence (see repository)Updated 8 days ago

    git clone https://github.com/sophgo/LLM-TPU.git
  10. tiny-llm

    AI repositories

    learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen

    Stale GitHub, skyzh 5,680 ★Apache-2.0Updated 10 days ago

    git clone https://github.com/skyzh/tiny-llm.git
  11. nanocoder

    AI repositories

    An open coding agent for your terminal, built by a community collective rather than a company. Bring your own model, keep your code on your machine, and owe nothing to anyone.

    Indexed GitHub, Nano-Collective 2,573 ★Custom licence (see repository)Updated yesterday

    git clone https://github.com/Nano-Collective/nanocoder.git
  12. neuron-ai

    AI repositories

    The Agentic Framework of the PHP ecosystem to build production-ready AI driven applications. Connect components (LLMs, Tools, vector DBs, memory) to agents that interact with your data and UI.

    Stale GitHub, neuron-core 2,162 ★MITUpdated 18 days ago

    git clone https://github.com/neuron-core/neuron-ai.git
  13. plano

    AI repositories

    Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.

    Stale GitHub, katanemo 7,099 ★Apache-2.0Updated 24 days ago

    git clone https://github.com/katanemo/plano.git
  14. rtp-llm

    AI repositories

    RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

    Indexed GitHub, alibaba 1,400 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/alibaba/rtp-llm.git
  15. kimi-k3-in-c

    AI repositories

    A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

    Stale GitHub, FareedKhan-dev 9,049 ★Apache-2.0Updated 46 days ago

    git clone https://github.com/FareedKhan-dev/kimi-k3-in-c.git
  16. openlake

    AI repositories

    OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

    Stale GitHub, openlake-project 2,851 ★Apache-2.0Updated 8 days ago

    git clone https://github.com/openlake-project/openlake.git
  17. MiniSearch

    AI repositories

    Minimalist web-searching platform with an AI assistant that runs directly from your browser. Demo: https://felladrin-minisearch.hf.space

    Indexed GitHub, felladrin 609 ★Apache-2.0Updated yesterday

    git clone https://github.com/felladrin/MiniSearch.git
  18. ome

    AI repositories

    Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

    Stale GitHub, ome-projects 523 ★Apache-2.0Updated 14 days ago

    git clone https://github.com/ome-projects/ome.git
  19. deepreasoning

    AI repositories

    A high-performance LLM inference API and Chat UI that integrates DeepSeek R1's CoT reasoning traces with Anthropic Claude models.

    Stale GitHub, winfunc 5,355 ★MITUpdated 13 months ago

    git clone https://github.com/winfunc/deepreasoning.git
  20. tokenspeed

    AI repositories

    TokenSpeed is a speed-of-light LLM inference engine.

    Indexed GitHub, lightseekorg 2,932 ★MITUpdated yesterday

    git clone https://github.com/lightseekorg/tokenspeed.git
  21. awesome-ai-web-search

    AI repositories

    List of software that allows searching the web with the assistance of AI: https://hf.co/spaces/felladrin/awesome-ai-web-search

    Stale GitHub, felladrin 1,506 ★CC0-1.0Updated 11 days ago

    git clone https://github.com/felladrin/awesome-ai-web-search.git
  22. Atomic-Chat

    AI repositories

    Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V

    Stale GitHub, AtomicBot-ai 1,808 ★Custom licence (see repository)Updated 31 days ago

    git clone https://github.com/AtomicBot-ai/Atomic-Chat.git
  23. jev

    AI repositories

    jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.

    Stale GitHub, feder-cr 1,342 ★MITUpdated 32 days ago

    git clone https://github.com/feder-cr/jev.git
  24. ShannonBase

    AI repositories

    The Next-Gen Database for AI—an infrastructure designed for data and AI. As the MySQL of the AI era.

    Indexed GitHub, Shannon-Data 200 ★Custom licence (see repository)Updated yesterday

    git clone https://github.com/Shannon-Data/ShannonBase.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.