InnerVault

Library

85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. inferssd

    AI repositories

    Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

    Indexed GitHub, wici-ai 1,025 ★Apache-2.0Updated 4 days ago

    git clone https://github.com/wici-ai/inferssd.git
  2. ai-infra-book

    AI repositories

    《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验

    Stale GitHub, bojieli 6,489 ★Apache-2.0Updated 47 days ago

    git clone https://github.com/bojieli/ai-infra-book.git
  3. kimi-k3-in-c

    AI repositories

    A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

    Stale GitHub, FareedKhan-dev 9,049 ★Apache-2.0Updated 46 days ago

    git clone https://github.com/FareedKhan-dev/kimi-k3-in-c.git
  4. BigMoeOnEdge

    AI repositories

    Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.

    Indexed GitHub, Helldez 924 ★Apache-2.0Updated yesterday

    git clone https://github.com/Helldez/BigMoeOnEdge.git
  5. halogen-flash-server

    AI repositories

    Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.

    Stale GitHub, peonist-ai 1,258 ★Custom licence (see repository)Updated 9 days ago

    git clone https://github.com/peonist-ai/halogen-flash-server.git
  6. project-maya

    AI repositories

    Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

    Stale GitHub, mw00 2,482 ★MITUpdated 13 days ago

    git clone https://github.com/mw00/project-maya.git
  7. janas

    AI repositories

    37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

    Indexed GitHub, prabanta-dev 717 ★GPL-3.0Updated yesterday

    git clone https://github.com/prabanta-dev/janas.git
  8. PolyStrata

    AI repositories

    Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

    Stale GitHub, VecSzn 722 ★MITUpdated 17 days ago

    git clone https://github.com/VecSzn/PolyStrata.git
  9. slotstream

    AI repositories

    Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.

    Indexed GitHub, carloslfu 673 ★MITUpdated yesterday

    git clone https://github.com/carloslfu/slotstream.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.