InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

  1. Firefly

    AI repositories

    Firefly: 大模型训练工具,支持训练Qwen2.5、Qwen2、Yi1.5、Phi-3、Llama3、Gemma、MiniCPM、Yi、Deepseek、Orion、Xverse、Mixtral-8x7B、Zephyr、Mistral、Baichuan2、Llma2、Llama、Qwen、Baichuan、ChatGLM2、InternLM、Ziya2、Vicuna、Bloom等大模型

    Stale GitHub, yangjianxin1 6,648 ★Updated 2 years ago

    git clone https://github.com/yangjianxin1/Firefly.git
  2. semantic-router

    AI repositories

    An open, programmable decision layer for models and compute.

    Indexed GitHub, vllm-project 6,481 ★Apache-2.0Updated today

    git clone https://github.com/vllm-project/semantic-router.git
  3. maxtext

    AI repositories

    A simple, performant, and scalable Jax LLM!

    Indexed GitHub, AI-Hypercomputer 2,461 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/AI-Hypercomputer/maxtext.git
  4. ai-infra-book

    AI repositories

    《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验

    Stale GitHub, bojieli 6,489 ★Apache-2.0Updated 46 days ago

    git clone https://github.com/bojieli/ai-infra-book.git
  5. inferssd

    AI repositories

    Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

    Indexed GitHub, wici-ai 1,025 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/wici-ai/inferssd.git
  6. nncf

    AI repositories

    Neural Network Compression Framework for enhanced OpenVINO™ inference

    Stale GitHub, openvinotoolkit 1,217 ★Apache-2.0Updated 12 days ago

    git clone https://github.com/openvinotoolkit/nncf.git
  7. aikit

    AI repositories

    🏗️ Fine-tune, build, and deploy open-source LLMs easily!

    Indexed GitHub, kaito-project 544 ★MITUpdated 2 days ago

    git clone https://github.com/kaito-project/aikit.git
  8. kimi-k3-in-c

    AI repositories

    A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

    Stale GitHub, FareedKhan-dev 9,049 ★Apache-2.0Updated 45 days ago

    git clone https://github.com/FareedKhan-dev/kimi-k3-in-c.git
  9. xTuring

    AI repositories

    Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6

    Stale GitHub, stochasticai 2,679 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/stochasticai/xTuring.git
  10. picollm

    AI repositories

    On-device LLM Inference Powered by X-Bit Quantization

    Indexed GitHub, Picovoice 320 ★Apache-2.0Updated 4 days ago

    git clone https://github.com/Picovoice/picollm.git
  11. h2ogpt

    AI repositories

    Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://gpt-docs.h2o.ai/

    Stale Archived GitHub, h2oai 11.9k ★Apache-2.0Updated 13 months ago 1 note

    git clone https://github.com/h2oai/h2ogpt.git
  12. BigMoeOnEdge

    AI repositories

    Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.

    Indexed GitHub, Helldez 924 ★Apache-2.0Updated today

    git clone https://github.com/Helldez/BigMoeOnEdge.git
  13. halogen-flash-server

    AI repositories

    Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.

    Stale GitHub, peonist-ai 1,258 ★Custom licence (see repository)Updated 8 days ago

    git clone https://github.com/peonist-ai/halogen-flash-server.git
  14. project-maya

    AI repositories

    Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

    Stale GitHub, mw00 2,482 ★MITUpdated 12 days ago

    git clone https://github.com/mw00/project-maya.git
  15. janas

    AI repositories

    37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

    Indexed GitHub, prabanta-dev 717 ★GPL-3.0Updated today

    git clone https://github.com/prabanta-dev/janas.git
  16. PolyStrata

    AI repositories

    Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

    Stale GitHub, VecSzn 722 ★MITUpdated 16 days ago

    git clone https://github.com/VecSzn/PolyStrata.git
  17. slotstream

    AI repositories

    Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.

    Indexed GitHub, carloslfu 673 ★MITUpdated today

    git clone https://github.com/carloslfu/slotstream.git
  18. ipex-llm

    AI repositories

    Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.

    Stale Archived GitHub, intel 8,846 ★Apache-2.0Updated 9 months ago 1 note

    git clone https://github.com/intel/ipex-llm.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.