InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. llm-action

    AI repositories

    本项目旨在分享大模型相关技术原理以及实战经验(大模型工程化、大模型应用落地)

    Stale GitHub, liguodongiot 25.2k ★Apache-2.0Updated 33 days ago

    git clone https://github.com/liguodongiot/llm-action.git
  2. TensorFold

    AI repositories

    LLM Inference Engine for Metal, CUDA and Vulkan.

    Indexed GitHub, ashhart 2,266 ★Apache-2.0Updated today

    git clone https://github.com/ashhart/TensorFold.git
  3. PowerInfer

    AI repositories

    High-speed Large Language Model Serving for Local Deployment

    Stale GitHub, Tiiny-AI 9,828 ★MITUpdated 6 months ago

    git clone https://github.com/Tiiny-AI/PowerInfer.git
  4. mistral.rs

    AI repositories

    Fast, flexible LLM inference

    Stale GitHub, EricLBuehler 7,821 ★MITUpdated 7 days ago

    git clone https://github.com/EricLBuehler/mistral.rs.git
  5. litgpt

    AI repositories

    20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

    Stale GitHub, Lightning-AI 13.7k ★Apache-2.0Updated 34 days ago

    git clone https://github.com/Lightning-AI/litgpt.git
  6. lmdeploy

    AI repositories

    LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

    Stale GitHub, InternLM 8,136 ★Apache-2.0Updated 6 days ago

    git clone https://github.com/InternLM/lmdeploy.git
  7. gitleaks

    AI repositories

    Find secrets with Gitleaks 🔑

    Stale GitHub, gitleaks 30.3k ★MITUpdated 24 days ago

    git clone https://github.com/gitleaks/gitleaks.git
  8. zero-to-sglang

    Docs & tutorials

    Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

    Indexed GitHub, datawhalechina 2,148 ★Updated today

    git clone https://github.com/datawhalechina/zero-to-sglang.git
  9. chatllm.cpp

    AI repositories

    Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)

    Stale GitHub, foldl 932 ★MITUpdated 29 days ago

    git clone https://github.com/foldl/chatllm.cpp.git
  10. GenerativeAIExamples

    AI repositories

    Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.

    Stale GitHub, NVIDIA 4,210 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/NVIDIA/GenerativeAIExamples.git
  11. web-llm

    AI repositories

    High-performance In-browser LLM Inference Engine

    Stale GitHub, mlc-ai 19.3k ★Apache-2.0Updated 36 days ago

    git clone https://github.com/mlc-ai/web-llm.git
  12. OlliteRT

    AI repositories

    Turn your Android phone into an OpenAI-compatible LLM inference server - Fully local, private and Open Source

    Stale GitHub, NightMean 597 ★Apache-2.0Updated 15 days ago

    git clone https://github.com/NightMean/OlliteRT.git
  13. engine

    AI repositories

    2bit quants, kernels and runtime for consumer devices.

    Stale GitHub, cactus-compute 6,181 ★Custom licence (see repository)Updated 11 days ago

    git clone https://github.com/cactus-compute/engine.git
  14. pegainfer

    AI repositories

    Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

    Stale GitHub, pegainfer-project 729 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/pegainfer-project/pegainfer.git
  15. turbo-fieldfare

    AI repositories

    Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook

    Stale GitHub, drumih 6,930 ★Apache-2.0Updated 33 days ago

    git clone https://github.com/drumih/turbo-fieldfare.git
  16. picollm

    AI repositories

    On-device LLM Inference Powered by X-Bit Quantization

    Indexed GitHub, Picovoice 320 ★Apache-2.0Updated 4 days ago

    git clone https://github.com/Picovoice/picollm.git
  17. AI-Compass

    AI repositories

    “AI-Compass”将为社区指引在 AI 技术海洋中航行的方向,无论你是初学者还是进阶开发者,都能在这里找到通往 AI 各大方向的路径。旨在帮助开发者系统性地了解 AI 的核心概念、主流技术、前沿趋势,并通过实践掌握从理论到落地的全过程。

    Stale GitHub, tingaicompass 992 ★Updated 17 days ago

    git clone https://github.com/tingaicompass/AI-Compass.git
  18. Your Cheat Sheet for AI Engineering Interview – Questions and Answers.

    Indexed GitHub, amitshekhariitbhu 3,659 ★Apache-2.0Updated today

    git clone https://github.com/amitshekhariitbhu/ai-engineering-interview-questions.git
  19. ray

    AI repositories

    Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

    Indexed GitHub, ray-project 44.3k ★Apache-2.0Updated today

    git clone https://github.com/ray-project/ray.git
  20. superduper

    AI repositories

    Superduper: End-to-end framework for building custom AI applications and agents.

    Stale GitHub, superduper-io 5,327 ★Apache-2.0Updated 14 months ago

    git clone https://github.com/superduper-io/superduper.git
  21. prompt-poet

    Prompts

    Streamlines and simplifies prompt design for both developers and non-technical users with a low code approach.

    Stale GitHub, character-ai 1,158 ★MITUpdated 9 months ago

    git clone https://github.com/character-ai/prompt-poet.git
  22. llmflows

    Prompts

    LLMFlows - Simple, Explicit and Transparent LLM Apps

    Stale GitHub, stoyan-stoyanov 708 ★MITUpdated 21 months ago

    git clone https://github.com/stoyan-stoyanov/llmflows.git
  23. ai-infra-book

    AI repositories

    《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验

    Stale GitHub, bojieli 6,489 ★Apache-2.0Updated 46 days ago

    git clone https://github.com/bojieli/ai-infra-book.git
  24. openvino

    AI repositories

    OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

    Indexed GitHub, openvinotoolkit 11.2k ★Apache-2.0Updated today

    git clone https://github.com/openvinotoolkit/openvino.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.