InnerVault

Library

85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. EmbedAnything

    AI repositories

    Highly Performant, Modular, Memory Safe and Production-ready Inference, Ingestion and Indexing built in Rust 🦀

    Indexed GitHub, StarlightSearch 1,324 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/StarlightSearch/EmbedAnything.git
  2. awsome-distributed-ai

    AI repositories

    Best practices, reference architectures, and examples for distributed AI training and inference on AWS.

    Indexed GitHub, awslabs 498 ★MIT-0Updated 3 days ago

    git clone https://github.com/awslabs/awsome-distributed-ai.git
  3. Halfrost-Field

    AI repositories

    ✍🏻 Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field 冰霜之地:源码解析、系统设计与工程实践笔记

    Stale GitHub, halfrost 13.2k ★CC-BY-SA-4.0Updated 48 days ago

    git clone https://github.com/halfrost/Halfrost-Field.git
  4. TensorSharp

    AI repositories

    A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability

    Stale GitHub, zhongkaifu 675 ★BSD-3-ClauseUpdated 5 days ago

    git clone https://github.com/zhongkaifu/TensorSharp.git
  5. SwiftLM

    AI repositories

    ⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.

    Indexed GitHub, SharpAI 799 ★MITUpdated yesterday

    git clone https://github.com/SharpAI/SwiftLM.git
  6. jev

    AI repositories

    jevos: an open-source alternative to Jev and Microsoft-Decision-1. Yes/no decisions, choices and scores with probabilities, on your laptop's CPU.

    Stale GitHub, feder-cr 1,342 ★MITUpdated 32 days ago

    git clone https://github.com/feder-cr/jev.git
  7. OlliteRT

    AI repositories

    Turn your Android phone into an OpenAI-compatible LLM inference server - Fully local, private and Open Source

    Stale GitHub, NightMean 597 ★Apache-2.0Updated 16 days ago

    git clone https://github.com/NightMean/OlliteRT.git
  8. LTX-2

    AI repositories

    Official Python inference and LoRA trainer package for the LTX-2 audio–video generative model.

    Indexed GitHub, Lightricks 10.2k ★Apache-2.0Updated 5 days ago

    git clone https://github.com/Lightricks/LTX-2.git
  9. mlx-serve

    AI repositories

    Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.

    Indexed GitHub, ddalcu 2,475 ★Custom licence (see repository)Updated yesterday

    git clone https://github.com/ddalcu/mlx-serve.git
  10. Your Cheat Sheet for AI Engineering Interview – Questions and Answers.

    Indexed GitHub, amitshekhariitbhu 3,659 ★Apache-2.0Updated yesterday

    git clone https://github.com/amitshekhariitbhu/ai-engineering-interview-questions.git
  11. airllm

    AI repositories

    AirLLM 2.8T LLM inference with single 4GB GPU

    Indexed GitHub, lyogavin 42.5k ★Apache-2.0Updated yesterday

    git clone https://github.com/lyogavin/airllm.git
  12. ai-infra-book

    AI repositories

    《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验

    Stale GitHub, bojieli 6,489 ★Apache-2.0Updated 47 days ago

    git clone https://github.com/bojieli/ai-infra-book.git
  13. AI-Compass

    AI repositories

    “AI-Compass”将为社区指引在 AI 技术海洋中航行的方向,无论你是初学者还是进阶开发者,都能在这里找到通往 AI 各大方向的路径。旨在帮助开发者系统性地了解 AI 的核心概念、主流技术、前沿趋势,并通过实践掌握从理论到落地的全过程。

    Stale GitHub, tingaicompass 992 ★Updated 18 days ago

    git clone https://github.com/tingaicompass/AI-Compass.git
  14. oumi

    AI repositories

    Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!

    Stale GitHub, oumi-ai 9,408 ★Apache-2.0Updated 25 days ago

    git clone https://github.com/oumi-ai/oumi.git
  15. nvidia_gpu_exporter

    AI repositories

    Nvidia GPU exporter for prometheus using nvidia-smi binary OR using NVML

    Stale GitHub, utkuozdemir 1,584 ★MITUpdated 7 days ago

    git clone https://github.com/utkuozdemir/nvidia_gpu_exporter.git
  16. Atomic-Chat

    AI repositories

    Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V

    Stale GitHub, AtomicBot-ai 1,808 ★Custom licence (see repository)Updated 31 days ago

    git clone https://github.com/AtomicBot-ai/Atomic-Chat.git
  17. vllm-mlx

    AI repositories

    High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.

    Stale GitHub, waybarrios 1,646 ★Apache-2.0Updated 12 days ago

    git clone https://github.com/waybarrios/vllm-mlx.git
  18. reasoning-from-scratch

    AI repositories

    Implement a reasoning LLM in PyTorch from scratch, step by step

    Stale GitHub, rasbt 5,404 ★Apache-2.0Updated 46 days ago

    git clone https://github.com/rasbt/reasoning-from-scratch.git
  19. wllama

    AI repositories

    WebAssembly binding for llama.cpp - Enabling on-browser LLM inference

    Stale GitHub, ngxson 1,363 ★MITUpdated 11 days ago

    git clone https://github.com/ngxson/wllama.git
  20. laya

    AI repositories

    Run Laya, the open-source Jev-compatible System-1 decision model, from Node.js / TypeScript via ONNX Runtime

    Stale GitHub, receptron 948 ★MITUpdated 31 days ago

    git clone https://github.com/receptron/laya.git
  21. larql

    AI repositories

    Run, serve, query and edit AI models — a Rust model runtime built on VINDEX3, an executable, inspectable model format.

    Indexed GitHub, chrishayuk 1,175 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/chrishayuk/larql.git
  22. AI-Engineer-Headquarters

    AI repositories

    A collection of scientific methods, processes, algorithms, and systems to build stories & models.

    Stale GitHub, hemansnation 3,776 ★Updated 20 days ago

    git clone https://github.com/hemansnation/AI-Engineer-Headquarters.git
  23. janas

    AI repositories

    37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.

    Indexed GitHub, prabanta-dev 717 ★GPL-3.0Updated yesterday

    git clone https://github.com/prabanta-dev/janas.git
  24. tiny-llm

    AI repositories

    learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen

    Stale GitHub, skyzh 5,680 ★Apache-2.0Updated 10 days ago

    git clone https://github.com/skyzh/tiny-llm.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.