InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

  1. SHP

    Datasets & benchmarks

    Hugging Face dataset by stanfordnlp

    Indexed Hugging Face Hub, stanfordnlp 333 ♥Updated 3 years ago

    stanfordnlp/SHP
  2. LightCompress

    Datasets & benchmarks

    [EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.

    Stale GitHub, ModelTC 751 ★Apache-2.0Updated 6 months ago

    git clone https://github.com/ModelTC/LightCompress.git
  3. skill-harness

    Skills

    How do you KNOW if a 'skill' is any good - deterministically? This runs controlled skill-vs-no-skill experiments. It doesn't just ask, “Did the skill work?” It asks “Did we actually give the skill to the model, did the comparison remain fair, what changed, what did it cost, and do we have enough evidence to say that confidently?”

    Stale GitHub, MrBinnacle 16 ★MITUpdated 14 days ago

    git clone https://github.com/MrBinnacle/skill-harness.git
  4. nautilus-compass

    Datasets & benchmarks

    Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform

    Indexed GitHub, chunxiaoxx 1,258 ★Custom licence (see repository)Updated 2 days ago

    git clone https://github.com/chunxiaoxx/nautilus-compass.git
  5. nemo-helix

    Agents & frameworks

    NeMo Helix: Make the agents you ship faster, more accurate, and safer.

    Indexed GitHub, NVIDIA-NeMo 145 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/NVIDIA-NeMo/nemo-helix.git
  6. palico-ai

    Agents & frameworks

    Build, Improve Performance, and Productionize your AI Application

    Stale GitHub, palico-ai 343 ★MITUpdated 24 months ago

    git clone https://github.com/palico-ai/palico-ai.git
  7. DyCodeEval

    Datasets & benchmarks

    Official repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”

    Stale GitHub, SeekingDream 222 ★Updated 10 months ago

    git clone https://github.com/SeekingDream/DyCodeEval.git
  8. d3-turbo

    Models

    zero shot classification model by vllm-sr · transformers

    Stale Hugging Face Hub, vllm-sr 130 ♥apache-2.0Updated 14 days ago

    vllm-sr/d3-turbo
  9. gamebot

    Datasets & benchmarks

    [ACL 2025] GAMEBoT: Transparent Assessment of LLM Reasoning in Games

    Stale GitHub, Visual-AI 36 ★Updated 10 days ago

    git clone https://github.com/Visual-AI/gamebot.git
  10. prism

    Datasets & benchmarks

    Открытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением в 1С — Claude, GPT, Gemini, DeepSeek, YandexGPT, GigaChat.

    Stale GitHub, genlab-1c 55 ★MITUpdated 18 days ago

    git clone https://github.com/genlab-1c/prism.git
  11. adk-python

    Agents & frameworks

    An open-source, code-first Python toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.

    Indexed GitHub, google 22.2k ★Apache-2.0Updated 2 days ago

    git clone https://github.com/google/adk-python.git
  12. image text to text model by lightonai · transformers

    Stale Hugging Face Hub, lightonai 107 ♥apache-2.0Updated 38 days ago

    lightonai/LightOnOCR-3-4B
  13. WFGY

    AI repositories

    WFGY is heading toward WFGY 5.0 Polaris Protocol, a major open-source release for AI reasoning, RAG, agents, and real-world workflows. Includes Problem Map, Global Debug Card, WFGY 4.0, and the CFV Easter Egg.

    Indexed GitHub, onestardao 1,805 ★Custom licence (see repository)Updated yesterday

    git clone https://github.com/onestardao/WFGY.git
  14. web-eval-agent

    MCP servers

    An MCP server that autonomously evaluates web applications.

    Stale Archived GitHub, refreshdotdev 1,235 ★Apache-2.0Updated 9 months ago 1 note

    git clone https://github.com/refreshdotdev/web-eval-agent.git
  15. chinese-llm-benchmark

    Agents & frameworks

    非线智能 NoneLinear - ReLE评测:中文AI大模型能力评测(持续更新):目前已囊括374个大模型,覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型, 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜,也提供规模超200万的大模型缺陷库!方便广大社区研究分析、改进大模型。

    Indexed GitHub, jeinlee1991 6,518 ★Updated yesterday

    git clone https://github.com/jeinlee1991/chinese-llm-benchmark.git
  16. deepresearch-agent

    Agents & frameworks

    一个生产级的深度研究 Agent 系统,从零构建多智能体编排、Red-Blue 对抗降噪、 语义级上下文压缩、跨 Agent 共享记忆四大核心能力,配套 165 次独立实验 + Bootstrap 统计显著性检验的完整评测体系。

    Stale GitHub, qiqihezh 165 ★Updated 6 months ago

    git clone https://github.com/qiqihezh/deepresearch-agent.git
  17. Adaptive-rag

    AI repositories

    按问题复杂度自动分流的电商客服 RAG:简单问题直接回答;复杂问题经过多跳检索并自我纠错;查单、退改通过工具调用完成,且高风险写操作必须先经人工确认。基于 FastAPI 和 LangGraph,混合检索并附带引用,内置离线评测与 Prometheus 指标。涵盖简历写法和项目面试拷打

    Stale GitHub, JEen-wang 137 ★MITUpdated 25 days ago

    git clone https://github.com/JEen-wang/Adaptive-rag.git
  18. d3-flash

    Models

    zero shot classification model by vllm-sr · transformers

    Stale Hugging Face Hub, vllm-sr 84 ♥apache-2.0Updated 30 days ago

    vllm-sr/d3-flash
  19. adk-go

    AI repositories

    An open-source, code-first Go toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.

    Stale GitHub, google 8,947 ★Apache-2.0Updated 7 days ago

    git clone https://github.com/google/adk-go.git
  20. TurtleBench

    Datasets & benchmarks

    TurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles.

    Stale GitHub, mazzzystar 161 ★Apache-2.0Updated 2 years ago

    git clone https://github.com/mazzzystar/TurtleBench.git
  21. confabulations

    Datasets & benchmarks

    Hallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.

    Stale GitHub, lechmazur 249 ★Updated 15 months ago

    git clone https://github.com/lechmazur/confabulations.git
  22. promptbench

    Prompts

    A unified evaluation framework for large language models

    Stale Archived GitHub, microsoftarchive 2,821 ★MITUpdated 8 months ago 1 note

    git clone https://github.com/microsoftarchive/promptbench.git
  23. Turn a production incident into a structured 9-section LLM response (severity, root cause, mitigation, postmortem). Ships with a 5-scenario regression suite + LLM-as-judge eval pipeline.

    Stale GitHub, YanpengQi7 83 ★Updated 29 days ago

    git clone https://github.com/YanpengQi7/ai-reliability-copilot.git
  24. Hands-on, framework-free Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set — model APIs, structured output, tool calling, RAG, evals-as-the-spine, agents (loop from scratch, tool design, guardrails, MCP, Skills), fine-tuning vs LoRA, prompt-injection/security, LLMOps, and customer craft. Runs on the free Groq API.

    Stale GitHub, calmrocks 743 ★MITUpdated 9 days ago

    git clone https://github.com/calmrocks/ai-engineer-notebooks.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.