Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
SHP
Datasets & benchmarksHugging Face dataset by stanfordnlp
stanfordnlp/SHP -
LightCompress
Datasets & benchmarks[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.
git clone https://github.com/ModelTC/LightCompress.git -
skill-harness
SkillsHow do you KNOW if a 'skill' is any good - deterministically? This runs controlled skill-vs-no-skill experiments. It doesn't just ask, “Did the skill work?” It asks “Did we actually give the skill to the model, did the comparison remain fair, what changed, what did it cost, and do we have enough evidence to say that confidently?”
git clone https://github.com/MrBinnacle/skill-harness.git -
nautilus-compass
Datasets & benchmarksOpen memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
git clone https://github.com/chunxiaoxx/nautilus-compass.git -
nemo-helix
Agents & frameworksNeMo Helix: Make the agents you ship faster, more accurate, and safer.
git clone https://github.com/NVIDIA-NeMo/nemo-helix.git -
palico-ai
Agents & frameworksBuild, Improve Performance, and Productionize your AI Application
git clone https://github.com/palico-ai/palico-ai.git -
DyCodeEval
Datasets & benchmarksOfficial repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”
git clone https://github.com/SeekingDream/DyCodeEval.git -
d3-turbo
Modelszero shot classification model by vllm-sr · transformers
vllm-sr/d3-turbo -
gamebot
Datasets & benchmarks[ACL 2025] GAMEBoT: Transparent Assessment of LLM Reasoning in Games
git clone https://github.com/Visual-AI/gamebot.git -
prism
Datasets & benchmarksОткрытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением в 1С — Claude, GPT, Gemini, DeepSeek, YandexGPT, GigaChat.
git clone https://github.com/genlab-1c/prism.git -
adk-python
Agents & frameworksAn open-source, code-first Python toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.
git clone https://github.com/google/adk-python.git -
LightOnOCR-3-4B
Modelsimage text to text model by lightonai · transformers
lightonai/LightOnOCR-3-4B -
WFGY
AI repositoriesWFGY is heading toward WFGY 5.0 Polaris Protocol, a major open-source release for AI reasoning, RAG, agents, and real-world workflows. Includes Problem Map, Global Debug Card, WFGY 4.0, and the CFV Easter Egg.
git clone https://github.com/onestardao/WFGY.git -
web-eval-agent
MCP serversAn MCP server that autonomously evaluates web applications.
git clone https://github.com/refreshdotdev/web-eval-agent.git -
chinese-llm-benchmark
Agents & frameworks非线智能 NoneLinear - ReLE评测:中文AI大模型能力评测(持续更新):目前已囊括374个大模型,覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型, 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜,也提供规模超200万的大模型缺陷库!方便广大社区研究分析、改进大模型。
git clone https://github.com/jeinlee1991/chinese-llm-benchmark.git -
deepresearch-agent
Agents & frameworks一个生产级的深度研究 Agent 系统,从零构建多智能体编排、Red-Blue 对抗降噪、 语义级上下文压缩、跨 Agent 共享记忆四大核心能力,配套 165 次独立实验 + Bootstrap 统计显著性检验的完整评测体系。
git clone https://github.com/qiqihezh/deepresearch-agent.git -
Adaptive-rag
AI repositories按问题复杂度自动分流的电商客服 RAG:简单问题直接回答;复杂问题经过多跳检索并自我纠错;查单、退改通过工具调用完成,且高风险写操作必须先经人工确认。基于 FastAPI 和 LangGraph,混合检索并附带引用,内置离线评测与 Prometheus 指标。涵盖简历写法和项目面试拷打
git clone https://github.com/JEen-wang/Adaptive-rag.git -
d3-flash
Modelszero shot classification model by vllm-sr · transformers
vllm-sr/d3-flash -
adk-go
AI repositoriesAn open-source, code-first Go toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.
git clone https://github.com/google/adk-go.git -
TurtleBench
Datasets & benchmarksTurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles.
git clone https://github.com/mazzzystar/TurtleBench.git -
confabulations
Datasets & benchmarksHallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.
git clone https://github.com/lechmazur/confabulations.git -
promptbench
PromptsA unified evaluation framework for large language models
git clone https://github.com/microsoftarchive/promptbench.git -
ai-reliability-copilot
PromptsTurn a production incident into a structured 9-section LLM response (severity, root cause, mitigation, postmortem). Ships with a 5-scenario regression suite + LLM-as-judge eval pipeline.
git clone https://github.com/YanpengQi7/ai-reliability-copilot.git -
ai-engineer-notebooks
PromptsHands-on, framework-free Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set — model APIs, structured output, tool calling, RAG, evals-as-the-spine, agents (loop from scratch, tool design, guardrails, MCP, Skills), fine-tuning vs LoRA, prompt-injection/security, LLMOps, and customer craft. Runs on the free Groq API.
git clone https://github.com/calmrocks/ai-engineer-notebooks.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.