Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
coder_eval
SkillsPlaywright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI gates.
git clone https://github.com/UiPath/coder_eval.git -
eval-genius
SkillsTeach your agent to work with evals: WHEN you actually need an eval or benchmark, HOW to build one that holds up, and how to read what it tells you. Deterministic-first, tool-agnostic.
git clone https://github.com/alexgreensh/eval-genius.git -
codex-keysmith
PromptsVersioned Codex instruction deployment with preview, ownership manifests, hook isolation, scenario evaluation, and recovery.
git clone https://github.com/Jia-Ethan/codex-keysmith.git -
prism
Datasets & benchmarksОткрытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением в 1С — Claude, GPT, Gemini, DeepSeek, YandexGPT, GigaChat.
git clone https://github.com/genlab-1c/prism.git -
nemo-helix
Agents & frameworksNeMo Helix: Make the agents you ship faster, more accurate, and safer.
git clone https://github.com/NVIDIA-NeMo/nemo-helix.git -
promptbench
PromptsA unified evaluation framework for large language models
git clone https://github.com/microsoftarchive/promptbench.git -
nasde-toolkit
SkillsCLI for benchmarks & evals of AI coding agents — on tasks you already understand, using your Claude / Codex / Gemini individual subscriptions or API keys.
git clone https://github.com/NoesisVision/nasde-toolkit.git -
generative-ai
Agents & frameworksComprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview preparation, and coding preparation.
git clone https://github.com/genieincodebottle/generative-ai.git -
llm-colosseum
Datasets & benchmarksBenchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM
git clone https://github.com/OpenGenerativeAI/llm-colosseum.git -
DyCodeEval
Datasets & benchmarksOfficial repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”
git clone https://github.com/SeekingDream/DyCodeEval.git -
SHP
Datasets & benchmarksHugging Face dataset by stanfordnlp
stanfordnlp/SHP -
palico-ai
Agents & frameworksBuild, Improve Performance, and Productionize your AI Application
git clone https://github.com/palico-ai/palico-ai.git -
LightCompress
Datasets & benchmarks[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.
git clone https://github.com/ModelTC/LightCompress.git -
deepresearch-agent
Agents & frameworks一个生产级的深度研究 Agent 系统,从零构建多智能体编排、Red-Blue 对抗降噪、 语义级上下文压缩、跨 Agent 共享记忆四大核心能力,配套 165 次独立实验 + Bootstrap 统计显著性检验的完整评测体系。
git clone https://github.com/qiqihezh/deepresearch-agent.git -
fast-agent
Agents & frameworksCode, Build and Evaluate agents - excellent Model and Skills/MCP/ACP/A2A Support
git clone https://github.com/evalstate/fast-agent.git -
nautilus-compass
Datasets & benchmarksOpen memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
git clone https://github.com/chunxiaoxx/nautilus-compass.git -
Adaptive-rag
AI repositories按问题复杂度自动分流的电商客服 RAG:简单问题直接回答;复杂问题经过多跳检索并自我纠错;查单、退改通过工具调用完成,且高风险写操作必须先经人工确认。基于 FastAPI 和 LangGraph,混合检索并附带引用,内置离线评测与 Prometheus 指标。涵盖简历写法和项目面试拷打
git clone https://github.com/JEen-wang/Adaptive-rag.git -
TurtleBench
Datasets & benchmarksTurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles.
git clone https://github.com/mazzzystar/TurtleBench.git -
DreamLayer-Eval
AI repositoriesDreamLayer Eval: open-source benchmarking for image and video diffusion models. Automate prompts, seeds, and metrics for reproducible results. A research project from DreamLayer AI (dreamlayer.io).
git clone https://github.com/TheDesignFounder/DreamLayer-Eval.git -
agents-last-exam
Datasets & benchmarksHugging Face dataset by agents-last-exam
agents-last-exam/agents-last-exam -
confabulations
Datasets & benchmarksHallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.
git clone https://github.com/lechmazur/confabulations.git -
Awesome-Scientific-LLM-Benchmarks
Datasets & benchmarksA curated, accuracy-first list of benchmarks for evaluating LLMs on scientific reasoning and discovery — math, physics, chemistry, materials, biology, and agentic science.
git clone https://github.com/subinium/Awesome-Scientific-LLM-Benchmarks.git -
WFGY
AI repositoriesWFGY is heading toward WFGY 5.0 Polaris Protocol, a major open-source release for AI reasoning, RAG, agents, and real-world workflows. Includes Problem Map, Global Debug Card, WFGY 4.0, and the CFV Easter Egg.
git clone https://github.com/onestardao/WFGY.git -
dokimi
SkillsDokimi (Greek δοκιμή, "test, proof") measures whether a skill, MCP doc source, agent profile or repository guidance actually helps a coding agent: paired on/off arm comparisons, novel-vs-known content analysis, and repository scans with graded agent runs.
git clone https://github.com/intel/dokimi.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.