Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
Looks like a goal we have a starting path for: I want to build an AI agent
-
TheAgentCompany
Datasets & benchmarksAn agent benchmark with tasks in a simulated software company.
git clone https://github.com/TheAgentCompany/TheAgentCompany.git -
ResearchClawBench
Datasets & benchmarks🦞 ResearchClawBench: Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery
git clone https://github.com/InternScience/ResearchClawBench.git -
AICGSecEval
Datasets & benchmarksA.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team.
git clone https://github.com/Tencent/AICGSecEval.git -
NoLiMa
Datasets & benchmarksOfficial repository for "NoLiMa: Long-Context Evaluation Beyond Literal Matching"
git clone https://github.com/adobe-research/NoLiMa.git -
nautilus-compass
Datasets & benchmarksOpen memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
git clone https://github.com/chunxiaoxx/nautilus-compass.git -
civ6-mcp
Datasets & benchmarksAn MCP server that lets LLM agents play Civilization VI.
git clone https://github.com/lmwilki/civ6-mcp.git -
GDPevo
Datasets & benchmarksA Benchmark for Evaluating Agent Self-Evolution on Real Business Tasks
git clone https://github.com/Prism-Shadow/GDPevo.git -
PhysGym
Datasets & benchmarksA benchmark suite for evaluating LLM-based interactive scientific reasoning.
git clone https://github.com/principia-ai/PhysGym.git -
ClawBench
Datasets & benchmarksOpen-source benchmark for browser AI agents on daily tasks.
git clone https://github.com/TIGER-AI-Lab/ClawBench.git -
mobilegym
Datasets & benchmarks[EMNLP 2026] MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research · 浏览器里运行的安卓模拟器 · Browser-hosted Android Simulator · Verifiable Evaluation · Scalable Online RL Training
git clone https://github.com/Purewhiter/mobilegym.git -
ChinaTravel
Datasets & benchmarksChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning
git clone https://github.com/LAMDA-NeSy/ChinaTravel.git -
Simreal-MLBench
Datasets & benchmarks[Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluation.
git clone https://github.com/Simreal-AI/Simreal-MLBench.git -
sacp
Datasets & benchmarksState-Aware Collaboration Protocol: auditable work receipts for AI agents.
git clone https://github.com/aDragon0707/sacp.git -
AIRTBench-Code
Datasets & benchmarksCode Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models
git clone https://github.com/dreadnode/AIRTBench-Code.git -
SGI-Bench
Datasets & benchmarksProbing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
git clone https://github.com/InternScience/SGI-Bench.git -
meta-agents-research-environments
Datasets & benchmarksMeta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.
git clone https://github.com/facebookresearch/meta-agents-research-environments.git -
VibeSearchBench
Datasets & benchmarks🔍 The hardest search benchmark in the wild — vague, multi-turn, proactive. 200 long-horizon tasks with persona-driven progressive disclosure, scored by verifiable schema-free knowledge-graph evaluation. No vibes, just triplet F1.
git clone https://github.com/VibeBench/VibeSearchBench.git -
Zero-To-CAD-1m
Datasets & benchmarksHugging Face dataset by ADSKAILab
ADSKAILab/Zero-To-CAD-1m -
agent-orchestra
Datasets & benchmarksAgents can work. They cannot declare success. A governed coordination protocol for multi-agent AI work: eight contracts (task, resource, ownership, handoff, verification, evidence, approval, recovery) enforced in code, plus a reproducible conformance benchmark. Apache-2.0 code, CC BY 4.0 specification, zero dependencies, Node 18+.
git clone https://github.com/3338902669-ops/agent-orchestra.git -
Awesome-AI-Scientist
Datasets & benchmarks🧪 Awesome list of AI Scientist papers, systems, benchmarks, datasets and open-source platforms.
git clone https://github.com/Omni-Scientist/Awesome-AI-Scientist.git -
Awesome-AI4AI
Datasets & benchmarksUpdated weekly — until humanity achieves AI4AI. 1148 curated papers across Benchmarks, Harness Design, and Model Design — from long-horizon agents to recursive self-improvement. Citation rankings, GitHub stars, and source-verified weekly news.
git clone https://github.com/KaiWU5/Awesome-AI4AI.git -
prism
Datasets & benchmarksОткрытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением в 1С — Claude, GPT, Gemini, DeepSeek, YandexGPT, GigaChat.
git clone https://github.com/genlab-1c/prism.git -
TraderHarness
Datasets & benchmarks面向 LLM 交易 Agent 的抗污染回测、确定性回放与训练轨迹生成,支持 A 股研究|Leakage-resistant backtesting, deterministic replay & training trajectory generation for LLM trading agents, with A-share data
git clone https://github.com/HephaestLab/TraderHarness.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.