Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
Looks like a goal we have a starting path for: I want to build an AI agent
-
Open-SWE-Traces
Datasets & benchmarksHugging Face dataset by nvidia
nvidia/Open-SWE-Traces -
DeepPlanning
Datasets & benchmarksHugging Face dataset by Qwen
Qwen/DeepPlanning -
One-Eval
Datasets & benchmarksAutomated system for LLM evaluation via agents. Doc as below:
git clone https://github.com/OpenDCAI/One-Eval.git -
WideSearch
Datasets & benchmarksWideSearch: Benchmarking Agentic Broad Info-Seeking
git clone https://github.com/ByteDance-Seed/WideSearch.git -
ToolScale
Datasets & benchmarksHugging Face dataset by nvidia
nvidia/ToolScale -
dataclaw-peteromallet
Datasets & benchmarksHugging Face dataset by peteromallet
peteromallet/dataclaw-peteromallet -
ParseBench
Datasets & benchmarksParseBench - A Document Parsing Benchmark for AI Agents
git clone https://github.com/run-llama/ParseBench.git -
aa-agentperf-local
Datasets & benchmarksBenchmark local LLM serving by replaying real agent trajectories
git clone https://github.com/ArtificialAnalysis/aa-agentperf-local.git -
Mind2Web
Datasets & benchmarksHugging Face dataset by osunlp
osunlp/Mind2Web -
reasoning-corpus-4K-5M-v1
Datasets & benchmarksHugging Face dataset by mithulaartigala
mithulaartigala/reasoning-corpus-4K-5M-v1 -
ChinaTravel
Datasets & benchmarksChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning
git clone https://github.com/LAMDA-NeSy/ChinaTravel.git -
doc2harness
Datasets & benchmarksOfficial Code for the paper "Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents "
git clone https://github.com/YuyaoGe/doc2harness.git -
claude_mythos_distilled_25k
Datasets & benchmarksHugging Face dataset by WithinUsAI
WithinUsAI/claude_mythos_distilled_25k -
sacp
Datasets & benchmarksState-Aware Collaboration Protocol: auditable work receipts for AI agents.
git clone https://github.com/aDragon0707/sacp.git -
openbench
Datasets & benchmarksSame model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost
git clone https://github.com/minghinmatthewlam/openbench.git -
SWE-rebench-openhands-trajectories
Datasets & benchmarksHugging Face dataset by nebius
nebius/SWE-rebench-openhands-trajectories -
NoLiMa
Datasets & benchmarksOfficial repository for "NoLiMa: Long-Context Evaluation Beyond Literal Matching"
git clone https://github.com/adobe-research/NoLiMa.git -
agent-orchestra
Datasets & benchmarksAgents can work. They cannot declare success. A governed coordination protocol for multi-agent AI work: eight contracts (task, resource, ownership, handoff, verification, evidence, approval, recovery) enforced in code, plus a reproducible conformance benchmark. Apache-2.0 code, CC BY 4.0 specification, zero dependencies, Node 18+.
git clone https://github.com/3338902669-ops/agent-orchestra.git -
fde-gym
Datasets & benchmarksThe online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.
git clone https://github.com/deeplethe/fde-gym.git -
VibeSearchBench
Datasets & benchmarks🔍 The hardest search benchmark in the wild — vague, multi-turn, proactive. 200 long-horizon tasks with persona-driven progressive disclosure, scored by verifiable schema-free knowledge-graph evaluation. No vibes, just triplet F1.
git clone https://github.com/VibeBench/VibeSearchBench.git -
swe-sweep
Datasets & benchmarksHow many bugs can LMs find & fix in large codebases?
git clone https://github.com/facebookresearch/swe-sweep.git -
PawBench
Datasets & benchmarksA benchmark for evaluating LLM × harness performance.
git clone https://github.com/agentscope-ai/PawBench.git -
Awesome-AI-Scientist
Datasets & benchmarks🧪 Awesome list of AI Scientist papers, systems, benchmarks, datasets and open-source platforms.
git clone https://github.com/Omni-Scientist/Awesome-AI-Scientist.git -
GDPevo
Datasets & benchmarksA Benchmark for Evaluating Agent Self-Evolution on Real Business Tasks
git clone https://github.com/Prism-Shadow/GDPevo.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.