Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
explicit-edit-benchmark
Datasets & benchmarksBenchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.
git clone https://github.com/alexshpunt/explicit-edit-benchmark.git -
Open-SWE-Traces
Datasets & benchmarksHugging Face dataset by nvidia
nvidia/Open-SWE-Traces -
LoopArena
Datasets & benchmarksBenchmarking models as runtime Controllers for Loop Engineering
git clone https://github.com/AMAP-ML/LoopArena.git -
TheAgentCompany
Datasets & benchmarksAn agent benchmark with tasks in a simulated software company.
git clone https://github.com/TheAgentCompany/TheAgentCompany.git -
SWE-rebench-openhands-trajectories
Datasets & benchmarksHugging Face dataset by nebius
nebius/SWE-rebench-openhands-trajectories -
computer-use-large
Datasets & benchmarksHugging Face dataset by markov-ai
markov-ai/computer-use-large -
developer-productivity-simulated-behavioral-data
Datasets & benchmarksHugging Face dataset by strova-ai
strova-ai/developer-productivity-simulated-behavioral-data -
VIBE
Datasets & benchmarksHugging Face dataset by MiniMaxAI
MiniMaxAI/VIBE -
Ultimate-Offensive-Red-Team
Datasets & benchmarksHugging Face dataset by WNT3D
WNT3D/Ultimate-Offensive-Red-Team -
AICGSecEval
Datasets & benchmarksA.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team.
git clone https://github.com/Tencent/AICGSecEval.git -
WebsiteBench
Datasets & benchmarksCan AI agents rebuild websites through browser exploration?
git clone https://github.com/TIGER-AI-Lab/WebsiteBench.git -
AgentLab
Datasets & benchmarksAgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.
git clone https://github.com/ServiceNow/AgentLab.git -
agent-duel
Datasets & benchmarksBlinded battles between coding agents: two agents, one prompt, you pick the winner. Desktop app, runs locally.
git clone https://github.com/bottomless/agent-duel.git -
inferock-bench
Datasets & benchmarksLocal LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity receipts.
git clone https://github.com/inferock/inferock-bench.git -
agent-orchestra
Datasets & benchmarksAgents can work. They cannot declare success. A governed coordination protocol for multi-agent AI work: eight contracts (task, resource, ownership, handoff, verification, evidence, approval, recovery) enforced in code, plus a reproducible conformance benchmark. Apache-2.0 code, CC BY 4.0 specification, zero dependencies, Node 18+.
git clone https://github.com/3338902669-ops/agent-orchestra.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.