Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
Looks like a goal we have a starting path for: I want to build an AI agent
-
AICGSecEval
Datasets & benchmarksA.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team.
git clone https://github.com/Tencent/AICGSecEval.git -
ChinaTravel
Datasets & benchmarksChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning
git clone https://github.com/LAMDA-NeSy/ChinaTravel.git -
sacp
Datasets & benchmarksState-Aware Collaboration Protocol: auditable work receipts for AI agents.
git clone https://github.com/aDragon0707/sacp.git -
agent-orchestra
Datasets & benchmarksAgents can work. They cannot declare success. A governed coordination protocol for multi-agent AI work: eight contracts (task, resource, ownership, handoff, verification, evidence, approval, recovery) enforced in code, plus a reproducible conformance benchmark. Apache-2.0 code, CC BY 4.0 specification, zero dependencies, Node 18+.
git clone https://github.com/3338902669-ops/agent-orchestra.git -
Toucan-1.5M
Datasets & benchmarksHugging Face dataset by Agent-Ark
Agent-Ark/Toucan-1.5M -
aa-agentperf-local
Datasets & benchmarksBenchmark local LLM serving by replaying real agent trajectories
git clone https://github.com/ArtificialAnalysis/aa-agentperf-local.git -
openbench
Datasets & benchmarksSame model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost
git clone https://github.com/minghinmatthewlam/openbench.git -
MiroVerse-v0.1
Datasets & benchmarksHugging Face dataset by miromind-ai
miromind-ai/MiroVerse-v0.1 -
Open-SWE-Traces
Datasets & benchmarksHugging Face dataset by nvidia
nvidia/Open-SWE-Traces -
fde-gym
Datasets & benchmarksThe online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.
git clone https://github.com/deeplethe/fde-gym.git -
Awesome-AI-Scientist
Datasets & benchmarks🧪 Awesome list of AI Scientist papers, systems, benchmarks, datasets and open-source platforms.
git clone https://github.com/Omni-Scientist/Awesome-AI-Scientist.git -
VibeSearchBench
Datasets & benchmarks🔍 The hardest search benchmark in the wild — vague, multi-turn, proactive. 200 long-horizon tasks with persona-driven progressive disclosure, scored by verifiable schema-free knowledge-graph evaluation. No vibes, just triplet F1.
git clone https://github.com/VibeBench/VibeSearchBench.git -
TrustLLM
Datasets & benchmarks[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.
git clone https://github.com/HowieHwong/TrustLLM.git -
hermes-function-calling-v1
Datasets & benchmarksHugging Face dataset by NousResearch
NousResearch/hermes-function-calling-v1 -
10Kh-RealOmin-OpenData
Datasets & benchmarksHugging Face dataset by genrobot2025
genrobot2025/10Kh-RealOmin-OpenData -
alem-env
Datasets & benchmarksCan LLM/MARL agents coordinate in long-horizon, open-ended tasks? NeurIPS 2026 (Evaluations and Datasets).
git clone https://github.com/alem-world/alem-env.git -
GDPevo
Datasets & benchmarksA Benchmark for Evaluating Agent Self-Evolution on Real Business Tasks
git clone https://github.com/Prism-Shadow/GDPevo.git -
Multi-User-LLM-Agent
Datasets & benchmarksOfficial code for the paper: "Multi-User Large Language Model Agents"
git clone https://github.com/Kordi-AI/Multi-User-LLM-Agent.git -
Fable-5.1-Max-Reasoning-Filtered-10000x
Datasets & benchmarksHugging Face dataset by MoreThought
MoreThought/Fable-5.1-Max-Reasoning-Filtered-10000x -
TwinFix
Datasets & benchmarksTwinFix — AI agent that diagnoses, reproduces and fixes bugs in a disposable twin environment. 在一次性孪生环境中自动诊断、复现并修复故障的 AI Agent。
git clone https://github.com/spacesky-cell/TwinFix.git -
agent-duel
Datasets & benchmarksBlinded battles between coding agents: two agents, one prompt, you pick the winner. Desktop app, runs locally.
git clone https://github.com/bottomless/agent-duel.git -
PawBench
Datasets & benchmarksA benchmark for evaluating LLM × harness performance.
git clone https://github.com/agentscope-ai/PawBench.git -
DeepPlanning
Datasets & benchmarksHugging Face dataset by Qwen
Qwen/DeepPlanning -
AIRTBench-Code
Datasets & benchmarksCode Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models
git clone https://github.com/dreadnode/AIRTBench-Code.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.