Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
Looks like a goal we have a starting path for: I want to build an AI agent
-
SWE-rebench-openhands-trajectories
Datasets & benchmarksHugging Face dataset by nebius
nebius/SWE-rebench-openhands-trajectories -
swe-sweep
Datasets & benchmarksHow many bugs can LMs find & fix in large codebases?
git clone https://github.com/facebookresearch/swe-sweep.git -
self-bench
Datasets & benchmarksBenchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.
git clone https://github.com/mupt-ai/self-bench.git -
reasoning-corpus-4K-5M-v1
Datasets & benchmarksHugging Face dataset by mithulaartigala
mithulaartigala/reasoning-corpus-4K-5M-v1 -
nagi-bench
Datasets & benchmarksOne-shot LLM eval cases by NAGI STUDIO - same prompt, different agents (model + harness), runnable artifacts side by side.
git clone https://github.com/nagi-studio/nagi-bench.git -
primitive-bench
Datasets & benchmarksThe marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it, refunded if the outcome isn't delivered. This repo is the open verification layer.
git clone https://github.com/primitive-bench/primitive-bench.git -
claude_mythos_distilled_25k
Datasets & benchmarksHugging Face dataset by WithinUsAI
WithinUsAI/claude_mythos_distilled_25k -
TerminalWorld
Datasets & benchmarksTerminalWorld: Benchmarking Agents on Real-World Terminal Tasks (NeurIPS 2026 Evaluations and Datasets Track)
git clone https://github.com/terminalworld/TerminalWorld.git -
o11y-bench
Datasets & benchmarksOpen benchmark on observability tasks built on Harbor
git clone https://github.com/grafana/o11y-bench.git -
PhysGym
Datasets & benchmarksA benchmark suite for evaluating LLM-based interactive scientific reasoning.
git clone https://github.com/principia-ai/PhysGym.git -
ParseBench
Datasets & benchmarksParseBench - A Document Parsing Benchmark for AI Agents
git clone https://github.com/run-llama/ParseBench.git -
SWE-ZERO-12M-trajectories
Datasets & benchmarksHugging Face dataset by AlienKevin
AlienKevin/SWE-ZERO-12M-trajectories -
deepsearchqa
Datasets & benchmarksHugging Face dataset by google
google/deepsearchqa -
dataclaw-peteromallet
Datasets & benchmarksHugging Face dataset by peteromallet
peteromallet/dataclaw-peteromallet -
xorcise
Datasets & benchmarksRun cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the claim.
git clone https://github.com/xorcise-ai/xorcise.git -
fable-5-premium
Datasets & benchmarksHugging Face dataset by saidutta69
saidutta69/fable-5-premium -
elimination_game
Datasets & benchmarksA multi-player tournament benchmark that tests LLMs in social reasoning, strategy, and deception. Players engage in public and private conversations, form alliances, and vote to eliminate each other
git clone https://github.com/lechmazur/elimination_game.git -
qwen38-strix-halo-harness
Datasets & benchmarksAI coding agent harness for any hardware, local or cloud (OpenAI-compatible engines). Tuned for AMD Strix Halo: Qwen3.8 125B on the iGPU, NPU side-jobs, crash recovery, logged benchmarks.
git clone https://github.com/aic0d3r/qwen38-strix-halo-harness.git -
SGI-Bench
Datasets & benchmarksProbing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
git clone https://github.com/InternScience/SGI-Bench.git -
Zero-To-CAD-1m
Datasets & benchmarksHugging Face dataset by ADSKAILab
ADSKAILab/Zero-To-CAD-1m -
DishwasherBench
Datasets & benchmarksA dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.
git clone https://github.com/composo-ai/DishwasherBench.git -
HaluMem
Datasets & benchmarksHaluMem is the first operation level hallucination evaluation benchmark tailored to agent memory systems.
git clone https://github.com/MemTensor/HaluMem.git -
seo-skill-bench
Datasets & benchmarksOpen benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic scoring, pre-registered rubric.
git clone https://github.com/aleclindz/seo-skill-bench.git -
prism
Datasets & benchmarksОткрытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением в 1С — Claude, GPT, Gemini, DeepSeek, YandexGPT, GigaChat.
git clone https://github.com/genlab-1c/prism.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.