Library
85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
HallusionBench
Datasets & benchmarks[CVPR'24] HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models
git clone https://github.com/tianyi-lab/HallusionBench.git -
elimination_game
Datasets & benchmarksA multi-player tournament benchmark that tests LLMs in social reasoning, strategy, and deception. Players engage in public and private conversations, form alliances, and vote to eliminate each other
git clone https://github.com/lechmazur/elimination_game.git -
TerminalWorld
Datasets & benchmarksTerminalWorld: Benchmarking Agents on Real-World Terminal Tasks (NeurIPS 2026 Evaluations and Datasets Track)
git clone https://github.com/terminalworld/TerminalWorld.git -
gamebot
Datasets & benchmarks[ACL 2025] GAMEBoT: Transparent Assessment of LLM Reasoning in Games
git clone https://github.com/Visual-AI/gamebot.git -
Awesome-Datasets-Hub
Datasets & benchmarksA curated collection of datasets for Large Language Models (LLMs), covering medical AI, NLP, multimodal learning, instruction tuning, reasoning, code generation, and evaluation benchmarks.
git clone https://github.com/ahammadmejbah/Awesome-Datasets-Hub.git -
multi-agents
Datasets & benchmarksMulti-agent teams for real work: run teams of coding agents (Claude Code, Codex, any model) in isolated git worktrees, and measure whether the team beats a solo agent on cost and results.
git clone https://github.com/BichengWang/multi-agents.git -
DishwasherBench
Datasets & benchmarksA dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.
git clone https://github.com/composo-ai/DishwasherBench.git -
seo-skill-bench
Datasets & benchmarksOpen benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic scoring, pre-registered rubric.
git clone https://github.com/aleclindz/seo-skill-bench.git -
Awesome-Scientific-LLM-Benchmarks
Datasets & benchmarksA curated, accuracy-first list of benchmarks for evaluating LLMs on scientific reasoning and discovery — math, physics, chemistry, materials, biology, and agentic science.
git clone https://github.com/subinium/Awesome-Scientific-LLM-Benchmarks.git -
xorcise
Datasets & benchmarksRun cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the claim.
git clone https://github.com/xorcise-ai/xorcise.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.