InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to build an AI agent

  1. LLM-Agent-Benchmark-List

    Datasets & benchmarks

    A banchmark list for evaluation of large language models.

    Stale GitHub, zhangxjohn 173 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/zhangxjohn/LLM-Agent-Benchmark-List.git
  2. ClawBench

    Datasets & benchmarks

    Open-source benchmark for browser AI agents on daily tasks.

    Stale GitHub, TIGER-AI-Lab 1,229 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/TIGER-AI-Lab/ClawBench.git
  3. WideSearch

    Datasets & benchmarks

    WideSearch: Benchmarking Agentic Broad Info-Seeking

    Stale GitHub, ByteDance-Seed 156 ★MITUpdated 13 months ago

    git clone https://github.com/ByteDance-Seed/WideSearch.git
  4. BALROG

    Datasets & benchmarks

    Benchmarking Agentic LLM and VLM Reasoning On Games

    Stale GitHub, balrog-ai 274 ★MITUpdated 7 months ago

    git clone https://github.com/balrog-ai/BALROG.git
  5. PawBench

    Datasets & benchmarks

    A benchmark for evaluating LLM × harness performance.

    Stale GitHub, agentscope-ai 118 ★Apache-2.0Updated 3 months ago

    git clone https://github.com/agentscope-ai/PawBench.git
  6. ClawProBench

    Datasets & benchmarks

    ClawProBench is a live-first benchmark harness for evaluating LLM agents in the OpenClaw runtime with deterministic grading and repeated-trial reliability.

    Stale GitHub, suyoumo 825 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/suyoumo/ClawProBench.git
  7. OctoCodingBench

    Datasets & benchmarks

    Hugging Face dataset by MiniMaxAI

    Indexed Hugging Face Hub, MiniMaxAI 465 ♥mitUpdated 10 months ago

    MiniMaxAI/OctoCodingBench
  8. explicit-edit-benchmark

    Datasets & benchmarks

    Benchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.

    Stale GitHub, alexshpunt 38 ★MITUpdated 7 days ago

    git clone https://github.com/alexshpunt/explicit-edit-benchmark.git
  9. OneMillion-Bench

    Datasets & benchmarks

    [NeurIPS 2026] $OneMillion-Bench: How Far are Language Agents from Human Experts?

    Stale GitHub, humanlaya 55 ★Apache-2.0Updated 27 days ago

    git clone https://github.com/humanlaya/OneMillion-Bench.git
  10. doc2harness

    Datasets & benchmarks

    Official Code for the paper "Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents "

    Stale GitHub, YuyaoGe 116 ★Custom licence (see repository)Updated 20 days ago

    git clone https://github.com/YuyaoGe/doc2harness.git
  11. One-Eval

    Datasets & benchmarks

    Automated system for LLM evaluation via agents. Doc as below:

    Stale GitHub, OpenDCAI 165 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/OpenDCAI/One-Eval.git
  12. AgentLab

    Datasets & benchmarks

    AgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.

    Stale GitHub, ServiceNow 653 ★Custom licence (see repository)Updated 41 days ago

    git clone https://github.com/ServiceNow/AgentLab.git
  13. swe-sweep

    Datasets & benchmarks

    How many bugs can LMs find & fix in large codebases?

    Indexed GitHub, facebookresearch 240 ★Custom licence (see repository)Updated 4 days ago

    git clone https://github.com/facebookresearch/swe-sweep.git
  14. TerminalWorld

    Datasets & benchmarks

    TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks (NeurIPS 2026 Evaluations and Datasets Track)

    Stale GitHub, terminalworld 54 ★Apache-2.0Updated 12 days ago

    git clone https://github.com/terminalworld/TerminalWorld.git
  15. TheAgentCompany

    Datasets & benchmarks

    An agent benchmark with tasks in a simulated software company.

    Stale GitHub, TheAgentCompany 795 ★MITUpdated 12 months ago

    git clone https://github.com/TheAgentCompany/TheAgentCompany.git
  16. Multi-User-LLM-Agent

    Datasets & benchmarks

    Official code for the paper: "Multi-User Large Language Model Agents"

    Stale GitHub, Kordi-AI 39 ★Apache-2.0Updated 7 days ago

    git clone https://github.com/Kordi-AI/Multi-User-LLM-Agent.git
  17. nagi-bench

    Datasets & benchmarks

    One-shot LLM eval cases by NAGI STUDIO - same prompt, different agents (model + harness), runnable artifacts side by side.

    Indexed GitHub, nagi-studio 78 ★MITUpdated 4 days ago

    git clone https://github.com/nagi-studio/nagi-bench.git
  18. agents-last-exam

    Datasets & benchmarks

    Hugging Face dataset by agents-last-exam

    Stale Hugging Face Hub, agents-last-exam 225 ♥cc-by-4.0Updated 33 days ago

    agents-last-exam/agents-last-exam
  19. alem-env

    Datasets & benchmarks

    Can LLM/MARL agents coordinate in long-horizon, open-ended tasks? NeurIPS 2026 (Evaluations and Datasets).

    Stale GitHub, alem-world 57 ★MITUpdated 11 days ago

    git clone https://github.com/alem-world/alem-env.git
  20. openbench

    Datasets & benchmarks

    Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost

    Stale GitHub, minghinmatthewlam 137 ★MITUpdated 2 months ago

    git clone https://github.com/minghinmatthewlam/openbench.git
  21. apex-agents

    Datasets & benchmarks

    Hugging Face dataset by mercor

    Stale Hugging Face Hub, mercor 207 ♥cc-by-4.0Updated 5 months ago 1 note

    mercor/apex-agents
  22. agent-duel

    Datasets & benchmarks

    Blinded battles between coding agents: two agents, one prompt, you pick the winner. Desktop app, runs locally.

    Stale GitHub, bottomless 35 ★Custom licence (see repository)Updated 5 days ago

    git clone https://github.com/bottomless/agent-duel.git
  23. WebsiteBench

    Datasets & benchmarks

    Can AI agents rebuild websites through browser exploration?

    Indexed GitHub, TIGER-AI-Lab 300 ★Apache-2.0Updated 6 days ago

    git clone https://github.com/TIGER-AI-Lab/WebsiteBench.git
  24. self-bench

    Datasets & benchmarks

    Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

    Indexed GitHub, mupt-ai 34 ★MITUpdated today

    git clone https://github.com/mupt-ai/self-bench.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.