InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to build an AI agent

  1. LLM-Agent-Benchmark-List

    Datasets & benchmarks

    A banchmark list for evaluation of large language models.

    Stale GitHub, zhangxjohn 173 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/zhangxjohn/LLM-Agent-Benchmark-List.git
  2. BALROG

    Datasets & benchmarks

    Benchmarking Agentic LLM and VLM Reasoning On Games

    Stale GitHub, balrog-ai 274 ★MITUpdated 7 months ago

    git clone https://github.com/balrog-ai/BALROG.git
  3. civ6-mcp

    Datasets & benchmarks

    An MCP server that lets LLM agents play Civilization VI.

    Stale GitHub, lmwilki 218 ★MITUpdated 34 days ago

    git clone https://github.com/lmwilki/civ6-mcp.git
  4. Simreal-MLBench

    Datasets & benchmarks

    [Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluation.

    Stale GitHub, Simreal-AI 301 ★Apache-2.0Updated 53 days ago

    git clone https://github.com/Simreal-AI/Simreal-MLBench.git
  5. Long-Horizon-Terminal-Bench

    Datasets & benchmarks

    Hugging Face dataset by IntelligenceLab

    Stale Hugging Face Hub, IntelligenceLab 140 ♥apache-2.0Updated 36 days ago

    IntelligenceLab/Long-Horizon-Terminal-Bench
  6. Awesome-LLM-Long-Context-Modeling

    Datasets & benchmarks

    📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

    Indexed GitHub, Xnhyacinth 2,192 ★MITUpdated 3 days ago

    git clone https://github.com/Xnhyacinth/Awesome-LLM-Long-Context-Modeling.git
  7. ResearchClawBench

    Datasets & benchmarks

    🦞 ResearchClawBench: Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery

    Stale GitHub, InternScience 275 ★MITUpdated 19 days ago

    git clone https://github.com/InternScience/ResearchClawBench.git
  8. One-Eval

    Datasets & benchmarks

    Automated system for LLM evaluation via agents. Doc as below:

    Stale GitHub, OpenDCAI 165 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/OpenDCAI/One-Eval.git
  9. OctoCodingBench

    Datasets & benchmarks

    Hugging Face dataset by MiniMaxAI

    Indexed Hugging Face Hub, MiniMaxAI 465 ♥mitUpdated 10 months ago

    MiniMaxAI/OctoCodingBench
  10. WideSearch

    Datasets & benchmarks

    WideSearch: Benchmarking Agentic Broad Info-Seeking

    Stale GitHub, ByteDance-Seed 156 ★MITUpdated 13 months ago

    git clone https://github.com/ByteDance-Seed/WideSearch.git
  11. Step-3.5-Flash-SFT

    Datasets & benchmarks

    Hugging Face dataset by stepfun-ai

    Indexed Hugging Face Hub, stepfun-ai 364 ♥apache-2.0Updated 8 months ago

    stepfun-ai/Step-3.5-Flash-SFT
  12. xlam-function-calling-60k

    Datasets & benchmarks

    Hugging Face dataset by Salesforce

    Indexed Hugging Face Hub, Salesforce 764 ♥cc-by-4.0Updated 22 months ago 1 note

    Salesforce/xlam-function-calling-60k
  13. MobileWorld

    Datasets & benchmarks

    Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments (ACL 2026)

    Stale GitHub, Tongyi-MAI 300 ★Apache-2.0Updated 13 days ago

    git clone https://github.com/Tongyi-MAI/MobileWorld.git
  14. polymarket-paper-trader

    Datasets & benchmarks

    Paper trading simulator for Polymarket

    Stale GitHub, agent-next 495 ★MITUpdated 15 days ago

    git clone https://github.com/agent-next/polymarket-paper-trader.git
  15. doc2harness

    Datasets & benchmarks

    Official Code for the paper "Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents "

    Stale GitHub, YuyaoGe 116 ★Custom licence (see repository)Updated 20 days ago

    git clone https://github.com/YuyaoGe/doc2harness.git
  16. AICGSecEval

    Datasets & benchmarks

    A.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team.

    Stale GitHub, Tencent 664 ★Custom licence (see repository)Updated 5 months ago

    git clone https://github.com/Tencent/AICGSecEval.git
  17. ChinaTravel

    Datasets & benchmarks

    ChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning

    Stale GitHub, LAMDA-NeSy 133 ★MITUpdated 2 months ago

    git clone https://github.com/LAMDA-NeSy/ChinaTravel.git
  18. sacp

    Datasets & benchmarks

    State-Aware Collaboration Protocol: auditable work receipts for AI agents.

    Stale GitHub, aDragon0707 107 ★MITUpdated 32 days ago

    git clone https://github.com/aDragon0707/sacp.git
  19. agent-orchestra

    Datasets & benchmarks

    Agents can work. They cannot declare success. A governed coordination protocol for multi-agent AI work: eight contracts (task, resource, ownership, handoff, verification, evidence, approval, recovery) enforced in code, plus a reproducible conformance benchmark. Apache-2.0 code, CC BY 4.0 specification, zero dependencies, Node 18+.

    Stale GitHub, 3338902669-ops 93 ★Apache-2.0Updated 24 days ago

    git clone https://github.com/3338902669-ops/agent-orchestra.git
  20. Toucan-1.5M

    Datasets & benchmarks

    Hugging Face dataset by Agent-Ark

    Indexed Hugging Face Hub, Agent-Ark 250 ♥apache-2.0Updated 13 months ago

    Agent-Ark/Toucan-1.5M
  21. aa-agentperf-local

    Datasets & benchmarks

    Benchmark local LLM serving by replaying real agent trajectories

    Stale GitHub, ArtificialAnalysis 105 ★Apache-2.0Updated 31 days ago

    git clone https://github.com/ArtificialAnalysis/aa-agentperf-local.git
  22. openbench

    Datasets & benchmarks

    Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost

    Stale GitHub, minghinmatthewlam 137 ★MITUpdated 2 months ago

    git clone https://github.com/minghinmatthewlam/openbench.git
  23. MiroVerse-v0.1

    Datasets & benchmarks

    Hugging Face dataset by miromind-ai

    Indexed Hugging Face Hub, miromind-ai 242 ♥cc-by-nc-4.0Updated 10 months ago 1 note

    miromind-ai/MiroVerse-v0.1
  24. Open-SWE-Traces

    Datasets & benchmarks

    Hugging Face dataset by nvidia

    Stale Hugging Face Hub, nvidia 150 ♥cc-by-4.0Updated 34 days ago

    nvidia/Open-SWE-Traces

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.