InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to build an AI agent

  1. pi-mono

    Datasets & benchmarks

    Hugging Face dataset by badlogicgames

    Stale Hugging Face Hub, badlogicgames 219 ♥otherUpdated 7 months ago

    badlogicgames/pi-mono
  2. multi-agents

    Datasets & benchmarks

    Multi-agent teams for real work: run teams of coding agents (Claude Code, Codex, any model) in isolated git worktrees, and measure whether the team beats a solo agent on cost and results.

    Stale GitHub, BichengWang 158 ★MITUpdated 8 days ago

    git clone https://github.com/BichengWang/multi-agents.git
  3. claude-fable-5-claude-code

    Datasets & benchmarks

    Hugging Face dataset by armand0e

    Indexed Hugging Face Hub, armand0e 416 ♥mitUpdated 2 months ago

    armand0e/claude-fable-5-claude-code
  4. jupyter-agent-dataset

    Datasets & benchmarks

    Hugging Face dataset by jupyter-agent

    Stale Hugging Face Hub, jupyter-agent 177 ♥apache-2.0Updated 14 months ago

    jupyter-agent/jupyter-agent-dataset
  5. explicit-edit-benchmark

    Datasets & benchmarks

    Benchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.

    Stale GitHub, alexshpunt 38 ★MITUpdated 7 days ago

    git clone https://github.com/alexshpunt/explicit-edit-benchmark.git
  6. AIRTBench-Code

    Datasets & benchmarks

    Code Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

    Stale GitHub, dreadnode 111 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/dreadnode/AIRTBench-Code.git
  7. doc2harness

    Datasets & benchmarks

    Official Code for the paper "Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents "

    Stale GitHub, YuyaoGe 116 ★Custom licence (see repository)Updated 20 days ago

    git clone https://github.com/YuyaoGe/doc2harness.git
  8. OSWorld

    Datasets & benchmarks

    [NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

    Indexed GitHub, xlang-ai 3,262 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/xlang-ai/OSWorld.git
  9. openbench

    Datasets & benchmarks

    Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost

    Stale GitHub, minghinmatthewlam 137 ★MITUpdated 2 months ago

    git clone https://github.com/minghinmatthewlam/openbench.git
  10. agent-duel

    Datasets & benchmarks

    Blinded battles between coding agents: two agents, one prompt, you pick the winner. Desktop app, runs locally.

    Stale GitHub, bottomless 35 ★Custom licence (see repository)Updated 5 days ago

    git clone https://github.com/bottomless/agent-duel.git
  11. AICGSecEval

    Datasets & benchmarks

    A.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team.

    Stale GitHub, Tencent 664 ★Custom licence (see repository)Updated 5 months ago

    git clone https://github.com/Tencent/AICGSecEval.git
  12. Open-SWE-Traces

    Datasets & benchmarks

    Hugging Face dataset by nvidia

    Stale Hugging Face Hub, nvidia 150 ♥cc-by-4.0Updated 34 days ago

    nvidia/Open-SWE-Traces
  13. dataclaw-peteromallet

    Datasets & benchmarks

    Hugging Face dataset by peteromallet

    Indexed Hugging Face Hub, peteromallet 312 ♥mitUpdated 8 months ago

    peteromallet/dataclaw-peteromallet
  14. nautilus-compass

    Datasets & benchmarks

    Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform

    Indexed GitHub, chunxiaoxx 1,258 ★Custom licence (see repository)Updated 2 days ago

    git clone https://github.com/chunxiaoxx/nautilus-compass.git
  15. LoopArena

    Datasets & benchmarks

    Benchmarking models as runtime Controllers for Loop Engineering

    Stale GitHub, AMAP-ML 151 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/AMAP-ML/LoopArena.git
  16. AgentTrove

    Datasets & benchmarks

    Hugging Face dataset by open-thoughts

    Stale Hugging Face Hub, open-thoughts 202 ♥apache-2.0Updated 6 months ago

    open-thoughts/AgentTrove
  17. Step-3.5-Flash-SFT

    Datasets & benchmarks

    Hugging Face dataset by stepfun-ai

    Indexed Hugging Face Hub, stepfun-ai 364 ♥apache-2.0Updated 8 months ago

    stepfun-ai/Step-3.5-Flash-SFT
  18. self-bench

    Datasets & benchmarks

    Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

    Indexed GitHub, mupt-ai 34 ★MITUpdated today

    git clone https://github.com/mupt-ai/self-bench.git
  19. OctoCodingBench

    Datasets & benchmarks

    Hugging Face dataset by MiniMaxAI

    Indexed Hugging Face Hub, MiniMaxAI 465 ♥mitUpdated 10 months ago

    MiniMaxAI/OctoCodingBench
  20. agent-orchestra

    Datasets & benchmarks

    Agents can work. They cannot declare success. A governed coordination protocol for multi-agent AI work: eight contracts (task, resource, ownership, handoff, verification, evidence, approval, recovery) enforced in code, plus a reproducible conformance benchmark. Apache-2.0 code, CC BY 4.0 specification, zero dependencies, Node 18+.

    Stale GitHub, 3338902669-ops 93 ★Apache-2.0Updated 24 days ago

    git clone https://github.com/3338902669-ops/agent-orchestra.git
  21. fde-gym

    Datasets & benchmarks

    The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.

    Stale GitHub, deeplethe 111 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/deeplethe/fde-gym.git
  22. UltraData-SFT-Agent-2609

    Datasets & benchmarks

    Hugging Face dataset by openbmb

    Indexed Hugging Face Hub, openbmb 322 ♥apache-2.0Updated 2 months ago

    openbmb/UltraData-SFT-Agent-2609
  23. ResearchClawBench

    Datasets & benchmarks

    🦞 ResearchClawBench: Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery

    Stale GitHub, InternScience 275 ★MITUpdated 19 days ago

    git clone https://github.com/InternScience/ResearchClawBench.git
  24. SGI-Bench

    Datasets & benchmarks

    Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows

    Stale GitHub, InternScience 172 ★MITUpdated 5 months ago

    git clone https://github.com/InternScience/SGI-Bench.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.