InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to build an AI agent

  1. SWE-rebench-openhands-trajectories

    Datasets & benchmarks

    Hugging Face dataset by nebius

    Stale Hugging Face Hub, nebius 147 ♥cc-by-4.0Updated 10 months ago

    nebius/SWE-rebench-openhands-trajectories
  2. swe-sweep

    Datasets & benchmarks

    How many bugs can LMs find & fix in large codebases?

    Indexed GitHub, facebookresearch 240 ★Custom licence (see repository)Updated 4 days ago

    git clone https://github.com/facebookresearch/swe-sweep.git
  3. self-bench

    Datasets & benchmarks

    Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

    Indexed GitHub, mupt-ai 34 ★MITUpdated today

    git clone https://github.com/mupt-ai/self-bench.git
  4. reasoning-corpus-4K-5M-v1

    Datasets & benchmarks

    Hugging Face dataset by mithulaartigala

    Stale Hugging Face Hub, mithulaartigala 208 ♥apache-2.0Updated 36 days ago

    mithulaartigala/reasoning-corpus-4K-5M-v1
  5. nagi-bench

    Datasets & benchmarks

    One-shot LLM eval cases by NAGI STUDIO - same prompt, different agents (model + harness), runnable artifacts side by side.

    Indexed GitHub, nagi-studio 78 ★MITUpdated 4 days ago

    git clone https://github.com/nagi-studio/nagi-bench.git
  6. primitive-bench

    Datasets & benchmarks

    The marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it, refunded if the outcome isn't delivered. This repo is the open verification layer.

    Stale GitHub, primitive-bench 122 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/primitive-bench/primitive-bench.git
  7. claude_mythos_distilled_25k

    Datasets & benchmarks

    Hugging Face dataset by WithinUsAI

    Stale Hugging Face Hub, WithinUsAI 191 ♥apache-2.0Updated 54 days ago

    WithinUsAI/claude_mythos_distilled_25k
  8. TerminalWorld

    Datasets & benchmarks

    TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks (NeurIPS 2026 Evaluations and Datasets Track)

    Stale GitHub, terminalworld 54 ★Apache-2.0Updated 12 days ago

    git clone https://github.com/terminalworld/TerminalWorld.git
  9. o11y-bench

    Datasets & benchmarks

    Open benchmark on observability tasks built on Harbor

    Indexed GitHub, grafana 80 ★AGPL-3.0Updated today

    git clone https://github.com/grafana/o11y-bench.git
  10. PhysGym

    Datasets & benchmarks

    A benchmark suite for evaluating LLM-based interactive scientific reasoning.

    Indexed GitHub, principia-ai 103 ★MITUpdated 3 days ago

    git clone https://github.com/principia-ai/PhysGym.git
  11. ParseBench

    Datasets & benchmarks

    ParseBench - A Document Parsing Benchmark for AI Agents

    Indexed GitHub, run-llama 640 ★Apache-2.0Updated today

    git clone https://github.com/run-llama/ParseBench.git
  12. SWE-ZERO-12M-trajectories

    Datasets & benchmarks

    Hugging Face dataset by AlienKevin

    Stale Hugging Face Hub, AlienKevin 130 ♥apache-2.0Updated 6 months ago

    AlienKevin/SWE-ZERO-12M-trajectories
  13. deepsearchqa

    Datasets & benchmarks

    Hugging Face dataset by google

    Stale Hugging Face Hub, google 133 ♥apache-2.0Updated 11 months ago

    google/deepsearchqa
  14. dataclaw-peteromallet

    Datasets & benchmarks

    Hugging Face dataset by peteromallet

    Indexed Hugging Face Hub, peteromallet 312 ♥mitUpdated 8 months ago

    peteromallet/dataclaw-peteromallet
  15. xorcise

    Datasets & benchmarks

    Run cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the claim.

    Stale GitHub, xorcise-ai 33 ★Custom licence (see repository)Updated 17 days ago

    git clone https://github.com/xorcise-ai/xorcise.git
  16. fable-5-premium

    Datasets & benchmarks

    Hugging Face dataset by saidutta69

    Stale Hugging Face Hub, saidutta69 136 ♥mitUpdated 2 months ago

    saidutta69/fable-5-premium
  17. elimination_game

    Datasets & benchmarks

    A multi-player tournament benchmark that tests LLMs in social reasoning, strategy, and deception. Players engage in public and private conversations, form alliances, and vote to eliminate each other

    Stale GitHub, lechmazur 298 ★Updated 10 months ago

    git clone https://github.com/lechmazur/elimination_game.git
  18. qwen38-strix-halo-harness

    Datasets & benchmarks

    AI coding agent harness for any hardware, local or cloud (OpenAI-compatible engines). Tuned for AMD Strix Halo: Qwen3.8 125B on the iGPU, NPU side-jobs, crash recovery, logged benchmarks.

    Stale GitHub, aic0d3r 38 ★MITUpdated 8 days ago

    git clone https://github.com/aic0d3r/qwen38-strix-halo-harness.git
  19. SGI-Bench

    Datasets & benchmarks

    Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows

    Stale GitHub, InternScience 172 ★MITUpdated 5 months ago

    git clone https://github.com/InternScience/SGI-Bench.git
  20. Zero-To-CAD-1m

    Datasets & benchmarks

    Hugging Face dataset by ADSKAILab

    Stale Hugging Face Hub, ADSKAILab 135 ♥apache-2.0Updated 6 months ago

    ADSKAILab/Zero-To-CAD-1m
  21. DishwasherBench

    Datasets & benchmarks

    A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.

    Stale GitHub, composo-ai 71 ★MITUpdated 30 days ago

    git clone https://github.com/composo-ai/DishwasherBench.git
  22. HaluMem

    Datasets & benchmarks

    HaluMem is the first operation level hallucination evaluation benchmark tailored to agent memory systems.

    Stale GitHub, MemTensor 178 ★Custom licence (see repository)Updated 19 days ago

    git clone https://github.com/MemTensor/HaluMem.git
  23. seo-skill-bench

    Datasets & benchmarks

    Open benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic scoring, pre-registered rubric.

    Stale GitHub, aleclindz 51 ★MITUpdated 32 days ago

    git clone https://github.com/aleclindz/seo-skill-bench.git
  24. prism

    Datasets & benchmarks

    Открытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением в 1С — Claude, GPT, Gemini, DeepSeek, YandexGPT, GigaChat.

    Stale GitHub, genlab-1c 55 ★MITUpdated 18 days ago

    git clone https://github.com/genlab-1c/prism.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.