InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to build an AI agent

  1. xorcise

    Datasets & benchmarks

    Run cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the claim.

    Stale GitHub, xorcise-ai 33 ★Custom licence (see repository)Updated 17 days ago

    git clone https://github.com/xorcise-ai/xorcise.git
  2. DishwasherBench

    Datasets & benchmarks

    A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.

    Stale GitHub, composo-ai 71 ★MITUpdated 30 days ago

    git clone https://github.com/composo-ai/DishwasherBench.git
  3. TrustLLM

    Datasets & benchmarks

    [ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.

    Stale GitHub, HowieHwong 632 ★MITUpdated 43 days ago

    git clone https://github.com/HowieHwong/TrustLLM.git
  4. fde-gym

    Datasets & benchmarks

    The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.

    Stale GitHub, deeplethe 111 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/deeplethe/fde-gym.git
  5. primitive-bench

    Datasets & benchmarks

    The marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it, refunded if the outcome isn't delivered. This repo is the open verification layer.

    Stale GitHub, primitive-bench 122 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/primitive-bench/primitive-bench.git
  6. meta-agents-research-environments

    Datasets & benchmarks

    Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.

    Stale GitHub, facebookresearch 567 ★MITUpdated 47 days ago

    git clone https://github.com/facebookresearch/meta-agents-research-environments.git
  7. multi-agents

    Datasets & benchmarks

    Multi-agent teams for real work: run teams of coding agents (Claude Code, Codex, any model) in isolated git worktrees, and measure whether the team beats a solo agent on cost and results.

    Stale GitHub, BichengWang 158 ★MITUpdated 8 days ago

    git clone https://github.com/BichengWang/multi-agents.git
  8. bigcodebench

    Datasets & benchmarks

    [ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI

    Stale Archived GitHub, bigcode-project 516 ★Apache-2.0Updated 10 months ago 1 note

    git clone https://github.com/bigcode-project/bigcodebench.git
  9. Awesome-AI4AI

    Datasets & benchmarks

    Updated weekly — until humanity achieves AI4AI. 1148 curated papers across Benchmarks, Harness Design, and Model Design — from long-horizon agents to recursive self-improvement. Citation rankings, GitHub stars, and source-verified weekly news.

    Stale GitHub, KaiWU5 218 ★MITUpdated 8 days ago

    git clone https://github.com/KaiWU5/Awesome-AI4AI.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.