InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

Looks like a goal we have a starting path for: I want to build an AI agent

  1. ClawBench

    Datasets & benchmarks

    Open-source benchmark for browser AI agents on daily tasks.

    Stale GitHub, TIGER-AI-Lab 1,229 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/TIGER-AI-Lab/ClawBench.git
  2. ai-agents-the-definitive-guide

    Agents & frameworks

    Repo for AI Agents The Definitive Guide

    Stale GitHub, Nicolepcx 2,747 ★Updated 3 months ago

    git clone https://github.com/Nicolepcx/ai-agents-the-definitive-guide.git
  3. GLM-5.3-Flash × J-Space capability realization — benchmark presentation of the J-Space Cognition Suite

    Stale GitHub, Tiger3807861189 1,019 ★Custom licence (see repository)Updated 2 months ago

    git clone https://github.com/Tiger3807861189/GLM-5.3-Flash-J-Space-Capability-Realization-Report.git
  4. swe-sweep

    Datasets & benchmarks

    How many bugs can LMs find & fix in large codebases?

    Indexed GitHub, facebookresearch 240 ★Custom licence (see repository)Updated 4 days ago

    git clone https://github.com/facebookresearch/swe-sweep.git
  5. openbench

    Datasets & benchmarks

    Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost

    Stale GitHub, minghinmatthewlam 137 ★MITUpdated 2 months ago

    git clone https://github.com/minghinmatthewlam/openbench.git
  6. Create, test, and optimize OpenCode skills. Supports OpenCode V2 and V1 ≥ 1.18.29. Adapted from Anthropic's skill-creator.

    Stale GitHub, antongulin 174 ★Apache-2.0Updated 29 days ago

    git clone https://github.com/antongulin/opencode-skill-creator.git
  7. agent-duel

    Datasets & benchmarks

    Blinded battles between coding agents: two agents, one prompt, you pick the winner. Desktop app, runs locally.

    Stale GitHub, bottomless 35 ★Custom licence (see repository)Updated 5 days ago

    git clone https://github.com/bottomless/agent-duel.git
  8. WebsiteBench

    Datasets & benchmarks

    Can AI agents rebuild websites through browser exploration?

    Indexed GitHub, TIGER-AI-Lab 300 ★Apache-2.0Updated 6 days ago

    git clone https://github.com/TIGER-AI-Lab/WebsiteBench.git
  9. overtime

    Agents & frameworks

    Give a coding agent a brief, not a chat, and it works on its own across sessions. Includes an example run: an open-world city built in a real game engine with no human help. In early development.

    Stale GitHub, ukanwat 399 ★MITUpdated 30 days ago

    git clone https://github.com/ukanwat/overtime.git
  10. context-diamond

    MCP servers

    Auditable context capsules for LLM handoffs, coding agents, and OpenCode MCP workflows.

    Stale GitHub, RainCherb 20 ★MITUpdated 24 days ago

    git clone https://github.com/RainCherb/context-diamond.git
  11. self-bench

    Datasets & benchmarks

    Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

    Indexed GitHub, mupt-ai 34 ★MITUpdated today

    git clone https://github.com/mupt-ai/self-bench.git
  12. xorcise

    Datasets & benchmarks

    Run cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the claim.

    Stale GitHub, xorcise-ai 33 ★Custom licence (see repository)Updated 17 days ago

    git clone https://github.com/xorcise-ai/xorcise.git
  13. embodied-first-exam

    Agents & frameworks

    The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.

    Stale GitHub, mll-lab-nu 236 ★MITUpdated 10 days ago

    git clone https://github.com/mll-lab-nu/embodied-first-exam.git
  14. DishwasherBench

    Datasets & benchmarks

    A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.

    Stale GitHub, composo-ai 71 ★MITUpdated 30 days ago

    git clone https://github.com/composo-ai/DishwasherBench.git
  15. SkillCorpus

    Skills

    Open-source infrastructure that turns scattered SKILL.md files into curated, retrieval-ready agent-skill corpora—with retrieval and evaluation tooling included.

    Stale GitHub, EverMind-AI 700 ★Apache-2.0Updated 7 days ago

    git clone https://github.com/EverMind-AI/SkillCorpus.git
  16. TrustLLM

    Datasets & benchmarks

    [ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.

    Stale GitHub, HowieHwong 632 ★MITUpdated 43 days ago

    git clone https://github.com/HowieHwong/TrustLLM.git
  17. explicit-edit-benchmark

    Datasets & benchmarks

    Benchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.

    Stale GitHub, alexshpunt 38 ★MITUpdated 7 days ago

    git clone https://github.com/alexshpunt/explicit-edit-benchmark.git
  18. fde-gym

    Datasets & benchmarks

    The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.

    Stale GitHub, deeplethe 111 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/deeplethe/fde-gym.git
  19. primitive-bench

    Datasets & benchmarks

    The marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it, refunded if the outcome isn't delivered. This repo is the open verification layer.

    Stale GitHub, primitive-bench 122 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/primitive-bench/primitive-bench.git
  20. multi-agents

    Datasets & benchmarks

    Multi-agent teams for real work: run teams of coding agents (Claude Code, Codex, any model) in isolated git worktrees, and measure whether the team beats a solo agent on cost and results.

    Stale GitHub, BichengWang 158 ★MITUpdated 8 days ago

    git clone https://github.com/BichengWang/multi-agents.git
  21. skillcheck

    Prompts

    Blind A/B testing for AI agent skills, .cursorrules, CLAUDE.md, and system prompts. Detects placebo and harmful instructions.

    Stale GitHub, sx4im 30 ★MITUpdated 13 days ago

    git clone https://github.com/sx4im/skillcheck.git
  22. dokimi

    Skills

    Dokimi (Greek δοκιμή, "test, proof") measures whether a skill, MCP doc source, agent profile or repository guidance actually helps a coding agent: paired on/off arm comparisons, novel-vs-known content analysis, and repository scans with graded agent runs.

    Indexed GitHub, intel 37 ★Apache-2.0Updated today

    git clone https://github.com/intel/dokimi.git
  23. ultrametric

    Tools & apps

    The open startup repo — everything a founder needs to start and run a company in the agentic era: the comprehensive founder processes from an agentic lens, open documents, situations founders might find themselves in, law that governs the business logic, startup lore, open modules, artifacts and schema.

    Indexed GitHub, ultrametricai 87 ★Apache-2.0Updated today

    git clone https://github.com/ultrametricai/ultrametric.git
  24. eval-genius

    Skills

    Teach your agent to work with evals: WHEN you actually need an eval or benchmark, HOW to build one that holds up, and how to read what it tells you. Deterministic-first, tool-agnostic.

    Stale GitHub, alexgreensh 24 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/alexgreensh/eval-genius.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.