InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. lasagna

    Datasets & benchmarks

    Swap one RAG pipeline stage, hold the rest fixed, and measure whether the gain survives composition. The framework behind "Retrieval Augmented Generation Compositionality": 450 configurations, eight benchmarks, corpora from 5K to 200K documents.

    Indexed GitHub, dolev31 35 ★Apache-2.0Updated yesterday

    git clone https://github.com/dolev31/lasagna.git
  2. AIRTBench-Code

    Datasets & benchmarks

    Code Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

    Stale GitHub, dreadnode 111 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/dreadnode/AIRTBench-Code.git
  3. AICGSecEval

    Datasets & benchmarks

    A.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team.

    Stale GitHub, Tencent 664 ★Custom licence (see repository)Updated 5 months ago

    git clone https://github.com/Tencent/AICGSecEval.git
  4. LLM-Agent-Benchmark-List

    Datasets & benchmarks

    A banchmark list for evaluation of large language models.

    Stale GitHub, zhangxjohn 173 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/zhangxjohn/LLM-Agent-Benchmark-List.git
  5. LLM-RGB

    Datasets & benchmarks

    LLM Reasoning and Generation Benchmark. Evaluate LLMs in complex scenarios systematically.

    Stale GitHub, babelcloud 165 ★MITUpdated 18 months ago

    git clone https://github.com/babelcloud/LLM-RGB.git
  6. autoguardrails

    Datasets & benchmarks

    Alignment-research scaffold (autoresearch-style) for LLM guardrails: search over a single policy.md surface

    Stale GitHub, SantanderAI 129 ★Apache-2.0Updated 46 days ago

    git clone https://github.com/SantanderAI/autoguardrails.git
  7. SGI-Bench

    Datasets & benchmarks

    Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows

    Stale GitHub, InternScience 172 ★MITUpdated 5 months ago

    git clone https://github.com/InternScience/SGI-Bench.git
  8. humanities_data_benchmark

    Datasets & benchmarks

    LLM Benchmark Suite for Humanities Data

    Stale GitHub, RISE-UNIBAS 32 ★GPL-3.0Updated 12 days ago

    git clone https://github.com/RISE-UNIBAS/humanities_data_benchmark.git
  9. PhysGym

    Datasets & benchmarks

    A benchmark suite for evaluating LLM-based interactive scientific reasoning.

    Indexed GitHub, principia-ai 103 ★MITUpdated 3 days ago

    git clone https://github.com/principia-ai/PhysGym.git
  10. GDPevo

    Datasets & benchmarks

    A Benchmark for Evaluating Agent Self-Evolution on Real Business Tasks

    Stale GitHub, Prism-Shadow 119 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/Prism-Shadow/GDPevo.git
  11. DishwasherBench

    Datasets & benchmarks

    A dishwasher-loading game on a physical simulation of one real machine, built to benchmark language models and playable in a browser. The game inside is called Spotless.

    Stale GitHub, composo-ai 71 ★MITUpdated 30 days ago

    git clone https://github.com/composo-ai/DishwasherBench.git
  12. sec-code-bench

    Datasets & benchmarks

    SecCodeBench is a benchmark suite focusing on evaluating the security of code generated by large language models (LLMs).

    Stale GitHub, alibaba 135 ★Apache-2.0Updated 35 days ago

    git clone https://github.com/alibaba/sec-code-bench.git
  13. fde-gym

    Datasets & benchmarks

    The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.

    Stale GitHub, deeplethe 111 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/deeplethe/fde-gym.git
  14. primitive-bench

    Datasets & benchmarks

    The marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it, refunded if the outcome isn't delivered. This repo is the open verification layer.

    Stale GitHub, primitive-bench 122 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/primitive-bench/primitive-bench.git
  15. explicit-edit-benchmark

    Datasets & benchmarks

    Benchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.

    Stale GitHub, alexshpunt 38 ★MITUpdated 7 days ago

    git clone https://github.com/alexshpunt/explicit-edit-benchmark.git
  16. seo-skill-bench

    Datasets & benchmarks

    Open benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic scoring, pre-registered rubric.

    Stale GitHub, aleclindz 51 ★MITUpdated 32 days ago

    git clone https://github.com/aleclindz/seo-skill-bench.git
  17. llm-bulls-and-cows-benchmark

    Datasets & benchmarks

    A mini-framework for evaluating LLM performance on the Bulls and Cows number guessing game, supporting multiple LLM providers.

    Stale GitHub, stalkermustang 235 ★MITUpdated 21 months ago

    git clone https://github.com/stalkermustang/llm-bulls-and-cows-benchmark.git
  18. alem-env

    Datasets & benchmarks

    Can LLM/MARL agents coordinate in long-horizon, open-ended tasks? NeurIPS 2026 (Evaluations and Datasets).

    Stale GitHub, alem-world 57 ★MITUpdated 11 days ago

    git clone https://github.com/alem-world/alem-env.git
  19. mobilegym

    Datasets & benchmarks

    [EMNLP 2026] MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research · 浏览器里运行的安卓模拟器 · Browser-hosted Android Simulator · Verifiable Evaluation · Scalable Online RL Training

    Indexed GitHub, Purewhiter 866 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/Purewhiter/mobilegym.git
  20. VibeSearchBench

    Datasets & benchmarks

    🔍 The hardest search benchmark in the wild — vague, multi-turn, proactive. 200 long-horizon tasks with persona-driven progressive disclosure, scored by verifiable schema-free knowledge-graph evaluation. No vibes, just triplet F1.

    Stale GitHub, VibeBench 112 ★MITUpdated 5 months ago

    git clone https://github.com/VibeBench/VibeSearchBench.git
  21. ReForm-Eval

    Datasets & benchmarks

    An benchmark for evaluating the capabilities of large vision-language models (LVLMs)

    Stale GitHub, FudanDISC 145 ★Apache-2.0Updated 3 years ago

    git clone https://github.com/FudanDISC/ReForm-Eval.git
  22. DeepResearch-Bench-II

    Datasets & benchmarks

    DeepResearch Bench II (DRB2) is the follow-up to DeepResearch Bench, with a stronger focus on measuring the gap between deep research systems and human experts. It does so by decomposing expert-written reports into hierarchical rubrics covering presentation, analysis, and evidence, and using them to evaluate model-generated.

    Stale GitHub, imlrz 96 ★Apache-2.0Updated 28 days ago

    git clone https://github.com/imlrz/DeepResearch-Bench-II.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.