InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

  1. MMLU-Pro

    Datasets & benchmarks

    Hugging Face dataset by TIGER-Lab

    Indexed Hugging Face Hub, TIGER-Lab 602 ♥mitUpdated 6 months ago

    TIGER-Lab/MMLU-Pro
  2. sec-code-bench

    Datasets & benchmarks

    SecCodeBench is a benchmark suite focusing on evaluating the security of code generated by large language models (LLMs).

    Stale GitHub, alibaba 135 ★Apache-2.0Updated 35 days ago

    git clone https://github.com/alibaba/sec-code-bench.git
  3. CL-bench

    Datasets & benchmarks

    Hugging Face dataset by tencent

    Stale Hugging Face Hub, tencent 148 ♥otherUpdated 9 months ago

    tencent/CL-bench
  4. LongBench

    Datasets & benchmarks

    LongBench v2 and LongBench (ACL 25'&24')

    Stale GitHub, THUDM 1,246 ★MITUpdated 22 months ago

    git clone https://github.com/THUDM/LongBench.git
  5. BizFinBench

    Datasets & benchmarks

    A Business-Driven Real-World Financial Benchmark for Evaluating LLMs

    Stale GitHub, HiThink-Research 169 ★Updated 6 months ago

    git clone https://github.com/HiThink-Research/BizFinBench.git
  6. apex-agents

    Datasets & benchmarks

    Hugging Face dataset by mercor

    Stale Hugging Face Hub, mercor 207 ♥cc-by-4.0Updated 5 months ago 1 note

    mercor/apex-agents
  7. o11y-bench

    Datasets & benchmarks

    Open benchmark on observability tasks built on Harbor

    Indexed GitHub, grafana 80 ★AGPL-3.0Updated today

    git clone https://github.com/grafana/o11y-bench.git
  8. swt-bench

    Datasets & benchmarks

    [NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation

    Stale GitHub, logic-star-ai 92 ★MITUpdated 12 days ago

    git clone https://github.com/logic-star-ai/swt-bench.git
  9. kg-gen

    Datasets & benchmarks

    [NeurIPS '25] Knowledge Graph Generation from Any Text

    Stale GitHub, stair-lab 1,286 ★Updated 7 months ago

    git clone https://github.com/stair-lab/kg-gen.git
  10. nyt-connections

    Datasets & benchmarks

    Benchmark that evaluates LLMs using 759 NYT Connections puzzles extended with extra trick words

    Indexed GitHub, lechmazur 250 ★Updated 5 days ago

    git clone https://github.com/lechmazur/nyt-connections.git
  11. nautilus-compass

    Datasets & benchmarks

    Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform

    Indexed GitHub, chunxiaoxx 1,258 ★Custom licence (see repository)Updated 2 days ago

    git clone https://github.com/chunxiaoxx/nautilus-compass.git
  12. gpqa

    Datasets & benchmarks

    Hugging Face dataset by Idavidrein

    Indexed Hugging Face Hub, Idavidrein 642 ♥cc-by-4.0Updated 56 days ago 1 note

    Idavidrein/gpqa
  13. BALROG

    Datasets & benchmarks

    Benchmarking Agentic LLM and VLM Reasoning On Games

    Stale GitHub, balrog-ai 274 ★MITUpdated 7 months ago

    git clone https://github.com/balrog-ai/BALROG.git
  14. SWE-bench

    Datasets & benchmarks

    Hugging Face dataset by princeton-nlp

    Stale Hugging Face Hub, princeton-nlp 152 ♥Updated 20 months ago

    princeton-nlp/SWE-bench
  15. OneMillion-Bench

    Datasets & benchmarks

    [NeurIPS 2026] $OneMillion-Bench: How Far are Language Agents from Human Experts?

    Stale GitHub, humanlaya 55 ★Apache-2.0Updated 27 days ago

    git clone https://github.com/humanlaya/OneMillion-Bench.git
  16. ParseBench

    Datasets & benchmarks

    Hugging Face dataset by llamaindex

    Stale Hugging Face Hub, llamaindex 137 ♥apache-2.0Updated 7 months ago

    llamaindex/ParseBench
  17. AICGSecEval

    Datasets & benchmarks

    A.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team.

    Stale GitHub, Tencent 664 ★Custom licence (see repository)Updated 5 months ago

    git clone https://github.com/Tencent/AICGSecEval.git
  18. WideSearch

    Datasets & benchmarks

    WideSearch: Benchmarking Agentic Broad Info-Seeking

    Stale GitHub, ByteDance-Seed 156 ★MITUpdated 13 months ago

    git clone https://github.com/ByteDance-Seed/WideSearch.git
  19. DyCodeEval

    Datasets & benchmarks

    Official repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”

    Stale GitHub, SeekingDream 222 ★Updated 10 months ago

    git clone https://github.com/SeekingDream/DyCodeEval.git
  20. ClawProBench

    Datasets & benchmarks

    ClawProBench is a live-first benchmark harness for evaluating LLM agents in the OpenClaw runtime with deterministic grading and repeated-trial reliability.

    Stale GitHub, suyoumo 825 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/suyoumo/ClawProBench.git
  21. MobileWorld

    Datasets & benchmarks

    Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments (ACL 2026)

    Stale GitHub, Tongyi-MAI 300 ★Apache-2.0Updated 13 days ago

    git clone https://github.com/Tongyi-MAI/MobileWorld.git
  22. confabulations

    Datasets & benchmarks

    Hallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.

    Stale GitHub, lechmazur 249 ★Updated 15 months ago

    git clone https://github.com/lechmazur/confabulations.git
  23. ParseBench

    Datasets & benchmarks

    ParseBench - A Document Parsing Benchmark for AI Agents

    Indexed GitHub, run-llama 640 ★Apache-2.0Updated today

    git clone https://github.com/run-llama/ParseBench.git
  24. ChemLLMBench

    Datasets & benchmarks

    Official Code for What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks (In NeurIPS 2023)

    Stale GitHub, ChemFoundationModels 176 ★Updated 2 years ago

    git clone https://github.com/ChemFoundationModels/ChemLLMBench.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.