InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. MMLU-Pro

    Datasets & benchmarks

    Hugging Face dataset by TIGER-Lab

    Indexed Hugging Face Hub, TIGER-Lab 602 ♥mitUpdated 6 months ago

    TIGER-Lab/MMLU-Pro
  2. OSWorld

    Datasets & benchmarks

    [NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

    Indexed GitHub, xlang-ai 3,262 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/xlang-ai/OSWorld.git
  3. apex-agents

    Datasets & benchmarks

    Hugging Face dataset by mercor

    Stale Hugging Face Hub, mercor 207 ♥cc-by-4.0Updated 5 months ago 1 note

    mercor/apex-agents
  4. LongBench

    Datasets & benchmarks

    LongBench v2 and LongBench (ACL 25'&24')

    Stale GitHub, THUDM 1,246 ★MITUpdated 22 months ago

    git clone https://github.com/THUDM/LongBench.git
  5. nyt-connections

    Datasets & benchmarks

    Benchmark that evaluates LLMs using 759 NYT Connections puzzles extended with extra trick words

    Indexed GitHub, lechmazur 250 ★Updated 5 days ago

    git clone https://github.com/lechmazur/nyt-connections.git
  6. kg-gen

    Datasets & benchmarks

    [NeurIPS '25] Knowledge Graph Generation from Any Text

    Stale GitHub, stair-lab 1,286 ★Updated 7 months ago

    git clone https://github.com/stair-lab/kg-gen.git
  7. BALROG

    Datasets & benchmarks

    Benchmarking Agentic LLM and VLM Reasoning On Games

    Stale GitHub, balrog-ai 274 ★MITUpdated 7 months ago

    git clone https://github.com/balrog-ai/BALROG.git
  8. ADR

    MCP servers

    ADR secures enterprise AI agents through observability, security benchmarking, and threat detection. Deployed at Uber.

    Stale GitHub, uber 2,090 ★Apache-2.0Updated 16 days ago

    git clone https://github.com/uber/ADR.git
  9. WideSearch

    Datasets & benchmarks

    WideSearch: Benchmarking Agentic Broad Info-Seeking

    Stale GitHub, ByteDance-Seed 156 ★MITUpdated 13 months ago

    git clone https://github.com/ByteDance-Seed/WideSearch.git
  10. SWE-bench_Pro

    Datasets & benchmarks

    Hugging Face dataset by ScaleAI

    Stale Hugging Face Hub, ScaleAI 197 ♥Updated 55 days ago

    ScaleAI/SWE-bench_Pro
  11. DyCodeEval

    Datasets & benchmarks

    Official repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”

    Stale GitHub, SeekingDream 222 ★Updated 10 months ago

    git clone https://github.com/SeekingDream/DyCodeEval.git
  12. ParseBench

    Datasets & benchmarks

    Hugging Face dataset by llamaindex

    Stale Hugging Face Hub, llamaindex 137 ♥apache-2.0Updated 7 months ago

    llamaindex/ParseBench
  13. gpqa

    Datasets & benchmarks

    Hugging Face dataset by Idavidrein

    Indexed Hugging Face Hub, Idavidrein 642 ♥cc-by-4.0Updated 56 days ago 1 note

    Idavidrein/gpqa
  14. AICGSecEval

    Datasets & benchmarks

    A.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team.

    Stale GitHub, Tencent 664 ★Custom licence (see repository)Updated 5 months ago

    git clone https://github.com/Tencent/AICGSecEval.git
  15. SWE-bench_Verified

    Datasets & benchmarks

    Hugging Face dataset by SWE-bench

    Stale Hugging Face Hub, SWE-bench 188 ♥Updated 3 months ago

    SWE-bench/SWE-bench_Verified
  16. MobileWorld

    Datasets & benchmarks

    Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments (ACL 2026)

    Stale GitHub, Tongyi-MAI 300 ★Apache-2.0Updated 13 days ago

    git clone https://github.com/Tongyi-MAI/MobileWorld.git
  17. confabulations

    Datasets & benchmarks

    Hallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.

    Stale GitHub, lechmazur 249 ★Updated 15 months ago

    git clone https://github.com/lechmazur/confabulations.git
  18. chinese-llm-benchmark

    Agents & frameworks

    非线智能 NoneLinear - ReLE评测:中文AI大模型能力评测(持续更新):目前已囊括374个大模型,覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型, 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜,也提供规模超200万的大模型缺陷库!方便广大社区研究分析、改进大模型。

    Indexed GitHub, jeinlee1991 6,518 ★Updated yesterday

    git clone https://github.com/jeinlee1991/chinese-llm-benchmark.git
  19. ClawProBench

    Datasets & benchmarks

    ClawProBench is a live-first benchmark harness for evaluating LLM agents in the OpenClaw runtime with deterministic grading and repeated-trial reliability.

    Stale GitHub, suyoumo 825 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/suyoumo/ClawProBench.git
  20. leaderboard

    Tools & apps

    Hugging Face space by agent-memory-leaderboard

    Indexed Hugging Face Hub, agent-memory-leaderboard 564 ♥Updated 3 months ago 1 note

    agent-memory-leaderboard/leaderboard
  21. ChemLLMBench

    Datasets & benchmarks

    Official Code for What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks (In NeurIPS 2023)

    Stale GitHub, ChemFoundationModels 176 ★Updated 2 years ago

    git clone https://github.com/ChemFoundationModels/ChemLLMBench.git
  22. chembench

    Datasets & benchmarks

    How good are LLMs at chemistry?

    Stale GitHub, lamalab-org 147 ★MITUpdated 9 months ago

    git clone https://github.com/lamalab-org/chembench.git
  23. ParseBench

    Datasets & benchmarks

    ParseBench - A Document Parsing Benchmark for AI Agents

    Indexed GitHub, run-llama 640 ★Apache-2.0Updated today

    git clone https://github.com/run-llama/ParseBench.git
  24. Long-Horizon-Terminal-Bench

    Datasets & benchmarks

    Hugging Face dataset by IntelligenceLab

    Stale Hugging Face Hub, IntelligenceLab 140 ♥apache-2.0Updated 36 days ago

    IntelligenceLab/Long-Horizon-Terminal-Bench

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.