Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
MMLU-Pro
Datasets & benchmarksHugging Face dataset by TIGER-Lab
TIGER-Lab/MMLU-Pro -
OSWorld
Datasets & benchmarks[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
git clone https://github.com/xlang-ai/OSWorld.git -
apex-agents
Datasets & benchmarksHugging Face dataset by mercor
mercor/apex-agents -
LongBench
Datasets & benchmarksLongBench v2 and LongBench (ACL 25'&24')
git clone https://github.com/THUDM/LongBench.git -
nyt-connections
Datasets & benchmarksBenchmark that evaluates LLMs using 759 NYT Connections puzzles extended with extra trick words
git clone https://github.com/lechmazur/nyt-connections.git -
kg-gen
Datasets & benchmarks[NeurIPS '25] Knowledge Graph Generation from Any Text
git clone https://github.com/stair-lab/kg-gen.git -
BALROG
Datasets & benchmarksBenchmarking Agentic LLM and VLM Reasoning On Games
git clone https://github.com/balrog-ai/BALROG.git -
ADR
MCP serversADR secures enterprise AI agents through observability, security benchmarking, and threat detection. Deployed at Uber.
git clone https://github.com/uber/ADR.git -
WideSearch
Datasets & benchmarksWideSearch: Benchmarking Agentic Broad Info-Seeking
git clone https://github.com/ByteDance-Seed/WideSearch.git -
SWE-bench_Pro
Datasets & benchmarksHugging Face dataset by ScaleAI
ScaleAI/SWE-bench_Pro -
DyCodeEval
Datasets & benchmarksOfficial repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”
git clone https://github.com/SeekingDream/DyCodeEval.git -
ParseBench
Datasets & benchmarksHugging Face dataset by llamaindex
llamaindex/ParseBench -
gpqa
Datasets & benchmarksHugging Face dataset by Idavidrein
Idavidrein/gpqa -
AICGSecEval
Datasets & benchmarksA.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team.
git clone https://github.com/Tencent/AICGSecEval.git -
SWE-bench_Verified
Datasets & benchmarksHugging Face dataset by SWE-bench
SWE-bench/SWE-bench_Verified -
MobileWorld
Datasets & benchmarksBenchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments (ACL 2026)
git clone https://github.com/Tongyi-MAI/MobileWorld.git -
confabulations
Datasets & benchmarksHallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.
git clone https://github.com/lechmazur/confabulations.git -
chinese-llm-benchmark
Agents & frameworks非线智能 NoneLinear - ReLE评测:中文AI大模型能力评测(持续更新):目前已囊括374个大模型,覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型, 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜,也提供规模超200万的大模型缺陷库!方便广大社区研究分析、改进大模型。
git clone https://github.com/jeinlee1991/chinese-llm-benchmark.git -
ClawProBench
Datasets & benchmarksClawProBench is a live-first benchmark harness for evaluating LLM agents in the OpenClaw runtime with deterministic grading and repeated-trial reliability.
git clone https://github.com/suyoumo/ClawProBench.git -
leaderboard
Tools & appsHugging Face space by agent-memory-leaderboard
agent-memory-leaderboard/leaderboard -
ChemLLMBench
Datasets & benchmarksOfficial Code for What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks (In NeurIPS 2023)
git clone https://github.com/ChemFoundationModels/ChemLLMBench.git -
chembench
Datasets & benchmarksHow good are LLMs at chemistry?
git clone https://github.com/lamalab-org/chembench.git -
ParseBench
Datasets & benchmarksParseBench - A Document Parsing Benchmark for AI Agents
git clone https://github.com/run-llama/ParseBench.git -
Long-Horizon-Terminal-Bench
Datasets & benchmarksHugging Face dataset by IntelligenceLab
IntelligenceLab/Long-Horizon-Terminal-Bench
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.