Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
rStar-Coder
Datasets & benchmarksHugging Face dataset by microsoft
microsoft/rStar-Coder -
AIRTBench-Code
Datasets & benchmarksCode Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models
git clone https://github.com/dreadnode/AIRTBench-Code.git -
self-bench
Datasets & benchmarksBenchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.
git clone https://github.com/mupt-ai/self-bench.git -
openbench
Datasets & benchmarksSame model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost
git clone https://github.com/minghinmatthewlam/openbench.git -
seo-skill-bench
Datasets & benchmarksOpen benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic scoring, pre-registered rubric.
git clone https://github.com/aleclindz/seo-skill-bench.git -
opencompass
Datasets & benchmarksOpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.
git clone https://github.com/open-compass/opencompass.git -
LLM-eval-survey
Datasets & benchmarksThe official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".
git clone https://github.com/MLGroupJLU/LLM-eval-survey.git -
Awesome-LLM-Eval
Datasets & benchmarksAwesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. 一个由工具、基准/数据、演示、排行榜和大模型等组成的精选列表,主要面向基础大模型评测,旨在探求生成式AI的技术边界.
git clone https://github.com/onejune2018/Awesome-LLM-Eval.git -
One-Eval
Datasets & benchmarksAutomated system for LLM evaluation via agents. Doc as below:
git clone https://github.com/OpenDCAI/One-Eval.git -
olmOCR-bench
Datasets & benchmarksHugging Face dataset by allenai
allenai/olmOCR-bench -
tweet_eval
Datasets & benchmarksHugging Face dataset by cardiffnlp
cardiffnlp/tweet_eval -
ReForm-Eval
Datasets & benchmarksAn benchmark for evaluating the capabilities of large vision-language models (LVLMs)
git clone https://github.com/FudanDISC/ReForm-Eval.git -
gsm8k
Datasets & benchmarksHugging Face dataset by openai
openai/gsm8k -
MMLU-Pro
Datasets & benchmarksHugging Face dataset by TIGER-Lab
TIGER-Lab/MMLU-Pro -
SWE-bench_Pro
Datasets & benchmarksHugging Face dataset by ScaleAI
ScaleAI/SWE-bench_Pro -
SWE-bench_Verified
Datasets & benchmarksHugging Face dataset by SWE-bench
SWE-bench/SWE-bench_Verified -
gamebot
Datasets & benchmarks[ACL 2025] GAMEBoT: Transparent Assessment of LLM Reasoning in Games
git clone https://github.com/Visual-AI/gamebot.git -
github-code
Datasets & benchmarksHugging Face dataset by codeparrot
codeparrot/github-code -
gpqa
Datasets & benchmarksHugging Face dataset by Idavidrein
Idavidrein/gpqa -
agents-last-exam
Datasets & benchmarksHugging Face dataset by agents-last-exam
agents-last-exam/agents-last-exam -
typed-decisions
Datasets & benchmarksHugging Face dataset by LocalLLaMA
LocalLLaMA/typed-decisions -
apex-agents
Datasets & benchmarksHugging Face dataset by mercor
mercor/apex-agents -
UltraData-Code
Datasets & benchmarksHugging Face dataset by openbmb
openbmb/UltraData-Code -
python_code_instructions_18k_alpaca
Datasets & benchmarksHugging Face dataset by iamtarun
iamtarun/python_code_instructions_18k_alpaca
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.