Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
HallusionBench
Datasets & benchmarks[CVPR'24] HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models
git clone https://github.com/tianyi-lab/HallusionBench.git -
Static-to-Dynamic-LLMEval
Datasets & benchmarksThe official GitHub repository of the paper "Recent advances in large language model benchmarks against data contamination: From static to dynamic evaluation"
git clone https://github.com/SeekingDream/Static-to-Dynamic-LLMEval.git -
the_pile_books3
Datasets & benchmarksHugging Face dataset by defunct-datasets
defunct-datasets/the_pile_books3 -
SkyPile-150B
Datasets & benchmarksHugging Face dataset by Skywork
Skywork/SkyPile-150B -
LLM-Agent-Benchmark-List
Datasets & benchmarksA banchmark list for evaluation of large language models.
git clone https://github.com/zhangxjohn/LLM-Agent-Benchmark-List.git -
ParseBench
Datasets & benchmarksParseBench - A Document Parsing Benchmark for AI Agents
git clone https://github.com/run-llama/ParseBench.git -
ocr
Datasets & benchmarksHugging Face dataset by uv-scripts
uv-scripts/ocr -
CharacterCodex
Datasets & benchmarksHugging Face dataset by NousResearch
NousResearch/CharacterCodex -
xVerify
Datasets & benchmarksxVerify: Efficient Answer Verifier for Reasoning Model Evaluations
git clone https://github.com/IAAR-Shanghai/xVerify.git -
oscar
Datasets & benchmarksHugging Face dataset by oscar-corpus
oscar-corpus/oscar -
ping_pong_bench
Datasets & benchmarksA benchmark for role-playing language models
git clone https://github.com/IlyaGusev/ping_pong_bench.git -
Awesome-LLM-Eval
Datasets & benchmarksAwesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. 一个由工具、基准/数据、演示、排行榜和大模型等组成的精选列表,主要面向基础大模型评测,旨在探求生成式AI的技术边界.
git clone https://github.com/onejune2018/Awesome-LLM-Eval.git -
apps
Datasets & benchmarksHugging Face dataset by codeparrot
codeparrot/apps -
pubmed
Datasets & benchmarksHugging Face dataset by ncbi
ncbi/pubmed -
AI_Diplomacy
Datasets & benchmarksFrontier Models playing the board game Diplomacy.
git clone https://github.com/GoodStartLabs/AI_Diplomacy.git -
Nectar
Datasets & benchmarksHugging Face dataset by berkeley-nest
berkeley-nest/Nectar -
minipile
Datasets & benchmarksHugging Face dataset by JeanKaddour
JeanKaddour/minipile -
xFinder
Datasets & benchmarks[ICLR 2025] xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation
git clone https://github.com/IAAR-Shanghai/xFinder.git -
DyCodeEval
Datasets & benchmarksOfficial repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”
git clone https://github.com/SeekingDream/DyCodeEval.git -
ReForm-Eval
Datasets & benchmarksAn benchmark for evaluating the capabilities of large vision-language models (LVLMs)
git clone https://github.com/FudanDISC/ReForm-Eval.git -
openbench
Datasets & benchmarksSame model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost
git clone https://github.com/minghinmatthewlam/openbench.git -
TrustLLM
Datasets & benchmarks[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.
git clone https://github.com/HowieHwong/TrustLLM.git -
mc4
Datasets & benchmarksHugging Face dataset by legacy-datasets
legacy-datasets/mc4 -
ChemLLMBench
Datasets & benchmarksOfficial Code for What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks (In NeurIPS 2023)
git clone https://github.com/ChemFoundationModels/ChemLLMBench.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.