Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
opencompass
Datasets & benchmarksOpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.
git clone https://github.com/open-compass/opencompass.git -
LLM-eval-survey
Datasets & benchmarksThe official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".
git clone https://github.com/MLGroupJLU/LLM-eval-survey.git -
ClawBench
Datasets & benchmarksOpen-source benchmark for browser AI agents on daily tasks.
git clone https://github.com/TIGER-AI-Lab/ClawBench.git -
Awesome-LLM-Eval
Datasets & benchmarksAwesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. 一个由工具、基准/数据、演示、排行榜和大模型等组成的精选列表,主要面向基础大模型评测,旨在探求生成式AI的技术边界.
git clone https://github.com/onejune2018/Awesome-LLM-Eval.git -
MMLU-Pro
Datasets & benchmarksHugging Face dataset by TIGER-Lab
TIGER-Lab/MMLU-Pro -
WizardLM_evol_instruct_V2_196k
Datasets & benchmarksHugging Face dataset by WizardLMTeam
WizardLMTeam/WizardLM_evol_instruct_V2_196k -
Evol-Instruct-Code-80k-v1
Datasets & benchmarksHugging Face dataset by nickrosh
nickrosh/Evol-Instruct-Code-80k-v1 -
Static-to-Dynamic-LLMEval
Datasets & benchmarksThe official GitHub repository of the paper "Recent advances in large language model benchmarks against data contamination: From static to dynamic evaluation"
git clone https://github.com/SeekingDream/Static-to-Dynamic-LLMEval.git -
ClawProBench
Datasets & benchmarksClawProBench is a live-first benchmark harness for evaluating LLM agents in the OpenClaw runtime with deterministic grading and repeated-trial reliability.
git clone https://github.com/suyoumo/ClawProBench.git -
WizardLM_evol_instruct_70k
Datasets & benchmarksHugging Face dataset by WizardLMTeam
WizardLMTeam/WizardLM_evol_instruct_70k -
gsm8k
Datasets & benchmarksHugging Face dataset by openai
openai/gsm8k -
One-Eval
Datasets & benchmarksAutomated system for LLM evaluation via agents. Doc as below:
git clone https://github.com/OpenDCAI/One-Eval.git -
Magicoder-Evol-Instruct-110K
Datasets & benchmarksHugging Face dataset by ise-uiuc
ise-uiuc/Magicoder-Evol-Instruct-110K -
TrustLLM
Datasets & benchmarks[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.
git clone https://github.com/HowieHwong/TrustLLM.git -
evol-codealpaca-v1
Datasets & benchmarksHugging Face dataset by theblackcat102
theblackcat102/evol-codealpaca-v1 -
xFinder
Datasets & benchmarks[ICLR 2025] xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation
git clone https://github.com/IAAR-Shanghai/xFinder.git -
typed-decisions
Datasets & benchmarksHugging Face dataset by LocalLLaMA
LocalLLaMA/typed-decisions -
WizardLM_alpaca_evol_instruct_70k_unfiltered
Datasets & benchmarksHugging Face dataset by QuixiAI
QuixiAI/WizardLM_alpaca_evol_instruct_70k_unfiltered -
agents-last-exam
Datasets & benchmarksHugging Face dataset by agents-last-exam
agents-last-exam/agents-last-exam -
everyday-conversations-llama3.1-2k
Datasets & benchmarksHugging Face dataset by HuggingFaceTB
HuggingFaceTB/everyday-conversations-llama3.1-2k -
FastEval
Datasets & benchmarksFast & more realistic evaluation of chat language models. Includes leaderboard.
git clone https://github.com/FastEval/FastEval.git -
UHGEval
Datasets & benchmarks[ACL 2024] User-friendly evaluation framework: Eval Suite & Benchmarks: UHGEval, HaluEval, HalluQA, etc.
git clone https://github.com/IAAR-Shanghai/UHGEval.git -
olmOCR-bench
Datasets & benchmarksHugging Face dataset by allenai
allenai/olmOCR-bench -
Awesome-LLM-Long-Context-Modeling
Datasets & benchmarks📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥
git clone https://github.com/Xnhyacinth/Awesome-LLM-Long-Context-Modeling.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.