Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
TACO
Datasets & benchmarksHugging Face dataset by BAAI
BAAI/TACO -
ping_pong_bench
Datasets & benchmarksA benchmark for role-playing language models
git clone https://github.com/IlyaGusev/ping_pong_bench.git -
Awesome-LLM-Long-Context-Modeling
Datasets & benchmarks📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥
git clone https://github.com/Xnhyacinth/Awesome-LLM-Long-Context-Modeling.git -
LLM-Agent-Benchmark-List
Datasets & benchmarksA banchmark list for evaluation of large language models.
git clone https://github.com/zhangxjohn/LLM-Agent-Benchmark-List.git -
OSWorld-V2
Datasets & benchmarksOSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks
git clone https://github.com/xlang-ai/OSWorld-V2.git -
CharacterCodex
Datasets & benchmarksHugging Face dataset by NousResearch
NousResearch/CharacterCodex -
pile-of-law
Datasets & benchmarksHugging Face dataset by pile-of-law
pile-of-law/pile-of-law -
tau2-bench
Datasets & benchmarksτ-Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
git clone https://github.com/sierra-research/tau2-bench.git -
SkyPile-150B
Datasets & benchmarksHugging Face dataset by Skywork
Skywork/SkyPile-150B -
Matrix
Datasets & benchmarksHugging Face dataset by m-a-p
m-a-p/Matrix -
ReForm-Eval
Datasets & benchmarksAn benchmark for evaluating the capabilities of large vision-language models (LVLMs)
git clone https://github.com/FudanDISC/ReForm-Eval.git -
OSCAR-2201
Datasets & benchmarksHugging Face dataset by oscar-corpus
oscar-corpus/OSCAR-2201 -
Static-to-Dynamic-LLMEval
Datasets & benchmarksThe official GitHub repository of the paper "Recent advances in large language model benchmarks against data contamination: From static to dynamic evaluation"
git clone https://github.com/SeekingDream/Static-to-Dynamic-LLMEval.git -
HallusionBench
Datasets & benchmarks[CVPR'24] HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models
git clone https://github.com/tianyi-lab/HallusionBench.git -
DyCodeEval
Datasets & benchmarksOfficial repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”
git clone https://github.com/SeekingDream/DyCodeEval.git -
WebInstructSub
Datasets & benchmarksHugging Face dataset by TIGER-Lab
TIGER-Lab/WebInstructSub -
ChemLLMBench
Datasets & benchmarksOfficial Code for What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks (In NeurIPS 2023)
git clone https://github.com/ChemFoundationModels/ChemLLMBench.git -
ocr
Datasets & benchmarksHugging Face dataset by uv-scripts
uv-scripts/ocr -
finephrase
Datasets & benchmarksHugging Face dataset by HuggingFaceFW
HuggingFaceFW/finephrase -
JBB-Behaviors
Datasets & benchmarksHugging Face dataset by JailbreakBench
JailbreakBench/JBB-Behaviors -
LightCompress
Datasets & benchmarks[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.
git clone https://github.com/ModelTC/LightCompress.git -
TrustLLM
Datasets & benchmarks[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.
git clone https://github.com/HowieHwong/TrustLLM.git -
Omni-SafetyBench
Datasets & benchmarks[ACM MM 2026 Oral] Code for paper "Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models".
git clone https://github.com/THU-BPM/Omni-SafetyBench.git -
ParseBench
Datasets & benchmarksParseBench - A Document Parsing Benchmark for AI Agents
git clone https://github.com/run-llama/ParseBench.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.