Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
Nectar
Datasets & benchmarksHugging Face dataset by berkeley-nest
berkeley-nest/Nectar -
minipile
Datasets & benchmarksHugging Face dataset by JeanKaddour
JeanKaddour/minipile -
xFinder
Datasets & benchmarks[ICLR 2025] xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation
git clone https://github.com/IAAR-Shanghai/xFinder.git -
DyCodeEval
Datasets & benchmarksOfficial repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”
git clone https://github.com/SeekingDream/DyCodeEval.git -
ReForm-Eval
Datasets & benchmarksAn benchmark for evaluating the capabilities of large vision-language models (LVLMs)
git clone https://github.com/FudanDISC/ReForm-Eval.git -
openbench
Datasets & benchmarksSame model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost
git clone https://github.com/minghinmatthewlam/openbench.git -
TrustLLM
Datasets & benchmarks[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.
git clone https://github.com/HowieHwong/TrustLLM.git -
mc4
Datasets & benchmarksHugging Face dataset by legacy-datasets
legacy-datasets/mc4 -
ChemLLMBench
Datasets & benchmarksOfficial Code for What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks (In NeurIPS 2023)
git clone https://github.com/ChemFoundationModels/ChemLLMBench.git -
pile-of-law
Datasets & benchmarksHugging Face dataset by pile-of-law
pile-of-law/pile-of-law -
LawBench
Datasets & benchmarksBenchmarking Legal Knowledge of Large Language Models
git clone https://github.com/open-compass/LawBench.git -
OSCAR-2301
Datasets & benchmarksHugging Face dataset by oscar-corpus
oscar-corpus/OSCAR-2301 -
SHP
Datasets & benchmarksHugging Face dataset by stanfordnlp
stanfordnlp/SHP -
polymarket-paper-trader
Datasets & benchmarksPaper trading simulator for Polymarket
git clone https://github.com/agent-next/polymarket-paper-trader.git -
WebInstructSub
Datasets & benchmarksHugging Face dataset by TIGER-Lab
TIGER-Lab/WebInstructSub -
Matrix
Datasets & benchmarksHugging Face dataset by m-a-p
m-a-p/Matrix -
TACO
Datasets & benchmarksHugging Face dataset by BAAI
BAAI/TACO -
Omni-SafetyBench
Datasets & benchmarks[ACM MM 2026 Oral] Code for paper "Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models".
git clone https://github.com/THU-BPM/Omni-SafetyBench.git -
JBB-Behaviors
Datasets & benchmarksHugging Face dataset by JailbreakBench
JailbreakBench/JBB-Behaviors -
claude_mythos_distilled_25k
Datasets & benchmarksHugging Face dataset by WithinUsAI
WithinUsAI/claude_mythos_distilled_25k -
aurora
Datasets & benchmarks[ICLR 2025] AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark
git clone https://github.com/wenhaochai/aurora.git -
NextTransformer
Datasets & benchmarks:sunrise: A comprehensive, cutting-edge open-source framework for next-generation Transformers/LLM, with compute-matched benchmarking and a platform for autonomous architecture research.
git clone https://github.com/Interactive-Active-AI/NextTransformer.git -
confabulations
Datasets & benchmarksHallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.
git clone https://github.com/lechmazur/confabulations.git -
FastEval
Datasets & benchmarksFast & more realistic evaluation of chat language models. Includes leaderboard.
git clone https://github.com/FastEval/FastEval.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.