Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
mt_bench_human_judgments
Datasets & benchmarksHugging Face dataset by lmsys
lmsys/mt_bench_human_judgments -
nagi-bench
Datasets & benchmarksOne-shot LLM eval cases by NAGI STUDIO - same prompt, different agents (model + harness), runnable artifacts side by side.
git clone https://github.com/nagi-studio/nagi-bench.git -
AIRTBench-Code
Datasets & benchmarksCode Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models
git clone https://github.com/dreadnode/AIRTBench-Code.git -
SGI-Bench
Datasets & benchmarksProbing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
git clone https://github.com/InternScience/SGI-Bench.git -
role-play-bench
Datasets & benchmarksHugging Face dataset by MiniMaxAI
MiniMaxAI/role-play-bench -
explicit-edit-benchmark
Datasets & benchmarksBenchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.
git clone https://github.com/alexshpunt/explicit-edit-benchmark.git -
HallusionBench
Datasets & benchmarks[CVPR'24] HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models
git clone https://github.com/tianyi-lab/HallusionBench.git -
SDGym
AI repositoriesBenchmarking synthetic data generation methods.
git clone https://github.com/sdv-dev/SDGym.git -
typed-decisions
Datasets & benchmarksHugging Face dataset by LocalLLaMA
LocalLLaMA/typed-decisions -
DeepResearch-Bench-II
Datasets & benchmarksDeepResearch Bench II (DRB2) is the follow-up to DeepResearch Bench, with a stronger focus on measuring the gap between deep research systems and human experts. It does so by decomposing expert-written reports into hierarchical rubrics covering presentation, analysis, and evidence, and using them to evaluate model-generated.
git clone https://github.com/imlrz/DeepResearch-Bench-II.git -
ArtifactsBenchmark
Datasets & benchmarksgit clone https://github.com/Tencent-Hunyuan/ArtifactsBenchmark.git -
PawBench
Datasets & benchmarksA benchmark for evaluating LLM × harness performance.
git clone https://github.com/agentscope-ai/PawBench.git -
docext
AI repositoriesAn on-premises, OCR-free unstructured data extraction, markdown conversion and benchmarking toolkit. (https://idp-leaderboard.org/)
git clone https://github.com/NanoNets/docext.git -
MMLU-CF
Datasets & benchmarksA Contamination-free Multi-task Language Understanding Benchmark [Official, ACL 2025]
git clone https://github.com/microsoft/MMLU-CF.git -
LoopArena
Datasets & benchmarksBenchmarking models as runtime Controllers for Loop Engineering
git clone https://github.com/AMAP-ML/LoopArena.git -
LLM-RGB
Datasets & benchmarksLLM Reasoning and Generation Benchmark. Evaluate LLMs in complex scenarios systematically.
git clone https://github.com/babelcloud/LLM-RGB.git -
ai-agents-the-definitive-guide
Agents & frameworksRepo for AI Agents The Definitive Guide
git clone https://github.com/Nicolepcx/ai-agents-the-definitive-guide.git -
toon
AI repositories🎒 Token-Oriented Object Notation (TOON) – compact, human-readable serialization of JSON data for LLM prompts. TypeScript SDK, CLI, benchmarks.
git clone https://github.com/toon-format/toon.git -
WritingBench
Datasets & benchmarksWritingBench: A Comprehensive Benchmark for Generative Writing
git clone https://github.com/X-PLUG/WritingBench.git -
self-bench
Datasets & benchmarksBenchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.
git clone https://github.com/mupt-ai/self-bench.git -
aya-101
Modelsmodel by CohereLabs · transformers
CohereLabs/aya-101 -
Gym
AI repositoriesEvaluate and improve models and agents using environments
git clone https://github.com/NVIDIA-NeMo/Gym.git -
awesome-web-agents
Docs & tutorials🔥 A list of tools, frameworks, and resources for building AI web agents
git clone https://github.com/steel-dev/awesome-web-agents.git -
benchmarks
Agents & frameworksReproducible voice-AI benchmarking — TTS / STT / S2S latency and accuracy.
git clone https://github.com/coval-ai/benchmarks.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.