Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
nyt-connections
Datasets & benchmarksBenchmark that evaluates LLMs using 759 NYT Connections puzzles extended with extra trick words
git clone https://github.com/lechmazur/nyt-connections.git -
xVerify
Datasets & benchmarksxVerify: Efficient Answer Verifier for Reasoning Model Evaluations
git clone https://github.com/IAAR-Shanghai/xVerify.git -
llm-bulls-and-cows-benchmark
Datasets & benchmarksA mini-framework for evaluating LLM performance on the Bulls and Cows number guessing game, supporting multiple LLM providers.
git clone https://github.com/stalkermustang/llm-bulls-and-cows-benchmark.git -
GPT-5.5-Gemini-3.1-Pro-Grok-4-Claude-Fable-5-Mythos-5-Qwen-3.7-Max-and-more-Distillation-Dataset
Datasets & benchmarksHugging Face dataset by Manusagents
Manusagents/GPT-5.5-Gemini-3.1-Pro-Grok-4-Claude-Fable-5-Mythos-5-Qwen-3.7-Max-and-more-Distillation-Dataset -
Nemotron-Math-Proofs-v1
Datasets & benchmarksHugging Face dataset by nvidia
nvidia/Nemotron-Math-Proofs-v1 -
FastEval
Datasets & benchmarksFast & more realistic evaluation of chat language models. Includes leaderboard.
git clone https://github.com/FastEval/FastEval.git -
TurtleBench
Datasets & benchmarksTurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles.
git clone https://github.com/mazzzystar/TurtleBench.git -
MedXpertQA
Datasets & benchmarks[ICML 2025] MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding
git clone https://github.com/TsinghuaC3I/MedXpertQA.git -
active_genie
Datasets & benchmarksThe Lodash for GenAI: Real Value + Consistent + Model-Agnostic
git clone https://github.com/Roriz/active_genie.git -
ChartVerse-SFT-1.8M
Datasets & benchmarksHugging Face dataset by opendatalab
opendatalab/ChartVerse-SFT-1.8M -
fde-gym
Datasets & benchmarksThe online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.
git clone https://github.com/deeplethe/fde-gym.git -
LongPage
Datasets & benchmarksHugging Face dataset by Pageshift-Entertainment
Pageshift-Entertainment/LongPage -
BizFinBench
Datasets & benchmarksA Business-Driven Real-World Financial Benchmark for Evaluating LLMs
git clone https://github.com/HiThink-Research/BizFinBench.git -
LLM-RGB
Datasets & benchmarksLLM Reasoning and Generation Benchmark. Evaluate LLMs in complex scenarios systematically.
git clone https://github.com/babelcloud/LLM-RGB.git -
OSWorld-V2
Datasets & benchmarksOSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks
git clone https://github.com/xlang-ai/OSWorld-V2.git -
SGI-Bench
Datasets & benchmarksProbing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
git clone https://github.com/InternScience/SGI-Bench.git -
ai-model-world
Datasets & benchmarksAI 大模型世界,把 556 个大模型拟人化成像素小人的可视化站点。进来就能看到此刻谁最聪明、谁最会写代码、谁最便宜、谁刚发布,往下是国内与国外分区的厂商广场、完整的发布时间线和多维排行榜。搜索认模型名、厂商和能力,输入「多模态」会直接列出全部多模态模型。数据取自 Epoch AI、models.dev、LiveBench 与 Hugging Face,每小时自动同步,所有文案由真实数据生成,不调用任何 LLM。Next.js 静态导出,零后端。
git clone https://github.com/liyupi/ai-model-world.git -
TwinFix
Datasets & benchmarksTwinFix — AI agent that diagnoses, reproduces and fixes bugs in a disposable twin environment. 在一次性孪生环境中自动诊断、复现并修复故障的 AI Agent。
git clone https://github.com/spacesky-cell/TwinFix.git -
DyCodeEval
Datasets & benchmarksOfficial repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”
git clone https://github.com/SeekingDream/DyCodeEval.git -
GDPevo
Datasets & benchmarksA Benchmark for Evaluating Agent Self-Evolution on Real Business Tasks
git clone https://github.com/Prism-Shadow/GDPevo.git -
aa-agentperf-local
Datasets & benchmarksBenchmark local LLM serving by replaying real agent trajectories
git clone https://github.com/ArtificialAnalysis/aa-agentperf-local.git -
PhysGym
Datasets & benchmarksA benchmark suite for evaluating LLM-based interactive scientific reasoning.
git clone https://github.com/principia-ai/PhysGym.git -
Simreal-MLBench
Datasets & benchmarks[Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluation.
git clone https://github.com/Simreal-AI/Simreal-MLBench.git -
ChinaTravel
Datasets & benchmarksChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning
git clone https://github.com/LAMDA-NeSy/ChinaTravel.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.