Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
Long-Horizon-Terminal-Bench
Datasets & benchmarksHugging Face dataset by IntelligenceLab
IntelligenceLab/Long-Horizon-Terminal-Bench -
llm-bulls-and-cows-benchmark
Datasets & benchmarksA mini-framework for evaluating LLM performance on the Bulls and Cows number guessing game, supporting multiple LLM providers.
git clone https://github.com/stalkermustang/llm-bulls-and-cows-benchmark.git -
LawBench
Datasets & benchmarksBenchmarking Legal Knowledge of Large Language Models
git clone https://github.com/open-compass/LawBench.git -
vidore-benchmark
AI repositoriesVision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.
git clone https://github.com/illuin-tech/vidore-benchmark.git -
ping_pong_bench
Datasets & benchmarksA benchmark for role-playing language models
git clone https://github.com/IlyaGusev/ping_pong_bench.git -
ollama-benchmark
Datasets & benchmarksOllama based Benchmark with detail I/O token per second. Python with Deepseek R1 example.
git clone https://github.com/LarHope/ollama-benchmark.git -
folkbench
AI repositories帮你的 AI 编程工具找到不降智、不断连、高性价比的好线路 · Live LLM relay rankings, model integrity probes & Tauri/Rust switcher for Claude Code, Codex & Gemini CLI
git clone https://github.com/Folkbench/folkbench.git -
LLM-Agent-Benchmark-List
Datasets & benchmarksA banchmark list for evaluation of large language models.
git clone https://github.com/zhangxjohn/LLM-Agent-Benchmark-List.git -
OSWorld
Datasets & benchmarks[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
git clone https://github.com/xlang-ai/OSWorld.git -
MMLU-Pro
Datasets & benchmarksHugging Face dataset by TIGER-Lab
TIGER-Lab/MMLU-Pro -
OpenLLM
AI repositoriesRun any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
git clone https://github.com/bentoml/OpenLLM.git -
sec-code-bench
Datasets & benchmarksSecCodeBench is a benchmark suite focusing on evaluating the security of code generated by large language models (LLMs).
git clone https://github.com/alibaba/sec-code-bench.git -
CL-bench
Datasets & benchmarksHugging Face dataset by tencent
tencent/CL-bench -
LongBench
Datasets & benchmarksLongBench v2 and LongBench (ACL 25'&24')
git clone https://github.com/THUDM/LongBench.git -
BizFinBench
Datasets & benchmarksA Business-Driven Real-World Financial Benchmark for Evaluating LLMs
git clone https://github.com/HiThink-Research/BizFinBench.git -
apex-agents
Datasets & benchmarksHugging Face dataset by mercor
mercor/apex-agents -
o11y-bench
Datasets & benchmarksOpen benchmark on observability tasks built on Harbor
git clone https://github.com/grafana/o11y-bench.git -
swt-bench
Datasets & benchmarks[NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation
git clone https://github.com/logic-star-ai/swt-bench.git -
kg-gen
Datasets & benchmarks[NeurIPS '25] Knowledge Graph Generation from Any Text
git clone https://github.com/stair-lab/kg-gen.git -
ADR
MCP serversADR secures enterprise AI agents through observability, security benchmarking, and threat detection. Deployed at Uber.
git clone https://github.com/uber/ADR.git -
nyt-connections
Datasets & benchmarksBenchmark that evaluates LLMs using 759 NYT Connections puzzles extended with extra trick words
git clone https://github.com/lechmazur/nyt-connections.git -
nautilus-compass
Datasets & benchmarksOpen memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
git clone https://github.com/chunxiaoxx/nautilus-compass.git -
chinese-llm-benchmark
Agents & frameworks非线智能 NoneLinear - ReLE评测:中文AI大模型能力评测(持续更新):目前已囊括374个大模型,覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型, 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜,也提供规模超200万的大模型缺陷库!方便广大社区研究分析、改进大模型。
git clone https://github.com/jeinlee1991/chinese-llm-benchmark.git -
gpqa
Datasets & benchmarksHugging Face dataset by Idavidrein
Idavidrein/gpqa
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.