Library
85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
UltraData-SFT-Agent-2609
Datasets & benchmarksHugging Face dataset by openbmb
openbmb/UltraData-SFT-Agent-2609 -
UltraData-Code
Datasets & benchmarksHugging Face dataset by openbmb
openbmb/UltraData-Code -
PawBench
Datasets & benchmarksA benchmark for evaluating LLM × harness performance.
git clone https://github.com/agentscope-ai/PawBench.git -
AgentTrove
Datasets & benchmarksHugging Face dataset by open-thoughts
open-thoughts/AgentTrove -
squad_v2
Datasets & benchmarksHugging Face dataset by rajpurkar
rajpurkar/squad_v2 -
Ultra-FineWeb-L1
Datasets & benchmarksHugging Face dataset by openbmb
openbmb/Ultra-FineWeb-L1 -
Big-Math-RL-Verified
Datasets & benchmarksHugging Face dataset by SynthLabsAI
SynthLabsAI/Big-Math-RL-Verified -
awesome-ai-gateway
Datasets & benchmarks⚡ Awesome AI Gateway — pick an AI gateway from 240+ (LiteLLM, OpenRouter, Portkey, Kong, Higress, new-api, Bifrost) by cost, security & compliance, see what consolidated in 2026, learn how they actually work from an 8-chapter handbook, and verify every number yourself. Reproducible benchmarks, bilingual; stars & releases refreshed daily.
git clone https://github.com/cuihuan/awesome-ai-gateway.git -
llm-bulls-and-cows-benchmark
Datasets & benchmarksA mini-framework for evaluating LLM performance on the Bulls and Cows number guessing game, supporting multiple LLM providers.
git clone https://github.com/stalkermustang/llm-bulls-and-cows-benchmark.git -
Awesome-LLM-Eval
Datasets & benchmarksAwesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on LLMs. 一个由工具、基准/数据、演示、排行榜和大模型等组成的精选列表,主要面向基础大模型评测,旨在探求生成式AI的技术边界.
git clone https://github.com/onejune2018/Awesome-LLM-Eval.git -
RLAIF-V-Dataset
Datasets & benchmarksHugging Face dataset by openbmb
openbmb/RLAIF-V-Dataset -
llm-colosseum
Datasets & benchmarksBenchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM
git clone https://github.com/OpenGenerativeAI/llm-colosseum.git -
empathetic_dialogues
Datasets & benchmarksHugging Face dataset by facebook
facebook/empathetic_dialogues -
SlimOrca
Datasets & benchmarksHugging Face dataset by Open-Orca
Open-Orca/SlimOrca -
librispeech_asr
Datasets & benchmarksHugging Face dataset by openslr
openslr/librispeech_asr -
LawBench
Datasets & benchmarksBenchmarking Legal Knowledge of Large Language Models
git clone https://github.com/open-compass/LawBench.git -
TrustLLM
Datasets & benchmarks[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.
git clone https://github.com/HowieHwong/TrustLLM.git -
commonsense_qa
Datasets & benchmarksHugging Face dataset by tau
tau/commonsense_qa -
truthful_qa
Datasets & benchmarksHugging Face dataset by truthfulqa
truthfulqa/truthful_qa -
product-database
Datasets & benchmarksHugging Face dataset by openfoodfacts
openfoodfacts/product-database -
AgentLab
Datasets & benchmarksAgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.
git clone https://github.com/ServiceNow/AgentLab.git -
NIMStats
Datasets & benchmarks📊 Automated hourly benchmarks for 20+ NVIDIA NIM models — interactive dashboard, zero infra, self-hostable. Open-source & community-driven.
git clone https://github.com/MauroDruwel/NIMStats.git -
flores_plus
Datasets & benchmarksHugging Face dataset by openlanguagedata
openlanguagedata/flores_plus -
nautilus-compass
Datasets & benchmarksOpen memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
git clone https://github.com/chunxiaoxx/nautilus-compass.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.