Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
xVerify
Datasets & benchmarksxVerify: Efficient Answer Verifier for Reasoning Model Evaluations
git clone https://github.com/IAAR-Shanghai/xVerify.git -
TrustLLM
Datasets & benchmarks[ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.
git clone https://github.com/HowieHwong/TrustLLM.git -
MegaMath
Datasets & benchmarksHugging Face dataset by IFM
IFM/MegaMath -
spider
Datasets & benchmarksHugging Face dataset by xlangai
xlangai/spider -
wikisql
Datasets & benchmarksHugging Face dataset by Salesforce
Salesforce/wikisql -
distilabel-capybara-dpo-7k-binarized
Datasets & benchmarksHugging Face dataset by argilla
argilla/distilabel-capybara-dpo-7k-binarized -
jupyter-agent-dataset
Datasets & benchmarksHugging Face dataset by jupyter-agent
jupyter-agent/jupyter-agent-dataset -
soda
Datasets & benchmarksHugging Face dataset by allenai
allenai/soda -
finephrase
Datasets & benchmarksHugging Face dataset by HuggingFaceFW
HuggingFaceFW/finephrase -
Simreal-MLBench
Datasets & benchmarks[Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluation.
git clone https://github.com/Simreal-AI/Simreal-MLBench.git -
twitter-financial-news-sentiment
Datasets & benchmarksHugging Face dataset by zeroshot
zeroshot/twitter-financial-news-sentiment -
spotify-tracks-dataset
Datasets & benchmarksHugging Face dataset by maharshipandya
maharshipandya/spotify-tracks-dataset -
LongPage
Datasets & benchmarksHugging Face dataset by Pageshift-Entertainment
Pageshift-Entertainment/LongPage -
apple-silicon-llm-bench
Datasets & benchmarksReproducible on-device LLM benchmarks for Apple Silicon (iPhone 17 Pro, M4 Max): Apple Core AI, MLX, llama.cpp, LiteRT-LM and Core ML on the same model and harness, every number with its quantization and capture session; hybrid Mamba-2 models (Nemotron-3 Nano, Granite-4.0-H, Falcon-H1) included.
git clone https://github.com/john-rocky/apple-silicon-llm-bench.git -
NoLiMa
Datasets & benchmarksOfficial repository for "NoLiMa: Long-Context Evaluation Beyond Literal Matching"
git clone https://github.com/adobe-research/NoLiMa.git -
CodeX-2M-Thinking
Datasets & benchmarksHugging Face dataset by Modotte
Modotte/CodeX-2M-Thinking -
OSCAR-2201
Datasets & benchmarksHugging Face dataset by oscar-corpus
oscar-corpus/OSCAR-2201 -
DecisionPivot
Datasets & benchmarksDecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.
git clone https://github.com/StartLuxLabs/DecisionPivot.git -
swe-sweep
Datasets & benchmarksHow many bugs can LMs find & fix in large codebases?
git clone https://github.com/facebookresearch/swe-sweep.git -
NextTransformer
Datasets & benchmarks:sunrise: A comprehensive, cutting-edge open-source framework for next-generation Transformers/LLM, with compute-matched benchmarking and a platform for autonomous architecture research.
git clone https://github.com/Interactive-Active-AI/NextTransformer.git -
tulu-3-sft-mixture
Datasets & benchmarksHugging Face dataset by allenai
allenai/tulu-3-sft-mixture -
MobileWorld
Datasets & benchmarksBenchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments (ACL 2026)
git clone https://github.com/Tongyi-MAI/MobileWorld.git -
alem-env
Datasets & benchmarksCan LLM/MARL agents coordinate in long-horizon, open-ended tasks? NeurIPS 2026 (Evaluations and Datasets).
git clone https://github.com/alem-world/alem-env.git -
autoguardrails
Datasets & benchmarksAlignment-research scaffold (autoresearch-style) for LLM guardrails: search over a single policy.md surface
git clone https://github.com/SantanderAI/autoguardrails.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.