Library
85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
LightCompress
Datasets & benchmarks[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.
git clone https://github.com/ModelTC/LightCompress.git -
nautilus-compass
Datasets & benchmarksOpen memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
git clone https://github.com/chunxiaoxx/nautilus-compass.git -
DyCodeEval
Datasets & benchmarksOfficial repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”
git clone https://github.com/SeekingDream/DyCodeEval.git -
gamebot
Datasets & benchmarks[ACL 2025] GAMEBoT: Transparent Assessment of LLM Reasoning in Games
git clone https://github.com/Visual-AI/gamebot.git -
prism
Datasets & benchmarksОткрытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением в 1С — Claude, GPT, Gemini, DeepSeek, YandexGPT, GigaChat.
git clone https://github.com/genlab-1c/prism.git -
TurtleBench
Datasets & benchmarksTurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles.
git clone https://github.com/mazzzystar/TurtleBench.git -
confabulations
Datasets & benchmarksHallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.
git clone https://github.com/lechmazur/confabulations.git -
apex-agents
Datasets & benchmarksHugging Face dataset by mercor
mercor/apex-agents -
Awesome-Scientific-LLM-Benchmarks
Datasets & benchmarksA curated, accuracy-first list of benchmarks for evaluating LLMs on scientific reasoning and discovery — math, physics, chemistry, materials, biology, and agentic science.
git clone https://github.com/subinium/Awesome-Scientific-LLM-Benchmarks.git -
Multi-User-LLM-Agent
Datasets & benchmarksOfficial code for the paper: "Multi-User Large Language Model Agents"
git clone https://github.com/Kordi-AI/Multi-User-LLM-Agent.git -
openbench
Datasets & benchmarksSame model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost
git clone https://github.com/minghinmatthewlam/openbench.git -
PawBench
Datasets & benchmarksA benchmark for evaluating LLM × harness performance.
git clone https://github.com/agentscope-ai/PawBench.git -
AICGSecEval
Datasets & benchmarksA.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team.
git clone https://github.com/Tencent/AICGSecEval.git -
lasagna
Datasets & benchmarksSwap one RAG pipeline stage, hold the rest fixed, and measure whether the gain survives composition. The framework behind "Retrieval Augmented Generation Compositionality": 450 configurations, eight benchmarks, corpora from 5K to 200K documents.
git clone https://github.com/dolev31/lasagna.git -
LLM-Agent-Benchmark-List
Datasets & benchmarksA banchmark list for evaluation of large language models.
git clone https://github.com/zhangxjohn/LLM-Agent-Benchmark-List.git -
LLM-RGB
Datasets & benchmarksLLM Reasoning and Generation Benchmark. Evaluate LLMs in complex scenarios systematically.
git clone https://github.com/babelcloud/LLM-RGB.git -
autoguardrails
Datasets & benchmarksAlignment-research scaffold (autoresearch-style) for LLM guardrails: search over a single policy.md surface
git clone https://github.com/SantanderAI/autoguardrails.git -
elimination_game
Datasets & benchmarksA multi-player tournament benchmark that tests LLMs in social reasoning, strategy, and deception. Players engage in public and private conversations, form alliances, and vote to eliminate each other
git clone https://github.com/lechmazur/elimination_game.git -
Long-Horizon-Terminal-Bench
Datasets & benchmarksHugging Face dataset by IntelligenceLab
IntelligenceLab/Long-Horizon-Terminal-Bench -
SGI-Bench
Datasets & benchmarksProbing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows
git clone https://github.com/InternScience/SGI-Bench.git -
PhysGym
Datasets & benchmarksA benchmark suite for evaluating LLM-based interactive scientific reasoning.
git clone https://github.com/principia-ai/PhysGym.git -
GDPevo
Datasets & benchmarksA Benchmark for Evaluating Agent Self-Evolution on Real Business Tasks
git clone https://github.com/Prism-Shadow/GDPevo.git -
humanities_data_benchmark
Datasets & benchmarksLLM Benchmark Suite for Humanities Data
git clone https://github.com/RISE-UNIBAS/humanities_data_benchmark.git -
sec-code-bench
Datasets & benchmarksSecCodeBench is a benchmark suite focusing on evaluating the security of code generated by large language models (LLMs).
git clone https://github.com/alibaba/sec-code-bench.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.