Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
tau2-bench
Datasets & benchmarksτ-Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
git clone https://github.com/sierra-research/tau2-bench.git -
olmOCR-bench
Datasets & benchmarksHugging Face dataset by allenai
allenai/olmOCR-bench -
BentoML
AI repositoriesThe easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
git clone https://github.com/bentoml/BentoML.git -
TheAgentCompany
Datasets & benchmarksAn agent benchmark with tasks in a simulated software company.
git clone https://github.com/TheAgentCompany/TheAgentCompany.git -
ollama-benchmark
Datasets & benchmarksLLM Benchmark for Throughput via Ollama (Local LLMs)
git clone https://github.com/aidatatools/ollama-benchmark.git -
SWE-bench_Verified
Datasets & benchmarksHugging Face dataset by SWE-bench
SWE-bench/SWE-bench_Verified -
AI_Diplomacy
Datasets & benchmarksFrontier Models playing the board game Diplomacy.
git clone https://github.com/GoodStartLabs/AI_Diplomacy.git -
BenchMARL
Agents & frameworksBenchMARL is a library for benchmarking Multi-Agent Reinforcement Learning (MARL). BenchMARL allows to quickly compare different MARL algorithms, tasks, and models while being systematically grounded in its two core tenets: reproducibility and standardization.
git clone https://github.com/facebookresearch/BenchMARL.git -
SWE-bench-Live
Datasets & benchmarks[NeurIPS 2025 D&B] 🚀 SWE-bench Goes Live!
git clone https://github.com/microsoft/SWE-bench-Live.git -
swe-sweep
Datasets & benchmarksHow many bugs can LMs find & fix in large codebases?
git clone https://github.com/facebookresearch/swe-sweep.git -
SWE-bench_Pro
Datasets & benchmarksHugging Face dataset by ScaleAI
ScaleAI/SWE-bench_Pro -
ClawBench
Datasets & benchmarksOpen-source benchmark for browser AI agents on daily tasks.
git clone https://github.com/TIGER-AI-Lab/ClawBench.git -
gsm8k
Datasets & benchmarksHugging Face dataset by openai
openai/gsm8k -
WebsiteBench
Datasets & benchmarksCan AI agents rebuild websites through browser exploration?
git clone https://github.com/TIGER-AI-Lab/WebsiteBench.git -
agents-last-exam
Datasets & benchmarksHugging Face dataset by agents-last-exam
agents-last-exam/agents-last-exam -
agent-memory-benchmark
Agents & frameworksAgent Memory Benchmark
git clone https://github.com/vectorize-io/agent-memory-benchmark.git -
leaderboard
Tools & appsHugging Face space by gaia-benchmark
gaia-benchmark/leaderboard -
SWE-bench_Verified
Datasets & benchmarksHugging Face dataset by princeton-nlp
princeton-nlp/SWE-bench_Verified -
mempalace
AI repositoriesThe best-benchmarked open-source AI memory system. And it's free.
git clone https://github.com/MemPalace/mempalace.git -
SciCode
Datasets & benchmarksA benchmark that challenges language models to code solutions for scientific problems
git clone https://github.com/scicode-bench/SciCode.git -
EnterpriseRAG-Bench
AI repositoriesDataset and benchmark for RAG on company internal documents.
git clone https://github.com/onyx-dot-app/EnterpriseRAG-Bench.git -
llm-colosseum
Datasets & benchmarksBenchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM
git clone https://github.com/OpenGenerativeAI/llm-colosseum.git -
benchmark-qed
AI repositoriesAutomated benchmarking of Retrieval-Augmented Generation (RAG) systems
git clone https://github.com/microsoft/benchmark-qed.git -
minebench
Datasets & benchmarksMinecraft-style voxel benchmark for comparing AI models (Arena + Sandbox)
git clone https://github.com/Ammaar-Alam/minebench.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.