Library
85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.
Results
-
WebInstructSub
Datasets & benchmarksHugging Face dataset by TIGER-Lab
TIGER-Lab/WebInstructSub -
Omni-SafetyBench
Datasets & benchmarks[ACM MM 2026 Oral] Code for paper "Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models".
git clone https://github.com/THU-BPM/Omni-SafetyBench.git -
JBB-Behaviors
Datasets & benchmarksHugging Face dataset by JailbreakBench
JailbreakBench/JBB-Behaviors -
claude_mythos_distilled_25k
Datasets & benchmarksHugging Face dataset by WithinUsAI
WithinUsAI/claude_mythos_distilled_25k -
confabulations
Datasets & benchmarksHallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.
git clone https://github.com/lechmazur/confabulations.git -
FastEval
Datasets & benchmarksFast & more realistic evaluation of chat language models. Includes leaderboard.
git clone https://github.com/FastEval/FastEval.git -
swt-bench
Datasets & benchmarks[NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation
git clone https://github.com/logic-star-ai/swt-bench.git -
lasagna
Datasets & benchmarksSwap one RAG pipeline stage, hold the rest fixed, and measure whether the gain survives composition. The framework behind "Retrieval Augmented Generation Compositionality": 450 configurations, eight benchmarks, corpora from 5K to 200K documents.
git clone https://github.com/dolev31/lasagna.git -
minebench
Datasets & benchmarksMinecraft-style voxel benchmark for comparing AI models (Arena + Sandbox)
git clone https://github.com/Ammaar-Alam/minebench.git -
TurtleBench
Datasets & benchmarksTurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles.
git clone https://github.com/mazzzystar/TurtleBench.git -
SciCode
Datasets & benchmarksA benchmark that challenges language models to code solutions for scientific problems
git clone https://github.com/scicode-bench/SciCode.git -
active_genie
Datasets & benchmarksThe Lodash for GenAI: Real Value + Consistent + Model-Agnostic
git clone https://github.com/Roriz/active_genie.git -
pii-masking-200k
Datasets & benchmarksHugging Face dataset by ai4privacy
ai4privacy/pii-masking-200k -
UHGEval
Datasets & benchmarks[ACL 2024] User-friendly evaluation framework: Eval Suite & Benchmarks: UHGEval, HaluEval, HalluQA, etc.
git clone https://github.com/IAAR-Shanghai/UHGEval.git -
meta-agents-research-environments
Datasets & benchmarksMeta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.
git clone https://github.com/facebookresearch/meta-agents-research-environments.git -
LoopArena
Datasets & benchmarksBenchmarking models as runtime Controllers for Loop Engineering
git clone https://github.com/AMAP-ML/LoopArena.git -
sloptotal
Datasets & benchmarksOpen-source AI text detector and ChatGPT detector: 23 engines, one calibrated score, every number measured on a public benchmark. Self-hosted, runs on CPU.
git clone https://github.com/pablocaeg/sloptotal.git -
PyHealth
Datasets & benchmarksA deep learning toolkit for healthcare AI: EHR (MIMIC, eICU, OMOP), clinical text, signals and imaging. 30+ models, reproducible pipelines.
git clone https://github.com/sunlabuiuc/PyHealth.git -
geobench
Datasets & benchmarksGeoGuessr benchmark for language models
git clone https://github.com/ccmdi/geobench.git -
debate
Datasets & benchmarksAdversarial multi-turn benchmark for LLM debate quality, using side-swapped matchups and multi-model judging to rank models by judged debate performance.
git clone https://github.com/lechmazur/debate.git -
ai-hardware-fit
Datasets & benchmarksCan my GPU run this? Pick a GPU to get 3 fitting AI models, quantization, VRAM, speed ranges, and run commands. 한국어·English.
git clone https://github.com/jaeseok614/ai-hardware-fit.git -
Ditto
Datasets & benchmarksA self-ailgnment method for role-play. Benchmark for role-play. Resources for "Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment".
git clone https://github.com/OFA-Sys/Ditto.git -
humanities_data_benchmark
Datasets & benchmarksLLM Benchmark Suite for Humanities Data
git clone https://github.com/RISE-UNIBAS/humanities_data_benchmark.git -
Awesome-AI-Scientist
Datasets & benchmarks🧪 Awesome list of AI Scientist papers, systems, benchmarks, datasets and open-source platforms.
git clone https://github.com/Omni-Scientist/Awesome-AI-Scientist.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.