Library
85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.
Results
-
ReForm-Eval
Datasets & benchmarksAn benchmark for evaluating the capabilities of large vision-language models (LVLMs)
git clone https://github.com/FudanDISC/ReForm-Eval.git -
meta-agents-research-environments
Datasets & benchmarksMeta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.
git clone https://github.com/facebookresearch/meta-agents-research-environments.git -
One-Eval
Datasets & benchmarksAutomated system for LLM evaluation via agents. Doc as below:
git clone https://github.com/OpenDCAI/One-Eval.git -
VibeSearchBench
Datasets & benchmarks🔍 The hardest search benchmark in the wild — vague, multi-turn, proactive. 200 long-horizon tasks with persona-driven progressive disclosure, scored by verifiable schema-free knowledge-graph evaluation. No vibes, just triplet F1.
git clone https://github.com/VibeBench/VibeSearchBench.git -
DeepPlanning
Datasets & benchmarksHugging Face dataset by Qwen
Qwen/DeepPlanning -
polymarket-paper-trader
Datasets & benchmarksPaper trading simulator for Polymarket
git clone https://github.com/agent-next/polymarket-paper-trader.git -
strix-halo-guide
Datasets & benchmarksEvidence-backed AMD Strix Halo local-AI setup and benchmarks: Qwen3.8, Ollama, llama.cpp, Vulkan/ROCm, large GGUFs, and cross-OEM results.
git clone https://github.com/hogeheer499-commits/strix-halo-guide.git -
civ6-mcp
Datasets & benchmarksAn MCP server that lets LLM agents play Civilization VI.
git clone https://github.com/lmwilki/civ6-mcp.git -
llmspeed
Datasets & benchmarksFeel how fast a local LLM runs before you buy the hardware
git clone https://github.com/nicorobinreal/llmspeed.git -
debate
Datasets & benchmarksAdversarial multi-turn benchmark for LLM debate quality, using side-swapped matchups and multi-model judging to rank models by judged debate performance.
git clone https://github.com/lechmazur/debate.git -
arena-ai-leaderboards
Datasets & benchmarks📊 Daily auto-updated snapshots of all Arena AI (LMSYS Chatbot Arena) leaderboards — LLM, Vision, Code, Video, Image & more. Structured JSON with historical tracking.
git clone https://github.com/oolong-tea-2026/arena-ai-leaderboards.git -
BusterX
Datasets & benchmarks[NeurIPS '26] BusterX & [EACL '27 Findings] BusterX++
git clone https://github.com/l8cv/BusterX.git -
tmmluplus
Datasets & benchmarksHugging Face dataset by ikala
ikala/tmmluplus -
VIBE
Datasets & benchmarksHugging Face dataset by MiniMaxAI
MiniMaxAI/VIBE -
lora-speedrun
Datasets & benchmarksSpeedrunning LoRA fine-tuning: frozen task, frozen hardware, public wall-clock leaderboard. modded-nanogpt for fine-tuning.
git clone https://github.com/Saivineeth147/lora-speedrun.git -
LightCompress
Datasets & benchmarks[EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.
git clone https://github.com/ModelTC/LightCompress.git -
gamebot
Datasets & benchmarks[ACL 2025] GAMEBoT: Transparent Assessment of LLM Reasoning in Games
git clone https://github.com/Visual-AI/gamebot.git -
legalbench
Datasets & benchmarksHugging Face dataset by nguha
nguha/legalbench -
prism
Datasets & benchmarksОткрытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением в 1С — Claude, GPT, Gemini, DeepSeek, YandexGPT, GigaChat.
git clone https://github.com/genlab-1c/prism.git -
PyHealth
Datasets & benchmarksA deep learning toolkit for healthcare AI: EHR (MIMIC, eICU, OMOP), clinical text, signals and imaging. 30+ models, reproducible pipelines.
git clone https://github.com/sunlabuiuc/PyHealth.git -
Glyd
Datasets & benchmarksLossless AI compression: 33% less GPU memory, bit for bit.
git clone https://github.com/surya-koritala/Glyd.git -
deepsearchqa
Datasets & benchmarksHugging Face dataset by google
google/deepsearchqa -
sacp
Datasets & benchmarksState-Aware Collaboration Protocol: auditable work receipts for AI agents.
git clone https://github.com/aDragon0707/sacp.git -
ds4-on-spark
Datasets & benchmarksEntrpi/ds4, a Blackwell CUDA perf fork of antirez/ds4 on NVIDIA DGX Spark: one-command install, ~3x upstream prefill, ~1.5x decode, DSpark, and full continuous batch support
git clone https://github.com/Entrpi/ds4-on-spark.git
Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.