InnerVault

Library

85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. LightCompress

    Datasets & benchmarks

    [EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.

    Stale GitHub, ModelTC 751 ★Apache-2.0Updated 6 months ago

    git clone https://github.com/ModelTC/LightCompress.git
  2. WritingBench

    Datasets & benchmarks

    WritingBench: A Comprehensive Benchmark for Generative Writing

    Stale GitHub, X-PLUG 202 ★Apache-2.0Updated 11 months ago

    git clone https://github.com/X-PLUG/WritingBench.git
  3. LoopArena

    Datasets & benchmarks

    Benchmarking models as runtime Controllers for Loop Engineering

    Stale GitHub, AMAP-ML 151 ★Apache-2.0Updated 33 days ago

    git clone https://github.com/AMAP-ML/LoopArena.git
  4. self-bench

    Datasets & benchmarks

    Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

    Indexed GitHub, mupt-ai 34 ★MITUpdated yesterday

    git clone https://github.com/mupt-ai/self-bench.git
  5. xorcise

    Datasets & benchmarks

    Run cyber-AI agents against real missions in isolated environments. Record everything as OpenTelemetry evidence. Grade the evidence, not the claim.

    Stale GitHub, xorcise-ai 33 ★Custom licence (see repository)Updated 18 days ago

    git clone https://github.com/xorcise-ai/xorcise.git
  6. xVerify

    Datasets & benchmarks

    xVerify: Efficient Answer Verifier for Reasoning Model Evaluations

    Stale GitHub, IAAR-Shanghai 150 ★Custom licence (see repository)Updated 20 days ago

    git clone https://github.com/IAAR-Shanghai/xVerify.git
  7. Omni-SafetyBench

    Datasets & benchmarks

    [ACM MM 2026 Oral] Code for paper "Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models".

    Stale GitHub, THU-BPM 115 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/THU-BPM/Omni-SafetyBench.git
  8. TerminalWorld

    Datasets & benchmarks

    TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks (NeurIPS 2026 Evaluations and Datasets Track)

    Stale GitHub, terminalworld 54 ★Apache-2.0Updated 13 days ago

    git clone https://github.com/terminalworld/TerminalWorld.git
  9. apex-agents

    Datasets & benchmarks

    Hugging Face dataset by mercor

    Stale Hugging Face Hub, mercor 207 ♥cc-by-4.0Updated 5 months ago 1 note

    mercor/apex-agents
  10. DyCodeEval

    Datasets & benchmarks

    Official repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”

    Stale GitHub, SeekingDream 222 ★Updated 10 months ago

    git clone https://github.com/SeekingDream/DyCodeEval.git
  11. Awesome-Datasets-Hub

    Datasets & benchmarks

    A curated collection of datasets for Large Language Models (LLMs), covering medical AI, NLP, multimodal learning, instruction tuning, reasoning, code generation, and evaluation benchmarks.

    Stale GitHub, ahammadmejbah 149 ★Updated 5 months ago

    git clone https://github.com/ahammadmejbah/Awesome-Datasets-Hub.git
  12. AICGSecEval

    Datasets & benchmarks

    A.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team.

    Stale GitHub, Tencent 664 ★Custom licence (see repository)Updated 5 months ago

    git clone https://github.com/Tencent/AICGSecEval.git
  13. confabulations

    Datasets & benchmarks

    Hallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.

    Stale GitHub, lechmazur 249 ★Updated 15 months ago

    git clone https://github.com/lechmazur/confabulations.git
  14. TurtleBench

    Datasets & benchmarks

    TurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles.

    Stale GitHub, mazzzystar 161 ★Apache-2.0Updated 2 years ago

    git clone https://github.com/mazzzystar/TurtleBench.git
  15. fde-gym

    Datasets & benchmarks

    The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.

    Stale GitHub, deeplethe 111 ★Apache-2.0Updated 19 days ago

    git clone https://github.com/deeplethe/fde-gym.git
  16. seo-skill-bench

    Datasets & benchmarks

    Open benchmark for Claude Code SEO skills — real headless execution against fixture sites with planted-defect answer keys. Deterministic scoring, pre-registered rubric.

    Stale GitHub, aleclindz 51 ★MITUpdated 33 days ago

    git clone https://github.com/aleclindz/seo-skill-bench.git
  17. prism

    Datasets & benchmarks

    Открытый бенчмарк LLM: какая нейросеть лучше пишет код 1С:Предприятие (BSL). Объективная оценка LLM по методике SMOP с реальным исполнением в 1С — Claude, GPT, Gemini, DeepSeek, YandexGPT, GigaChat.

    Stale GitHub, genlab-1c 55 ★MITUpdated 19 days ago

    git clone https://github.com/genlab-1c/prism.git
  18. openbench

    Datasets & benchmarks

    Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost

    Stale GitHub, minghinmatthewlam 137 ★MITUpdated 2 months ago

    git clone https://github.com/minghinmatthewlam/openbench.git
  19. elimination_game

    Datasets & benchmarks

    A multi-player tournament benchmark that tests LLMs in social reasoning, strategy, and deception. Players engage in public and private conversations, form alliances, and vote to eliminate each other

    Stale GitHub, lechmazur 298 ★Updated 10 months ago

    git clone https://github.com/lechmazur/elimination_game.git
  20. Long-Horizon-Terminal-Bench

    Datasets & benchmarks

    Hugging Face dataset by IntelligenceLab

    Stale Hugging Face Hub, IntelligenceLab 140 ♥apache-2.0Updated 37 days ago

    IntelligenceLab/Long-Horizon-Terminal-Bench
  21. gamebot

    Datasets & benchmarks

    [ACL 2025] GAMEBoT: Transparent Assessment of LLM Reasoning in Games

    Stale GitHub, Visual-AI 36 ★Updated 11 days ago

    git clone https://github.com/Visual-AI/gamebot.git
  22. PawBench

    Datasets & benchmarks

    A benchmark for evaluating LLM × harness performance.

    Stale GitHub, agentscope-ai 118 ★Apache-2.0Updated 3 months ago

    git clone https://github.com/agentscope-ai/PawBench.git
  23. LLM-Agent-Benchmark-List

    Datasets & benchmarks

    A banchmark list for evaluation of large language models.

    Stale GitHub, zhangxjohn 173 ★Apache-2.0Updated 12 days ago

    git clone https://github.com/zhangxjohn/LLM-Agent-Benchmark-List.git
  24. LLM-RGB

    Datasets & benchmarks

    LLM Reasoning and Generation Benchmark. Evaluate LLMs in complex scenarios systematically.

    Stale GitHub, babelcloud 165 ★MITUpdated 18 months ago

    git clone https://github.com/babelcloud/LLM-RGB.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.