InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. mt_bench_human_judgments

    Datasets & benchmarks

    Hugging Face dataset by lmsys

    Stale Hugging Face Hub, lmsys 149 ♥cc-by-4.0Updated 3 years ago

    lmsys/mt_bench_human_judgments
  2. nagi-bench

    Datasets & benchmarks

    One-shot LLM eval cases by NAGI STUDIO - same prompt, different agents (model + harness), runnable artifacts side by side.

    Indexed GitHub, nagi-studio 78 ★MITUpdated 4 days ago

    git clone https://github.com/nagi-studio/nagi-bench.git
  3. AIRTBench-Code

    Datasets & benchmarks

    Code Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

    Stale GitHub, dreadnode 111 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/dreadnode/AIRTBench-Code.git
  4. SGI-Bench

    Datasets & benchmarks

    Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows

    Stale GitHub, InternScience 172 ★MITUpdated 5 months ago

    git clone https://github.com/InternScience/SGI-Bench.git
  5. role-play-bench

    Datasets & benchmarks

    Hugging Face dataset by MiniMaxAI

    Stale Hugging Face Hub, MiniMaxAI 155 ♥apache-2.0Updated 9 months ago

    MiniMaxAI/role-play-bench
  6. explicit-edit-benchmark

    Datasets & benchmarks

    Benchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.

    Stale GitHub, alexshpunt 38 ★MITUpdated 7 days ago

    git clone https://github.com/alexshpunt/explicit-edit-benchmark.git
  7. HallusionBench

    Datasets & benchmarks

    [CVPR'24] HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models

    Stale GitHub, tianyi-lab 344 ★BSD-3-ClauseUpdated 13 months ago

    git clone https://github.com/tianyi-lab/HallusionBench.git
  8. SDGym

    AI repositories

    Benchmarking synthetic data generation methods.

    Indexed GitHub, sdv-dev 313 ★Custom licence (see repository)Updated today

    git clone https://github.com/sdv-dev/SDGym.git
  9. typed-decisions

    Datasets & benchmarks

    Hugging Face dataset by LocalLLaMA

    Indexed Hugging Face Hub, LocalLLaMA 277 ♥apache-2.0Updated today

    LocalLLaMA/typed-decisions
  10. DeepResearch-Bench-II

    Datasets & benchmarks

    DeepResearch Bench II (DRB2) is the follow-up to DeepResearch Bench, with a stronger focus on measuring the gap between deep research systems and human experts. It does so by decomposing expert-written reports into hierarchical rubrics covering presentation, analysis, and evidence, and using them to evaluate model-generated.

    Stale GitHub, imlrz 96 ★Apache-2.0Updated 28 days ago

    git clone https://github.com/imlrz/DeepResearch-Bench-II.git
  11. ArtifactsBenchmark

    Datasets & benchmarks

    Stale GitHub, Tencent-Hunyuan 280 ★Custom licence (see repository)Updated 10 months ago

    git clone https://github.com/Tencent-Hunyuan/ArtifactsBenchmark.git
  12. PawBench

    Datasets & benchmarks

    A benchmark for evaluating LLM × harness performance.

    Stale GitHub, agentscope-ai 118 ★Apache-2.0Updated 3 months ago

    git clone https://github.com/agentscope-ai/PawBench.git
  13. docext

    AI repositories

    An on-premises, OCR-free unstructured data extraction, markdown conversion and benchmarking toolkit. (https://idp-leaderboard.org/)

    Stale GitHub, NanoNets 2,083 ★Apache-2.0Updated 8 months ago

    git clone https://github.com/NanoNets/docext.git
  14. MMLU-CF

    Datasets & benchmarks

    A Contamination-free Multi-task Language Understanding Benchmark [Official, ACL 2025]

    Stale GitHub, microsoft 127 ★MITUpdated 18 months ago

    git clone https://github.com/microsoft/MMLU-CF.git
  15. LoopArena

    Datasets & benchmarks

    Benchmarking models as runtime Controllers for Loop Engineering

    Stale GitHub, AMAP-ML 151 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/AMAP-ML/LoopArena.git
  16. LLM-RGB

    Datasets & benchmarks

    LLM Reasoning and Generation Benchmark. Evaluate LLMs in complex scenarios systematically.

    Stale GitHub, babelcloud 165 ★MITUpdated 18 months ago

    git clone https://github.com/babelcloud/LLM-RGB.git
  17. ai-agents-the-definitive-guide

    Agents & frameworks

    Repo for AI Agents The Definitive Guide

    Stale GitHub, Nicolepcx 2,747 ★Updated 3 months ago

    git clone https://github.com/Nicolepcx/ai-agents-the-definitive-guide.git
  18. toon

    AI repositories

    🎒 Token-Oriented Object Notation (TOON) – compact, human-readable serialization of JSON data for LLM prompts. TypeScript SDK, CLI, benchmarks.

    Stale GitHub, toon-format 25.5k ★MITUpdated 38 days ago

    git clone https://github.com/toon-format/toon.git
  19. WritingBench

    Datasets & benchmarks

    WritingBench: A Comprehensive Benchmark for Generative Writing

    Stale GitHub, X-PLUG 202 ★Apache-2.0Updated 11 months ago

    git clone https://github.com/X-PLUG/WritingBench.git
  20. self-bench

    Datasets & benchmarks

    Benchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.

    Indexed GitHub, mupt-ai 34 ★MITUpdated today

    git clone https://github.com/mupt-ai/self-bench.git
  21. aya-101

    Models

    model by CohereLabs · transformers

    Stale Hugging Face Hub, CohereLabs 665 ♥apache-2.0Updated 14 months ago

    CohereLabs/aya-101
  22. Gym

    AI repositories

    Evaluate and improve models and agents using environments

    Indexed GitHub, NVIDIA-NeMo 1,334 ★Apache-2.0Updated today

    git clone https://github.com/NVIDIA-NeMo/Gym.git
  23. awesome-web-agents

    Docs & tutorials

    🔥 A list of tools, frameworks, and resources for building AI web agents

    Indexed GitHub, steel-dev 1,609 ★Custom licence (see repository)Updated 6 days ago

    git clone https://github.com/steel-dev/awesome-web-agents.git
  24. benchmarks

    Agents & frameworks

    Reproducible voice-AI benchmarking — TTS / STT / S2S latency and accuracy.

    Stale GitHub, coval-ai 104 ★Custom licence (see repository)Updated 24 days ago

    git clone https://github.com/coval-ai/benchmarks.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.