InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. Simreal-MLBench

    Datasets & benchmarks

    [Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluation.

    Stale GitHub, Simreal-AI 301 ★Apache-2.0Updated 53 days ago

    git clone https://github.com/Simreal-AI/Simreal-MLBench.git
  2. MedXpertQA

    Datasets & benchmarks

    [ICML 2025] MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding

    Stale GitHub, TsinghuaC3I 182 ★MITUpdated 2 months ago

    git clone https://github.com/TsinghuaC3I/MedXpertQA.git
  3. GLM-5.3-Flash × J-Space capability realization — benchmark presentation of the J-Space Cognition Suite

    Stale GitHub, Tiger3807861189 1,019 ★Custom licence (see repository)Updated 2 months ago

    git clone https://github.com/Tiger3807861189/GLM-5.3-Flash-J-Space-Capability-Realization-Report.git
  4. deepmark

    Datasets & benchmarks

    Deepmark AI enables a unique testing environment for language models (LLM) assessment on task-specific metrics and on your own data so your GenAI-powered solution has predictable and reliable performance.

    Stale GitHub, IngestAI 104 ★AGPL-3.0Updated 2 years ago

    git clone https://github.com/IngestAI/deepmark.git
  5. beir

    AI repositories

    A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

    Stale GitHub, beir-cellar 2,310 ★Apache-2.0Updated 13 months ago

    git clone https://github.com/beir-cellar/beir.git
  6. explicit-edit-benchmark

    Datasets & benchmarks

    Benchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.

    Stale GitHub, alexshpunt 38 ★MITUpdated 7 days ago

    git clone https://github.com/alexshpunt/explicit-edit-benchmark.git
  7. AIRTBench-Code

    Datasets & benchmarks

    Code Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

    Stale GitHub, dreadnode 111 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/dreadnode/AIRTBench-Code.git
  8. hle

    Datasets & benchmarks

    Hugging Face dataset by cais

    Indexed Hugging Face Hub, cais 1,106 ♥mitUpdated 10 months ago 1 note

    cais/hle
  9. FDE-Gym

    Datasets & benchmarks

    Hugging Face dataset by DeepLethe

    Stale Hugging Face Hub, DeepLethe 437 ♥otherUpdated 31 days ago 1 note

    DeepLethe/FDE-Gym
  10. AgentLab

    Datasets & benchmarks

    AgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.

    Stale GitHub, ServiceNow 653 ★Custom licence (see repository)Updated 41 days ago

    git clone https://github.com/ServiceNow/AgentLab.git
  11. ping_pong_bench

    Datasets & benchmarks

    A benchmark for role-playing language models

    Stale GitHub, IlyaGusev 122 ★Apache-2.0Updated 18 months ago

    git clone https://github.com/IlyaGusev/ping_pong_bench.git
  12. SDGym

    AI repositories

    Benchmarking synthetic data generation methods.

    Indexed GitHub, sdv-dev 313 ★Custom licence (see repository)Updated today

    git clone https://github.com/sdv-dev/SDGym.git
  13. PawBench

    Datasets & benchmarks

    A benchmark for evaluating LLM × harness performance.

    Stale GitHub, agentscope-ai 118 ★Apache-2.0Updated 3 months ago

    git clone https://github.com/agentscope-ai/PawBench.git
  14. HallusionBench

    Datasets & benchmarks

    [CVPR'24] HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models

    Stale GitHub, tianyi-lab 344 ★BSD-3-ClauseUpdated 13 months ago

    git clone https://github.com/tianyi-lab/HallusionBench.git
  15. MMLU-CF

    Datasets & benchmarks

    A Contamination-free Multi-task Language Understanding Benchmark [Official, ACL 2025]

    Stale GitHub, microsoft 127 ★MITUpdated 18 months ago

    git clone https://github.com/microsoft/MMLU-CF.git
  16. ArtifactsBenchmark

    Datasets & benchmarks

    Stale GitHub, Tencent-Hunyuan 280 ★Custom licence (see repository)Updated 10 months ago

    git clone https://github.com/Tencent-Hunyuan/ArtifactsBenchmark.git
  17. LoopArena

    Datasets & benchmarks

    Benchmarking models as runtime Controllers for Loop Engineering

    Stale GitHub, AMAP-ML 151 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/AMAP-ML/LoopArena.git
  18. LLM-RGB

    Datasets & benchmarks

    LLM Reasoning and Generation Benchmark. Evaluate LLMs in complex scenarios systematically.

    Stale GitHub, babelcloud 165 ★MITUpdated 18 months ago

    git clone https://github.com/babelcloud/LLM-RGB.git
  19. typed-decisions

    Datasets & benchmarks

    Hugging Face dataset by LocalLLaMA

    Indexed Hugging Face Hub, LocalLLaMA 277 ♥apache-2.0Updated today

    LocalLLaMA/typed-decisions
  20. WritingBench

    Datasets & benchmarks

    WritingBench: A Comprehensive Benchmark for Generative Writing

    Stale GitHub, X-PLUG 202 ★Apache-2.0Updated 11 months ago

    git clone https://github.com/X-PLUG/WritingBench.git
  21. docext

    AI repositories

    An on-premises, OCR-free unstructured data extraction, markdown conversion and benchmarking toolkit. (https://idp-leaderboard.org/)

    Stale GitHub, NanoNets 2,083 ★Apache-2.0Updated 8 months ago

    git clone https://github.com/NanoNets/docext.git
  22. geobench

    Datasets & benchmarks

    GeoGuessr benchmark for language models

    Indexed GitHub, ccmdi 65 ★MITUpdated 3 days ago

    git clone https://github.com/ccmdi/geobench.git
  23. benchmarks

    Agents & frameworks

    Reproducible voice-AI benchmarking — TTS / STT / S2S latency and accuracy.

    Stale GitHub, coval-ai 104 ★Custom licence (see repository)Updated 24 days ago

    git clone https://github.com/coval-ai/benchmarks.git
  24. ai-agents-the-definitive-guide

    Agents & frameworks

    Repo for AI Agents The Definitive Guide

    Stale GitHub, Nicolepcx 2,747 ★Updated 3 months ago

    git clone https://github.com/Nicolepcx/ai-agents-the-definitive-guide.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.