InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

  1. swt-bench

    Datasets & benchmarks

    [NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation

    Stale GitHub, logic-star-ai 92 ★MITUpdated 12 days ago

    git clone https://github.com/logic-star-ai/swt-bench.git
  2. WritingBench

    Datasets & benchmarks

    WritingBench: A Comprehensive Benchmark for Generative Writing

    Stale GitHub, X-PLUG 202 ★Apache-2.0Updated 11 months ago

    git clone https://github.com/X-PLUG/WritingBench.git
  3. ping_pong_bench

    Datasets & benchmarks

    A benchmark for role-playing language models

    Stale GitHub, IlyaGusev 122 ★Apache-2.0Updated 18 months ago

    git clone https://github.com/IlyaGusev/ping_pong_bench.git
  4. LoopArena

    Datasets & benchmarks

    Benchmarking models as runtime Controllers for Loop Engineering

    Stale GitHub, AMAP-ML 151 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/AMAP-ML/LoopArena.git
  5. Awesome-Scientific-LLM-Benchmarks

    Datasets & benchmarks

    A curated, accuracy-first list of benchmarks for evaluating LLMs on scientific reasoning and discovery — math, physics, chemistry, materials, biology, and agentic science.

    Stale GitHub, subinium 42 ★MITUpdated 14 days ago

    git clone https://github.com/subinium/Awesome-Scientific-LLM-Benchmarks.git
  6. MMLU-CF

    Datasets & benchmarks

    A Contamination-free Multi-task Language Understanding Benchmark [Official, ACL 2025]

    Stale GitHub, microsoft 127 ★MITUpdated 18 months ago

    git clone https://github.com/microsoft/MMLU-CF.git
  7. apple-silicon-llm-bench

    Datasets & benchmarks

    Reproducible on-device LLM benchmarks for Apple Silicon (iPhone 17 Pro, M4 Max): Apple Core AI, MLX, llama.cpp, LiteRT-LM and Core ML on the same model and harness, every number with its quantization and capture session; hybrid Mamba-2 models (Nemotron-3 Nano, Granite-4.0-H, Falcon-H1) included.

    Indexed GitHub, john-rocky 73 ★MITUpdated today

    git clone https://github.com/john-rocky/apple-silicon-llm-bench.git
  8. UltraData-SFT-2605

    Datasets & benchmarks

    Hugging Face dataset by openbmb

    Indexed Hugging Face Hub, openbmb 419 ♥apache-2.0Updated 5 months ago 1 note

    openbmb/UltraData-SFT-2605
  9. Finance-Instruct-500k

    Datasets & benchmarks

    Hugging Face dataset by Josephgflowers

    Indexed Hugging Face Hub, Josephgflowers 251 ♥apache-2.0Updated 8 months ago

    Josephgflowers/Finance-Instruct-500k
  10. guidelines

    Datasets & benchmarks

    Hugging Face dataset by epfl-llm

    Stale Hugging Face Hub, epfl-llm 163 ♥otherUpdated 2 years ago

    epfl-llm/guidelines
  11. Ultra-FineWeb-L3

    Datasets & benchmarks

    Hugging Face dataset by openbmb

    Indexed Hugging Face Hub, openbmb 346 ♥apache-2.0Updated 2 months ago

    openbmb/Ultra-FineWeb-L3
  12. SGI-Bench

    Datasets & benchmarks

    Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows

    Stale GitHub, InternScience 172 ★MITUpdated 5 months ago

    git clone https://github.com/InternScience/SGI-Bench.git
  13. ollama-benchmark

    Datasets & benchmarks

    Ollama based Benchmark with detail I/O token per second. Python with Deepseek R1 example.

    Stale GitHub, LarHope 174 ★MITUpdated 7 months ago

    git clone https://github.com/LarHope/ollama-benchmark.git
  14. OSWorld-V2

    Datasets & benchmarks

    OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks

    Indexed GitHub, xlang-ai 421 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/xlang-ai/OSWorld-V2.git
  15. nagi-bench

    Datasets & benchmarks

    One-shot LLM eval cases by NAGI STUDIO - same prompt, different agents (model + harness), runnable artifacts side by side.

    Indexed GitHub, nagi-studio 78 ★MITUpdated 4 days ago

    git clone https://github.com/nagi-studio/nagi-bench.git
  16. llm-value-rankings

    Datasets & benchmarks

    Daily LLM value rankings - compare 300+ models by intelligence, speed and price. OpenRouter + Artificial Analysis. 大模型性价比排行榜

    Indexed GitHub, yyh-001 30 ★MITUpdated yesterday

    git clone https://github.com/yyh-001/llm-value-rankings.git
  17. UltraData-RL-2609

    Datasets & benchmarks

    Hugging Face dataset by openbmb

    Stale Hugging Face Hub, openbmb 184 ♥apache-2.0Updated 2 months ago

    openbmb/UltraData-RL-2609
  18. synthetic-healthcare-admissions

    Datasets & benchmarks

    Hugging Face dataset by strova-ai

    Indexed Hugging Face Hub, strova-ai 244 ♥apache-2.0Updated 14 months ago

    strova-ai/synthetic-healthcare-admissions
  19. UltraData-Code

    Datasets & benchmarks

    Hugging Face dataset by openbmb

    Stale Hugging Face Hub, openbmb 205 ♥apache-2.0Updated 35 days ago

    openbmb/UltraData-Code
  20. Awesome-AI4AI

    Datasets & benchmarks

    Updated weekly — until humanity achieves AI4AI. 1148 curated papers across Benchmarks, Harness Design, and Model Design — from long-horizon agents to recursive self-improvement. Citation rankings, GitHub stars, and source-verified weekly news.

    Stale GitHub, KaiWU5 218 ★MITUpdated 8 days ago

    git clone https://github.com/KaiWU5/Awesome-AI4AI.git
  21. geobench

    Datasets & benchmarks

    GeoGuessr benchmark for language models

    Indexed GitHub, ccmdi 65 ★MITUpdated 3 days ago

    git clone https://github.com/ccmdi/geobench.git
  22. Ultra-FineWeb-L1

    Datasets & benchmarks

    Hugging Face dataset by openbmb

    Stale Hugging Face Hub, openbmb 201 ♥apache-2.0Updated 2 months ago

    openbmb/Ultra-FineWeb-L1
  23. gamebot

    Datasets & benchmarks

    [ACL 2025] GAMEBoT: Transparent Assessment of LLM Reasoning in Games

    Stale GitHub, Visual-AI 36 ★Updated 10 days ago

    git clone https://github.com/Visual-AI/gamebot.git
  24. o11y-bench

    Datasets & benchmarks

    Open benchmark on observability tasks built on Harbor

    Indexed GitHub, grafana 80 ★AGPL-3.0Updated today

    git clone https://github.com/grafana/o11y-bench.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.