InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

  1. NoLiMa

    Datasets & benchmarks

    Official repository for "NoLiMa: Long-Context Evaluation Beyond Literal Matching"

    Stale GitHub, adobe-research 205 ★Custom licence (see repository)Updated 16 months ago

    git clone https://github.com/adobe-research/NoLiMa.git
  2. mac-ml-speed-test

    Datasets & benchmarks

    A few quick scripts focused on testing TensorFlow/PyTorch/Llama 2 on macOS.

    Stale GitHub, mrdbourke 203 ★Updated 2 years ago

    git clone https://github.com/mrdbourke/mac-ml-speed-test.git
  3. WritingBench

    Datasets & benchmarks

    WritingBench: A Comprehensive Benchmark for Generative Writing

    Stale GitHub, X-PLUG 202 ★Apache-2.0Updated 11 months ago

    git clone https://github.com/X-PLUG/WritingBench.git
  4. FastEval

    Datasets & benchmarks

    Fast & more realistic evaluation of chat language models. Includes leaderboard.

    Stale GitHub, FastEval 187 ★Apache-2.0Updated 2 years ago

    git clone https://github.com/FastEval/FastEval.git
  5. UHGEval

    Datasets & benchmarks

    [ACL 2024] User-friendly evaluation framework: Eval Suite & Benchmarks: UHGEval, HaluEval, HalluQA, etc.

    Stale GitHub, IAAR-Shanghai 183 ★Apache-2.0Updated 17 months ago

    git clone https://github.com/IAAR-Shanghai/UHGEval.git
  6. MedXpertQA

    Datasets & benchmarks

    [ICML 2025] MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding

    Stale GitHub, TsinghuaC3I 182 ★MITUpdated 2 months ago

    git clone https://github.com/TsinghuaC3I/MedXpertQA.git
  7. xFinder

    Datasets & benchmarks

    [ICLR 2025] xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation

    Stale GitHub, IAAR-Shanghai 178 ★Custom licence (see repository)Updated 12 months ago

    git clone https://github.com/IAAR-Shanghai/xFinder.git
  8. HaluMem

    Datasets & benchmarks

    HaluMem is the first operation level hallucination evaluation benchmark tailored to agent memory systems.

    Stale GitHub, MemTensor 178 ★Custom licence (see repository)Updated 19 days ago

    git clone https://github.com/MemTensor/HaluMem.git
  9. ChemLLMBench

    Datasets & benchmarks

    Official Code for What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks (In NeurIPS 2023)

    Stale GitHub, ChemFoundationModels 176 ★Updated 2 years ago

    git clone https://github.com/ChemFoundationModels/ChemLLMBench.git
  10. ollama-benchmark

    Datasets & benchmarks

    Ollama based Benchmark with detail I/O token per second. Python with Deepseek R1 example.

    Stale GitHub, LarHope 174 ★MITUpdated 7 months ago

    git clone https://github.com/LarHope/ollama-benchmark.git
  11. LLM-Agent-Benchmark-List

    Datasets & benchmarks

    A banchmark list for evaluation of large language models.

    Stale GitHub, zhangxjohn 173 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/zhangxjohn/LLM-Agent-Benchmark-List.git
  12. SGI-Bench

    Datasets & benchmarks

    Probing Scientific General Intelligence of LLMs with Scientist-Aligned Workflows

    Stale GitHub, InternScience 172 ★MITUpdated 5 months ago

    git clone https://github.com/InternScience/SGI-Bench.git
  13. BizFinBench

    Datasets & benchmarks

    A Business-Driven Real-World Financial Benchmark for Evaluating LLMs

    Stale GitHub, HiThink-Research 169 ★Updated 6 months ago

    git clone https://github.com/HiThink-Research/BizFinBench.git
  14. active_genie

    Datasets & benchmarks

    The Lodash for GenAI: Real Value + Consistent + Model-Agnostic

    Stale GitHub, Roriz 168 ★Apache-2.0Updated 3 months ago

    git clone https://github.com/Roriz/active_genie.git
  15. One-Eval

    Datasets & benchmarks

    Automated system for LLM evaluation via agents. Doc as below:

    Stale GitHub, OpenDCAI 165 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/OpenDCAI/One-Eval.git
  16. LLM-RGB

    Datasets & benchmarks

    LLM Reasoning and Generation Benchmark. Evaluate LLMs in complex scenarios systematically.

    Stale GitHub, babelcloud 165 ★MITUpdated 18 months ago

    git clone https://github.com/babelcloud/LLM-RGB.git
  17. TurtleBench

    Datasets & benchmarks

    TurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles.

    Stale GitHub, mazzzystar 161 ★Apache-2.0Updated 2 years ago

    git clone https://github.com/mazzzystar/TurtleBench.git
  18. awesome-ai-gateway

    Datasets & benchmarks

    ⚡ Awesome AI Gateway — pick an AI gateway from 240+ (LiteLLM, OpenRouter, Portkey, Kong, Higress, new-api, Bifrost) by cost, security & compliance, see what consolidated in 2026, learn how they actually work from an 8-chapter handbook, and verify every number yourself. Reproducible benchmarks, bilingual; stars & releases refreshed daily.

    Indexed GitHub, cuihuan 158 ★CC0-1.0Updated yesterday

    git clone https://github.com/cuihuan/awesome-ai-gateway.git
  19. multi-agents

    Datasets & benchmarks

    Multi-agent teams for real work: run teams of coding agents (Claude Code, Codex, any model) in isolated git worktrees, and measure whether the team beats a solo agent on cost and results.

    Stale GitHub, BichengWang 158 ★MITUpdated 8 days ago

    git clone https://github.com/BichengWang/multi-agents.git
  20. WideSearch

    Datasets & benchmarks

    WideSearch: Benchmarking Agentic Broad Info-Seeking

    Stale GitHub, ByteDance-Seed 156 ★MITUpdated 13 months ago

    git clone https://github.com/ByteDance-Seed/WideSearch.git
  21. llmspeed

    Datasets & benchmarks

    Feel how fast a local LLM runs before you buy the hardware

    Stale GitHub, nicorobinreal 152 ★MITUpdated 27 days ago

    git clone https://github.com/nicorobinreal/llmspeed.git
  22. LoopArena

    Datasets & benchmarks

    Benchmarking models as runtime Controllers for Loop Engineering

    Stale GitHub, AMAP-ML 151 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/AMAP-ML/LoopArena.git
  23. xVerify

    Datasets & benchmarks

    xVerify: Efficient Answer Verifier for Reasoning Model Evaluations

    Stale GitHub, IAAR-Shanghai 150 ★Custom licence (see repository)Updated 19 days ago

    git clone https://github.com/IAAR-Shanghai/xVerify.git
  24. lora-speedrun

    Datasets & benchmarks

    Speedrunning LoRA fine-tuning: frozen task, frozen hardware, public wall-clock leaderboard. modded-nanogpt for fine-tuning.

    Stale GitHub, Saivineeth147 149 ★MITUpdated 2 months ago

    git clone https://github.com/Saivineeth147/lora-speedrun.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.