InnerVault

Library

85,340 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. Long-Horizon-Terminal-Bench

    Datasets & benchmarks

    Hugging Face dataset by IntelligenceLab

    Stale Hugging Face Hub, IntelligenceLab 140 ♥apache-2.0Updated 36 days ago

    IntelligenceLab/Long-Horizon-Terminal-Bench
  2. llm-bulls-and-cows-benchmark

    Datasets & benchmarks

    A mini-framework for evaluating LLM performance on the Bulls and Cows number guessing game, supporting multiple LLM providers.

    Stale GitHub, stalkermustang 235 ★MITUpdated 21 months ago

    git clone https://github.com/stalkermustang/llm-bulls-and-cows-benchmark.git
  3. LawBench

    Datasets & benchmarks

    Benchmarking Legal Knowledge of Large Language Models

    Stale GitHub, open-compass 455 ★Apache-2.0Updated 3 years ago

    git clone https://github.com/open-compass/LawBench.git
  4. vidore-benchmark

    AI repositories

    Vision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.

    Stale GitHub, illuin-tech 278 ★MITUpdated 27 days ago

    git clone https://github.com/illuin-tech/vidore-benchmark.git
  5. ping_pong_bench

    Datasets & benchmarks

    A benchmark for role-playing language models

    Stale GitHub, IlyaGusev 122 ★Apache-2.0Updated 18 months ago

    git clone https://github.com/IlyaGusev/ping_pong_bench.git
  6. ollama-benchmark

    Datasets & benchmarks

    Ollama based Benchmark with detail I/O token per second. Python with Deepseek R1 example.

    Stale GitHub, LarHope 174 ★MITUpdated 7 months ago

    git clone https://github.com/LarHope/ollama-benchmark.git
  7. folkbench

    AI repositories

    帮你的 AI 编程工具找到不降智、不断连、高性价比的好线路 · Live LLM relay rankings, model integrity probes & Tauri/Rust switcher for Claude Code, Codex & Gemini CLI

    Indexed GitHub, Folkbench 3,883 ★Custom licence (see repository)Updated today

    git clone https://github.com/Folkbench/folkbench.git
  8. LLM-Agent-Benchmark-List

    Datasets & benchmarks

    A banchmark list for evaluation of large language models.

    Stale GitHub, zhangxjohn 173 ★Apache-2.0Updated 11 days ago

    git clone https://github.com/zhangxjohn/LLM-Agent-Benchmark-List.git
  9. OSWorld

    Datasets & benchmarks

    [NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

    Indexed GitHub, xlang-ai 3,262 ★Apache-2.0Updated 2 days ago

    git clone https://github.com/xlang-ai/OSWorld.git
  10. MMLU-Pro

    Datasets & benchmarks

    Hugging Face dataset by TIGER-Lab

    Indexed Hugging Face Hub, TIGER-Lab 602 ♥mitUpdated 6 months ago

    TIGER-Lab/MMLU-Pro
  11. OpenLLM

    AI repositories

    Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

    Stale GitHub, bentoml 12.6k ★Apache-2.0Updated 25 days ago

    git clone https://github.com/bentoml/OpenLLM.git
  12. sec-code-bench

    Datasets & benchmarks

    SecCodeBench is a benchmark suite focusing on evaluating the security of code generated by large language models (LLMs).

    Stale GitHub, alibaba 135 ★Apache-2.0Updated 35 days ago

    git clone https://github.com/alibaba/sec-code-bench.git
  13. CL-bench

    Datasets & benchmarks

    Hugging Face dataset by tencent

    Stale Hugging Face Hub, tencent 148 ♥otherUpdated 9 months ago

    tencent/CL-bench
  14. LongBench

    Datasets & benchmarks

    LongBench v2 and LongBench (ACL 25'&24')

    Stale GitHub, THUDM 1,246 ★MITUpdated 22 months ago

    git clone https://github.com/THUDM/LongBench.git
  15. BizFinBench

    Datasets & benchmarks

    A Business-Driven Real-World Financial Benchmark for Evaluating LLMs

    Stale GitHub, HiThink-Research 169 ★Updated 6 months ago

    git clone https://github.com/HiThink-Research/BizFinBench.git
  16. apex-agents

    Datasets & benchmarks

    Hugging Face dataset by mercor

    Stale Hugging Face Hub, mercor 207 ♥cc-by-4.0Updated 5 months ago 1 note

    mercor/apex-agents
  17. o11y-bench

    Datasets & benchmarks

    Open benchmark on observability tasks built on Harbor

    Indexed GitHub, grafana 80 ★AGPL-3.0Updated today

    git clone https://github.com/grafana/o11y-bench.git
  18. swt-bench

    Datasets & benchmarks

    [NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation

    Stale GitHub, logic-star-ai 92 ★MITUpdated 12 days ago

    git clone https://github.com/logic-star-ai/swt-bench.git
  19. kg-gen

    Datasets & benchmarks

    [NeurIPS '25] Knowledge Graph Generation from Any Text

    Stale GitHub, stair-lab 1,286 ★Updated 7 months ago

    git clone https://github.com/stair-lab/kg-gen.git
  20. ADR

    MCP servers

    ADR secures enterprise AI agents through observability, security benchmarking, and threat detection. Deployed at Uber.

    Stale GitHub, uber 2,090 ★Apache-2.0Updated 16 days ago

    git clone https://github.com/uber/ADR.git
  21. nyt-connections

    Datasets & benchmarks

    Benchmark that evaluates LLMs using 759 NYT Connections puzzles extended with extra trick words

    Indexed GitHub, lechmazur 250 ★Updated 5 days ago

    git clone https://github.com/lechmazur/nyt-connections.git
  22. nautilus-compass

    Datasets & benchmarks

    Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform

    Indexed GitHub, chunxiaoxx 1,258 ★Custom licence (see repository)Updated 2 days ago

    git clone https://github.com/chunxiaoxx/nautilus-compass.git
  23. chinese-llm-benchmark

    Agents & frameworks

    非线智能 NoneLinear - ReLE评测:中文AI大模型能力评测(持续更新):目前已囊括374个大模型,覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-X1.1、ERNIE-5.0、qwen3.6-max、qwen3.6-plus、百川、讯飞星火、商汤senseChat等商用模型, 以及step3.5-flash、kimi-k2.6、ernie4.5、MiniMax-M2.7、deepseek-v4、Qwen3.6、llama4、智谱GLM-5.1、MiMo-V2、LongCat、gemma4、mistral等开源大模型。不仅提供排行榜,也提供规模超200万的大模型缺陷库!方便广大社区研究分析、改进大模型。

    Indexed GitHub, jeinlee1991 6,518 ★Updated yesterday

    git clone https://github.com/jeinlee1991/chinese-llm-benchmark.git
  24. gpqa

    Datasets & benchmarks

    Hugging Face dataset by Idavidrein

    Indexed Hugging Face Hub, Idavidrein 642 ♥cc-by-4.0Updated 56 days ago 1 note

    Idavidrein/gpqa

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.