InnerVault

Library

85,843 resources from 11 public sources. 51 checked by hand; the rest are unreviewed.

Results

757 results with any of your words Clear search and filters
  1. ollama-benchmark

    Datasets & benchmarks

    LLM Benchmark for Throughput via Ollama (Local LLMs)

    Stale GitHub, aidatatools 397 ★MITUpdated 21 days ago

    git clone https://github.com/aidatatools/ollama-benchmark.git
  2. FastEval

    Datasets & benchmarks

    Fast & more realistic evaluation of chat language models. Includes leaderboard.

    Stale GitHub, FastEval 187 ★Apache-2.0Updated 2 years ago

    git clone https://github.com/FastEval/FastEval.git
  3. llm-colosseum

    Datasets & benchmarks

    Benchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM

    Stale GitHub, OpenGenerativeAI 1,483 ★MITUpdated 20 months ago

    git clone https://github.com/OpenGenerativeAI/llm-colosseum.git
  4. BALROG

    Datasets & benchmarks

    Benchmarking Agentic LLM and VLM Reasoning On Games

    Stale GitHub, balrog-ai 274 ★MITUpdated 7 months ago

    git clone https://github.com/balrog-ai/BALROG.git
  5. TurtleBench

    Datasets & benchmarks

    TurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles.

    Stale GitHub, mazzzystar 161 ★Apache-2.0Updated 2 years ago

    git clone https://github.com/mazzzystar/TurtleBench.git
  6. MedXpertQA

    Datasets & benchmarks

    [ICML 2025] MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding

    Stale GitHub, TsinghuaC3I 182 ★MITUpdated 2 months ago

    git clone https://github.com/TsinghuaC3I/MedXpertQA.git
  7. LongCite

    Datasets & benchmarks

    LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

    Stale GitHub, THUDM 523 ★Apache-2.0Updated 22 months ago

    git clone https://github.com/THUDM/LongCite.git
  8. active_genie

    Datasets & benchmarks

    The Lodash for GenAI: Real Value + Consistent + Model-Agnostic

    Stale GitHub, Roriz 168 ★Apache-2.0Updated 3 months ago

    git clone https://github.com/Roriz/active_genie.git
  9. LLM-eval-survey

    Datasets & benchmarks

    The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".

    Stale GitHub, MLGroupJLU 1,615 ★Updated 8 days ago

    git clone https://github.com/MLGroupJLU/LLM-eval-survey.git
  10. MobileWorld

    Datasets & benchmarks

    Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments (ACL 2026)

    Stale GitHub, Tongyi-MAI 300 ★Apache-2.0Updated 14 days ago

    git clone https://github.com/Tongyi-MAI/MobileWorld.git
  11. geobench

    Datasets & benchmarks

    GeoGuessr benchmark for language models

    Indexed GitHub, ccmdi 65 ★MITUpdated 4 days ago

    git clone https://github.com/ccmdi/geobench.git
  12. SciCode

    Datasets & benchmarks

    A benchmark that challenges language models to code solutions for scientific problems

    Stale GitHub, scicode-bench 246 ★Apache-2.0Updated 15 days ago

    git clone https://github.com/scicode-bench/SciCode.git
  13. ping_pong_bench

    Datasets & benchmarks

    A benchmark for role-playing language models

    Stale GitHub, IlyaGusev 122 ★Apache-2.0Updated 18 months ago

    git clone https://github.com/IlyaGusev/ping_pong_bench.git
  14. PawBench

    Datasets & benchmarks

    A benchmark for evaluating LLM × harness performance.

    Stale GitHub, agentscope-ai 118 ★Apache-2.0Updated 3 months ago

    git clone https://github.com/agentscope-ai/PawBench.git
  15. MMLU-CF

    Datasets & benchmarks

    A Contamination-free Multi-task Language Understanding Benchmark [Official, ACL 2025]

    Stale GitHub, microsoft 127 ★MITUpdated 18 months ago

    git clone https://github.com/microsoft/MMLU-CF.git
  16. minebench

    Datasets & benchmarks

    Minecraft-style voxel benchmark for comparing AI models (Arena + Sandbox)

    Indexed GitHub, Ammaar-Alam 343 ★MITUpdated 5 days ago

    git clone https://github.com/Ammaar-Alam/minebench.git
  17. BizFinBench

    Datasets & benchmarks

    A Business-Driven Real-World Financial Benchmark for Evaluating LLMs

    Stale GitHub, HiThink-Research 169 ★Updated 6 months ago

    git clone https://github.com/HiThink-Research/BizFinBench.git
  18. OSWorld

    Datasets & benchmarks

    [NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

    Indexed GitHub, xlang-ai 3,262 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/xlang-ai/OSWorld.git
  19. o11y-bench

    Datasets & benchmarks

    Open benchmark on observability tasks built on Harbor

    Indexed GitHub, grafana 80 ★AGPL-3.0Updated yesterday

    git clone https://github.com/grafana/o11y-bench.git
  20. AICGSecEval

    Datasets & benchmarks

    A.S.E (AICGSecEval) is a repository-level AI-generated code security evaluation benchmark developed by Tencent Wukong Code Security Team.

    Stale GitHub, Tencent 664 ★Custom licence (see repository)Updated 5 months ago

    git clone https://github.com/Tencent/AICGSecEval.git
  21. LoopArena

    Datasets & benchmarks

    Benchmarking models as runtime Controllers for Loop Engineering

    Stale GitHub, AMAP-ML 151 ★Apache-2.0Updated 33 days ago

    git clone https://github.com/AMAP-ML/LoopArena.git
  22. chembench

    Datasets & benchmarks

    How good are LLMs at chemistry?

    Stale GitHub, lamalab-org 147 ★MITUpdated 9 months ago

    git clone https://github.com/lamalab-org/chembench.git
  23. LLM-Agent-Benchmark-List

    Datasets & benchmarks

    A banchmark list for evaluation of large language models.

    Stale GitHub, zhangxjohn 173 ★Apache-2.0Updated 12 days ago

    git clone https://github.com/zhangxjohn/LLM-Agent-Benchmark-List.git
  24. turnkeyml

    Datasets & benchmarks

    No-code CLI designed for accelerating ONNX workflows

    Stale GitHub, onnx 247 ★Apache-2.0Updated 4 months ago

    git clone https://github.com/onnx/turnkeyml.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.