InnerVault

Library

85,762 resources from 11 public sources. 53 checked by hand; the rest are unreviewed.

Results

  1. Awesome-Datasets-Hub

    Datasets & benchmarks

    A curated collection of datasets for Large Language Models (LLMs), covering medical AI, NLP, multimodal learning, instruction tuning, reasoning, code generation, and evaluation benchmarks.

    Stale GitHub, ahammadmejbah 149 ★Updated 4 months ago

    git clone https://github.com/ahammadmejbah/Awesome-Datasets-Hub.git
  2. aurora

    Datasets & benchmarks

    [ICLR 2025] AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark

    Stale GitHub, wenhaochai 149 ★Apache-2.0Updated 17 months ago

    git clone https://github.com/wenhaochai/aurora.git
  3. chembench

    Datasets & benchmarks

    How good are LLMs at chemistry?

    Stale GitHub, lamalab-org 147 ★MITUpdated 9 months ago

    git clone https://github.com/lamalab-org/chembench.git
  4. ReForm-Eval

    Datasets & benchmarks

    An benchmark for evaluating the capabilities of large vision-language models (LVLMs)

    Stale GitHub, FudanDISC 145 ★Apache-2.0Updated 3 years ago

    git clone https://github.com/FudanDISC/ReForm-Eval.git
  5. DecisionPivot

    Datasets & benchmarks

    DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.

    Stale GitHub, StartLuxLabs 144 ★Apache-2.0Updated 26 days ago

    git clone https://github.com/StartLuxLabs/DecisionPivot.git
  6. openbench

    Datasets & benchmarks

    Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost

    Stale GitHub, minghinmatthewlam 137 ★MITUpdated 2 months ago

    git clone https://github.com/minghinmatthewlam/openbench.git
  7. sec-code-bench

    Datasets & benchmarks

    SecCodeBench is a benchmark suite focusing on evaluating the security of code generated by large language models (LLMs).

    Stale GitHub, alibaba 135 ★Apache-2.0Updated 35 days ago

    git clone https://github.com/alibaba/sec-code-bench.git
  8. ChinaTravel

    Datasets & benchmarks

    ChinaTravel: A Real-World Benchmark for Language Agents in Chinese Travel Planning

    Stale GitHub, LAMDA-NeSy 133 ★MITUpdated 2 months ago

    git clone https://github.com/LAMDA-NeSy/ChinaTravel.git
  9. autoguardrails

    Datasets & benchmarks

    Alignment-research scaffold (autoresearch-style) for LLM guardrails: search over a single policy.md surface

    Stale GitHub, SantanderAI 129 ★Apache-2.0Updated 46 days ago

    git clone https://github.com/SantanderAI/autoguardrails.git
  10. MMLU-CF

    Datasets & benchmarks

    A Contamination-free Multi-task Language Understanding Benchmark [Official, ACL 2025]

    Stale GitHub, microsoft 127 ★MITUpdated 18 months ago

    git clone https://github.com/microsoft/MMLU-CF.git
  11. ping_pong_bench

    Datasets & benchmarks

    A benchmark for role-playing language models

    Stale GitHub, IlyaGusev 122 ★Apache-2.0Updated 18 months ago

    git clone https://github.com/IlyaGusev/ping_pong_bench.git
  12. primitive-bench

    Datasets & benchmarks

    The marketplace for verifiable AI outcomes: state a task, get a fixed price upfront, and Primitive Bench routes across tools to complete it, refunded if the outcome isn't delivered. This repo is the open verification layer.

    Stale GitHub, primitive-bench 122 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/primitive-bench/primitive-bench.git
  13. GDPevo

    Datasets & benchmarks

    A Benchmark for Evaluating Agent Self-Evolution on Real Business Tasks

    Stale GitHub, Prism-Shadow 119 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/Prism-Shadow/GDPevo.git
  14. PawBench

    Datasets & benchmarks

    A benchmark for evaluating LLM × harness performance.

    Stale GitHub, agentscope-ai 118 ★Apache-2.0Updated 3 months ago

    git clone https://github.com/agentscope-ai/PawBench.git
  15. TwinFix

    Datasets & benchmarks

    TwinFix — AI agent that diagnoses, reproduces and fixes bugs in a disposable twin environment. 在一次性孪生环境中自动诊断、复现并修复故障的 AI Agent。

    Indexed GitHub, spacesky-cell 117 ★Apache-2.0Updated 3 days ago

    git clone https://github.com/spacesky-cell/TwinFix.git
  16. doc2harness

    Datasets & benchmarks

    Official Code for the paper "Doc2Harness: Benchmarking LLMs on Long-Horizon Harness Engineering of Coding Agents "

    Stale GitHub, YuyaoGe 116 ★Custom licence (see repository)Updated 20 days ago

    git clone https://github.com/YuyaoGe/doc2harness.git
  17. Omni-SafetyBench

    Datasets & benchmarks

    [ACM MM 2026 Oral] Code for paper "Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Models".

    Stale GitHub, THU-BPM 115 ★Apache-2.0Updated 2 months ago

    git clone https://github.com/THU-BPM/Omni-SafetyBench.git
  18. VibeSearchBench

    Datasets & benchmarks

    🔍 The hardest search benchmark in the wild — vague, multi-turn, proactive. 200 long-horizon tasks with persona-driven progressive disclosure, scored by verifiable schema-free knowledge-graph evaluation. No vibes, just triplet F1.

    Stale GitHub, VibeBench 112 ★MITUpdated 5 months ago

    git clone https://github.com/VibeBench/VibeSearchBench.git
  19. AIRTBench-Code

    Datasets & benchmarks

    Code Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

    Stale GitHub, dreadnode 111 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/dreadnode/AIRTBench-Code.git
  20. fde-gym

    Datasets & benchmarks

    The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.

    Stale GitHub, deeplethe 111 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/deeplethe/fde-gym.git
  21. inferock-bench

    Datasets & benchmarks

    Local LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity receipts.

    Stale GitHub, inferock 107 ★Custom licence (see repository)Updated 3 months ago

    git clone https://github.com/inferock/inferock-bench.git
  22. sacp

    Datasets & benchmarks

    State-Aware Collaboration Protocol: auditable work receipts for AI agents.

    Stale GitHub, aDragon0707 107 ★MITUpdated 32 days ago

    git clone https://github.com/aDragon0707/sacp.git
  23. VisualNews-Repository

    Datasets & benchmarks

    [EMNLP'21] Visual News: Benchmark and Challenges in News Image Captioning

    Stale GitHub, FuxiaoLiu 106 ★Updated 2 years ago

    git clone https://github.com/FuxiaoLiu/VisualNews-Repository.git
  24. Awesome-AI-Scientist

    Datasets & benchmarks

    🧪 Awesome list of AI Scientist papers, systems, benchmarks, datasets and open-source platforms.

    Indexed GitHub, Omni-Scientist 105 ★Custom licence (see repository)Updated 3 days ago

    git clone https://github.com/Omni-Scientist/Awesome-AI-Scientist.git

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.