InnerVault

Library

85,762 resources from 11 public sources. 54 checked by hand; the rest are unreviewed.

Results

  1. OctoCodingBench

    Datasets & benchmarks

    Hugging Face dataset by MiniMaxAI

    Indexed Hugging Face Hub, MiniMaxAI 465 ♥mitUpdated 10 months ago

    MiniMaxAI/OctoCodingBench
  2. NoLiMa

    Datasets & benchmarks

    Official repository for "NoLiMa: Long-Context Evaluation Beyond Literal Matching"

    Stale GitHub, adobe-research 205 ★Custom licence (see repository)Updated 16 months ago

    git clone https://github.com/adobe-research/NoLiMa.git
  3. meta-agents-research-environments

    Datasets & benchmarks

    Meta Agents Research Environments is a comprehensive platform designed to evaluate AI agents in dynamic, realistic scenarios. Unlike static benchmarks, this platform introduces evolving environments where agents must adapt their strategies as new information becomes available, mirroring real-world challenges.

    Stale GitHub, facebookresearch 567 ★MITUpdated 47 days ago

    git clone https://github.com/facebookresearch/meta-agents-research-environments.git
  4. BizFinBench

    Datasets & benchmarks

    A Business-Driven Real-World Financial Benchmark for Evaluating LLMs

    Stale GitHub, HiThink-Research 169 ★Updated 6 months ago

    git clone https://github.com/HiThink-Research/BizFinBench.git
  5. GDPevo

    Datasets & benchmarks

    A Benchmark for Evaluating Agent Self-Evolution on Real Business Tasks

    Stale GitHub, Prism-Shadow 119 ★Apache-2.0Updated 18 days ago

    git clone https://github.com/Prism-Shadow/GDPevo.git
  6. ParseBench

    Datasets & benchmarks

    Hugging Face dataset by llamaindex

    Stale Hugging Face Hub, llamaindex 137 ♥apache-2.0Updated 7 months ago

    llamaindex/ParseBench
  7. nyt-connections

    Datasets & benchmarks

    Benchmark that evaluates LLMs using 759 NYT Connections puzzles extended with extra trick words

    Indexed GitHub, lechmazur 250 ★Updated 5 days ago

    git clone https://github.com/lechmazur/nyt-connections.git
  8. WebsiteBench

    Datasets & benchmarks

    Can AI agents rebuild websites through browser exploration?

    Indexed GitHub, TIGER-AI-Lab 300 ★Apache-2.0Updated 6 days ago

    git clone https://github.com/TIGER-AI-Lab/WebsiteBench.git
  9. ParseBench

    Datasets & benchmarks

    ParseBench - A Document Parsing Benchmark for AI Agents

    Indexed GitHub, run-llama 640 ★Apache-2.0Updated today

    git clone https://github.com/run-llama/ParseBench.git
  10. ReForm-Eval

    Datasets & benchmarks

    An benchmark for evaluating the capabilities of large vision-language models (LVLMs)

    Stale GitHub, FudanDISC 145 ★Apache-2.0Updated 3 years ago

    git clone https://github.com/FudanDISC/ReForm-Eval.git
  11. tweet_eval

    Datasets & benchmarks

    Hugging Face dataset by cardiffnlp

    Stale Hugging Face Hub, cardiffnlp 158 ♥unknownUpdated 2 years ago

    cardiffnlp/tweet_eval
  12. gpqa

    Datasets & benchmarks

    Hugging Face dataset by Idavidrein

    Indexed Hugging Face Hub, Idavidrein 642 ♥cc-by-4.0Updated 56 days ago 1 note

    Idavidrein/gpqa
  13. swt-bench

    Datasets & benchmarks

    [NeurIPS 2024] Evaluation harness for SWT-Bench, a benchmark for evaluating LLM repository-level test-generation

    Stale GitHub, logic-star-ai 92 ★MITUpdated 12 days ago

    git clone https://github.com/logic-star-ai/swt-bench.git
  14. AgentLab

    Datasets & benchmarks

    AgentLab: An open-source framework for developing, testing, and benchmarking web agents on diverse tasks, designed for scalability and reproducibility.

    Stale GitHub, ServiceNow 653 ★Custom licence (see repository)Updated 41 days ago

    git clone https://github.com/ServiceNow/AgentLab.git
  15. Simreal-MLBench

    Datasets & benchmarks

    [Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluation.

    Stale GitHub, Simreal-AI 301 ★Apache-2.0Updated 53 days ago

    git clone https://github.com/Simreal-AI/Simreal-MLBench.git
  16. DecisionPivot

    Datasets & benchmarks

    DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.

    Stale GitHub, StartLuxLabs 144 ★Apache-2.0Updated 26 days ago

    git clone https://github.com/StartLuxLabs/DecisionPivot.git
  17. llm-colosseum

    Datasets & benchmarks

    Benchmark LLMs by fighting in Street Fighter 3! The new way to evaluate the quality of an LLM

    Stale GitHub, OpenGenerativeAI 1,483 ★MITUpdated 20 months ago

    git clone https://github.com/OpenGenerativeAI/llm-colosseum.git
  18. ResearchClawBench

    Datasets & benchmarks

    🦞 ResearchClawBench: Evaluating AI Agents for Automated Research from Re-Discovery to New-Discovery

    Stale GitHub, InternScience 275 ★MITUpdated 19 days ago

    git clone https://github.com/InternScience/ResearchClawBench.git
  19. HaluMem

    Datasets & benchmarks

    HaluMem is the first operation level hallucination evaluation benchmark tailored to agent memory systems.

    Stale GitHub, MemTensor 178 ★Custom licence (see repository)Updated 19 days ago

    git clone https://github.com/MemTensor/HaluMem.git
  20. SWE-bench_Pro

    Datasets & benchmarks

    Hugging Face dataset by ScaleAI

    Stale Hugging Face Hub, ScaleAI 197 ♥Updated 55 days ago

    ScaleAI/SWE-bench_Pro
  21. nautilus-compass

    Datasets & benchmarks

    Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform

    Indexed GitHub, chunxiaoxx 1,258 ★Custom licence (see repository)Updated 2 days ago

    git clone https://github.com/chunxiaoxx/nautilus-compass.git
  22. AIRTBench-Code

    Datasets & benchmarks

    Code Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models

    Stale GitHub, dreadnode 111 ★Apache-2.0Updated 32 days ago

    git clone https://github.com/dreadnode/AIRTBench-Code.git
  23. SWE-bench_Verified

    Datasets & benchmarks

    Hugging Face dataset by SWE-bench

    Stale Hugging Face Hub, SWE-bench 188 ♥Updated 3 months ago

    SWE-bench/SWE-bench_Verified
  24. SHP

    Datasets & benchmarks

    Hugging Face dataset by stanfordnlp

    Indexed Hugging Face Hub, stanfordnlp 333 ♥Updated 3 years ago

    stanfordnlp/SHP

Indexed entries aren't reviewed or tested by InnerVault. Details come straight from each source. Anything that can reach your accounts, files or browser can do damage, so read the source before you install it.