InnerVault

debate

Stale GitHub, lechmazur

Adversarial multi-turn benchmark for LLM debate quality, using side-swapped matchups and multi-model judging to rank models by judged debate performance.

Open the source (opens github.com) https://github.com/lechmazur/debate

What it is

Datasets & benchmarks from lechmazur.

The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.

How to get it

Get the code

git clone https://github.com/lechmazur/debate.git

Standard git access to a public repository

Setup steps: not provided by the source. Follow the instructions on the source.

Related

  1. geobench

    Datasets & benchmarks

    GeoGuessr benchmark for language models

    Indexed GitHub, ccmdi 65 ★MITUpdated 3 days ago

    git clone https://github.com/ccmdi/geobench.git
  2. confabulations

    Datasets & benchmarks

    Hallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.

    Stale GitHub, lechmazur 249 ★Updated 15 months ago

    git clone https://github.com/lechmazur/confabulations.git
  3. nyt-connections

    Datasets & benchmarks

    Benchmark that evaluates LLMs using 759 NYT Connections puzzles extended with extra trick words

    Indexed GitHub, lechmazur 250 ★Updated 5 days ago

    git clone https://github.com/lechmazur/nyt-connections.git
  4. inferock-bench

    Datasets & benchmarks

    Local LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity receipts.

    Stale GitHub, inferock 107 ★Custom licence (see repository)Updated 3 months ago

    git clone https://github.com/inferock/inferock-bench.git
  5. nagi-bench

    Datasets & benchmarks

    One-shot LLM eval cases by NAGI STUDIO - same prompt, different agents (model + harness), runnable artifacts side by side.

    Indexed GitHub, nagi-studio 78 ★MITUpdated 4 days ago

    git clone https://github.com/nagi-studio/nagi-bench.git
  6. bigcodebench

    Datasets & benchmarks

    [ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI

    Stale Archived GitHub, bigcode-project 516 ★Apache-2.0Updated 10 months ago 1 note

    git clone https://github.com/bigcode-project/bigcodebench.git