debate
Adversarial multi-turn benchmark for LLM debate quality, using side-swapped matchups and multi-model judging to rank models by judged debate performance.
Open the source (opens github.com) https://github.com/lechmazur/debate
What it is
Datasets & benchmarks from lechmazur.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/lechmazur/debate.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
geobench
Datasets & benchmarksGeoGuessr benchmark for language models
git clone https://github.com/ccmdi/geobench.git -
confabulations
Datasets & benchmarksHallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.
git clone https://github.com/lechmazur/confabulations.git -
nyt-connections
Datasets & benchmarksBenchmark that evaluates LLMs using 759 NYT Connections puzzles extended with extra trick words
git clone https://github.com/lechmazur/nyt-connections.git -
inferock-bench
Datasets & benchmarksLocal LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity receipts.
git clone https://github.com/inferock/inferock-bench.git -
nagi-bench
Datasets & benchmarksOne-shot LLM eval cases by NAGI STUDIO - same prompt, different agents (model + harness), runnable artifacts side by side.
git clone https://github.com/nagi-studio/nagi-bench.git -
bigcodebench
Datasets & benchmarks[ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI
git clone https://github.com/bigcode-project/bigcodebench.git