MMLU-CF
A Contamination-free Multi-task Language Understanding Benchmark [Official, ACL 2025]
Open the source (opens github.com) https://github.com/microsoft/mmlu-cf
What it is
Datasets & benchmarks from microsoft.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/microsoft/MMLU-CF.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
DyCodeEval
Datasets & benchmarksOfficial repository of the ICML2025 paper “Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contamination”
git clone https://github.com/SeekingDream/DyCodeEval.git -
MMLU-Pro
Datasets & benchmarksHugging Face dataset by TIGER-Lab
TIGER-Lab/MMLU-Pro -
Static-to-Dynamic-LLMEval
Datasets & benchmarksThe official GitHub repository of the paper "Recent advances in large language model benchmarks against data contamination: From static to dynamic evaluation"
git clone https://github.com/SeekingDream/Static-to-Dynamic-LLMEval.git -
mmlu
Datasets & benchmarksHugging Face dataset by cais
cais/mmlu -
Global-MMLU
Datasets & benchmarksHugging Face dataset by CohereLabs
CohereLabs/Global-MMLU -
ollama-benchmark
Datasets & benchmarksLLM Benchmark for Throughput via Ollama (Local LLMs)
git clone https://github.com/aidatatools/ollama-benchmark.git