Simreal-MLBench
[Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluation.
Open the source (opens github.com) https://github.com/simreal-ai/simreal-mlbench
What it is
Datasets & benchmarks from Simreal-AI.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/Simreal-AI/Simreal-MLBench.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
confabulations
Datasets & benchmarksHallucinations (Confabulations) Document-Based Benchmark for RAG. Includes human-verified questions and answers.
git clone https://github.com/lechmazur/confabulations.git -
TheAgentCompany
Datasets & benchmarksAn agent benchmark with tasks in a simulated software company.
git clone https://github.com/TheAgentCompany/TheAgentCompany.git -
swe-sweep
Datasets & benchmarksHow many bugs can LMs find & fix in large codebases?
git clone https://github.com/facebookresearch/swe-sweep.git -
agents-last-exam
Datasets & benchmarksHugging Face dataset by agents-last-exam
agents-last-exam/agents-last-exam -
WebsiteBench
Datasets & benchmarksCan AI agents rebuild websites through browser exploration?
git clone https://github.com/TIGER-AI-Lab/WebsiteBench.git -
LLM-Agent-Benchmark-List
Datasets & benchmarksA banchmark list for evaluation of large language models.
git clone https://github.com/zhangxjohn/LLM-Agent-Benchmark-List.git