fde-gym
The online judge for the LLM era. Every case is a real customer delivery: you work it with a coding agent, and your system is judged once it is live.
Open the source (opens github.com) https://github.com/deeplethe/fde-gym
What it is
Datasets & benchmarks from deeplethe.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/deeplethe/fde-gym.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
self-bench
Datasets & benchmarksBenchmark coding agents and models on your own repository with private Harbor-formatted evals built from its merged pull requests.
git clone https://github.com/mupt-ai/self-bench.git -
FDE-Gym
Datasets & benchmarksHugging Face dataset by DeepLethe
DeepLethe/FDE-Gym -
LoopArena
Datasets & benchmarksBenchmarking models as runtime Controllers for Loop Engineering
git clone https://github.com/AMAP-ML/LoopArena.git -
ClawBench
Datasets & benchmarksOpen-source benchmark for browser AI agents on daily tasks.
git clone https://github.com/TIGER-AI-Lab/ClawBench.git -
WebsiteBench
Datasets & benchmarksCan AI agents rebuild websites through browser exploration?
git clone https://github.com/TIGER-AI-Lab/WebsiteBench.git -
nautilus-compass
Datasets & benchmarksOpen memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
git clone https://github.com/chunxiaoxx/nautilus-compass.git