swe-sweep
How many bugs can LMs find & fix in large codebases?
Open the source (opens github.com) https://github.com/facebookresearch/swe-sweep
What it is
Datasets & benchmarks from facebookresearch.
The problem it solves and who it suits haven't been reviewed by InnerVault. The description above is the source's own.
How to get it
Get the code
git clone https://github.com/facebookresearch/swe-sweep.git
Standard git access to a public repository
Setup steps: not provided by the source. Follow the instructions on the source.
Related
-
ClawBench
Datasets & benchmarksOpen-source benchmark for browser AI agents on daily tasks.
git clone https://github.com/TIGER-AI-Lab/ClawBench.git -
WebsiteBench
Datasets & benchmarksCan AI agents rebuild websites through browser exploration?
git clone https://github.com/TIGER-AI-Lab/WebsiteBench.git -
explicit-edit-benchmark
Datasets & benchmarksBenchmark coding agents and agent harnesses on 226 deterministic file-editing tasks with byte-exact verification and a public leaderboard.
git clone https://github.com/alexshpunt/explicit-edit-benchmark.git -
AIRTBench-Code
Datasets & benchmarksCode Repository for: AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models
git clone https://github.com/dreadnode/AIRTBench-Code.git -
aa-agentperf-local
Datasets & benchmarksBenchmark local LLM serving by replaying real agent trajectories
git clone https://github.com/ArtificialAnalysis/aa-agentperf-local.git -
ParseBench
Datasets & benchmarksParseBench - A Document Parsing Benchmark for AI Agents
git clone https://github.com/run-llama/ParseBench.git