DeepSWE: A Contamination-Resistant Coding Benchmark — James Shi, Datacurve
DeepSWE is a contamination-resistant benchmark featuring 113 original software engineering tasks. It tests LLMs on long-horizon problems by focusing on observable behavior rather than PR-derived implementations, revealing critical gaps in model capabilities.
Read article