1 Comment
User's avatar
Ruby slippers's avatar

SWE-bench is not a good benchmark. It's been contaminated as well. DeepSWE is much better, although it hasn't been run on a lot of models yet.