datacurve-ai/deep-swe
Measuring frontier coding agents on original, long-horizon engineering tasks
View on GitHub →Where this sits in the catalog
Most-wanted open issues
Ranked by 👍 reactions. 65 open issues total, checked 2026-08-30.
| Issue | 👍 | Comments | Opened |
|---|---|---|---|
| 66 | 25 | 3 mo | |
| 30 | 0 | 2 mo | |
| 23 | 2 | 3 mo | |
| 22 | 9 | 2 mo | |
| 16 | 3 | 3 mo |
What Hacker News made of it
Threads that link to or name this project, ranked by points. The comments are where the objections live — the README never argues back.
- DeepSWE results are unreliable – 3/3 DSv4 "failed" tasks solved with same model
3 points · 0 comments · 3 mo
- DeepSWE Audit: DeepSeek-v4-pro results are unreliable
3 points · 0 comments · 3 mo
Alternatives in Evals & benchmarks
非线智能 NoneLinear - ReLE评测：中文AI大模型能力评测（持续更新）：目前已囊括374个大模型，覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-…
A model-mediated harness for reliable agentic software development.
An open-source Harness Engineering platform for coding agents—define harnesses as code, run controlled experim…
Measuring how well CLI agents like Claude Code or Codex CLI can post-train base LLMs on a single H100 GPU in 1…
An open-source game coding agent environment and benchmark for Godot.