datacurve-ai/deep-swe
Measuring frontier coding agents on original, long-horizon engineering tasks
View on GitHub →Where this sits in the catalog
Most-wanted open issues
Ranked by 👍 reactions. 85 open issues total, checked 2026-10-18.
| Issue | 👍 | Comments | Opened |
|---|---|---|---|
| 66 | 25 | 4 mo | |
| 30 | 0 | 4 mo | |
| 29 | 2 | 4 wk | |
| 23 | 2 | 4 mo | |
| 22 | 9 | 4 mo |
What Hacker News made of it
Threads that link to or name this project, ranked by points. The comments are where the objections live — the README never argues back.
- DeepSWE results are unreliable – 3/3 DSv4 "failed" tasks solved with same model
3 points · 0 comments · 4 mo
- DeepSWE Audit: DeepSeek-v4-pro results are unreliable
3 points · 0 comments · 4 mo
Head to head with its peers
The same six axes and the same star history, two projects at a time.
Alternatives in Evals & benchmarks
非线智能 NoneLinear - ReLE评测：中文AI大模型能力评测（持续更新）：目前已囊括374个大模型，覆盖chatgpt、gpt-5.4、谷歌gemini-3.1-pro、Claude-4.6、文心ERNIE-…
A model-driven harness for AI coding agents: semantic modeling, app generation, progressive API Assist and evi…
An open-source Harness Engineering platform for coding agents—define harnesses as code, run controlled experim…
Skills that guide AI coding agents to help you build product-specific AI evals.
Measuring how well CLI agents like Claude Code or Codex CLI can post-train base LLMs on a single H100 GPU in 1…