ProgramBench: Can Language Models Rebuild Programs From Scratch? · HackerPapers discussions

ProgramBench: Can Language Models Rebuild Programs From Scratch? — mentioned by westurner

Sat, 31 Oct 2026 00:27:17 GMT

ScholarlyArticle: "ProgramBench: Can Language Models Rebuild Programs From Scratch?" (2026) https://arxiv.org/abs/2605.03546 Is it that agents don't do the human-in-the-loop engineering management tasks like refactoring and docs and stopping to push for branch coverage before technical debt accrues? How many tokens per insufficient solution?

Paper and discussion history. A mention is not an endorsement.

ProgramBench: Can Language Models Rebuild Programs From Scratch? — mentioned by adrian_b

Tue, 12 May 2026 22:15:08 GMT

I have found it: https://news.ycombinator.com/item?id=48045174 The study paper: https://arxiv.org/abs/2605.03546 Look at Table 3, where the cheating rates of Claude Sonnet, Claude Opus and Gemini were between 20% and 36%, during the coding benchmarks.

Paper and discussion history. A mention is not an endorsement.

ProgramBench: Can Language Models Rebuild Programs From Scratch? — HN submission

Thu, 07 May 2026 03:46:15 GMT

ProgramBench: Can language models rebuild programs from scratch?

150 points; 80 discussion replies on HN. Paper and source history. A submission is not an endorsement.