Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents · HackerPapers discussions

Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents — mentioned by gregwebs

Sun, 04 Oct 2026 12:39:03 GMT

The value of TDD is it does ensure that tests are written and that will help with future regressions. However, I found the same problems with agents that it has with humans (usually with humans it isn't TDD but code coverage requirements). The problem is the tests are now written to satisfy a bureaucracy rather than to properly verify the code. And there are real studies that point out the low value of these types of bureaucratic testing requirements. For example, when models are just required to write a test file, they don't produce a better result: https://arxiv.org/abs/2602.07900 I wrote a…

Paper and discussion history. A mention is not an endorsement.

Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents — mentioned by zuzululu

Thu, 25 Jun 2026 15:30:02 GMT

yup and I find it weird that people still remain so defensive of the Church of TDD even against empirical studies that show its limited benefits https://arxiv.org/abs/2602.07900

Paper and discussion history. A mention is not an endorsement.

Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents — mentioned by rsalus

Sat, 06 Jun 2026 18:31:25 GMT

for LLMs, TDD amounts to little more than ceremony. there is a study on this exact topic: https://arxiv.org/pdf/2602.07900

Paper and discussion history. A mention is not an endorsement.

Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents — HN submission

Sat, 06 Jun 2026 01:39:40 GMT

Rethinking the Value of Generated Tests for LLM Software Engineering Agents

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Rethinking the Value of Agent-Generated Tests for LLM-Based Software Engineering Agents — mentioned by rsalus

Fri, 05 Jun 2026 23:24:47 GMT

I was a big proponent of encoding TDD red-green-refactor methodology into my agent workflows until recently when I made the same realization after reading this study: https://arxiv.org/pdf/2602.07900 TLDR; it found test-writing volume only weakly correlates with success and that encoding test-writing principles did not move resolution rates but _did_ materially change cost. Encouraging tests cost +19.8% output tokens for 0% gain; discouraging them saved 33–49% input tokens for ≤2.6pp accuracy loss. Separately, imposing the TDD procedure specifically seems like it can backfire: it actually…

Paper and discussion history. A mention is not an endorsement.