Rethinking Verification for LLM Code Generation: From Generation to Testing · HackerPapers discussions

Rethinking Verification for LLM Code Generation: From Generation to Testing — mentioned by troupo

Fri, 29 Aug 2025 12:29:59 GMT

> the extrapolation should have been straightforward, Indeed."By late next month you'll have over four dozen husbands" https://xkcd.com/605/ > So, here's some support for my second claim: I don't think any of these links support the claim that "No SOTA model was trained without verified reward training for math and programming" https://arxiv.org/abs/2507.06920: "We hope this work contributes to building a scalable foundation for reliable LLM code evaluation" https://arxiv.org/abs/2506.11425: A custom agent with a custom environment and a custom training dataset on ~800 predetermined problems.…

Paper and discussion history. A mention is not an endorsement.

Rethinking Verification for LLM Code Generation: From Generation to Testing — mentioned by khafra

Fri, 29 Aug 2025 07:42:31 GMT

My two claims: 1. OpenAI has been doing verifier-guided training since last year. 2. No SOTA model was trained without verified reward training for math and programming. I supported the first claim with a document describing what OpenAI was doing last year; the extrapolation should have been straightforward, but it's easy for people who aren't tracking AI progress to underestimate the rate at which it occurs. So, here's some support for my second claim: https://arxiv.org/abs/2507.06920 https://arxiv.org/abs/2506.11425 https://arxiv.org/abs/2502.06807

Paper and discussion history. A mention is not an endorsement.