Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2312.08935/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTMath-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations — mentioned by dimitry12https://news.ycombinator.com/item?id=42475263urn:hackerpapers:mention:2312.08935:42475263Fri, 20 Dec 2024 21:32:38 GMT<p>Verifier is trained with soft values of reward-to-go for each solution-prefix, obtained from monte-carlo rollouts of step-by-step solutions sampled from the &quot;base&quot; model. In other words: 1) sample step-by-step solutions from &quot;base&quot; model; 2) do it at non-zero temperature so that you can get multiple continuation from each solution-prefix; 3) use MATH-labels to decide if full solution (leaf/terminal node in MC rolloout) has reward `1` or `0`; 4) roll up these rewards to calculate reward-to-go for each intermediate step. Yes, verifier trained in this manner can be used to score…</p><p><a href="https://hackerpaper.kaipfister.de/p/2312.08935/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.AI