Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate — mentioned by Vetch

Fri, 07 Feb 2025 08:21:01 GMT

The essence of the article is that self-correction exists as a nascent ability in base models already (more robustly in some like Qwen than others). This is highly reminiscent of Chain of Thought, which was found to be a capability already present in base models too. The result of RL is to reinforce already present authentic self-correction patterns and down weight superficial self-correction. Thoughts: - An analogy you shouldn't zoom too close into is going from CoT to reasoning traces is like going from purely ballistic trajectories to including navigation and thrusters. RL is for learning…

Paper and discussion history. A mention is not an endorsement.

Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate · HackerPapers discussions

Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate — mentioned by Vetch