Reinforcement Learning from Reflective Feedback (RLRF): Aligning and Improving LLMs via Fine-Grained Self-Reflection · HackerPapers discussions

Reinforcement Learning from Reflective Feedback (RLRF): Aligning and Improving LLMs via Fine-Grained Self-Reflection — mentioned by lossolo

Mon, 30 Jun 2025 19:33:12 GMT

I wrote about it around a year ago here: "There weren't really any advancements from around 2018. The majority of the 'advancements' were in the amount of parameters, training data, and its applications. What was the GPT-3 to ChatGPT transition? It involved fine-tuning, using specifically crafted training data. What changed from GPT-3 to GPT-4? It was the increase in the number of parameters, improved training data, and the addition of another modality. From GPT-4 to GPT-40? There was more optimization and the introduction of a new modality. The only thing left that could further improve…

Paper and discussion history. A mention is not an endorsement.

Reinforcement Learning from Reflective Feedback (RLRF): Aligning and Improving LLMs via Fine-Grained Self-Reflection — mentioned by lossolo

Fri, 13 Sep 2024 13:36:39 GMT

O1 seems like a variant of RLRF https://arxiv.org/abs/2403.14238

Paper and discussion history. A mention is not an endorsement.

Reinforcement Learning from Reflective Feedback (RLRF): Aligning and Improving LLMs via Fine-Grained Self-Reflection — mentioned by lossolo

Thu, 12 Sep 2024 21:58:39 GMT

It's because there is nothing novel here from an architectural point of view. Again, the secret sauce is only in the training data. O1 seems like a variant of RLRF https://arxiv.org/abs/2403.14238 Soon you will see similar models from competitors.

Paper and discussion history. A mention is not an endorsement.