Deep reinforcement learning from human preferences · HackerPapers discussions

Deep reinforcement learning from human preferences — mentioned by reducesuffering

Sat, 12 Sep 2026 21:05:33 GMT

This thread is a testament to the travesty of how bad HN discourse has gotten. It's become an echo chamber of knee-jerk cynical sneering. All the heads of the labs (Sama, Dario, Demis Hassabis, Ilya, Musk) agree with the risks to humanity's existence. Most of the core employees agree. Yoshua Bengio, Geoffrey Hinton, Bernie Sanders agree. These issues have been discussed in more intelligent channels for a decade now, and still after all the incredible mathematics and coding progress that is improving incredibly fast every 6 months, this forum is fundamentally unserious. You will not find…

Paper and discussion history. A mention is not an endorsement.

Deep reinforcement learning from human preferences — mentioned by godelski

Mon, 11 Aug 2025 01:11:29 GMT

Jack has a lot of really bad takes and frequently makes lots of mistakes. Honestly, I don't know why people take him seriously. I mean you can go read his blog post that's pinned where he argues that there's no new ideas and it is all data. He makes the argument that architecture doesn't matter, which is just so demonstrably false that it is laughable. He's a scale maximalist. I also expect an AI researcher from a top university to not make such wild mistakes > 3. RLHF: first proposed (to my knowledge) in the InstructGPT paper from OpenAI in 2022 I mean if you go read the instruct paper on…

Paper and discussion history. A mention is not an endorsement.

Deep reinforcement learning from human preferences — mentioned by p1esk

Wed, 21 Jun 2023 18:45:22 GMT

Original RLHF paper: https://arxiv.org/abs/1706.03741

Paper and discussion history. A mention is not an endorsement.

Deep reinforcement learning from human preferences — mentioned by jefftk

Tue, 14 Mar 2023 12:51:01 GMT

Reinforcement learning in general didn't come out of AI alignment work, but RHLF in particular did. The initial idea and paper [1] were from AI alignment folks, as was most of the later development [2][3][4][5]. Overview: https://www.alignmentforum.org/posts/vwu4kegAEZTBtpT6p/thoug... [1] Deep Reinforcement Learning from Human Preferences https://arxiv.org/abs/1706.03741 [2] Fine-Tuning Language Models from Human Preferences https://arxiv.org/abs/1909.08593 [3] Learning to summarize from human feedback https://arxiv.org/abs/2009.01325 [4] Recursively Summarizing Books with Human Feedback…

Paper and discussion history. A mention is not an endorsement.

Deep reinforcement learning from human preferences — mentioned by sanxiyn

Sat, 19 Oct 2019 17:18:04 GMT

The obvious answer is to learn the metrics itself too. OpenAI does interesting work in this area: "Deep reinforcement learning from human preferences", for example. https://arxiv.org/abs/1706.03741

Paper and discussion history. A mention is not an endorsement.

Deep reinforcement learning from human preferences — mentioned by sgillen

Thu, 07 Mar 2019 01:10:10 GMT

Thanks for your response, I had actually heard about Yuri before and was hoping you would respond to this post! I have a lot of questions for you if you don't mind, maybe you don't want to answer any/all of them that's fine. 1. How do you handle constraining the AI such that it's enjoyable? If you read through the comments here you'll notice just about everyone is saying the hard part is not making a good AI but in making a fun AI. Do you know of any research in this area? most examples I see are all about making the best AI possible. It seems like it's really hard to find a good reward…

Paper and discussion history. A mention is not an endorsement.

Deep reinforcement learning from human preferences — mentioned by gwern

Tue, 31 Jul 2018 17:19:14 GMT

> Qualitative items such as the ones listed are important for customer focused environments, however, I'm not sure if AI can account for such factors. One of the nice things you can do with optimization problems is plug humans into the loop as oracles. Often, 'we know it when we see it', and we can do pairwise comparisons of 2 possibilities. So you can train a ML model based on win/loss comparisons and it'll learn to take into account the softer qualitative aspects via preference learning. A recent example you might remember from the press: "Deep reinforcement learning from human preferences"…

Paper and discussion history. A mention is not an endorsement.

Deep reinforcement learning from human preferences — HN submission

Fri, 23 Feb 2018 18:00:07 GMT

Deep reinforcement learning from human preferences

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.