Measuring Progress on Scalable Oversight for Large Language Models — mentioned by agucova

Sat, 18 May 2024 23:34:42 GMT

For context, the point of the Superalignment team was to work on a problem known as scalable oversight: the problem of aligning models in a way that holds up as models become more capable [1]. The reason behind this is that current alignment techniques (like RLHF), have limitations which are expected to worsen as models are scaled up [2]. This is to say, the objective of the Superalignment team was precisely to work on techniques that would work for models which don't yet exist. They are of course aware that they don't yet have superintelligence. [1]: This paper by Anthropic is a good…

Paper and discussion history. A mention is not an endorsement.

Measuring Progress on Scalable Oversight for Large Language Models — mentioned by nmca

Fri, 05 May 2023 10:29:11 GMT

It's important that humans are still able to understand the actions of these systems for safety reasons; the area focused on the problem you identify is "scalable oversight", see for example: https://arxiv.org/abs/2206.05802 https://arxiv.org/abs/2211.03540

Paper and discussion history. A mention is not an endorsement.

Measuring Progress on Scalable Oversight for Large Language Models · HackerPapers discussions

Measuring Progress on Scalable Oversight for Large Language Models — mentioned by agucova

Measuring Progress on Scalable Oversight for Large Language Models — mentioned by nmca