Goal Misgeneralization in Deep Reinforcement Learning — mentioned by diziet

Wed, 05 Apr 2023 00:08:46 GMT

Here is an example of a paper on inner alignment misalignment with a relatively simple level of complexity and even reward/punishment layers visualized. https://arxiv.org/pdf/2105.14111.pdf The authors show specific examples of goal misalignment. To say that we know how something works because we know the underlying principles is not correct. What is going on in a 150gb~ 65B parameter model (ie, llama 65B) of matrix multiplications is not just complexity, it is tough to say "we know how they work inside". We can explain backpropagation and transformers and RLHF principles. When you scale that…

Paper and discussion history. A mention is not an endorsement.

Goal Misgeneralization in Deep Reinforcement Learning — mentioned by philbert101

Tue, 12 Oct 2021 05:29:39 GMT

Links to articles https://distill.pub/2020/understanding-rl-vision/ https://arxiv.org/pdf/2105.14111.pdf

Paper and discussion history. A mention is not an endorsement.

Goal Misgeneralization in Deep Reinforcement Learning · HackerPapers discussions

Goal Misgeneralization in Deep Reinforcement Learning — mentioned by diziet

Goal Misgeneralization in Deep Reinforcement Learning — mentioned by philbert101