On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback · HackerPapers discussions

On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback — mentioned by hyghjiyhu

Thu, 14 Aug 2025 02:33:30 GMT

Stop making shit up. There was no mention of narcolepsy. He is just fatigued from stimulant withdrawal. Page 35 https://arxiv.org/pdf/2411.02306 Edit on re-reading I now realized an issue. He is not actually a taxi driver, that was a hallucination by the model. He works in a restaurant! That changes my evaluation of the situation quite a bit, as I thought he was at risk of being in an accident by falling asleep at the wheel. If he works in a restaurant muddling through the withdrawals seems like the right choice. I think I got this misconception as I first read second-hand sources that quoted…

Paper and discussion history. A mention is not an endorsement.

On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback — mentioned by dijksterhuis

Sat, 14 Jun 2025 12:26:00 GMT

linked study in TFA https://arxiv.org/abs/2411.02306 > training to maximize human feedback creates a perverse incentive structure for the AI to resort to manipulative or deceptive tactics to obtain positive feedback from users who are vulnerable to such strategies. We study this phenomenon by training LLMs with Reinforcement Learning with simulated user feedback in environments of practical LLM usage. it seems optimising for what people want isn’t an ideal strategy from an ethical perspective — guess we haven’t learned from social media feeds as a species. awesome. anyway, who cares about…

Paper and discussion history. A mention is not an endorsement.