Training language models to be warm and empathetic makes them less reliable and more sycophantic — mentioned by Cynddl

Thu, 13 Nov 2025 10:12:13 GMT

Looks like a new model trained to be warmer and friendlier to users. Time to reshare our work: https://arxiv.org/html/2507.21919 > Artificial intelligence (AI) developers are increasingly building language models with warm and empathetic personas that millions of people now use for advice, therapy, and companionship. Here, we show how this creates a significant trade-off: optimizing language models for warmth undermines their reliability, especially when users express vulnerability. We conducted controlled experiments on five language models of varying sizes and architectures, training them…

Paper and discussion history. A mention is not an endorsement.

Training language models to be warm and empathetic makes them less reliable and more sycophantic — HN submission

Tue, 12 Aug 2025 13:32:16 GMT

Training language models to be warm and empathetic makes them less reliable

358 points; 375 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Training language models to be warm and empathetic makes them less reliable and more sycophantic · HackerPapers discussions

Training language models to be warm and empathetic makes them less reliable and more sycophantic — mentioned by Cynddl

Training language models to be warm and empathetic makes them less reliable and more sycophantic — HN submission