Phare: A Safety Probe for Large Language Models — HN submission

Wed, 21 May 2025 10:08:05 GMT

Show HN: Phare: A Safety Probe for Large Language Models

We've just published a benchmark and accompanying paper on arXiv that challenges conventional leaderboard-driven LLM evaluation. Phare focuses on factual reliability, prompt sensitivity, multilingual support, and how models handle false premises like issues that actually matter when you're building serious applications. Some insights: - Preference scores ≠ factual correctness. - Framing effects can cause models to miss obvious falsehoods. - Safety metrics like sycophancy and stereotype reproduction show surprising results across popular models. Would love feedback from the community.

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Phare: A Safety Probe for Large Language Models · HackerPapers discussions

Phare: A Safety Probe for Large Language Models — HN submission