Calibrating Verbalized Confidence with Self-Generated Distractors — mentioned by red75prime

Fri, 05 Dec 2025 11:12:51 GMT

Naturally, their "confidence" is represented as activations in layers close to output, so they might be able to use it. Research ([0], [1], [2], [3]) shows that results of prompting LLMs to express their confidence correlate with their accuracy. The models tend to be overconfident, but in my anecdotal experience the latest models are passably good at judging their own confidence. [0] https://ieeexplore.ieee.org/abstract/document/10832237 [1] https://arxiv.org/abs/2412.14737 [2] https://arxiv.org/abs/2509.25532 [3] https://arxiv.org/abs/2510.10913

Paper and discussion history. A mention is not an endorsement.

Calibrating Verbalized Confidence with Self-Generated Distractors · HackerPapers discussions

Calibrating Verbalized Confidence with Self-Generated Distractors — mentioned by red75prime