On Verbalized Confidence Scores for LLMs — mentioned by red75prime

Fri, 05 Dec 2025 11:12:51 GMT

Naturally, their "confidence" is represented as activations in layers close to output, so they might be able to use it. Research ([0], [1], [2], [3]) shows that results of prompting LLMs to express their confidence correlate with their accuracy. The models tend to be overconfident, but in my anecdotal experience the latest models are passably good at judging their own confidence. [0] https://ieeexplore.ieee.org/abstract/document/10832237 [1] https://arxiv.org/abs/2412.14737 [2] https://arxiv.org/abs/2509.25532 [3] https://arxiv.org/abs/2510.10913

Paper and discussion history. A mention is not an endorsement.

On Verbalized Confidence Scores for LLMs — mentioned by ogrisel

Tue, 11 Mar 2025 10:17:44 GMT

According to the following paper, it's possible to get calibrated confidence scores by directly asking the LLM to verbalize a confidence level, but it strongly depends on how you prompt it to do so: https://arxiv.org/abs/2412.14737

Paper and discussion history. A mention is not an endorsement.

On Verbalized Confidence Scores for LLMs · HackerPapers discussions

On Verbalized Confidence Scores for LLMs — mentioned by red75prime

On Verbalized Confidence Scores for LLMs — mentioned by ogrisel