On Calibration of Modern Neural Networks · HackerPapers discussions

On Calibration of Modern Neural Networks — mentioned by andy99

Mon, 05 Oct 2026 11:58:15 GMT

Softmax over logits doesn’t give calibrated probabilities either as a rule. I don’t want to comment specifically on Jev but as a rule it’s very hard to get good calibration because it’s somewhat in tension with minimizing training loss for neural networks, e.g. Guo et al (2017) https://arxiv.org/abs/1706.04599 While I know there are ways to improve calibration, I’d personally want to see a lot of evidence the probabilities were actually more meaningful before trusting them. I agree of course that asking an LLM to provide a confidence estimate is meaningless.

Paper and discussion history. A mention is not an endorsement.

On Calibration of Modern Neural Networks — mentioned by ainch

Wed, 23 Sep 2026 08:28:04 GMT

In my experience as well using logprobs to try to quantify uncertainty, LLMs are a poor fit. Neural nets in general struggle with 'calibration' --- ie. if a prediction is truly 50/50, neural nets are often prone to predicting overconfidently [0]. I ran some tests using GPT-4 to do some basic classification a couple years ago. On ambiguous options which had to be escalated to a human, the LLM would regularly output something like a 99.8% probability, compared to 99.99% for a correct answer. 0: https://arxiv.org/pdf/1706.04599

Paper and discussion history. A mention is not an endorsement.

On Calibration of Modern Neural Networks — mentioned by minihat

Wed, 06 Sep 2023 04:18:12 GMT

Cross-entropy loss can start getting worse due to the model becoming less calibrated, even as the classification accuracy continues to improve. I first heard that here: https://arxiv.org/abs/1706.04599 Is this 'overconfidence' the leading explanation as to why LLMs continue to show qualitative improvement even after their test loss levels off?

Paper and discussion history. A mention is not an endorsement.

On Calibration of Modern Neural Networks — mentioned by version_five

Mon, 23 Jan 2023 00:31:32 GMT

Overconfidence has always been a big problem with deep learning. And in fact, better calibration ends up leading to worse overall performance. So ML models are incentivized to be overconfident. To be fair, the same is true of people. See e.g Guo et al, but don't trust their advice about how to improve calibration https://arxiv.org/abs/1706.04599

Paper and discussion history. A mention is not an endorsement.

On Calibration of Modern Neural Networks — mentioned by andy99

Fri, 23 Apr 2021 00:25:25 GMT

The concept of calibration - actually being right at the rate that matches your estimated percent confidence - is important for machine learning. It's important to know how well you can trust the prediction of a ML model (ironically to that you can do something like send low confidence predictions for manual review, which apparently might not be that calibrated either)... Deep learning models in particular are not good at accurately estimating confidence, so there is a whole body of work on getting better confidence estimates, e.g. [0]. It's nice to see that people make the same kind of…

Paper and discussion history. A mention is not an endorsement.

On Calibration of Modern Neural Networks — mentioned by Eruditass

Thu, 12 Apr 2018 19:48:19 GMT

Have you actually worked on applying neural networks in the real world? Calibrating the probabilities of machine learning algorithms is an old problem. By nature of disciminative algorithms and increasing model capacities, yes training typically pushes outputs to one extreme. There is a ton of information still maintained which can be properly calibrated for downstream ingestion, which anyone actually trying to integrate these into actual applications should be doing. Most recently: https://arxiv.org/abs/1706.04599 https://arxiv.org/abs/1802.03916

Paper and discussion history. A mention is not an endorsement.