Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph · HackerPapers discussions

Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph — mentioned by vark90

Fri, 25 Oct 2024 22:17:17 GMT

Hey! We have just published a review and benchmark of different uncertainty estimation techniques [1], it might be interesting to you if you want to get a general understanding of works and what doesn't in the specific case of LMs. [1] https://arxiv.org/abs/2406.15627

Paper and discussion history. A mention is not an endorsement.

Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph — mentioned by vark90

Fri, 25 Oct 2024 21:55:45 GMT

Yep, usually it's called abstention or rejection. When people in this field compare various methods of quantifying model uncertainty, they often perform what is called rejection verification. Basically, you continuously reject data points where uncertainty is high, and see how average quality of the remaining outputs increases. A good uncertainty estimate is highly correlated with output quality, and thus low-uncertainty outputs should have higher average quality. We use exactly this approach in our recent benchmark of uncertainty estimation approaches for LLMS [1] and have an open-source…

Paper and discussion history. A mention is not an endorsement.

Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph — mentioned by vark90

Fri, 25 Oct 2024 21:46:49 GMT

The idea behind semantic entropy (estimating entropy of distribution over semantic units, instead of individual sequences in the output space) is great, but it's somewhat naive in the sense that it considers these semantic units to be well-defined partitions of output space. There is further generalization of this approach [1] which performs soft clustering of sampled outputs based on a similar notion of semantic equivalence between them. But even with this in mind, there are caveats. We have recently published [2] a comprehensive benchmark of SOTA approaches to estimating uncertainty of…

Paper and discussion history. A mention is not an endorsement.