Uncertainty in Neural Networks: Approximately Bayesian Ensembling · HackerPapers discussions

Uncertainty in Neural Networks: Approximately Bayesian Ensembling — mentioned by steppi

Sat, 17 Jun 2023 12:52:10 GMT

I’m not sure if you are aware, but Bayesian neural networks can be actually be well approximated by appropriate ensembles of standard neural networks [0]. The strength of Bayesian nets (including the approximating ensembles) is that they are able to estimate the uncertainty in their own predictions (by generating a probability distribution of possible predictions), at the cost of more computation needed for training and inference. I don’t think it’s ever going to be a matter of Bayesian nets outright outcompeting standard nets though, it’s just another tool in the toolbox if you want a model…

Paper and discussion history. A mention is not an endorsement.

Uncertainty in Neural Networks: Approximately Bayesian Ensembling — mentioned by datastoat

Sun, 15 Jan 2023 21:05:28 GMT

Indeed, there are Bayesian neural networks and there are non-Bayesian neural networks, and I shouldn't have implied that all neural networks are non-Bayesian. I'm just trying to point out that there is a dichotomy between the Bayesian and the non-Bayesian, and that the standard neural network models are non-Bayesian, and that we need Bayesianism (or something like it) to talk about (epistemic) uncertainty. Standard neural networks are non-Bayesian, because they do not treat the neural network parameters as random variables. This includes most of the examples that have been mentioned in this…

Paper and discussion history. A mention is not an endorsement.