Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One · HackerPapers discussions

Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One — mentioned by maurits

Thu, 20 Feb 2025 13:07:15 GMT

For people interested in the softmax, log sum exp and energy models, have a look at "Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One" [1] [1]: https://arxiv.org/abs/1912.03263

Paper and discussion history. A mention is not an endorsement.

Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One — mentioned by mpmisko

Sat, 17 Aug 2024 00:06:21 GMT

EBMs show up all over the place, apparently even your classifier is an EBM :) (https://arxiv.org/abs/1912.03263).

Paper and discussion history. A mention is not an endorsement.

Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One — mentioned by godelski

Wed, 07 Aug 2024 21:21:25 GMT

Thanks. I am passionate haha. I do like to educate, but it is often difficult to do on these forms as I do not know peoples backgrounds. If you throw too high level at them, I find it does more harm than good. I also find that while I know many here took things like calculus, that we also need to recognize that skills degrade with time (though can be regained more easily). I do understand that it is difficult to find the math pathways in ML. What I do suggest is focusing on generative models[0], but the important part is to keep asking "why" and "what does this actually mean?" I think it is…

Paper and discussion history. A mention is not an endorsement.

Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One — mentioned by godelski

Tue, 06 Aug 2024 08:26:48 GMT

> The rule in ML There's definitely attempts to revive things (in the general sense, not just GANs), but most successes appear to come from large labs dedicating equal computational resources to the older models and often by changing names. This can make things more confusing and make things appear to be changing faster, but once you can see this, you'll have an easier time keeping up (so being new, watch out for this). I'll give some examples that are easier to read[0-2] (i.e. don't need expert knowledge to understand the nuances). As an insider (ML researcher), my complaint isn't so much…

Paper and discussion history. A mention is not an endorsement.

Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One — mentioned by rubatuga

Fri, 05 Mar 2021 04:33:12 GMT

But BERT isn't a generative model? Unless you mean it's trained similar to a generative model by predicting the missing tokens? Also an interesting paper about hybrid energy based models: https://arxiv.org/pdf/1912.03263.pdf It's possible to have a model be both a classifier + generative, and the performance is comparable (although lower) to the SOTA of single-purpose models.

Paper and discussion history. A mention is not an endorsement.

Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One — mentioned by blackcat201

Thu, 17 Sep 2020 04:21:23 GMT

For neural networks unless it's calibrated otherwise the confidence score has little meaning to the accuracy. [1] see figure 4 and you will understand what I mean [1] https://arxiv.org/pdf/1912.03263.pdf

Paper and discussion history. A mention is not an endorsement.

Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One — mentioned by rjeli

Thu, 03 Sep 2020 22:28:55 GMT

Reminds me of JEMs from Grathwohl et al last year [0]. They train a generative model by treating it an an Energy Based Model using stochastic langevin gradients. I’m curious as to how it relates to the models in this post. [0] https://arxiv.org/abs/1912.03263

Paper and discussion history. A mention is not an endorsement.