Universal Language Model Fine-tuning for Text Classification · HackerPapers discussions

Universal Language Model Fine-tuning for Text Classification — HN submission

Sun, 30 Mar 2025 05:08:06 GMT

The algo that advanced natural language processing and influenced OpenAI's LLMs

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Universal Language Model Fine-tuning for Text Classification — mentioned by AlexanderDhoore

Sat, 18 Nov 2023 08:14:20 GMT

"That guy" has a pretty good idea when it comes to NLP https://arxiv.org/abs/1801.06146

Paper and discussion history. A mention is not an endorsement.

Universal Language Model Fine-tuning for Text Classification — mentioned by yunusabd

Tue, 07 Jul 2020 16:49:24 GMT

I agree with the other posters that the intro to NLP part is unnecessary. It reads like those recipe websites where they tell you their whole life story before the actual recipe. I get that it's good for SEO, but it's still annoying to read. Did you try other solutions like ULMFiT [1]? Seems like the exact use case for that. Although it might be overkill for just 4 categories. [1] https://arxiv.org/abs/1801.06146

Paper and discussion history. A mention is not an endorsement.

Universal Language Model Fine-tuning for Text Classification — mentioned by phowon

Mon, 04 Feb 2019 20:45:25 GMT

ULMFiT is LSTM-based. https://arxiv.org/pdf/1801.06146.pdf

Paper and discussion history. A mention is not an endorsement.

Universal Language Model Fine-tuning for Text Classification — mentioned by cs702

Fri, 12 Oct 2018 01:45:45 GMT

These guys have done what looks like an impressive amount of work... but I was disappointed to see there's no mention of generative language modeling at all and only a very brief mention of key advances in image generation over the past few years. Examples of generative language modeling ignored by this paper (not even mentioned): * Unsupervised Transformer - https://blog.openai.com/language-unsupervised/ * ELMo - https://arxiv.org/abs/1802.05365 * ULMFit - https://arxiv.org/abs/1801.06146 Examples of generative image modeling ignored by this paper (some are mentioned only in passing): * Glow…

Paper and discussion history. A mention is not an endorsement.

Universal Language Model Fine-tuning for Text Classification — mentioned by cs702

Mon, 09 Jul 2018 18:01:32 GMT

For those who don't know, Sebastian Ruder is a coauthor of the ULMFiT paper: https://arxiv.org/abs/1801.06146

Paper and discussion history. A mention is not an endorsement.

Universal Language Model Fine-tuning for Text Classification — mentioned by cs702

Mon, 11 Jun 2018 19:09:35 GMT

This is fabulous, great work, with broad applicability. Train this transformer model on a good amount of text (or grab a pretrained model), and then, with minimal fuss and very little tweaking, you can repurpose it to obtain state-of-the-art (or near state-of-the-art) results in a wide range of tasks, from document classification to textual entailment to semantic similarity. This stands in contrast to prior approaches that involve much more tweaking and/or careful discriminative finetuning of the pretrained model, such as as Jeremy Howard and Sebastian Ruder's also-impressive ULMFit.[a] The…

Paper and discussion history. A mention is not an endorsement.

Universal Language Model Fine-tuning for Text Classification — mentioned by rsmith49

Tue, 29 May 2018 23:22:45 GMT

There is definitely a feedback loop at play in the NLP community. Since most NLP research is based on English, most applications that build off that research are also tailored to English. Our approach to classifying feedback in other languages is to simply use Google Translate's API and then proceed to classify the English translation. However, you are right in that many aspects of the base feedback are lost in this approach, and our accuracy is lower than if we had a language specific model for each language. That being said, there is a promising new research paper from fast.ai…

Paper and discussion history. A mention is not an endorsement.

Universal Language Model Fine-tuning for Text Classification — mentioned by dreaminvm

Tue, 15 May 2018 18:55:36 GMT

Link to paper for those that are curious: https://arxiv.org/abs/1801.06146

Paper and discussion history. A mention is not an endorsement.

Universal Language Model Fine-tuning for Text Classification — mentioned by laughingman2

Sat, 24 Feb 2018 02:34:02 GMT

Yes, recent work on transfer learning https://arxiv.org/abs/1801.06146 " Fine-tuned Language Models for Text Classification" suggests that training a lstm on a large corpora for language modeling, then using it as sentence encoder after tuning it in specific dataset gives a considerable perfomance. This is covered in fast.ai 's new course 1.

Paper and discussion history. A mention is not an endorsement.

Universal Language Model Fine-tuning for Text Classification — HN submission

Mon, 22 Jan 2018 00:42:54 GMT

Fine-Tuned Language Models for Text Classification [pdf]

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Universal Language Model Fine-tuning for Text Classification — HN submission

Fri, 19 Jan 2018 10:18:22 GMT

Language Models for Classification (state of art on everything)

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.