Open, Closed, or Small Language Models for Text Classification? — mentioned by miven

Sat, 02 Dec 2023 19:54:21 GMT

Well, LLMs don't consistently beat purpose-trained BERT-type models just yet, for example in [1] the authors show RoBERTa (fine-tuned for each individual task separately of course) going basically toe-to-toe with GPT-4 on quite a few somewhat conventional NLP tasks, while open-source LLaMA 2 models are getting severely outclassed. [1] https://arxiv.org/abs/2308.10092

Paper and discussion history. A mention is not an endorsement.

Open, Closed, or Small Language Models for Text Classification? — mentioned by syllogism

Sat, 09 Sep 2023 10:43:44 GMT

(Original author of spaCy and Explosion CTO here) Okay so, first some terminology. LLMs can mean a bunch of different things, people call models the size of BERT LLMs sometimes. So let's talk specifically about in-context learning (ICL) with either zero or a few examples. So we'll say LLM ICL, and contrast that with techniques where you annotate enough data to train with, which might only be something like 10-40 hours of annotation. The something you do with that data is probably training a task-specific classification model initialised with weights from a language modelling objective. This…

Paper and discussion history. A mention is not an endorsement.

Open, Closed, or Small Language Models for Text Classification? · HackerPapers discussions

Open, Closed, or Small Language Models for Text Classification? — mentioned by miven

Open, Closed, or Small Language Models for Text Classification? — mentioned by syllogism