Finetuned Language Models Are Zero-Shot Learners · HackerPapers discussions

Finetuned Language Models Are Zero-Shot Learners — mentioned by radarsat1

Tue, 11 Jul 2023 20:27:45 GMT

It comes from this paper [0], and I believe the idea is that the LLM was not trained on the task in question, but is able to do it with only instructions (zero shot) or with one or a few examples (few shot). The paper rightfully points out the unexpected fact that the model is only trained to predict the next word and yet can follow arbitrary instructions and perform tasks that it was not explicitly trained to do. [0]: https://arxiv.org/abs/2109.01652

Paper and discussion history. A mention is not an endorsement.

Finetuned Language Models Are Zero-Shot Learners — mentioned by make3

Sun, 13 Nov 2022 20:31:15 GMT

https://arxiv.org/abs/2109.01652 https://arxiv.org/abs/2206.04615

Paper and discussion history. A mention is not an endorsement.

Finetuned Language Models Are Zero-Shot Learners — mentioned by joshuamorton

Mon, 02 May 2022 16:09:39 GMT

The controversy here is specifically around Stochastic Parrots, by among others Timnit Gebru and "Schmargaret Smitchell". Timnit had recently been fired by Google for a controversy that started when leadership prevented her and Margaret from being coauthors on the paper (which is critical of Google in some ways) and Mitchell was fired a few months later. Google[0] claims the paper didn't meet the bar for publication and ignored too much relevant research. The authors obviously dispute these claims (I generally side with the authors that the "ignored research" line is a stretch). The then…

Paper and discussion history. A mention is not an endorsement.