It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners · HackerPapers discussions

It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners — HN submission

Mon, 08 Mar 2021 23:43:29 GMT

It's Not Just Size That Matters Small Language Models Are Also Few-Shot Learners

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners — mentioned by Veedrac

Wed, 07 Oct 2020 19:31:14 GMT

The results reported there are what most people would call ‘semi-supervised learning’, not ‘few-shot’. The true few-shot results are in a few places in the paper, https://arxiv.org/abs/2009.07118, labeled with ‘- dist’.

Paper and discussion history. A mention is not an endorsement.

It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners — mentioned by m3at

Wed, 07 Oct 2020 04:51:04 GMT

If you want to read the source, I recommend skipping to their follow-up paper (same authors): https://arxiv.org/abs/2009.07118 Edit: tangentially related but for those who like to have a glance from their phone, arxiv-vanity is great instead of squinting at a pdf: https://www.arxiv-vanity.com/papers/2009.07118/

Paper and discussion history. A mention is not an endorsement.

It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners — HN submission

Thu, 24 Sep 2020 08:10:46 GMT

It's Not Just Size That Matters: Small Models with Performance Similar to GPT-3

9 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners — mentioned by polymorph1sm

Thu, 24 Sep 2020 06:49:14 GMT

I recently found this paper[1] claiming near GPT-3 performance with only a fraction of parameters. They seems to simply reformulate the input sequence to change classification to a sequence generation task. Disclaimer, I am not affiliated to any of the authors [1] https://arxiv.org/pdf/2009.07118.pdf

Paper and discussion history. A mention is not an endorsement.

It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners — mentioned by andreyk

Tue, 22 Sep 2020 21:52:03 GMT

Anyone else feel like this idea of commercializing GPT-3 is bound to go nowhere as the research community figures out how to replicate the same capabilities in smaller cheaper open models within a few months or even a year? Not to mention, what commercial applications actually require the model to be so few-shot? (see eg this recent paper that achieves similar results with a bit more data https://arxiv.org/pdf/2009.07118.pdf)

Paper and discussion history. A mention is not an endorsement.

It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners — HN submission

Tue, 22 Sep 2020 07:23:22 GMT

It's Not Just Size That Matters:Small Language Models Are Also Few-Shot Learners

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.