What learning algorithm is in-context learning? Investigations with linear models · HackerPapers discussions

What learning algorithm is in-context learning? Investigations with linear models — mentioned by tehf0x

Sat, 08 Apr 2023 01:25:10 GMT

Ah the good old "it's not me it's the test" argument. These systems are not just next token predictors, they learn complex algorithms and can perform general computation, its just so happens that by asking them to next-token predict the internet they learn a bunch of smart ways to compress everything, potentially in a way similar to how we might use a general concept to avoid memorizing a lookup table. Please have a look at https://arxiv.org/pdf/2211.15661 and https://mobile.twitter.com/DimitrisPapail/status/16208344092.... We don't understand everything that's going on yet but it would be…

Paper and discussion history. A mention is not an endorsement.

What learning algorithm is in-context learning? Investigations with linear models — mentioned by O__________O

Fri, 24 Mar 2023 22:45:28 GMT

Related paper: “What learning algorithm is in-context learning? Investigations with linear models” https://arxiv.org/abs/2211.15661

Paper and discussion history. A mention is not an endorsement.

What learning algorithm is in-context learning? Investigations with linear models — mentioned by LargoLasskhyfv

Sat, 11 Feb 2023 11:08:36 GMT

Link to rendered paper https://ar5iv.labs.arxiv.org/html/2211.15661

Paper and discussion history. A mention is not an endorsement.

What learning algorithm is in-context learning? Investigations with linear models — mentioned by benlivengood

Sat, 11 Feb 2023 07:27:11 GMT

The article doesn't understand large language models. It makes an assumption about how they work and uses that assumption to draw a conclusion. Most likely, as found in small models [0], large autoregressive transformers understand machine learning algorithms well enough to apply them to their inputs to achieve in-context learning (few-shot prompting) by performing gradient descent within the overall network [1]. This is fundamentally different from static predictors; LLMs are dynamic modelers that learn from context in order to better predict tokens. [0] https://arxiv.org/pdf/2211.15661 [1]…

Paper and discussion history. A mention is not an endorsement.

What learning algorithm is in-context learning? Investigations with linear models — mentioned by dr_dshiv

Sat, 11 Feb 2023 07:20:08 GMT

The author says: “Large Language Models don’t understand anything — and they can’t get better at something they can’t do.” Yet, LLM’s can understand (unless one claims that understanding requires consciousness): https://arxiv.org/abs/2101.06573 And, strangely enough, LLM’s can get better at something they can’t do. https://news.mit.edu/2023/large-language-models-in-context-l... https://arxiv.org/pdf/2211.15661.pdf

Paper and discussion history. A mention is not an endorsement.

What learning algorithm is in-context learning? Investigations with linear models — HN submission

Sat, 11 Feb 2023 01:08:02 GMT

What Learning Algorithm Is In-Context Learning? Linear-Models Investigations

7 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

What learning algorithm is in-context learning? Investigations with linear models — mentioned by dctoedt

Sat, 11 Feb 2023 01:05:37 GMT

Link to paper: https://arxiv.org/pdf/2211.15661.pdf

Paper and discussion history. A mention is not an endorsement.

What learning algorithm is in-context learning? Investigations with linear models — HN submission

Fri, 10 Feb 2023 14:34:39 GMT

What Learning Algorithm Is IN-Context Learning?

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.