Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regression — mentioned by famouswaffles

Sat, 04 Nov 2023 19:23:15 GMT

>how many books is 500gb at, say, a few bits per character? In what reality is a character taking up only a few bits ? Certainly isn't this one. Your denial is so weird. Why does ICL use Higher-Order Optimization Methods when it's just a lookup table ? https://arxiv.org/abs/2310.17086 How do you explain small language models that have much less space than the text they were trained on even with your nonsensical calculations ? LLMs are not lookup tables and there's plenty evidence to support that. You look insane insisting they are. >even, just ask chatgpt to repeat the first paragraph of some…

Paper and discussion history. A mention is not an endorsement.

Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regression · HackerPapers discussions

Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regression — mentioned by famouswaffles