Can language models learn from explanations in context? · HackerPapers discussions

Can language models learn from explanations in context? — mentioned by famouswaffles

Mon, 22 May 2023 23:25:33 GMT

I don't think you took more than a passing glance, if any at those papers. What you describe is impossible with these 3. https://arxiv.org/abs/2212.09196 - new evaluation set introduced with the paper. modelled after tests that previously only had visual equivalents. contamination literally impossible https://arxiv.org/abs/2204.02329 - effect of explanations on questions introduced with the paper. dataset concerns make no sense. https://arxiv.org/abs/2211.09066 - new prompting method introduced to improve algorithmic calculations. dataset concerns make no sense. The Casual paper is the only…

Paper and discussion history. A mention is not an endorsement.

Can language models learn from explanations in context? — mentioned by famouswaffles

Mon, 22 May 2023 22:23:07 GMT

>It's not better at reasoning. It's barely even capable of it You are wrong. and there's many papers to show otherwise. Algorithmic, Casual, Inference, Analogical LLMs reason just fine https://arxiv.org/abs/2212.09196 https://arxiv.org/abs/2305.00050 https://arxiv.org/abs/2204.02329 https://arxiv.org/abs/2211.09066

Paper and discussion history. A mention is not an endorsement.

Can language models learn from explanations in context? — mentioned by famouswaffles

Sun, 21 May 2023 02:46:45 GMT

No world model ? A world model is so obvious papers like these are more confirmation than surprise https://arxiv.org/abs/2305.11169 https://arxiv.org/abs/2210.13382 There a certain sentiment that AGI however you wish to define it won't infact be a "We'll know it when we see it" situation but rather a "AGI will arrive long before consensus reaches its AGI". LLMs have made me believe this will 100% be the case, either way. It's one thing to argue over things we can't evaluate even now but man the 100th "They can't reason!" every week is pretty funny when you can basically take your pick of…

Paper and discussion history. A mention is not an endorsement.

Can language models learn from explanations in context? — mentioned by famouswaffles

Tue, 09 May 2023 01:03:35 GMT

LLMs are not very bad at reasoning. People say this but struggle to demonstrate how. There are many types of reasoning. For some like abstract analogical reasoning, LLMs are top % performers. https://arxiv.org/abs/2212.09196 There's many papers at this point demonstrating pretty strong reasoning. From Casual Reasoning, and explanatory reasoning to algorithmic reasoning. https://arxiv.org/abs/2305.00050 https://arxiv.org/abs/2204.02329 https://arxiv.org/abs/2211.09066 The only kind of reasoning that's somewhat lacking is logical reasoning. And even then it's not awful, just lagging behind…

Paper and discussion history. A mention is not an endorsement.