LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations · HackerPapers discussions

LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations — mentioned by famouswaffles

Mon, 19 Aug 2024 13:50:40 GMT

Oh i've read them. The claim doesn't match up to reality. It's as simple as that. You can claim anything you want to. https://arxiv.org/abs/2305.18354 All these papers you keep linking do is at best point out the shortcomings of current state of the art LLMs. They do not in any way disprove their ability to reason. I don't know when the word reason started having different standards for humans and machines but i don't care for it. Either your definition of reasoning also allows for the faulty kind humans display or humans don't reason either. You can't have your cake and eat it.

Paper and discussion history. A mention is not an endorsement.

LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations — HN submission

Sun, 10 Mar 2024 18:29:42 GMT

LLMs and the Abstraction and Reasoning Corpus

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations — mentioned by famouswaffles

Sun, 10 Mar 2024 18:23:27 GMT

For now, LLMs are a lot better when you express this kind of task in text, preferably as a 1D arrangement. GPT-4's performance on the Abstraction and Reasoning Corpus. https://arxiv.org/abs/2305.18354 Also, this model trained to predict image tokens seemed to be able to perform Raven's Matrices type questions when expressed as an inpainting task in context. https://yutongbai.com/lvm.html

Paper and discussion history. A mention is not an endorsement.

LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations — mentioned by famouswaffles

Fri, 24 Nov 2023 21:38:57 GMT

Late reply but, LLMs are not bad at abstraction in general, just spatially focused abstraction. https://arxiv.org/abs/2212.09196 Also, how the data is presented matters a lot. Currently, LLMs handle the benchmark much better presented linearly in 1 dimension https://arxiv.org/abs/2305.18354

Paper and discussion history. A mention is not an endorsement.

LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations — HN submission

Mon, 20 Nov 2023 20:12:04 GMT

LLMs and the Abstraction and Reasoning Corpus

5 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations — HN submission

Sun, 19 Nov 2023 20:29:50 GMT

LLMs and the Abstraction and Reasoning Corpus

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations — mentioned by famouswaffles

Sun, 19 Nov 2023 20:16:37 GMT

>It is, and it should, be able to find correlations across data no matter how the input is arranged. That doesn't mean it doesn't favor certain arrangements. https://arxiv.org/abs/2305.18354 Linear is in fact better for this benchmark

Paper and discussion history. A mention is not an endorsement.