Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap · HackerPapers discussions

Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap — mentioned by nopinsight

Sun, 03 Mar 2024 17:38:47 GMT

Latest models can solve some relatively challenging math problems, such as those in the MATH dataset. Assuming not all of the capability stems from memorization, that exhibits a certain level of fluid intelligence. Although recent research shows much of LLM’s reasoning capability are indeed based on memorization, some models can actually reason a bit. https://arxiv.org/pdf/2402.19450.pdf https://twitter.com/_saurabh/status/1763626711407816930

Paper and discussion history. A mention is not an endorsement.

Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap — mentioned by sgt101

Sun, 03 Mar 2024 17:21:09 GMT

I was reading twitter yesterday and there was a guy saying that Elon says that GTP-4 is an AGI so this blog is definitely wrong. Also : https://arxiv.org/abs/2402.19450 (which shows that there is a big gap between what LLM's look like they do and what they can actually do)

Paper and discussion history. A mention is not an endorsement.

Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap — HN submission

Sat, 02 Mar 2024 18:32:17 GMT

Fn Benchmarks for Robust Evaluation of Reasoning Performance, and Reasoning Gap

2 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.