Large Language Models are Zero-Shot Reasoners · HackerPapers discussions

Large Language Models are Zero-Shot Reasoners — mentioned by CamperBob2

Tue, 20 Jan 2026 19:10:24 GMT

These arguments usually seem to come down to disagreements about definitions, as you suggest. You've talked about what you don't consider evidence of intelligence, but you haven't said anything about the criteria you would apply. What evidence of intelligent reasoning would change your mind? It is unsupportable to claim that ML researchers at leading labs share your opinion. Since roughly 2022, they understand that they are working with systems capable of reasoning: https://arxiv.org/abs/2205.11916

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by LordDragonfang

Tue, 10 Jun 2025 22:29:06 GMT

"Thinking" is a term of art referring to the hidden/internal output of "reasoning" models where they output "chain of thought" before giving an answer[1]. This technique and name stem from the early observation that LLMs do better when explicitly told to "think step by step"[2]. Hope that helps clarify things for you for future constructive discussion. [1] https://arxiv.org/html/2410.10630v1 [2] https://arxiv.org/pdf/2205.11916

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by TeMPOraL

Sun, 23 Mar 2025 13:21:40 GMT

You're right about "living in the “overselling all up in your ears” epoch", but a good first defense against "being sold pffts as badabooms" is to blanket distrust all the marketing copy and whatever the salespeople say, and rely on your own understanding or experience. You may lose out on becoming an early adopter of some good things, but you'll also be spared wasting money on most garbage. With that in mind, I still don't get the dismissal. LLMs are broadly accessible - ever since the first ChatGPT, anyone could easily get access to a SOTA LLM and evaluate it for free; even the limited…

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by potatoman22

Thu, 26 Sep 2024 14:50:56 GMT

It's appending "think step-by-step" to the end of the prompt to elicit a chain-of-thought response. See: https://arxiv.org/abs/2205.11916

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by jdietrich

Sun, 21 Jul 2024 07:41:21 GMT

This isn't something I should convince you of. Just open up ChatGPT or Claude and try it for yourself. Think up a batch of your own questions and see how a modern LLM fares. I assure you that it'll do much better than chance. If you're so inclined, you can run enough tests to achieve statistical significance in the course of your lunch break. It depresses me that we seem to be spending more time arguing and hypothesising about LLMs than empirically testing them. The question of whether LLMs can think is completely settled, as their performance at zero-shot problems is simply impossible…

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by kromem

Wed, 01 Nov 2023 08:19:34 GMT

CoT doesn't require it to be a few shot prompt though. That was in the original paper, as you point out, but even something as simple as a zero shot prompt followed by "let's think step by step" has significant improvement: https://arxiv.org/abs/2205.11916 Do you still consider a zero-shot-CoT to be hinting? (And I don't mean the [0] example you shared - biasing results by continuing to ask for refinements in incorrect answers but not correct answers is an area where a number of papers have already had issues, but is not part of what I'm discussing - I mean in single prompt/response cycles).…

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by joshka

Sat, 07 Oct 2023 23:10:02 GMT

This seems like a slight variation of "Let’s think step by step" [1] or "Let’s work this out in a step by step way to be sure we have the right answer" [2]. There's a good review of a bunch of these in https://arxiv.org/abs/2305.02897 1: https://arxiv.org/abs/2205.11916 2: https://openreview.net/forum?id=92gvk82DE-

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by TeMPOraL

Wed, 12 Jul 2023 04:45:42 GMT

I appreciate you giving solid overview and references of the use and context of that term - however, in this case, I fear that ship has long sailed by now. The more relevant paper would be[0], the title of which is, I believe, the source of the "LLMs are zero-shot [learners / reasoners / whatever]" phrase that's been reaching meme status, and now shows up in training materials. -- [0] - https://arxiv.org/abs/2205.11916 - "Large Language Models are Zero-Shot Reasoners"

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by convexfunction

Fri, 05 May 2023 18:55:07 GMT

I'm not sure if "helpful answer" as opposed to "answer" makes much of a difference in answer quality -- I'd believe it helps a little, just don't know that it's been studied -- but a lot of silly stuff like that does definitely make a big difference in response quality on certain tasks. "Let's think step by step:" at the end of your prompt is probably the best-known one: https://arxiv.org/pdf/2205.11916.pdf

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by famouswaffles

Tue, 11 Apr 2023 03:46:10 GMT

chain of thought prompting. It's well known https://arxiv.org/abs/2205.11916 https://arxiv.org/abs/2201.11903

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by DougBTX

Sat, 25 Mar 2023 06:06:55 GMT

https://arxiv.org/pdf/2205.11916.pdf

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by andyk

Mon, 20 Mar 2023 23:03:53 GMT

My personal point was just to document my exploration of using prompts to generate new prompts, and more specifically the case where the prompts contain state and each recursively generated prompt updates that state to be closer to an end goal (which in this case I compared to the concept of a base case in recursion). For whatever reason Patrick H. Winston's MIT OCW lecture on Cognitive Architectures always stuck with me, and in particular his summary of the historical system from CMU called General Problem Solver (GPS) in which they try to identify a goal and then have the AI evaluate the…

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by vsareto

Sun, 26 Feb 2023 20:42:22 GMT

>Notably, chain of thought (CoT) prompting, a recent technique for eliciting complex multi-step reasoning through step-by-step answer examples, achieved the state-of-the-art performances in arithmetics and symbolic reasoning, difficult system-2 tasks that do not follow the standard scaling laws for LLMs https://arxiv.org/abs/2205.11916 >https://sites.google.com/view/automatic-prompt-engineer Not exactly a "toaster go brrrr" job, but it could be obsolete one day WaPo does need to chill though. There’s barely any Prompt Engineer jobs Edit: If anyone's curious, I've been following this for…

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by simonw

Tue, 24 Jan 2023 17:17:07 GMT

My personal excitement about language models is based on what they can do today. I'm a big believer in the "capability overhang" idea, which is that the existing language models still have a huge array of capabilities that we haven't discovered yet. That theory seems to be proved correct on a constant basis. Even the classic "let's think about this step by step" paper came out less than a year ago: https://arxiv.org/abs/2205.11916 - May 2022.

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — mentioned by Isinlor

Mon, 13 Jun 2022 08:00:12 GMT

Small disclaimer: I don't want to take position on sentience or consciousness. "LaMDA is not an agent with desires that it expresses in language." Some counter points: - being an agent is not a high bar to clear; a thermostat is an agent - "with desires" - LaMDA can be thought of as rational agent whose objective function is to produce text with high likelihood - "that it expresses in language" it expresses likelihood of next token in language "it can't fear anything because there is no time for it to do so." There is a time for it to experience something and that is when it does feed forward…

Paper and discussion history. A mention is not an endorsement.

Large Language Models are Zero-Shot Reasoners — HN submission

Sat, 28 May 2022 19:01:56 GMT

Large Language Models Are Zero-Shot Reasoners

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Large Language Models are Zero-Shot Reasoners — HN submission

Wed, 25 May 2022 21:51:01 GMT

Large Language Models are Zero-Shot Reasoners: Let's think step by step

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.