Let's Think Dot by Dot: Hidden Computation in Transformer Language Models · HackerPapers discussions

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by thefxperson

Tue, 06 Oct 2026 18:41:49 GMT

My understanding is that the extra token vectors generated as reasoning are still useful, but that their surface form (tokens themselves) do not necessarily reflect the underlying reasoning. i.e. reading the reasoning traces could be complete gibberish, but the hidden-dim vectors themselves still refine the latent probabilities and help in generating the correct answer. Not an expert in LLMs, but this seems supported by the abstract of the paper cited in the above article: it remains unclear to what extent these performance gains can be attributed to human-like task decomposition or simply…

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by wren6991

Thu, 24 Sep 2026 00:06:08 GMT

I think this is a known phenomenon: even in non-reasoning models, adding useless/filler tokens before an answer improves task performance. The model is doing some computation during the filler. See: https://arxiv.org/html/2404.15758v1

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by khalic

Thu, 03 Sep 2026 23:16:23 GMT

[2404.15758] Let's Think Dot by Dot: Hidden Computation in Transformer Language Models https://arxiv.org/abs/2404.15758

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by dannyw

Wed, 26 Aug 2026 14:50:35 GMT

You might already know this, but a large part of test-time compute / 'overthinking' is just letting the model do more passes, and refine its activation residuals more. For example, even if you make thinking tokens literally just '....' (absolutely meaningless; zero information), you still see significant performance improvements: https://arxiv.org/abs/2404.15758 and https://arxiv.org/abs/2607.22925 for some starters. Treat thinking more like a "loading screen message" that's been RL'd to somewhat resemble its actual internal state; which happens in its activations, not tokens.

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by neuroticnews25

Mon, 17 Aug 2026 06:40:49 GMT

I was pretty happy with Depseek Pro in Opencode util I discovered I can see the thinking trace by clicking on the "thinking..." communicate. All those seemingly unnecessary "but wait" messages are frustrating to read. But at least to some extent it's just model taking time thinking through the problem, and the trace produced doesn't have to be representative of what happens internally: https://arxiv.org/abs/2404.15758

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by Chance-Device

Sun, 05 Apr 2026 13:12:44 GMT

Let’s see, I think these pretty much map out a little chronology of the research: https://arxiv.org/abs/2112.00114 https://arxiv.org/abs/2406.06467 https://arxiv.org/abs/2404.15758 https://arxiv.org/abs/2512.12777 First that scratchpads matter, then why they matter, then that they don’t even need to be meaningful tokens, then a conceptual framework for the whole thing.

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by prodigycorp

Sun, 05 Apr 2026 12:08:59 GMT

Are you sure about that? Chain of thought does not need to be semantically useful to improve LLM performance. https://arxiv.org/abs/2404.15758

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by Majromax

Tue, 17 Feb 2026 14:34:36 GMT

Thinking happens in latent space, but the thinking trace is then the projection of that thinking onto tokens. Since autoregressive generation involves sampling a specific token and continuing the process, that sampling step is lossy. However, it is a genuine question whether the literal meanings of thinking blocks are important over their less-observable latent meanings. The ultimate latent state attributable to the last-generated thinking token is some combination of the actual token (literal meaning) and recurrent thinking thus far. The latter does have some value; a 2024 paper…

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by krackers

Sat, 05 Jul 2025 19:49:20 GMT

There are several such papers, off the top of my head one is https://arxiv.org/abs/2404.15758 It's a bit more subtle though, if I understand correctly this only works for parallelizable problems. Which makes intuitive sense since the model cannot pass information along with each dot. So in that sense COT can be seen as some form of sampling, which also tracks with findings that COT doesn't boost the "raw intelligence" but rather uncovers latent intelligence, converting pass@k to maj@k. Antirez touches upon this in [1]. On the other hand, I think problems with serial dependencies require…

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by layoric

Thu, 22 May 2025 22:37:19 GMT

There are several papers pointing towards 'thinking' output is meaningless to the final output, and using dots, or pause tokens enabling the same additional rounds of throughput result in similar improvements. So in a lot of regards the 'thinking' is mostly marketing. - "Think before you speak: Training Language Models With Pause Tokens" - https://arxiv.org/abs/2310.02226 - "Let's Think Dot by Dot: Hidden Computation in Transformer Language Models" - https://arxiv.org/abs/2404.15758 - "Do LLMs Really Think Step-by-step In Implicit Reasoning?" - https://arxiv.org/abs/2411.15862 - Video by…

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by andai

Sun, 23 Mar 2025 19:37:29 GMT

Here's one where just appending dots to the output improved quality: https://arxiv.org/abs/2404.15758

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by gardnr

Sun, 23 Mar 2025 06:25:36 GMT

There was a paper[1] from last year where the authors discovered getting the model to output anything during times of uncertainty, improved the generations overall. If all of the post-training alignment reasoning starts with the same tokens then I could see how it would condition the model to continue the reasoning phase. 1: https://arxiv.org/abs/2404.15758

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by mistercow

Sun, 26 Jan 2025 13:12:06 GMT

Has anyone done a benchmark on these reasoning models compared to simply prompting "non-reasoning" LLMs with massive chain of thought? For example, a go to test I've used (but will have to stop using soon) is: "Write some JS code to find the smallest four digit prime number whose digits are in strictly descending order" That prompt, on its own, usually leads to an incorrect response with non-reasoning models. They almost always forget the "smallest" part, and give the largest four digit prime with descending digits instead. If I prompt o1, it takes longer, but gives the correct answer. If I…

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by mistercow

Tue, 23 Jul 2024 15:04:29 GMT

Absolutely. I'm just making a more general point that "the same information in fewer tokens" does not mean "more comprehensible to an LLM". And we have more practical evidence that that's not the case, like the recent "Let's Think Dot by Dot" paper, which found that you can get many of the benefits of chain-of-thought simply by adding filler tokens to your context (if your model is trained to deal with filler tokens). For that matter, chain-of-thought itself is an example of increasing the tokens:information ratio, and generally improves LLM performance. That's not to say that I think that…

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by FeepingCreature

Sun, 26 May 2024 16:12:19 GMT

> A LLM is not busy humming away, thinking on it's own. Let's Think Dot by Dot: Hidden Computation in Transformer Language Models https://arxiv.org/abs/2404.15758 A LLM will readily hum away, thinking on its own, if given the option.

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by zargon

Sun, 26 May 2024 08:21:25 GMT

The tokens don't have to be related to the task at all. (From an outside perspective. The connections are internal in the model. That might raise transparency concerns.) A single designated 'compute token' repeated over and over can perform as well as traditional 'chain of thought.' See for example, Let's Think Dot by Dot (https://arxiv.org/abs/2404.15758).

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by spencerchubb

Sun, 26 May 2024 03:40:00 GMT

https://arxiv.org/abs/2404.15758

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by FeepingCreature

Sun, 12 May 2024 11:48:01 GMT

Saying that LLMs should be able to do problems without CoT is a bit like saying that humans should be able to write programs without thinking though. CoT is fundamentally an architectural necessity owed to the finite cognitive effort possible per token. Let it be CoT, let it be QuietSTaR ( https://arxiv.org/abs/2403.09629 ), let it be pause tokens ( https://arxiv.org/abs/2310.02226 ), let it even be dots ( https://arxiv.org/abs/2404.15758 ), but the thinking has to happen somewhere. I think mostly, LLMs at the moment are incredibly uneven. LLM assistants can pull obscure knowledge out of…

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — mentioned by richdougherty

Sat, 11 May 2024 19:59:38 GMT

Related? "Let's Think Dot by Dot: Hidden Computation in Transformer Language Models" https://arxiv.org/abs/2404.15758 > Chain-of-thought responses from language models improve performance across most benchmarks. However, it remains unclear to what extent these performance gains can be attributed to human-like task decomposition or simply the greater computation that additional tokens allow. We show that transformers can use meaningless filler tokens (e.g., '......') in place of a chain of thought to solve two hard algorithmic tasks they could not solve when responding without intermediate…

Paper and discussion history. A mention is not an endorsement.

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models — HN submission

Sat, 27 Apr 2024 19:28:26 GMT

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models

159 points; 32 discussion replies on HN. Paper and source history. A submission is not an endorsement.