Titans: Learning to Memorize at Test Time · HackerPapers discussions

Titans: Learning to Memorize at Test Time — mentioned by carlovalenti

Tue, 28 Apr 2026 11:09:29 GMT

don't miss the (formerly over-hyped) Google Research's Titans+Atlas thread, they are quite inspiring and informative on the subject; even if I don't think they released anything (open, at least) as a follow-up to their research. https://arxiv.org/abs/2501.00663 https://arxiv.org/abs/2505.23735

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by andy12_

Tue, 10 Mar 2026 16:28:47 GMT

Putting stuff you have learned into a markdown file is a very "shallow" version of continual learning. It can remember facts, yes, but I doubt a model can master new out-of-distribution tasks this way. If anything, I think that Google's Titans[1] and Hope[2] architectures are more aligned with true continual learning (without being actual continual learning still, which is why they call it "test-time memorization"). [1] https://arxiv.org/pdf/2501.00663 [2] https://arxiv.org/pdf/2512.24695

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by psb217

Thu, 08 Jan 2026 10:48:34 GMT

Per your point 4, some current hyped work is pushing hard in this direction [1, 2, 3]. The basic idea is to think of attention as a way of implementing an associative memory. Variants like SDPA or gated linear attention can then be derived as methods for optimizing this memory online such that a particular query will return a particular value. Different attention variants correspond to different ways of defining how the memory produces a value in response to a query, and how we measure how well the produced value matches the desired value. Some of the attention-like ops proposed in this new…

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by mycall

Fri, 26 Dec 2025 03:11:20 GMT

Have you looked at Titan and MIRAS where they use online/updating associative memory that happens to be read out via next-token prediction? https://research.google/blog/titans-miras-helping-ai-have-lo... https://arxiv.org/abs/2501.00663 https://arxiv.org/pdf/2504.13173 Much research is going into these directions, but I'm more interested in mind-wandering tangents, involving both attentional control and additional mechanisms (memory retrieval, self-referential processing).

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by okdood64

Sun, 07 Dec 2025 14:05:36 GMT

From the blog: https://arxiv.org/abs/2501.00663 https://arxiv.org/pdf/2504.13173 Is there any other company that's openly publishing their research on AI at this level? Google should get a lot of credit for this.

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by Alifatisk

Sun, 07 Dec 2025 12:27:48 GMT

Titans: Learning to Memorize at Test Time https://arxiv.org/abs/2501.00663

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by djrhails

Fri, 05 Dec 2025 16:37:10 GMT

There is TITANS - https://arxiv.org/abs/2501.00663

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by imjonse

Sat, 22 Feb 2025 07:12:17 GMT

Something like this, using Simon Willison's llm cli, will give you a better experience than such generated articles unless your goal is explicitly to read blogs. wget https://arxiv.org/pdf/2501.00663 -O - | pdftotext - - | llm -s "Explain the following paper assuming I am familiar with deep learning" -m gpro Advantages: * put in any prompt you like * pick any model (above gpro is alias for latest gemini pro if you have the llm-gemini plugin installed) * ask for clarifications using llm chat -c (continue last conversation) * no annoying random image For another approach for this particular…

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by tkgally

Sat, 22 Feb 2025 05:51:25 GMT

The paper the article is based on [1] treats “Titans” as a plural: “Titans are capable of solving problems beyond TC^0, meaning that Titans are theoretically more expressive than Transformers….” [1] https://arxiv.org/pdf/2501.00663

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by brrrrrm

Fri, 21 Feb 2025 23:27:10 GMT

Paper https://arxiv.org/abs/2501.00663

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by korbip

Sat, 15 Feb 2025 03:04:16 GMT

There is a LOT of effort in the research community currently: 1. Improving the Self-Attention in the Transformer as is, keeping the quadratic complexity, which has some theoretical advantage in principle[1]: The most hyped one probably DeepSeek's Multi-Latent Attention[15], which kind of is Attention still - but also somehow different. 2. Linear RNNs: This starts from Linear Attention[2], DeltaNet[3], RKWV[4], Retention[5], Gated Linear Attention[6], Mamba[7], Griffin[8], Based[9], xLSTM[10], TTT[11], Gated DeltaNet[12], Titans[13]. They all have an update like: C_{t} = F_{t} C_{t-1} + i_{t}…

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by SiddanthEmani

Fri, 14 Feb 2025 21:33:41 GMT

Titans has a new approach to longer and faster memory compared to transformers. https://arxiv.org/html/2501.00663v1

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by mkroman

Sun, 26 Jan 2025 21:15:28 GMT

The AI hardware race is still going strong, but with so many rapid changes to the fundamental architectures, it doesn't make sense to bet everything on specialized hardware just yet.. It's happening, but it's expensive and slow. There's just not enough capacity to build memory fast enough right now. Everyone needs the biggest and fastest modules they can get, since it directly impacts the performance of the models. There's still a lot of happening to improve memory, like the latest Titans paper: https://arxiv.org/abs/2501.00663 So I think until a breakthrough happens or the fabs catch up…

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by OutOfHere

Mon, 20 Jan 2025 04:12:50 GMT

Titans [1] goes sort of a bit in this direction. [1] https://arxiv.org/abs/2501.00663

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — HN submission

Wed, 15 Jan 2025 22:39:49 GMT

Titans: Learning to Memorize at Test Time

161 points; 35 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Titans: Learning to Memorize at Test Time — mentioned by rar00

Wed, 15 Jan 2025 21:01:11 GMT

Paper: https://arxiv.org/abs/2501.00663

Paper and discussion history. A mention is not an endorsement.

Titans: Learning to Memorize at Test Time — HN submission

Wed, 15 Jan 2025 13:16:03 GMT

Titans: Learning to Memorize at Test Time

5 points; 2 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Titans: Learning to Memorize at Test Time — HN submission

Mon, 13 Jan 2025 20:11:14 GMT

Titans: Learning to Memorize at Test Time

115 points; 15 discussion replies on HN. Paper and source history. A submission is not an endorsement.