Sleep-time Compute: Beyond Inference Scaling at Test-time · HackerPapers discussions

Sleep-time Compute: Beyond Inference Scaling at Test-time — mentioned by swyx

Tue, 26 May 2026 16:37:00 GMT

related preprint from the letta team https://arxiv.org/abs/2504.13171 Scaling test-time compute has emerged as a key ingredient for enabling large language models (LLMs) to solve difficult problems, but comes with high latency and inference cost. We introduce sleep-time compute, which allows models to "think" offline about contexts before queries are presented: by anticipating what queries users might ask and pre-computing useful quantities, we can significantly reduce the compute requirements at test-time. To demonstrate the efficacy of our method, we create modified versions of two…

Paper and discussion history. A mention is not an endorsement.

Sleep-time Compute: Beyond Inference Scaling at Test-time — mentioned by falcor84

Wed, 09 Jul 2025 14:37:25 GMT

An interesting idea, but I'm surprised that they only mention MemGPT/Letta [0] in passing; from what I've seen of Letta's work and e.g. their recent paper on "sleep-time compute", they are really pushing this space forward. [0] https://www.letta.com/ [1] https://arxiv.org/abs/2504.13171

Paper and discussion history. A mention is not an endorsement.

Sleep-time Compute: Beyond Inference Scaling at Test-time — mentioned by pacjam

Thu, 15 May 2025 20:26:28 GMT

check out Letta - the OSS codebase (https://github.com/letta-ai/letta) is basically focused on solving the memory/context problem in a generalized way (via "agentic context management"). if you're more interested in papers, we also worked on MemGPT and more recently sleep-time compute (https://arxiv.org/abs/2504.13171)

Paper and discussion history. A mention is not an endorsement.

Sleep-time Compute: Beyond Inference Scaling at Test-time — mentioned by x-complexity

Wed, 07 May 2025 02:19:10 GMT

Dummy calculations aren't even needed, if you allow the LLMs to pre-compute on the given context before inference: https://arxiv.org/abs/2504.13171 It should be noted that this type of inference is less useful on time-sensitive tasks, but most tasks truthfully don't require such time sensitivity (there exists slack time between when the task is given & when questions are asked).

Paper and discussion history. A mention is not an endorsement.

Sleep-time Compute: Beyond Inference Scaling at Test-time — HN submission

Sat, 26 Apr 2025 22:48:12 GMT

Sleep-Time Compute: Beyond Inference Scaling at Test-Time

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Sleep-time Compute: Beyond Inference Scaling at Test-time — HN submission

Thu, 24 Apr 2025 23:20:52 GMT

Sleep-Time Compute: Beyond Inference Scaling at Test-Time

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Sleep-time Compute: Beyond Inference Scaling at Test-time — HN submission

Mon, 21 Apr 2025 16:41:43 GMT

Sleep-Time Compute: Beyond Inference Scaling at Test-Time

5 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.