DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving · HackerPapers discussions

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving — mentioned by brrrrrm

Wed, 02 Sep 2026 01:50:45 GMT

perhaps its unfair to say this in hindsight, but it's a fairly straightforward application of little's law that's been around for some time https://arxiv.org/html/2401.09670v2

Paper and discussion history. A mention is not an endorsement.

DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving — mentioned by kevmo314

Sat, 01 Mar 2025 04:30:35 GMT

Interesting they chose to split prefilling into its own independent service, I hadn't heard of that technique before. I found this paper that researches why that could be beneficial: https://arxiv.org/abs/2401.09670v1

Paper and discussion history. A mention is not an endorsement.