Cloud Native System for LLM Inference Serving — HN submission

Thu, 28 Aug 2025 05:17:26 GMT

Ask HN: Do cloud provided LLMs get "dumber" during business hours

I have heard this mentioned, online and in casual conversation ** During business hours these endpoints may get so highly utilised that accuracy and quality decline (ignore latency) (effective available token capacity is reduced etc?** Has anyone run studies/experiments to show (true or false) - across a large dataset - that the performance of these endpoints during peak usage hours vs, low usage hours changes or does not change (I assume some statistical significance test required). I don't have the knowledge to answer this question or understand if it's a valid hypothesis. Anyone got…

4 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Cloud Native System for LLM Inference Serving · HackerPapers discussions

Cloud Native System for LLM Inference Serving — HN submission