https://hackerpaper.kaipfister.de/p/2408.14717/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=43756511urn:hackerpapers:mention:2408.14717:43756511Mon, 21 Apr 2025 21:09:36 GMT<p>Having done my masters on the topic of grammar-assisted text2sql let me add some additional context here: - first of all local inference can never beat cloud inference for the very simple reason that costs go down with batching. it took me two years to actually understand what batching is - the LLM tensors flowing through transformer layers has a dimension designed specifically for processing data in parallel. so no matter if you process a 1 sequence or 128 sequences the costs are the same. i&#x27;ve read very few articles overstating this, so bear in mind - this is the primary stopper for…</p><p><a href="https://hackerpaper.kaipfister.de/p/2408.14717/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.DB