https://hackerpaper.kaipfister.de/p/2307.08691/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=40940418urn:hackerpapers:mention:2307.08691:40940418Thu, 11 Jul 2024 20:38:30 GMT<p>There are a bunch of good answers, but I wanted to succinctly say &quot;practically, quite a bit&quot;. Here&#x27;s a good little rabbit-hole example: &gt; https://github.com/karpathy/nanoGPT/blob/master/model.py#L45 Karpathy&#x27;s nanoGPT calling flash attention by checking if torch.nn.functional.scaled_dot_product_attention exists &gt; https://pytorch.org/docs/stable/generated/torch.nn.functiona... Looking at the docs, in reality, most of the time you want this to call out to FA2 which optimizes the kernals on the device to split ops on the Softmax of the triangular matrix as well as reduce moving unnecessary…</p><p><a href="https://hackerpaper.kaipfister.de/p/2307.08691/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG