https://hackerpaper.kaipfister.de/p/2410.01131/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=42018030urn:hackerpapers:mention:2410.01131:42018030Fri, 01 Nov 2024 15:37:50 GMT<p>There is a particularly nice geometric interpretation of attention I just realised recently in a flash of enlightenment, best explained with an interactive Desmos plot (black dot is draggable): https://www.desmos.com/calculator/3rtqsyapxo The above assumes the columns of K are normalised but bear with me. K and V together form a vector database. V are the payloads, each row containing a vector of data. K describes the position of these points in space, on the surface of a hypershpere. The query vector describes the query into the database: the vector direction describes the point in space…</p><p><a href="https://hackerpaper.kaipfister.de/p/2410.01131/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LGhttps://news.ycombinator.com/item?id=41881478urn:hackerpapers:submission:2410.01131:41881478Fri, 18 Oct 2024 17:18:33 GMT<p>NGPT: Normalized Transformer with Representation Learning on the Hypersphere</p><p></p><p>4 points; 0 discussion replies on HN. <a href="https://hackerpaper.kaipfister.de/p/2410.01131/">Paper and source history</a>. A submission is not an endorsement.</p>cs.LG