https://hackerpaper.kaipfister.de/p/2403.10459/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=50301908urn:hackerpapers:mention:2403.10459:50301908Fri, 30 Oct 2026 10:13:50 GMT<p>AFAICT there are several things that affect the size of the model: 1. the number of transformer layers -- e.g. each layer adds the ability to provide more information to the residual stream [1], [2]; each layer is `tokens * features * active context` 2. the number of tokens -- the more tokens there are the more word fragments and Unicode characters you can represent 3. the number of features assigned for a token -- these are used to associate trained semantic information for the token; with the active context this can encode contextual information for that token (the idea being that the model…</p><p><a href="https://hackerpaper.kaipfister.de/p/2403.10459/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG