https://hackerpaper.kaipfister.de/p/2203.10790/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=32058497urn:hackerpapers:mention:2203.10790:32058497Mon, 11 Jul 2022 16:43:22 GMT<p>An open-source implementation of the ScalableViT: Rethinking the Context-oriented Generalization of Vision Transformer research paper in Google&#x27;s JAX and Flax. &quot;The vanilla self-attention mechanism inherently relies on pre-defined and steadfast computational dimensions. Such inflexibility restricts it from possessing context-oriented generalization that can bring more contextual cues and global representations. To mitigate this issue, we propose a Scalable Self-Attention (SSA) mechanism that leverages two scaling factors to release dimensions of query, key, and value matrix while unbinding…</p><p><a href="https://hackerpaper.kaipfister.de/p/2203.10790/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CV