Monarch: Expressive Structured Matrices for Efficient and Accurate Training · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2204.00595/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTMonarch: Expressive Structured Matrices for Efficient and Accurate Training — mentioned by sdenton4https://news.ycombinator.com/item?id=46012005urn:hackerpapers:mention:2204.00595:46012005Sat, 22 Nov 2025 04:04:21 GMT<p>A somewhat more beautiful matmul for neural networks is given by the Monarch paper: https://arxiv.org/abs/2204.00595 Generally, low-rank and block-diagonal matrices are both great strategies for producing expressive matmuls with fewer parameters. We can view the FFT as a particularly deft example of factorizing one big matmul into a number of block-diagonal matmuls, greatly reducing the overall number of multiplications by minimizing the block size. However, on a G/TPU, we have a lot more parallelism available, so the sweet spot for size of the blocks may be larger than 2x2... We can also mix…</p><p><a href="https://hackerpaper.kaipfister.de/p/2204.00595/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG