https://hackerpaper.kaipfister.de/p/2109.04838/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=34460786urn:hackerpapers:mention:2109.04838:34460786Fri, 20 Jan 2023 22:12:56 GMT<p>May I add another method: block fine-pruning of transformers (pruning while fine-tuning) ? https://arxiv.org/abs/2109.04838 Using blocks allows to keep good performence on GPUS, while giving some flexibility in the pruning pattern. And when removing entirely empty rows and columns the pruned matrices are actually pretty dense, so competitive with structured pruning for speedup, but less &quot;aggressive&quot; on the network during the pruning process. Disclaimer: I am the main co-author.</p><p><a href="https://hackerpaper.kaipfister.de/p/2109.04838/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.LG