Rigging the Lottery: Making All Tickets Winners · HackerPapers discussions

Rigging the Lottery: Making All Tickets Winners — mentioned by DARSHANFOFADIYA

Fri, 24 Apr 2026 01:15:40 GMT

Most "sparse training" in PyTorch today isn't actually sparse. A binary mask gets multiplied into a dense weight matrix, which means the zeros still consume memory, still move through the cache, and still get multiplied. That's pruning simulation, not sparse computation. SparseLab does the other thing: real sparse storage (a custom Padded-CSR layout), custom NEON kernels, in an nn.Linear-compatible layer. The premise. It's been known since the Lottery Ticket Hypothesis (https://arxiv.org/abs/1803.03635) and RigL (https://arxiv.org/abs/1911.11134) that most models train competitively with ~10%…

Paper and discussion history. A mention is not an endorsement.

Rigging the Lottery: Making All Tickets Winners — mentioned by cma

Tue, 16 May 2023 23:55:15 GMT

50% sparsity is almost certainly already being used given that it is accelerated in current nvidia hardware both at training time, usable dynamically through RigL ("Rigging the Lottery: Making All Tickets Winners" https://arxiv.org/pdf/1911.11134.pdf )--which also addresses your point about initial conditions being locked in-- and at accelerates 50% sparsity at inference time.

Paper and discussion history. A mention is not an endorsement.

Rigging the Lottery: Making All Tickets Winners — HN submission

Sun, 08 Mar 2020 09:13:38 GMT

Rigging the Lottery: Making All Tickets Winners (2019)

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Rigging the Lottery: Making All Tickets Winners — mentioned by ekelsen

Sat, 07 Mar 2020 15:55:04 GMT

The implications are unclear to me. We already know how to prune models for inference. For example https://arxiv.org/abs/1710.01878, along with earlier work (and more recent work). There's also work showing that you can take advantage of the sparsity to achieve practical speed gains: https://arxiv.org/abs/1911.09723. We can also train networks that are sparse from the beginning of training (without requiring any special knowledge of the solution): https://arxiv.org/abs/1911.11134. It remains to be shown that this can be done with a speed advantage.

Paper and discussion history. A mention is not an endorsement.