Full deep neural network training on a pruned weight budget · HackerPapers discussions

Full deep neural network training on a pruned weight budget — mentioned by buildbot

Fri, 08 May 2020 22:24:57 GMT

If they are simply masking out low values weights after doing full training, they’ve basically recreated what I worked on for my graduate thesis. In contrast to this work, what we worked on starts from iteration 0, and constantly adapts the lottery mask. You can reset weights back to their initial value, or set them back to their initial value tied to a decay term. This will zero initial values slowly to get computational sparsity if needed. https://arxiv.org/abs/1806.06949 https://open.library.ubc.ca/cIRcle/collections/ubctheses/24/...

Paper and discussion history. A mention is not an endorsement.

Full deep neural network training on a pruned weight budget — mentioned by buildbot

Wed, 20 Mar 2019 05:38:17 GMT

Neural networks (At least CNNs) have so many parameters, you can hold nearly all of them at there initial values! Checkout this paper if you are interested: https://arxiv.org/abs/1806.06949 Disclaimer: I am an author and that version is somewhat out of date.

Paper and discussion history. A mention is not an endorsement.

Full deep neural network training on a pruned weight budget — HN submission

Fri, 22 Jun 2018 00:39:39 GMT

DropBack: Continuous Pruning During Training

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.