Learning both Weights and Connections for Efficient Neural Networks · HackerPapers discussions

Learning both Weights and Connections for Efficient Neural Networks — mentioned by trhway

Wed, 13 Dec 2017 08:25:30 GMT

similar work from couple of years ago https://arxiv.org/pdf/1506.02626.pdf "our method prunes redundant connections using a three-step method. First, we train the network to learn which connections are important. Next, we prune the unim- portant connections. Finally, we retrain the network to fine tune the weights of the remaining connections. On the ImageNet dataset, our method reduced the number of parameters of AlexNet by a factor of 9 × , from 61 million to 6.7 million, without incurring accuracy loss. Similar experiments with VGG-16 found that the total number of parameters can be…

Paper and discussion history. A mention is not an endorsement.

Learning both Weights and Connections for Efficient Neural Networks — mentioned by sanxiyn

Thu, 07 Dec 2017 08:17:18 GMT

This is the idea behind "Learning both Weights and Connections for Efficient Neural Networks (2015)", and yes, it works: https://arxiv.org/abs/1506.02626 How to figure out which ones are useful and which ones aren't? Why, you can try the simplest thing that could possibly work. Quoting the paper: "All connections with weights below a threshold are removed from the network". Is that all? Yes it is.

Paper and discussion history. A mention is not an endorsement.

Learning both Weights and Connections for Efficient Neural Networks — mentioned by cing

Tue, 13 Dec 2016 23:24:33 GMT

I believe several papers have examined efficiently pruning neural networks, but neural death would be better branding ;) (https://arxiv.org/pdf/1506.02626v3.pdf https://arxiv.org/pdf/1510.00149v5.pdf)

Paper and discussion history. A mention is not an endorsement.