On the importance of single directions for generalization · HackerPapers discussions

On the importance of single directions for generalization — HN submission

Mon, 26 Mar 2018 07:56:55 GMT

DeepMind – On the importance of single directions for generalization

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

On the importance of single directions for generalization — mentioned by arimorcos

Fri, 23 Mar 2018 14:02:11 GMT

Author here. Indeed, our work is closely related to dropout. However, as we discuss in the paper (https://arxiv.org/abs/1803.06959), dropout doesn't really encourage the network to be robust to deletion generally; it only encourages this robustness up until the dropout fraction used in training. So, for example, if you train the network with a 50% dropout rate, dropout will encourage the network to be robust to dropping 50% of the units, but the network could completely fail once 51% of the units are deleted and the training objective would be perfectly happy. As a result, dropout doesn't…

Paper and discussion history. A mention is not an endorsement.

On the importance of single directions for generalization — HN submission

Fri, 23 Mar 2018 05:25:42 GMT

Deep Mind – On the importance of single directions for generalization

7 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

On the importance of single directions for generalization — HN submission

Thu, 22 Mar 2018 17:04:50 GMT

DeepMind – On the importance of single directions for generalization

8 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.