Variational Dropout Sparsifies Deep Neural Networks — mentioned by tysam_and

Wed, 20 Sep 2023 16:06:04 GMT

DL researcher here, it's also hard I think because experimentally many of us have noted (also some research on this) that there's a critical early phase to learning that conditions the network a certain way, and adding and/or removing layers later seems to actually be quite difficult (removing easier than adding, by far, esp w/ something like variational dropout [yes, i cite the old deep magicks]: https://arxiv.org/abs/1701.05369)

Paper and discussion history. A mention is not an endorsement.

Variational Dropout Sparsifies Deep Neural Networks · HackerPapers discussions

Variational Dropout Sparsifies Deep Neural Networks — mentioned by tysam_and