AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks — HN submission

Wed, 19 Jan 2022 20:38:58 GMT

AutoInit Software for Model Initialization

5 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks — HN submission

Tue, 18 Jan 2022 20:06:07 GMT

AutoInit Software for Model Initialization

AutoInit is a weight initialization method that automatically adapts to different neural network architectures. It tracks the mean and variance of signals as they propagate through the network and initializes the weights at each layer to avoid exploding or vanishing signals. AutoInit can be used to improve performance of feedforward, convolutional, and residual networks; configured with different activation function, dropout, weight decay, learning rate, and normalizer settings; and applied to vision, language, tabular, multi-task, and transfer learning domains. The software package provides…

7 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks · HackerPapers discussions

AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks — HN submission

AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks — HN submission