Recurrent Highway Networks · HackerPapers discussions

Recurrent Highway Networks — mentioned by inlineint

Sat, 10 Jun 2017 10:38:27 GMT

I personally find recurrent highway networks (RHNs) as described in [1] to be easier to understand and remember the formulas for than the original LSTM. Because as they are generalizations of LSTM, if one understands RHNs, one can understand LSTMs as just a particular case of RHN. Instead of handwaving about "forgetting", it is IMO better to understand the problem of vanishing gradients and how can forget gates actually help with them. And Jürgen Schmidhuber, the inventor of LSTM, is a co-author of the RHN paper. [1] https://arxiv.org/abs/1607.03474

Paper and discussion history. A mention is not an endorsement.

Recurrent Highway Networks — HN submission

Sun, 17 Jul 2016 16:04:24 GMT

Recurrent Highway Networks

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Recurrent Highway Networks — HN submission

Wed, 13 Jul 2016 15:49:57 GMT

Recurrent Highway Networks

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.