Quasi-Recurrent Neural Networks · HackerPapers discussions

Quasi-Recurrent Neural Networks — mentioned by Smerity

Fri, 04 Oct 2024 01:31:57 GMT

Excited to see more people working on RNNs but wish their citations were better. In 2016 my team from Salesforce Research published our work on the Quasi-Recurrent Neural Network[1] (QRNN). The QRNN variants we describe are near identical (minGRU) or highly similar (minLSTM) to the work here. The QRNN was used, many years ago now, in the first version of Baidu's speech recognition system (Deep Voice [6]) and as part of Google's handwriting recognition system in Gboard[5] (2019). Even if there are expressivity trade-offs when using parallelizable RNNs they've shown historically they can work…

Paper and discussion history. A mention is not an endorsement.

Quasi-Recurrent Neural Networks — mentioned by bvm

Thu, 24 Sep 2020 08:52:45 GMT

Quasi: https://arxiv.org/abs/1611.01576

Paper and discussion history. A mention is not an endorsement.

Quasi-Recurrent Neural Networks — mentioned by Smerity

Wed, 01 Mar 2017 02:21:15 GMT

Disclosure: I'm one of the co-authors of the QRNN paper (James Bradbury, Stephen Merity, Caiming Xiong, Richard Socher) produced by Salesforce Research. There are many interesting advances that Deep Voice paper and implementation make but the part I'm excited by (and which might be transferable to other tasks that use RNNs) is showing that QRNNs are indeed generalizable to speech too - in this case in place of WaveNet. "WaveNet uses transposed convolutions for upsampling and conditioning. We find that our models perform better, train faster, and require fewer parameters if we instead first…

Paper and discussion history. A mention is not an endorsement.