Recurrent Neural Network Training with Dark Knowledge Transfer — mentioned by visarga

Sat, 26 Nov 2016 06:32:55 GMT

Yeah, but you don't want to run an ensemble of 10 RNNs on your phone, or in the cloud for that matter, when you got billions of queries. It's too expensive. In practice the ensemble model is compactly transferred into a single network. In order to do that, they train a new network to copy the outputs of the ensemble, exploiting "dark knowledge". Recurrent Neural Network Training with Dark Knowledge Transfer - https://arxiv.org/abs/1505.04630v5

Paper and discussion history. A mention is not an endorsement.

Recurrent Neural Network Training with Dark Knowledge Transfer · HackerPapers discussions

Recurrent Neural Network Training with Dark Knowledge Transfer — mentioned by visarga