Learning Simpler Language Models with the Differential State Framework — mentioned by ml_thoughts

Sun, 08 Oct 2017 00:05:10 GMT

I think an aspect of deep learning that is often overlooked is that it is still not clear how much of current algorithm performance is defined by local "obsession to detail" vs global "understanding" of the subject matter. The "tiramisu" layers here are an interesting example of this: they are built on dilated convolutions and one of their main selling points is that they can do calculations on a pixel-by-pixel basis, as opposed to standard methods which use are forced to compress information along the way through pooling/strided convolutions (basically taking multiple pixels and summarizing…

Paper and discussion history. A mention is not an endorsement.

Learning Simpler Language Models with the Differential State Framework · HackerPapers discussions

Learning Simpler Language Models with the Differential State Framework — mentioned by ml_thoughts