Learning Efficient Algorithms with Hierarchical Attentive Memory · HackerPapers discussions

Learning Efficient Algorithms with Hierarchical Attentive Memory — mentioned by Sergej-Shegurin

Mon, 14 Mar 2016 16:19:13 GMT

(1) Adversarial learning is unsupervised and works great. Most of language modeling is unsupervised (you predict next word, but it's not real supervision because it's self-supervision). There're many works in computer vision which are unsupervised and still give more or less reasonable performance. See f.e. http://arxiv.org/pdf/1511.05045v2.pdf for unsupervised learning in action recognition, also http://arxiv.org/pdf/1511.06434v2.pdf and http://www.arxiv-sanity.com/search?q=unsupervised (2) ImageNet supervision gives you much information to solve other computer vision tasks. So perhaps we…

Paper and discussion history. A mention is not an endorsement.

Learning Efficient Algorithms with Hierarchical Attentive Memory — mentioned by Smerity

Mon, 07 Mar 2016 06:24:51 GMT

Thanks for the great comment! For a direct link to the visual saliency maps from the paper (Figure 6) which @nicklo mentions: http://i.imgur.com/DRfaNxB.png Being able to see where a neural attention mechanism is fascinating and allows for far more than just introspection. Indeed, one of the earliest and coolest examples are neural attention mechanisms learning to align words between languages without any help[1]! http://i.imgur.com/J5zFZzN.png New papers on neural attention seem to be coming out every few weeks. I devoted much of my weekend to a naive implementation of hierarchical attentive…

Paper and discussion history. A mention is not an endorsement.