HackerPapers

arXiv:1603.01417cs.NE

Dynamic Memory Networks for Visual and Textual Question Answering

Caiming Xiong, Stephen Merity, Richard Socher

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

Neural network architectures with memory and attention mechanisms exhibit certain reasoning capabilities required for question answering. One such architecture, the dynamic memory network (DMN), obtained high accuracy on a variety of language tasks. However, it was not shown whether the architecture achieves strong results for question answering when supporting facts are not marked during training or whether it could be applied to other modalities such as images. Based on an analysis of the DMN, we propose several improvements to its memory and input modules. Together with these changes we introduce a novel input module for images in order to be able to answer visual questions. Our new DMN+ model improves the state of the art on both the Visual Question Answering dataset and the \babi-10k text question-answering dataset without supporting fact supervision.

arXiv comments
Journal reference
DOI
Subjects
cs.NENeural and Evolutionary Computing cs.CLComputation and Language cs.CVComputer Vision and Pattern Recognition

On Hacker News

Linked by 1 identifiable commenter in 2 comments across 2 threads, from Jul 2016 to Sep 2016. Found in 1 HN submission from Mar 2016 to Mar 2016. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments2 threads

    • I disagree. There exist examples of system displaying very non-trivial, open-ended behavior, and yet they are still not agi. A good example of this is https://arxiv.org/abs/1603.01417 . When trained it is able to answer free-form questions about images.

      sapphireblueView on HNParent context

    • 3 years ago many systems that exist today would be regarded as "scifi" or "maybe 10 years into the future". One of the most striking examples is Visual Question Answering with dynamic memory networks [1]. The model is given raw pixels and encoded question in natural language, it…

      sapphireblueView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.