HackerPapers

arXiv:1410.3916cs.AI

Memory Networks

Jason Weston, Sumit Chopra, Antoine Bordes

Abstract

We describe a new class of learning models called memory networks. Memory networks reason with inference components combined with a long-term memory component; they learn how to use these jointly. The long-term memory can be read and written to, with the goal of using it for prediction. We investigate these models in the context of question answering (QA) where the long-term memory effectively acts as a (dynamic) knowledge base, and the output is a textual response. We evaluate them on a large-scale QA task, and a smaller, but more complex, toy task generated from a simulated world. In the latter, we show the reasoning power of such models by chaining multiple supporting sentences to answer questions that require understanding the intension of verbs.

arXiv comments
Journal reference
DOI
Subjects
cs.AIArtificial Intelligence cs.CLComputation and Language stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 5 identifiable commenters in 7 comments across 6 threads, from Mar 2015 to Oct 2025. Found in 3 HN submissions from Nov 2014 to Mar 2015. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments6 threads

  1. A History of Large Language Models 1 comment

    • Are you sure? I wrote an essay at the end of 2016 about the state of AI research and at the time researchers were demolishing benchmarks like FAIR's bAbI which involved generating answers to questions. I wrote back then about story comprehension and programming robots by giving…

      mike_hearnView on HNParent context

  2. Hybrid computing using a neural network with dynamic external memory 1 comment

    • This is probably the most important research direction in modern neural network research. Neural networks are great at pattern recognition. Things like LSTMs allow pattern recognition through time, so they can develop "memories". This is useful in things like understanding text…

      nlView on HNParent context

  3. FANN.js – Fast Artificial Neural Network Compiled with Emscripten 1 comment

    • Jason Weston (Facebook) etc's work on Memory Nets[1] Anything to do with transfer learning (the 2013 Zero-Shot Learning Through Cross-Modal Transfer[2] paper is a good place to start) The increasing amount of demos around using NNs to generate "things" that look kinda-almost…

      nlView on HNParent context

  4. Are There Deep Reasons Underlying the Pathologies of Deep Learning Algorithms? [pdf] 1 comment

    • Semantic hashing by similarity is one way to create more sensible representations (Re: Proposition 2): http://www.cs.toronto.edu/~rsalakhu/papers/semantic_final.pd... This would also fit well into memory networks: http://arxiv.org/abs/1410.3916, SDM…

      bra-ketView on HNParent context

  5. By 2029 no computer or “machine intelligence” will have passed the Turing Test 1 comment

    • I was thinking mostly of this paper: http://arxiv.org/abs/1503.02364 Here is some other relevant work of recent: http://arxiv.org/abs/1502.05698 http://arxiv.org/abs/1410.3916 https://www.reddit.com/r/thisisthewayitwillbe/comments/2zi8i...…

      HoushalterView on HNParent context

  6. Facebook's Memory Networks Demo 2 comments

    • Memory Networks: http://arxiv.org/abs/1410.3916

      smhxView on HNParent context

    • It uses a class of RNN they call a MemNN to do something that looks a lot like inference using phrase embeddings. It's a pretty revolutionary approach - no one seems to have tried anything very similar since the 1990's. The related work section[1] is worth reading.…

      nlView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.