HackerPapers

arXiv:1502.05698cs.AI

Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks

Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M. Rush, Bart van Merriënboer, Armand Joulin, Tomas Mikolov

Abstract

One long-term goal of machine learning research is to produce methods that are applicable to reasoning and natural language, in particular building an intelligent dialogue agent. To measure progress towards that goal, we argue for the usefulness of a set of proxy tasks that evaluate reading comprehension via question answering. Our tasks measure understanding in several ways: whether a system is able to answer questions via chaining facts, simple induction, deduction and many more. The tasks are designed to be prerequisites for any system that aims to be capable of conversing with a human. We believe many existing learning systems can currently not solve them, and hence our aim is to classify these tasks into skill sets, so that researchers can identify (and then rectify) the failings of their systems. We also extend and improve the recently introduced Memory Networks model, and show it is able to solve some, but not all, of the tasks.

arXiv comments
Journal reference
DOI
Subjects
cs.AIArtificial Intelligence cs.CLComputation and Language stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 6 identifiable commenters in 8 comments across 8 threads, from Mar 2015 to Oct 2025. Found in 2 HN submissions from Feb 2015 to Feb 2015. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments8 threads

  1. A History of Large Language Models 1 comment

    • Are you sure? I wrote an essay at the end of 2016 about the state of AI research and at the time researchers were demolishing benchmarks like FAIR's bAbI which involved generating answers to questions. I wrote back then about story comprehension and programming robots by giving…

      mike_hearnView on HNParent context

  2. DeepText: Facebook's text understanding engine 1 comment

    • Facebook have one of the leading teams on doing Deep Learning question answering. Jason Weston et. al wrote a paper introducing a machine-generated textual dataset[1] similar to what you describe, eg: Daniel picked up the football. Daniel drops the football. Daniel got the milk.…

      nlView on HNParent context

  3. Inferring Algorithmic Patterns with Stack 1 comment

    • Here is a recent Facebook research paper that addresses the problem I believe this paper is intended to solve. Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks http://arxiv.org/pdf/1502.05698v6.pdf This recent paper is an algorithm to solve simple question…

      andrewtbhamView on HNParent context

  4. Towards Neural Network-based Reasoning 1 comment

    • It is true that this is an artificial test, but it seems to me that doing well at this is a necessary prerequisite to do well at more complex scenarios. It's worth noting that these tests were introduced by Jason Weston[1] as "A Set of Prerequisite Toy Tasks"[2] Also, they…

      nlView on HNParent context

  5. Hound voice search NLP demo [video] 1 comment

    • Here's a chronological reading list of techniques Traditional NLP -> 2012, IBM Watson papers[1] Watson is probably the pinnacle of "traditional" style NLP (ie, Tokenization/Lemmatization/Part of Speech Tagging/Framing/Knowledge Engineering/etc) Word2Vec -> 2013, Google paper[2]…

      nlView on HNParent context

  6. By 2029 no computer or “machine intelligence” will have passed the Turing Test 1 comment

    • I was thinking mostly of this paper: http://arxiv.org/abs/1503.02364 Here is some other relevant work of recent: http://arxiv.org/abs/1502.05698 http://arxiv.org/abs/1410.3916 https://www.reddit.com/r/thisisthewayitwillbe/comments/2zi8i...…

      HoushalterView on HNParent context

  7. Facebook's Memory Networks Demo 1 comment

    • More: https://research.facebook.com/researchers/1543934539189348 http://arxiv.org/abs/1502.05698 http://www.thespermwhale.com/jaseweston/babi/tasks_1-20_v1-1... So, anyone have open source code that can perform these tasks?

      solveView on HNParent context

  8. Facebook invents an intelligence test for machines 1 comment

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.