HackerPapers

arXiv:1506.05869cs.CL

A Neural Conversational Model

Oriol Vinyals, Quoc Le

Abstract

Conversational modeling is an important task in natural language understanding and machine intelligence. Although previous approaches exist, they are often restricted to specific domains (e.g., booking an airline ticket) and require hand-crafted rules. In this paper, we present a simple approach for this task which uses the recently proposed sequence to sequence framework. Our model converses by predicting the next sentence given the previous sentence or sentences in a conversation. The strength of our model is that it can be trained end-to-end and thus requires much fewer hand-crafted rules. We find that this straightforward model can generate simple conversations given a large conversational training dataset. Our preliminary results suggest that, despite optimizing the wrong objective function, the model is able to converse well. It is able extract knowledge from both a domain specific dataset, and from a large, noisy, and general domain dataset of movie subtitles. On a domain-specific IT helpdesk dataset, the model can find a solution to a technical problem via conversations. On a noisy open-domain movie transcript dataset, the model can perform simple forms of common sense reasoning. As expected, we also find that the lack of consistency is a common failure mode of our model.

arXiv comments
ICML Deep Learning Workshop 2015
Journal reference
DOI
Subjects
cs.CLComputation and Language

On Hacker News

Linked by 10 identifiable commenters in 12 comments across 12 threads, from Jun 2015 to Jun 2026. Found in 3 HN submissions from Jun 2015 to Jul 2015. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments12 threads

  1. Ask HN: What was your "oh shit" moment with GenAI? 1 comment

    • My personal "oh shit" moment was in 2015, when this paper came out: https://arxiv.org/abs/1506.05869 It showed me that a model trained only on movie subtitles data exhibited some (very primitive) reasoning. I have been working on Deep Learning and later LLMs ever since.

      ma2rtenView on HNParent context

  2. A man who invented VR goggles 50 years too soon (2016) 1 comment
  3. Towards a conversational agent that can chat about anything 1 comment

    • In 2015 Google claimed to have created a chat bot that can do common-sense reasoning and do tech support just by reading a sufficiently large dataset: https://arxiv.org/pdf/1506.05869v1.pdf

      gamblerView on HNParent context

  4. AI learns to write its own code by stealing from other programs 1 comment

    • I'm not sure if you're aware, but this has already been done, pretty much literally: https://arxiv.org/abs/1506.05869

      p1eskView on HNParent context

  5. Show HN: Spamnesty: Waste spammers' time 1 comment

    • Using machine learning like this https://arxiv.org/pdf/1506.05869.pdf would be fun for this and avoid(?) canned responses...

      mallamanisView on HNParent context

  6. The truth about deep learning 1 comment

    • > If this is the state of the art in machine translation, how on Earth can you expect a machine that can converse on human level in three years? Here are 2 good papers--one from last year, one very recent--that might change your mind. https://arxiv.org/pdf/1506.05869.pdf…

      dave_sullivanView on HNParent context

  7. Why AlphaGo Is Such a Big Deal 1 comment

    • Exactly, because it has a shot. Writing that it doesn't have a chance is just effect of lack of knowledge in ML field. You can see some work that was already done in this research paper, before AlphaGo success. http://arxiv.org/pdf/1506.05869v1.pdf

      lossoloView on HNParent context

  8. Generating Factoid Questions with RNNs: The 30M Factoid Question-Answer Corpus 1 comment

    • Read this paper from July of last year and it'll make more sense as to why this research is interesting: http://arxiv.org/pdf/1506.05869.pdf. Seeding a dataset with human labels, then generating new, better data is pretty cool. Similar to deep mind watching go games, then…

      dave_sullivanView on HNParent context

  9. BotLibre – Free Open Artificial Intelligence for Everyone 1 comment

    • It seems like an old way to solve the problem. I would be quite surprised if it gave good results compared to recent advances such as the LSTM-based paper “A Neural Conversational Model” [0]. In fact, I doubt it is even competitive compared to Cleverbot. It does try to compare…

      espadrineView on HNParent context

  10. Facebook M – The Anti-Turing Test 1 comment

    • Simulating bad typing is only necessary to fake a human. Here, having bad typing when faking an AI is stranger. That said, AIs trained through the chat transcripts of a large number of conversations may produce mistakes. I remember reading a paper that gave good results that…

      espadrineView on HNParent context

  11. A Beginner’s Guide to Restricted Boltzmann Machines 1 comment

    • summary: http://www.businessinsider.com/google-tests-new-artificial-i... paper (worth reading): http://arxiv.org/pdf/1506.05869v2.pdf

      taimur38View on HNParent context

  12. Artificial Intelligence Machine Gets Testy with Its Programmer 1 comment

    • The actual paper is actually way more interesting, with lots of examples of conversations: http://arxiv.org/pdf/1506.05869v2.pdf

      cocoflunchyView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.