HackerPapers

arXiv:1603.06042cs.CL

Globally Normalized Transition-Based Neural Networks

Daniel Andor, Chris Alberti, David Weiss, Aliaksei Severyn, Alessandro Presta, Kuzman Ganchev, Slav Petrov, Michael Collins

Abstract

We introduce a globally normalized transition-based neural network model that achieves state-of-the-art part-of-speech tagging, dependency parsing and sentence compression results. Our model is a simple feed-forward neural network that operates on a task-specific transition system, yet achieves comparable or better accuracies than recurrent models. We discuss the importance of global as opposed to local normalization: a key insight is that the label bias problem implies that globally normalized models can be strictly more expressive than locally normalized models.

arXiv comments
Journal reference
DOI
Subjects
cs.CLComputation and Language cs.LGMachine Learning cs.NENeural and Evolutionary Computing

On Hacker News

Linked by 4 identifiable commenters in 4 comments across 3 threads, from May 2016 to Jun 2016.

Comments linking this paper, per year.

Discussed in comments3 threads

  1. Intro to Hidden Markov Models (2010) [pdf] 1 comment

    • Still, beam search can be simulated and improved by running decision processes in parallel. For example, instead of learning the sequence labeling as a sequence of n decisions (where n is the length of the sequence) you can learn sequence labeling as a sequence of 3n+1 decisions…

      supercarrotView on HNParent context

  2. SyntaxNet: Neural Models of Syntax 1 comment

    • http://arxiv.org/abs/1603.06042 this is the paper. the idea is based on the incredible incremental perceptron coupled with beam search for having different hypotheses at the same time trying to find a policy which will build them and select the best one…

      vagabondvectorView on HNParent context

  3. Announcing SyntaxNet: The World’s Most Accurate Natural Language Parser 2 comments

    • Evidence that this is the most accurate parser is here; the previous approach mentioned is a March 2016 paper, "Globally Normalized Transition-Based Neural Networks," http://arxiv.org/abs/1603.06042 "On a standard benchmark consisting of randomly drawn English newswire sentences…

      xigencyView on HNParent context

    • According to their paper (http://arxiv.org/pdf/1603.06042v1.pdf), the technique can also be applied to sentence compression. It would be cool if Google publishes that example code/training-data as well.

      the_deciderView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.