HackerPapers

arXiv:1511.05392stat.ML

Learning the Dimensionality of Word Embeddings

Eric Nalisnick, Sachin Ravi

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

We describe a method for learning word embeddings with data-dependent dimensionality. Our Stochastic Dimensionality Skip-Gram (SD-SG) and Stochastic Dimensionality Continuous Bag-of-Words (SD-CBOW) are nonparametric analogs of Mikolov et al.'s (2013) well-known 'word2vec' models. Vector dimensionality is made dynamic by employing techniques used by Cote & Larochelle (2016) to define an RBM with an infinite number of hidden units. We show qualitatively and quantitatively that SD-SG and SD-CBOW are competitive with their fixed-dimension counterparts while providing a distribution over embedding dimensionalities, which offers a window into how semantics distribute across dimensions.

arXiv comments
Journal reference
DOI
Subjects
stat.MLMachine Learning (Statistics) cs.CLComputation and Language cs.LGMachine Learning

On Hacker News

Linked by 1 identifiable commenter in 1 comment across 1 thread, from Nov 2015 to Nov 2015.

Comments linking this paper, per year.

Discussed in comments1 thread

  1. Sense2vec – A Fast and Accurate Method for Word Sense Disambiguation 1 comment

    • Since this appears to be the 'related works' thread, I'll add my links here. A reasonable amount of work has already been done in disambiguation of the meanings of words with word embeddings, some of it trained, some untrained. These particular papers that have recently caught…

      benten10View on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.