Exploring the Limits of Language Modeling · HackerPapers discussions

Exploring the Limits of Language Modeling — mentioned by visarga

Mon, 12 Jun 2017 07:46:51 GMT

This is the state of the art text generation result from Google: https://arxiv.org/abs/1602.02410 * < S > With even more new technologies coming onto the market quickly during the past three years , an increasing number of companies now must tackle the ever-changing and ever-changing environmental challenges online . < S > Check back for updates on this breaking news story . < S > About 800 people gathered at Hever Castle on Long Beach from noon to 2pm , three to four times that of the funeral cortege . ` < S > We are aware of written instructions from the copyright holder not to , in any way…

Paper and discussion history. A mention is not an endorsement.

Exploring the Limits of Language Modeling — mentioned by ma2rten

Sun, 26 Feb 2017 02:44:08 GMT

If I understand you correctly you are comparing Markov Chains on words to a RNN over characters. That's not fair. This paper shows a large LSTM outperform n-gram models: "In this paper we have shown that RNN LMs can be trained on large amounts of data, and outperform competing models including carefully tuned N-grams. [...] Unlike previous work, we do not require to interpolate both the RNN LM and the N-gram, and the gains of doing so are rather marginal." https://arxiv.org/abs/1602.02410

Paper and discussion history. A mention is not an endorsement.

Exploring the Limits of Language Modeling — mentioned by Smerity

Fri, 30 Sep 2016 08:05:45 GMT

$0.9 per K80 GPU per hour, while expensive, opens up so many opportunities - especially when you can get a properly connected machine. Just as an example of the change this entails for deep learning, the recent "Exploring the Limits of Language Modeling"[1] paper from Google used 32 K40 GPUs. While the K40 / K80 are not the most recent generation of GPU, they're still powerful beasts, and finding a large number of them set up well is a challenge for most. In only 2 hours, their model beat previous state of the art results. Their new best result was achieved after three weeks of compute. With…

Paper and discussion history. A mention is not an endorsement.

Exploring the Limits of Language Modeling — mentioned by nl

Wed, 14 Sep 2016 23:24:47 GMT

Most modern language modelling is RNN based now; eg: https://arxiv.org/abs/1602.02410 (which I think is SOTA).

Paper and discussion history. A mention is not an endorsement.

Exploring the Limits of Language Modeling — mentioned by nl

Wed, 17 Feb 2016 02:31:14 GMT

Which data are you after? The ImageNet data is public and they released the pretrained model. They've promised to release (or already have released) the models for Exploring the Limits of Language Modeling[1] which was trained on the 1 B Word Benchmark corpus[2] which is also public data. Note that for these, the trained models are often more immediately useful. The language modelling model was trained for 3 weeks on 32 Tesla K40s. That's not something many can replicate casually. [1] http://arxiv.org/pdf/1602.02410v2.pdf [2] http://www.statmt.org/lm-benchmark/

Paper and discussion history. A mention is not an endorsement.

Exploring the Limits of Language Modeling — HN submission

Thu, 11 Feb 2016 05:11:42 GMT

Exploring the Limits of Language Modeling

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Exploring the Limits of Language Modeling — HN submission

Tue, 09 Feb 2016 18:14:26 GMT

Exploring the Limits of Language Modeling

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.