Regularizing and Optimizing LSTM Language Models — mentioned by jszymborski

Wed, 24 Jul 2024 20:57:22 GMT

Salesforce has long been involved in publishing quality NLP papers, especially during Stephen Merity's tenure. Smerity's papers are some of my favourite. Check out https://ar5iv.labs.arxiv.org/html/1708.02182 And my all-time favourite https://ar5iv.labs.arxiv.org/html/1911.11423

Paper and discussion history. A mention is not an endorsement.

Regularizing and Optimizing LSTM Language Models — mentioned by cs702

Fri, 04 May 2018 12:47:14 GMT

This is the kind of problem for which LSTM RNNs -- and more recently, fully-attention-based deep neural nets -- produce state-of-the-art results. I wonder if the author ever tried using, say, an AWD LSTM RNN[a] or a Transformer-like model[b] for this task. Using an RNN or an attention model for this would eliminate the need for feature engineering such as: feature_1 = 1 if x_t is capitalized and y_t equals "NAME"; 0 otherwise. This is one of seven carefully engineered feature functions listed in the article, and the author states that the seven are only a partial list. Moreover, using a…

Paper and discussion history. A mention is not an endorsement.

Regularizing and Optimizing LSTM Language Models · HackerPapers discussions

Regularizing and Optimizing LSTM Language Models — mentioned by jszymborski

Regularizing and Optimizing LSTM Language Models — mentioned by cs702