Deep contextualized word representations · HackerPapers discussions

Deep contextualized word representations — mentioned by NaiveBayesian

Thu, 07 Sep 2023 18:18:10 GMT

There was actually a series of language models named after Sesame Street characters back in 2018-2020, starting with ELMo, then BERT, ERNIE (a different model from 2019), Big Bird, ... There are likely some more that I missed. ELMo: https://arxiv.org/abs/1802.05365 BERT: https://arxiv.org/abs/1810.04805 ERNIE: https://arxiv.org/abs/1904.09223v1 Big Bird: https://arxiv.org/abs/2007.14062

Paper and discussion history. A mention is not an endorsement.

Deep contextualized word representations — mentioned by vikashrungta

Wed, 08 Feb 2023 16:19:46 GMT

I posted a list of papers on twitter, and will be posting a summary for each of them as well. here is the list https://twitter.com/vrungta/status/1623343807227105280 Unlocking the Secrets of AI: A Journey through the Foundational Papers by @vrungta (2023) 1. "Attention is All You Need" (2017) - https://arxiv.org/abs/1706.03762 (Google Brain) 2. "Generative Adversarial Networks" (2014) - https://arxiv.org/abs/1406.2661 (University of Montreal) 3. "Dynamic Routing Between Capsules" (2017) - https://arxiv.org/abs/1710.09829 (Google Brain) 4. "Unsupervised Representation Learning with Deep…

Paper and discussion history. A mention is not an endorsement.

Deep contextualized word representations — mentioned by cs702

Fri, 12 Oct 2018 01:45:45 GMT

These guys have done what looks like an impressive amount of work... but I was disappointed to see there's no mention of generative language modeling at all and only a very brief mention of key advances in image generation over the past few years. Examples of generative language modeling ignored by this paper (not even mentioned): * Unsupervised Transformer - https://blog.openai.com/language-unsupervised/ * ELMo - https://arxiv.org/abs/1802.05365 * ULMFit - https://arxiv.org/abs/1801.06146 Examples of generative image modeling ignored by this paper (some are mentioned only in passing): * Glow…

Paper and discussion history. A mention is not an endorsement.

Deep contextualized word representations — mentioned by visarga

Sat, 22 Sep 2018 06:34:21 GMT

There's a long list of papers on that - 'multi-sense word embeddings'. But more recently we have found that passing the raw character embeddings through a two layer BiLSTM will resolve the ambiguity of meaning from context - 'ElMO'. https://arxiv.org/abs/1802.05365 (state of the art)

Paper and discussion history. A mention is not an endorsement.