Dynamic Word Embeddings for Evolving Semantic Discovery · HackerPapers discussions

Dynamic Word Embeddings for Evolving Semantic Discovery — mentioned by yorwba

Sun, 05 Jan 2020 11:28:48 GMT

How about a model that can detect the semantic shift of "Amazon" from "river" to "company"? https://arxiv.org/abs/1703.00607 They're not even using transformers, just simple word embeddings.

Paper and discussion history. A mention is not an endorsement.

Dynamic Word Embeddings for Evolving Semantic Discovery — mentioned by yorwba

Wed, 23 Jan 2019 22:05:01 GMT

I think meaning isn't really separable from structure. But assuming you're talking about semantics vs. syntax, then continuous representations can certainly do a lot more than just syntax. They can model how words change in meaning over time [1], they can identify words that mean the same in different languages [2] and use that to translate whole sentences [3] without ever seeing a direct translation. Of course none of those tasks are solved to perfection yet, but some kind of meaning is definitely being captured. [1] https://arxiv.org/abs/1703.00607 [2] https://arxiv.org/abs/1710.04087 [3]…

Paper and discussion history. A mention is not an endorsement.

Dynamic Word Embeddings for Evolving Semantic Discovery — mentioned by yorwba

Sat, 24 Feb 2018 08:51:41 GMT

If you know beforehand what kind of document you are dealing with, you can refine the word vectors for your given task, or even train them from scratch if you have enough data. In general, though, you'll end up with a mixture of meanings. The vector for "Apple" would be somewhere between fruits and companies, while the vector for "Amazon" would be somewhere between rivers and companies. An interesting paper looked at how these associations changed over time [1]. It was also featured recently on The Morning Paper [2], in case you prefer a summary with added context. Although those ambiguities…

Paper and discussion history. A mention is not an endorsement.