Universal Language Model Fine-Tuning with Subword Tokenization for Polish — mentioned by mci

Sat, 29 Feb 2020 18:03:10 GMT

The term you are looking for may be "highly inflected". Side note: IMHO, you are exaggerating the ability of Polish to form long compounds. Dissecting the "Bezbarwne zielone idee wściekle śpią" example from https://arxiv.org/pdf/1810.10222.pdf#page=3 reveals no words longer than 4 morphemes: bez-BARW-n-e ZIEL-on-e IDE-e WŚCIEK-l-e ŚP-ią, where I put word roots in uppercase and bound morphemes in lowercase. The longest sequences of morphemes (for a loose definition of morpheme) I can think of are conditional mood of verbs with double prefixes like po-wy-CHODZI-ł-y-by-ście. However, the…

Paper and discussion history. A mention is not an endorsement.

Universal Language Model Fine-Tuning with Subword Tokenization for Polish — mentioned by jph00

Sat, 29 Feb 2020 15:47:26 GMT

Yes you're right, in our NLP course we used Turkish as our example. But for the book I mentioned Polish due to this paper: https://arxiv.org/abs/1810.10222 . But as you say, now the word "agglutinative" isn't technically correct. I'm actually not sure what the right word is to describe languages that have lots of big compounds with no spaces. (Which is the key issue here, as to why we need subword tokenization techniques).

Paper and discussion history. A mention is not an endorsement.

Universal Language Model Fine-Tuning with Subword Tokenization for Polish · HackerPapers discussions

Universal Language Model Fine-Tuning with Subword Tokenization for Polish — mentioned by mci

Universal Language Model Fine-Tuning with Subword Tokenization for Polish — mentioned by jph00