MASS: Masked Sequence to Sequence Pre-training for Language Generation — mentioned by alephnerd

Tue, 17 Sep 2024 03:07:30 GMT

> Care to give some examples In the 2010s they were the leader in NLP and the precursor of LLMs like GPT3/3.5/4/4o Machine Translation with Human Parity (2018) - https://arxiv.org/abs/1803.05567 MT-DNN (2019) - https://arxiv.org/abs/1901.11504 MASS (2019) - https://arxiv.org/abs/1905.02450 VALL-E (2023) - https://arxiv.org/abs/2301.02111 VALL-E 2 (2024) - https://arxiv.org/abs/2406.05370 While OpenAI was the first to monetize an LLM at scale via ChatGPT, it's still the early stages of this field, and there is a lot of innovation that can still be leveraged, especially in non-English language…

Paper and discussion history. A mention is not an endorsement.

MASS: Masked Sequence to Sequence Pre-training for Language Generation · HackerPapers discussions

MASS: Masked Sequence to Sequence Pre-training for Language Generation — mentioned by alephnerd