Data Scaling Laws in NMT: The Effect of Noise and Architecture — mentioned by momenti

Tue, 22 Mar 2022 15:07:07 GMT

That's not entirely true. Neural networks are fairly robust to noisy training data (a.k.a. garbage).[0] Well, stochastic gradient descent has the noise in its name. More training data can compensate for noisy data to some extent.[1] I'm not sure know if model size can also compensate for noisy data though, but would not be surprised if it did. [0] https://arxiv.org/abs/1705.10694 [1] https://arxiv.org/abs/2202.01994

Paper and discussion history. A mention is not an endorsement.

Data Scaling Laws in NMT: The Effect of Noise and Architecture · HackerPapers discussions

Data Scaling Laws in NMT: The Effect of Noise and Architecture — mentioned by momenti