Deep Learning is Robust to Massive Label Noise · HackerPapers discussions

Deep Learning is Robust to Massive Label Noise — mentioned by whiplash451

Thu, 25 Jul 2024 05:40:32 GMT

Deep learning is robust to massive label noise [1] Not to say that data quality does not matter, but these noisy sets are still very useful. [1] https://arxiv.org/abs/1705.10694

Paper and discussion history. A mention is not an endorsement.

Deep Learning is Robust to Massive Label Noise — mentioned by dougabug

Fri, 21 Oct 2022 14:05:53 GMT

You need to be very careful about making sweeping generalizations based on a single personal anecdote. The really large data sets typically have very high error rates and sample biases. For instance, Google’s JTF300M is far noisier than ImageNet, which itself is hardly free of errors and biases. Any data set with hundreds of millions to billions of images will generally contain a large proportion of images and labels scraped from the web, w/ automatic filtering or pseudolabeling, perhaps w/ some degree of sampled verification by human labelers. In fact, generally DL is quite tolerant to label…

Paper and discussion history. A mention is not an endorsement.

Deep Learning is Robust to Massive Label Noise — mentioned by momenti

Tue, 22 Mar 2022 15:07:07 GMT

That's not entirely true. Neural networks are fairly robust to noisy training data (a.k.a. garbage).[0] Well, stochastic gradient descent has the noise in its name. More training data can compensate for noisy data to some extent.[1] I'm not sure know if model size can also compensate for noisy data though, but would not be surprised if it did. [0] https://arxiv.org/abs/1705.10694 [1] https://arxiv.org/abs/2202.01994

Paper and discussion history. A mention is not an endorsement.