Confident Learning: Estimating Uncertainty in Dataset Labels · HackerPapers discussions

Confident Learning: Estimating Uncertainty in Dataset Labels — HN submission

Thu, 08 Sep 2022 15:23:22 GMT

Show HN: How simple (but clever) algorithms can find label issues in datasets

I built Vizzy as a hackathon project, with the goal of explaining how relatively simple (but clever) algorithms can be a powerful tool to automatically find issues in datasets, including label errors and out-of-distribution data. Vizzy uses a JavaScript port of (a part of) https://github.com/cleanlab/cleanlab, which implements the algorithms described in https://arxiv.org/abs/1911.00068. There are other neat technical nuggets in the implementation of Vizzy as well, including ML model training in the browser (using features from a pretrained ResNet-18, performing truncated SVD, and using an…

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Confident Learning: Estimating Uncertainty in Dataset Labels — HN submission

Wed, 17 Aug 2022 14:23:17 GMT

Show HN: Cleanlab Vizzy – automatically find label errors and bad data

Cleanlab (https://github.com/cleanlab/cleanlab) is a family of algorithms for automatically finding issues in datasets. It might seem surprising that it’s possible to automatically identify label errors and out-of-distribution data; Cleanlab does this using the algorithms published in https://arxiv.org/abs/1911.00068. Cleanlab’s algorithms, while clever, are actually relatively simple. To help myself (and others!) build intuition for how they work, I built Vizzy, an interactive demo that runs in the browser. Vizzy lets you experiment with an example dataset, tweak the labels, and run Cleanlab…

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Confident Learning: Estimating Uncertainty in Dataset Labels — mentioned by cgn

Sun, 04 Apr 2021 21:47:52 GMT

I am an author on the paper, and just saw this thread. Thanks for all of your comments. Note this work focuses on errors in test sets, not training labels. Test sets are foundation of benchmarking for machine learning. We study test sets (versus training sets), because training sets are already studied in-depth in confident learning (https://arxiv.org/pdf/1911.00068.pdf) and because before this paper, no one had studied (at scale, for lots of datasets) if these test sets had errors in the first place (beyond just looking at the well-known erroneous ImageNet dataset, for example) and do these…

Paper and discussion history. A mention is not an endorsement.

Confident Learning: Estimating Uncertainty in Dataset Labels — mentioned by cgn

Mon, 04 Nov 2019 14:29:39 GMT

Hi, Hackers. I'm excited to share confident learning for characterizing, finding, and learning with label errors in datasets. To promote and standardize future research in learning with noisy labels and weak supervision, I've also open-sourced the cleanlab Python package: https://pypi.org/project/cleanlab/ Post: https://l7.curtisnorthcutt.com/confident-learning Paper: https://arxiv.org/abs/1911.00068 Code: https://github.com/cgnorthcutt/cleanlab/ Abstract: Learning exists in the context of data, yet notions of confidence typically focus on model predictions, not label quality. Confident…

Paper and discussion history. A mention is not an endorsement.