A Less Biased Evaluation of Out-of-distribution Sample Detectors — mentioned by ericjang

Fri, 09 Nov 2018 03:13:31 GMT

Great question! This is actually a surprisingly deep problem in ML, known as "anomaly detection" or "out-of-distribution" (OoD) detection. Another way to formulate this question: "given training data that only tells you about digits, how do you know whether something is a digit or not?" Given that the training data never actually defines what isn't a digit, how can we ensure that the model actually sees a digit at test time? If we cannot ensure this (e.g. an adversary or the real world supplies inputs), how can we "filter out" bad inputs? A quick hack solution that works well in practice is…

Paper and discussion history. A mention is not an endorsement.

A Less Biased Evaluation of Out-of-distribution Sample Detectors · HackerPapers discussions

A Less Biased Evaluation of Out-of-distribution Sample Detectors — mentioned by ericjang