HackerPapers

arXiv:1902.11097cs.CV

Predictive Inequity in Object Detection

Benjamin Wilson, Judy Hoffman, Jamie Morgenstern

Abstract

In this work, we investigate whether state-of-the-art object detection systems have equitable predictive performance on pedestrians with different skin tones. This work is motivated by many recent examples of ML and vision systems displaying higher error rates for certain demographic groups than others. We annotate an existing large scale dataset which contains pedestrians, BDD100K, with Fitzpatrick skin tones in ranges [1-3] or [4-6]. We then provide an in-depth comparative analysis of performance between these two skin tone groupings, finding that neither time of day nor occlusion explain this behavior, suggesting this disparity is not merely the result of pedestrians in the 4-6 range appearing in more difficult scenes for detection. We investigate to what extent time of day, occlusion, and reweighting the supervised loss during training affect this predictive bias.

arXiv comments
Journal reference
DOI
Subjects
cs.CVComputer Vision and Pattern Recognition cs.LGMachine Learning stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 4 identifiable commenters in 4 comments across 4 threads, from Mar 2019 to Aug 2019. Found in 1 HN submission from Mar 2019 to Mar 2019. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments4 threads

  1. How to Teach an Iris Scanner That the Eye It’s Looking at Is Dead 1 comment

    • That progress is not made all at once. It's a problem with a known fix; but just like buffer overflows, a known fix doesn't always get implemented everywhere. For example, this 2019 paper: https://arxiv.org/pdf/1902.11097.pdf

      klyrsView on HNParent context

  2. MIT Researcher Exposing Bias in Facial Recognition Tech Triggers Amazon’s Wrath 1 comment

    • The story doesn’t cite the specific paper. But I think I submitted the paper last month: “Predictive Inequality in Object Detection” https://news.ycombinator.com/item?id=19379827 https://arxiv.org/abs/1902.11097 I’m not denying the conclusion of the paper, but it does have a lot…

      js2View on HNParent context

  3. Self-driving cars may be more likely to hit you if you have dark skin 1 comment

    • The article is very thin on details. It serves mostly to point to an abstract for a research paper : https://arxiv.org/abs/1902.11097 The abstract is unfortunately also very light on detail. It appears as though the researchers trained a number of classification systems using…

      ubermanView on HNParent context

  4. Autonomous Cars Can't Recognise Pedestrians with Darker Skin Tones 1 comment

    • The headline here is not a good description of the actual paper (https://arxiv.org/pdf/1902.11097.pdf). The difference in average precision in bounding boxes between the LS (light skin) and DS (dark skin) observed by the researchers was around 1-10%. Not "can't recognize".…

      czrView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.