HackerPapers

arXiv:1609.05807cs.LG

Inherent Trade-Offs in the Fair Determination of Risk Scores

Jon Kleinberg, Sendhil Mullainathan, Manish Raghavan

Abstract

Recent discussion in the public sphere about algorithmic classification has involved tension between competing notions of what it means for a probabilistic classification to be fair to different groups. We formalize three fairness conditions that lie at the heart of these debates, and we prove that except in highly constrained special cases, there is no method that can satisfy these three conditions simultaneously. Moreover, even satisfying all three conditions approximately requires that the data lie in an approximate version of one of the constrained special cases identified by our theorem. These results suggest some of the ways in which key notions of fairness are incompatible with each other, and hence provide a framework for thinking about the trade-offs between them.

arXiv comments
To appear in Proceedings of Innovations in Theoretical Computer Science (ITCS), 2017
Journal reference
DOI
Subjects
cs.LGMachine Learning cs.CYComputers and Society stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 7 identifiable commenters in 8 comments across 8 threads, from Nov 2016 to Jun 2024. Found in 1 HN submission from Sep 2017 to Sep 2017. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments8 threads

  1. Mozilla.ai did what? When silliness goes dangerous 1 comment

    • On the flip side: A lot of claims of AI bias are overblown and/or straight-up disingenuous. Remember when ProPublica claimed that recidivism prediction models were biased against black people? (https://www.propublica.org/article/how-we-analyzed-the-compa...) They rigged their…

      Legend2440View on HNParent context

  2. Making a racist AI without really trying (2017) 1 comment

    • > Some people expect that fighting algorithmic racism is going to come with some sort of trade-off. Um, that's because we know it comes with trade-offs once you have the most optimal algorithm. See for instance https://arxiv.org/pdf/1610.02413.pdf. If your best performing…

      lalaland1125View on HNParent context

  3. AI at Google: our principles 1 comment

    • >Avoid creating or reinforcing unfair bias. I recommend _The impossibility of “fairness”: a generalized impossibility result for decisions_[0] and _Inherent Trade-Offs in the Fair Determination of Risk Scores_[1] [0] https://arxiv.org/pdf/1707.01195.pdf [1]…

      bobcostas55View on HNParent context

  4. Algorithmic decision making and the cost of fairness (2017) 1 comment

    • >> there is no magic algorithm that bypasses this fact. > Maybe there is No, there isn't. We actually have a mathematical proof (which is quite simple) why this is impossible. Specifically, following conditions can't be true at the same time: 1. groups differ in base rate 2.…

      sanxiynView on HNParent context

  5. Courts are using risk-assessment software to sentence criminals 1 comment

    • There was a Washington Post article and arXiv paper on this topic several months ago. [1][2] The problem boils down to the definition of bias. Northpointe, the company that owns COMPAS, says the algorithm treats black and white defendants equally, which is true. Others claim…

      bbig3831View on HNParent context

  6. Unexpected sources of bias in artificial intelligence 1 comment

    • Not only that, there are impossibility theorems that say you can't meet everyone's definition of "nondiscriminatory" except in a few trivial cases. https://arxiv.org/pdf/1609.05807v1.pdf I.e. every algorithm will be "discriminatory" unless you predictor is perfect (i.e. gets…

      yummyfajitasView on HNParent context

  7. A machine learning approach to identifying thought markers of suicidal subjects 1 comment

    • Propublica has an "interesting" definition of "fairness". It turns out there is a beautiful mathematical result that shows that the different definitions of "fairness" are inherently incompatible with each other and cannot be satisfied at the same time. This is the best article…

      aub3bhatView on HNParent context

  8. Facebook is harming our democracy 1 comment

    • The conclusion that numerate people are coming to regarding algorithms for court sentencing is quite different from what you describe. The conclusion there is simply a theorem: it's mathematically impossible to be both well calibrated (a black and white person receiving the same…

      yummyfajitasView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.