Categorizing Variants of Goodhart's Law · HackerPapers discussions

Categorizing Variants of Goodhart's Law — HN submission

Sat, 17 Jan 2026 09:02:35 GMT

Categorizing Variants of Goodhart's Law

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Categorizing Variants of Goodhart's Law — mentioned by divan

Sat, 24 Feb 2024 16:17:30 GMT

Interesting links in the article and comments, but it seems like two different concepts are mixed here – the nature of bureaucracies and wrong incentives. I struggle to understand how cobra and rat cases are "institutions trying to preserve the problem". Cobra effect, in particular, is an immensely interesting topic of how rewarding the metrics can distort the system, otherwise known as Goodhart's effect. I highly recommend reading these two papers on the subject for anyone interested: - Categorizing Variants of Goodhart's Law [1] - Building less-flawed metrics: Understanding and creating…

Paper and discussion history. A mention is not an endorsement.

Categorizing Variants of Goodhart's Law — mentioned by divan

Sat, 17 Feb 2024 13:54:32 GMT

Must reads: - Categorizing Variants of Goodhart's Law [1] - Building less-flawed metrics: Understanding and creating better measurement and incentive systems [2] [1] https://arxiv.org/abs/1803.04585 [2] https://www.sciencedirect.com/science/article/pii/S266638992...

Paper and discussion history. A mention is not an endorsement.

Categorizing Variants of Goodhart's Law — mentioned by divan

Thu, 25 Jan 2024 11:17:29 GMT

There is a deeper and largely overlooked problem with sports judging, especially in the sports with artistic component (like gymnastics or figure skating) – Goodhart's effects. Goodhart's effect stands for "when the metric becomes a target it stops being a good metric". As soon as you make measurement part of the reward system, it changes the whole system. Take figure skating as an example. Since 2006, ISU has introduced a similar judging system - IJS – that uses points for elements and an increasingly complex code of rules on how to assign those points. Each element has a so-called "base…

Paper and discussion history. A mention is not an endorsement.

Categorizing Variants of Goodhart's Law — mentioned by divan

Thu, 01 Jun 2023 17:52:58 GMT

Right. Do you have any good recommendations to read on this matter? Here are a couple of good papers on how metrics distort systems and what can be done to mitigate this. [1] [2]. And then a new post by Cedric Chin on practical examples of mitigating Goodhart's law in Amazon. Really good read: [3] But generally, I feel like not many people or organizations care about this. The non-profit sports world is conservative and slow, organizations and corporations are also often slow, and there some psychological/social effects linked to the metrics-that-don't-work-anymore. --- - [1] Categorizing…

Paper and discussion history. A mention is not an endorsement.

Categorizing Variants of Goodhart's Law — mentioned by divan

Sat, 06 May 2023 11:15:33 GMT

Exams and sports competitions are a particularly nasty example of Campbell's Law (or Goodheart's Law) – collapsing the correlation between metric and the true goal. Exams, for example, are supposed to measure how much students progressed. "True goal" - learn some knowledge or master some skill. "Metric" is the quiz/tests/exam result. If you try to design an exam for any new skill or knowledge, you'll come up with a pretty good metric. But soon, people will start ignoring the true goal and focus on the metric. Instead of "measuring" how well they learned the skill, your metric measures "how…

Paper and discussion history. A mention is not an endorsement.

Categorizing Variants of Goodhart's Law — mentioned by davidmanheim

Sun, 17 Oct 2021 15:28:44 GMT

Others pointed out that it's hard to measure subjective factors well. I'd point to a different issue - if you have a metric for, say, wellbeing which is correlated with what you actually care about, putting pressure on other parts of the system - like maximizing engagement - will systemically warp those metrics to be less accurate. For an extensive discussion of how this can happen, see: https://arxiv.org/abs/1803.04585

Paper and discussion history. A mention is not an endorsement.

Categorizing Variants of Goodhart's Law — mentioned by pikwip

Fri, 17 Sep 2021 14:58:28 GMT

Here's a interesting paper I found that attempts to categorize the mechanisms by which Goodhart's Law operates in the real world. The variants are separated into Causal and Non-causal mechanisms. https://arxiv.org/abs/1803.04585

Paper and discussion history. A mention is not an endorsement.

Categorizing Variants of Goodhart's Law — mentioned by davidmanheim

Fri, 15 Jun 2018 17:27:28 GMT

That's often helpful, but not sufficient in general. See the dynamics discussed in my article here; https://arxiv.org/abs/1803.04585

Paper and discussion history. A mention is not an endorsement.

Categorizing Variants of Goodhart's Law — mentioned by davidmanheim

Fri, 15 Jun 2018 17:25:36 GMT

This is probably where I should mention my two longer articles that discuss this in depth; https://www.ribbonfarm.com/2016/06/09/goodharts-law-and-why-... https://www.ribbonfarm.com/2016/09/29/soft-bias-of-underspec... As well as my more recent paper categorizing the different ways it occurs; https://arxiv.org/abs/1803.04585

Paper and discussion history. A mention is not an endorsement.