HackerPapers

arXiv:1803.04585cs.AI

Categorizing Variants of Goodhart's Law

David Manheim, Scott Garrabrant

Abstract

There are several distinct failure modes for overoptimization of systems on the basis of metrics. This occurs when a metric which can be used to improve a system is used to an extent that further optimization is ineffective or harmful, and is sometimes termed Goodhart's Law. This class of failure is often poorly understood, partly because terminology for discussing them is ambiguous, and partly because discussion using this ambiguous terminology ignores distinctions between different failure modes of this general type. This paper expands on an earlier discussion by Garrabrant, which notes there are "(at least) four different mechanisms" that relate to Goodhart's Law. This paper is intended to explore these mechanisms further, and specify more clearly how they occur. This discussion should be helpful in better understanding these types of failures in economic regulation, in public policy, in machine learning, and in Artificial Intelligence alignment. The importance of Goodhart effects depends on the amount of power directed towards optimizing the proxy, and so the increased optimization power offered by artificial intelligence makes it especially critical for that field.

arXiv comments
10 pages
Journal reference
DOI
Subjects
cs.AIArtificial Intelligence q-fin.GNGeneral Finance stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 3 identifiable commenters in 9 comments across 8 threads, from Jun 2018 to Feb 2024. Found in 1 HN submission from Jan 2026 to Jan 2026. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments8 threads

  1. Institutions try to preserve the problem to which they are the solution 1 comment

    • Interesting links in the article and comments, but it seems like two different concepts are mixed here – the nature of bureaucracies and wrong incentives. I struggle to understand how cobra and rat cases are "institutions trying to preserve the problem". Cobra effect, in…

      divanView on HNParent context

  2. Automated Unit Test Improvement Using Large Language Models at Meta 1 comment

    • Must reads: - Categorizing Variants of Goodhart's Law [1] - Building less-flawed metrics: Understanding and creating better measurement and incentive systems [2] [1] https://arxiv.org/abs/1803.04585 [2] https://www.sciencedirect.com/science/article/pii/S266638992...

      divanView on HNParent context

  3. How AI is changing gymnastics judging 1 comment

    • There is a deeper and largely overlooked problem with sports judging, especially in the sports with artistic component (like gymnastics or figure skating) – Goodhart's effects. Goodhart's effect stands for "when the metric becomes a target it stops being a good metric". As soon…

      divanView on HNParent context

  4. Own-goal football (2022) 1 comment

    • Right. Do you have any good recommendations to read on this matter? Here are a couple of good papers on how metrics distort systems and what can be done to mitigate this. [1] [2]. And then a new post by Cedric Chin on practical examples of mitigating Goodhart's law in Amazon.…

      divanView on HNParent context

  5. How is it even possible to cheat on a musical performance practical exam? 1 comment

    • Exams and sports competitions are a particularly nasty example of Campbell's Law (or Goodheart's Law) – collapsing the correlation between metric and the true goal. Exams, for example, are supposed to measure how much students progressed. "True goal" - learn some knowledge or…

      divanView on HNParent context

  6. Whistleblower to Congress: Facebook products harm kids and democracy 1 comment

    • Others pointed out that it's hard to measure subjective factors well. I'd point to a different issue - if you have a metric for, say, wellbeing which is correlated with what you actually care about, putting pressure on other parts of the system - like maximizing engagement…

      davidmanheimView on HNParent context

  7. Goodhart's Law 1 comment

    • Here's a interesting paper I found that attempts to categorize the mechanisms by which Goodhart's Law operates in the real world. The variants are separated into Causal and Non-causal mechanisms. https://arxiv.org/abs/1803.04585

      pikwipView on HNParent context

  8. Goodhart's Law: When a measure becomes a target, it ceases to be a good measure 2 comments

    • This is probably where I should mention my two longer articles that discuss this in depth; https://www.ribbonfarm.com/2016/06/09/goodharts-law-and-why-... https://www.ribbonfarm.com/2016/09/29/soft-bias-of-underspec... As well as my more recent paper categorizing the different…

      davidmanheimView on HNParent context

    • That's often helpful, but not sufficient in general. See the dynamics discussed in my article here; https://arxiv.org/abs/1803.04585

      davidmanheimView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.