HackerPapers

arXiv:1606.03490cs.LG

The Mythos of Model Interpretability

Zachary C. Lipton

Abstract

Supervised machine learning models boast remarkable predictive capabilities. But can you trust your model? Will it work in deployment? What else can it tell you about the world? We want models to be not only good, but interpretable. And yet the task of interpretation appears underspecified. Papers provide diverse and sometimes non-overlapping motivations for interpretability, and offer myriad notions of what attributes render models interpretable. Despite this ambiguity, many papers proclaim interpretability axiomatically, absent further explanation. In this paper, we seek to refine the discourse on interpretability. First, we examine the motivations underlying interest in interpretability, finding them to be diverse and occasionally discordant. Then, we address model properties and techniques thought to confer interpretability, identifying transparency to humans and post-hoc explanations as competing notions. Throughout, we discuss the feasibility and desirability of different notions, and question the oft-made assertions that linear models are interpretable and that deep neural networks are not.

arXiv comments
presented at 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), New York, NY
Journal reference
DOI
Subjects
cs.LGMachine Learning cs.AIArtificial Intelligence cs.CVComputer Vision and Pattern Recognition cs.NENeural and Evolutionary Computing stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 5 identifiable commenters in 6 comments across 5 threads, from Feb 2017 to May 2020. Found in 1 HN submission from Jul 2016 to Jul 2016. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments5 threads

  1. Explainable Deep Learning: A Field Guide for the Uninitiated 1 comment

    • Zachary Lipton has a really good taxonomy of the different things people refer to when they talk about interpretability and explainability here: https://arxiv.org/pdf/1606.03490.pdf

      owenshen24View on HNParent context

  2. A watershed moment for protein structure prediction 1 comment

    • I work professionally in machine learning for over ten years now and I deeply dispute what you say, and many other practitioners do as well. > “not particularly clear how the features actually interact to reach the prediction.“ This is quite true of many models, such as linear…

      mlthoughts2018View on HNParent context

  3. Why are we using black box models in AI when we don’t need to? (2019) 2 comments

    • “Explainability” is a subjective idea that allows people to act as gatekeepers about the use of a model through arbitrary politics. “The Mythos of Model Interpretability” is good reading on this. https://arxiv.org/pdf/1606.03490.pdf

      mlthoughts2018View on HNParent context

    • This is a great point. There is a general lack of understanding about what it means for models to be interpretable & explainable. These words get thrown around often by people who don't understand the definition, and also the trade off with accuracy. Some papers i found…

      1_over_nView on HNParent context

  4. The Mythos of Model Interpretability in Machine Learning 1 comment
  5. Mark Cuban on Why You Need to Study Artificial Intelligence 1 comment

    • The following article has an interesting treatment of interpretability: https://arxiv.org/pdf/1606.03490.pdf

      ylemView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.