HackerPapers

arXiv:1412.1897cs.CV

Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images

Anh Nguyen, Jason Yosinski, Jeff Clune

Abstract

Deep neural networks (DNNs) have recently been achieving state-of-the-art performance on a variety of pattern-recognition tasks, most notably visual classification problems. Given that DNNs are now able to classify objects in images with near-human-level performance, questions naturally arise as to what differences remain between computer and human vision. A recent study revealed that changing an image (e.g. of a lion) in a way imperceptible to humans can cause a DNN to label the image as something else entirely (e.g. mislabeling a lion a library). Here we show a related result: it is easy to produce images that are completely unrecognizable to humans, but that state-of-the-art DNNs believe to be recognizable objects with 99.99% confidence (e.g. labeling with certainty that white noise static is a lion). Specifically, we take convolutional neural networks trained to perform well on either the ImageNet or MNIST datasets and then find images with evolutionary algorithms or gradient ascent that DNNs label with high confidence as belonging to each dataset class. It is possible to produce images totally unrecognizable to human eyes that DNNs believe with near certainty are familiar objects, which we call "fooling images" (more generally, fooling examples). Our results shed light on interesting differences between human vision and current DNNs, and raise questions about the generality of DNN computer vision.

arXiv comments
To appear at CVPR 2015
Journal reference
DOI
Subjects
cs.CVComputer Vision and Pattern Recognition cs.AIArtificial Intelligence cs.NENeural and Evolutionary Computing

On Hacker News

Linked by 17 identifiable commenters in 17 comments across 17 threads, from Dec 2014 to Jun 2019. Found in 2 HN submissions from Dec 2014 to Dec 2014. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments17 threads

    • It will always be a game of cat and mouse. Even with Neural networks involved :D. [0] https://arxiv.org/abs/1412.1897 [1] https://arxiv.org/abs/1710.08864

      penagwinView on HNParent context

  1. iPhone X 1 comment
    • For some background, see https://arxiv.org/pdf/1412.1897.pdf which has some cool visuals of pictures that make neural networks to strange things. Looks like this group might have solved adversarial examples (or rather, found that they're not a problem in practice after all) for…

      ogdanView on HNParent context

  2. Teaching Machines to Draw 1 comment
    • TLDR: The point is to correct a known flaw with image recognition algorithms. I'm not a professional researcher, but here's what I gather from reading other articles: One major problem with image recognition in machines is that while they are generally able to recognize real…

      jxcoleView on HNParent context

    • > I don't understand how Google's algorithm can be misled into finding sexiness in those. I'm reminded of a paper for which the authors generated different pictures of static that fooled neural network image classifiers into confidently identifying them as different objects…

      reView on HNParent context

    • Basically this video ignores the history of machine learning in general. Jumping from Expert Systems to Neural Networks and Deep Learning is actually ignoring 36 years (and billions of dollars) of research http://machinelearning.org/icml.html (Breiman, Quinlan, Mitchell…

      aficionadoView on HNParent context

    • That's a good point and it also brings to mind a way to fight it. This http://arxiv.org/pdf/1412.1897v4.pdf could be one way. You could generate images which are not of someone who wants to protect his privacy, but tweak them to strongly correlate to an image of that person by…

      feelixView on HNParent context

    • > given that we now have programs that look at a photograph and say "Women wearing a hat, sitting on a bar stool and drinking wine" That may be; but then there's also this: http://arxiv.org/abs/1412.1897

      discardoramaView on HNParent context

    • I find it less interesting in terms of applications. I am sure finding the causes of these illusions will help in understanding aspects of how the system, human or machine, subject to the illusion operates; however, so many of them depend on either a static image or a fixed…

      jkldotioView on HNParent context

    • Definitely agree here, want to add a link to this paper which shows how far we still have to go (and questions whether the current models will ever replicate human vision): http://arxiv.org/abs/1412.1897

      jhundalView on HNParent context

    • It's entirely possible for biometrics to spoofed. For example, folks have already figured out how to trick deep neural nets: http://arxiv.org/abs/1412.1897 Not to mention bio-engineering possibilities on the horizon. Of course, just because we can use a certain technology…

      cafebeenView on HNParent context

    • Algorithmic biases often reflect human biases I've been doing this for a while and I've literally never met a human who told an algorithm to overweight x[23] ("good looking"), x[48] ("is white") and x[873] ("is wealthy"), for x a 1,100-dimensional feature vector. Algorithms do…

      yummyfajitasView on HNParent context

  3. Deep Learning Reading List 1 comment
    • In (2)'s paper [2], the images are really neat. Staring at it for a few moments, I feel like I can see where the computer is coming from. A bit. The robin, armadillo, centipede, peacock, and bubble all actually have a little swirl of features that - to me at least - resemble the…

      HCIdivision17View on HNParent context

    • Automatic feature choice can actually lead to a set of features that resembles V1 receptive fields (as demonstrated by Olshausen and Field in https://courses.cs.washington.edu/courses/cse528/11sp/Olshau...). I recently attended a talk by Geoff Hinton on "capsules." He pointed…

      simonsterView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.