HackerPapers

arXiv:1412.6572stat.ML

Explaining and Harnessing Adversarial Examples

Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy

Abstract

Several machine learning models, including neural networks, consistently misclassify adversarial examples---inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed input results in the model outputting an incorrect answer with high confidence. Early attempts at explaining this phenomenon focused on nonlinearity and overfitting. We argue instead that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature. This explanation is supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets. Moreover, this view yields a simple and fast method of generating adversarial examples. Using this approach to provide examples for adversarial training, we reduce the test set error of a maxout network on the MNIST dataset.

arXiv comments
Journal reference
DOI
Subjects
stat.MLMachine Learning (Statistics) cs.LGMachine Learning

On Hacker News

Linked by 11 identifiable commenters in 17 comments across 14 threads, from Mar 2015 to Jul 2025. Found in 3 HN submissions from Dec 2014 to Dec 2015. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments14 threads

  1. Supabase MCP can leak your entire SQL database 1 comment

    • > prompt injection attacks are the LLM equivalent of social engineering, That's anthropomorphizing. Maybe some of the basic "ignore previous instructions" style attacks feel like that, but the category as a whole is just adversarial ML attacks that work because the LLM doesn't…

      andy99View on HNParent context

  2. Adversarial.io – Fighting mass image recognition 1 comment

    • I think it's unlikely that the noise will generally have any relation to the target class. I can't find anywhere they say exactly which attack method they use, so it's hard to say for certain, but all of the attacks I'm aware of don't generate noise that has any…

      ImnimoView on HNParent context

  3. A Guide to Synthesizing Adversarial Examples 1 comment

    • Yes: > Adversarial training seeks to improve the generalization of a model when presented with adversarial examples at test time by proactively generating adversarial examples as part of the training procedure. This idea was first introduced by Szegedy et al. [SZS13] but was not…

      moyixView on HNParent context

  4. Apollo – An open autonomous driving platform 1 comment

    • There are results showing that machine learning can be completely fooled by images that are slightly modified, so little that the human eye can not perceive any difference. Have a look at the images in this paper on "adversarial examples": https://arxiv.org/abs/1412.6572

      mhueView on HNParent context

  5. Understanding deep learning requires rethinking generalization 1 comment

    • Adversarial noise isn't specific to deep learning models. Most model working on high-dimensional input will confidently misclassify noise with high confidence if you construct the noise right. The original "adversarial pixel" paper demonstrates this with logistic regression.…

      dbeckerView on HNParent context

  6. Real and Stealthy Attacks on State-Of-the-Art Face Recognition [pdf] 1 comment

    • https://arxiv.org/abs/1412.6572 is a Goodfellow et. al. (ie, the person who's Tweet you note above) paper where they show adversarial examples do apply across architectures (as you state above), but they show that using the adversarial examples during training has a…

      nlView on HNParent context

  7. Universal adversarial perturbations 1 comment

    • Short answer: No. Computing these perturbations requires an expensive optimization with multiple passes through the dataset, and this would be prohibitively expensive to do in the inner-most loop of training. There are other work in the literature describing faster algorithms to…

      sherjilozairView on HNParent context

  8. Impressive Demonstration of Facebook's AI by Yann LeCun 1 comment

    • Humans can be easily fooled too: https://en.wikipedia.org/wiki/Optical_illusion Anyway this issue has been explained and fixed somewhat in this paper: http://arxiv.org/abs/1412.6572 And karpathy has a post which explains it…

      HoushalterView on HNParent context

  9. Suddenly, a leopard print sofa appears 1 comment

    • We spend so much effort trying to engineer intelligence, when we would get a lot farther reverse engineering intelligence. Whenever AI makes a big advance the analog was already known my neuroscientists. The problem with attempting to understand intelligence by reverse…

      abrichrView on HNParent context

  10. Are There Deep Reasons Underlying the Pathologies of Deep Learning Algorithms? [pdf] 1 comment

    • He's also referring to this follow up paper which attempts to explain the issue and show that it exists in other models too: http://arxiv.org/abs/1412.6572

      HoushalterView on HNParent context

  11. A Probabilistic Theory of Deep Learning 1 comment

    • > If this model can provide an explanation for the small noises impacting NN performance on images (from karpathy.github.io, posted to HN earlier today) then that would be rocking This paper [0] does a pretty good job of giving an explanation for that. [0]…

      NvnView on HNParent context

  12. Breaking Linear Classifiers on ImageNet 1 comment

    • The original paper found that nets pretrained as autoencoders were the least susceptible to it, but still failed. IIRC the follow up paper (http://arxiv.org/abs/1412.6572) tested it or at least claimed it wasn't a solution.

      HoushalterView on HNParent context

  13. By 2029 no computer or “machine intelligence” will have passed the Turing Test 1 comment

    • Those are images incredibly optimized to fool and exploit those neural networks. However on the actual image recognition tasks they were designed for they beat humans. Anyway this paper has a method for addressing this problem: http://arxiv.org/abs/1412.6572

      HoushalterView on HNParent context

  14. Images that fool computer vision raise security concerns 4 comments

    • A paper came out that explains this effect and a method of minimizing it: http://arxiv.org/abs/1412.6572 Basically neural networks and many other machine learning methods are highly linear and continuous. So changing an input just slightly should change the output just slightly.…

      HoushalterView on HNParent context

    • This paper tried that with positive results: http://arxiv.org/abs/1412.6572

      HoushalterView on HNParent context

    • Overfitting has nothing to do with it. See this paper: http://arxiv.org/abs/1412.6572 I believe the original paper tried ensembles and even got the images to work on different networks.

      HoushalterView on HNParent context

    • Very interesting. First of all here's a youtube video associated with the paper -- https://www.youtube.com/watch?v=M2IebCN9Ht4 . Second some on here have posted about the Svegedy, Goodfellow, and Shlens paper http://arxiv.org/abs/1412.6572 which discusses the opposite effect.…

      masterfresView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.