Intriguing properties of neural networks · HackerPapers discussions

Intriguing properties of neural networks — mentioned by YeGoblynQueenne

Fri, 07 Jan 2022 18:56:11 GMT

With the exception of counterfactual regret minimisation, which has nothing to do with deep learning, all those are applications of existing approaches that have become possible because of the increased expenditure of resources- processing power and data. There have been no theoretical advances, no algorithmic advances to speak of, no new knowledge learned in the last 20 years of reserach in deep learning. All that has been achieved is new SOTA on old benchmarks [1]. To make an analogy it's as if we could sort ever larger lists because we keep sorting them with ever bigger computers... but…

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by sennight

Fri, 19 Mar 2021 01:39:27 GMT

It isn't "science fiction" though, we have a mountain of examples that demonstrate how probabilistic AI fails when the training dataset doesn't perfectly align with the eventual application. It isn't restricted to AI either, there have been numerous studies showing that cross-race identification to be garbage data. <Insert other race> all look the same: https://onlinelibrary.wiley.com/doi/abs/10.1002/acp.898 This bus is an ostrich: https://arxiv.org/abs/1312.6199 Racist autofocus: https://sitn.hms.harvard.edu/flash/2020/racial-discriminatio... Google thinks black people look like gorillas…

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by verdverm

Sun, 08 Mar 2020 00:21:31 GMT

Original paper: https://arxiv.org/abs/1312.6199

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by LoSboccacc

Sun, 22 Dec 2019 09:45:08 GMT

the Uber incident investigation showed how a continuous input generated wildly varying output in practice. in theory the existence of adversarial images and adversarial training has been studied for a while, but the state of the art can still produce singularities around continuous inputs. start here: https://arxiv.org/pdf/1312.6199.pdf

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — HN submission

Thu, 31 Jan 2019 21:40:40 GMT

Intriguing properties of neural networks (2014)

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Intriguing properties of neural networks — mentioned by bagnalla

Sat, 09 Dec 2017 01:54:16 GMT

Verification of machine learning models, especially w.r.t. robustness to so-called "adversarial examples" [1], is a major research topic at the moment. There are certainly people worrying about it in the machine learning community, but it appears to be a very difficult problem. [1] https://arxiv.org/abs/1312.6199

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by pinouchon

Wed, 08 Nov 2017 16:16:55 GMT

Hi. Thanks for the great article. Can you elaborate on what you mean by "directions in activation space" ? If I understand it right: You take a few neurons in a layer, and you follow some linear combination of their weights; you are then walking along a random direction. You take a single neuron and walk along its weights; you walk along this neuron's direction. Is this correct? Also another comment: Szegedy et al [9] (https://arxiv.org/pdf/1312.6199.pdf) has the following abstract: > First, we find that there is no distinction between individual high level units and random linear…

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by haraldurt

Thu, 02 Nov 2017 23:50:44 GMT

Not necessarily. Adversarial examples have been shown to, for instance, be transferable across different networks with different hyperparameters (e.g., number of layers) trained on disjoint subsets of a training set [0, section 4.2]. There are more references from the paper linked by the OP. [0] https://arxiv.org/abs/1312.6199

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by anishathalye

Thu, 02 Nov 2017 19:10:25 GMT

It's probably referencing Szegedy et al. 2013 (https://arxiv.org/abs/1312.6199).

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by yorwba

Fri, 27 Oct 2017 13:42:55 GMT

Actually, no [1]. From the abstract: We can cause the network to misclassify an image by applying a certain imperceptible perturbation, which is found by maximizing the network's prediction error. In addition, the specific nature of these perturbations is not a random artifact of learning: the same perturbation can cause a different network, that was trained on a different subset of the dataset, to misclassify the same input. [1] https://arxiv.org/abs/1312.6199 EDIT: Also interesting: "Universal Adversarial Perturbations" https://arxiv.org/abs/1610.08401

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by moyix

Wed, 26 Jul 2017 01:39:55 GMT

Yes: > Adversarial training seeks to improve the generalization of a model when presented with adversarial examples at test time by proactively generating adversarial examples as part of the training procedure. This idea was first introduced by Szegedy et al. [SZS13] but was not yet practical because of the high computation cost of generating adversarial examples. Goodfellow et al. showed how to generate adversarial examples inexpensively with the fast gradient sign method and made it computationally efficient to generate large batches of adversarial examples during the training process…

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by mholt

Sat, 15 Apr 2017 16:22:39 GMT

This is called "adversarial training" and is ineffective because of transferability. Also the space of adversarial examples is thought to be large (especially as dimensionality increases), making robust adversarial training intractable in practice. See https://arxiv.org/abs/1312.6199 and https://arxiv.org/abs/1510.05328

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by BickNowstrom

Wed, 02 Nov 2016 21:48:36 GMT

Ensembles are also vulnerable to adversarial images. From one of the canonical papers on adversarial images: > In addition, the specific nature of these perturbations is not a random artifact of learning: the same perturbation can cause a different network, that was trained on a different subset of the dataset, to misclassify the same input. > ... suggest that adversarial examples are somewhat universal and not just the results of overfitting to a particular model or to the specific selection of the training set https://arxiv.org/abs/1312.6199 More recent research shows that one can make…

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by obastani

Thu, 20 Oct 2016 16:13:05 GMT

It's not at all the same -- self-driving cars can risk the lives not only of the passengers, but of innocent bystanders. Several commenters have pointed out problems such as rain, snow, darkness, etc., but there are more fundamental problems. For example, self-driving cars based on machine learning may be susceptible to adversarial attacks that cause the car to behave unpredictably [1,2]. How will self-driving cars know how to react when things go wrong, e.g., the stop light is broken? What about construction? I'm not saying humans react perfectly in these situations, but self-driving cars…

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by obastani

Thu, 20 Oct 2016 16:03:14 GMT

AI/ML algos cannot magically guarantee safety, for example, deep nets are known to be very un-robust: https://arxiv.org/abs/1312.6199. We are not yet at the point where AI/ML based algos can be guaranteed to be safe.

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by jkldotio

Tue, 10 May 2016 13:24:08 GMT

1. If that's the case perhaps another kind of blurring? "Intriguing properties of neural networks" (https://arxiv.org/pdf/1312.6199.pdf page 6) has examples where you get radically different classifications that I don't think would occur naturally or survive a blur with some random element, let alone two moving cameras and a sequence of images. As the title says it's an intriguing property, not necessarily a huge problem. 2. I honestly can't think of a situation where this could occur. It's the equivalent of kids shining lasers into the eyes of airline pilots, but the kids need a PhD in deep…

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by dllu

Sun, 13 Mar 2016 11:55:42 GMT

Failure to generalize is not always caused by overfitting. Even if there is no overfitting, deep neural networks seem to learn a surprisingly discontinuous function. Consequently, in rare cases, they can misclassify things with great confidence. [0] From the cited paper, > [Experimental results] suggest that adversarial examples are somewhat universal and not just the results of overfitting to a particular model or to the specific selection of the training set. Anyway, Monte Carlo Tree Search is bad at losing positions. In general, you want to delay the impending catastrophe as long as…

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by Houshalter

Mon, 23 Mar 2015 18:00:15 GMT

To be clear I meant this paper (http://arxiv.org/abs/1312.6199) as the original paper for adversarial images. I think they did try transferring them between very different NNs: >In addition, the specific nature of these perturbations is not a random artifact of learning: the same perturbation can cause a different network, that was trained on a different subset of the dataset, to misclassify the same input.

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by yummyfajitas

Mon, 16 Mar 2015 14:47:01 GMT

Algorithmic biases often reflect human biases I've been doing this for a while and I've literally never met a human who told an algorithm to overweight x[23] ("good looking"), x[48] ("is white") and x[873] ("is wealthy"), for x a 1,100-dimensional feature vector. Algorithms do have biases, but they are almost always orthogonal to the human ones. Witness, for example, all the recent "we can fool deep learning image recognition systems" papers. http://arxiv.org/abs/1412.1897 http://arxiv.org/abs/1312.6199 At this point I'm 90% sure you are a layperson who's never actually programmed such a…

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — HN submission

Wed, 04 Feb 2015 13:58:16 GMT

Intriguing Properties of Neural Networks [pdf]

19 points; 4 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Intriguing properties of neural networks — mentioned by colah3

Thu, 22 Jan 2015 18:16:45 GMT

You make some pretty strong attacks on the state of the art approach to computer vision. Could you please provide citations? As someone working on neural networks, although not really on computer vision, I do not believe several of the things you said are true. > > Pornographic images, for instance, are frequently identified not by the presence of particular body parts or structural features, but by the dominance of certain colors in the images > Neural networks This is definitely not what state of the art neural networks for computer vision do, although I'm sure there are many models out…

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — HN submission

Thu, 22 Jan 2015 04:37:33 GMT

Intriguing properties of neural networks

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Intriguing properties of neural networks — mentioned by AndrewOMartin

Tue, 30 Dec 2014 16:52:34 GMT

Don't forget "Intriguing properties of neural networks", otherwise known as "Does Deep Learning have deep flaws?". In sum, you can teach a network to say "that's a dog" when presented with a picture of a dog, but you'll also be able to (1) find an imperceptibly modified version of the image where it'll say "that's garbage" and (2) intelligently generate an image of noise that also gets recognised as a dog. (1) http://arxiv.org/abs/1312.6199 (2) http://www.newscientist.com/article/dn26691-optical-illusion...

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by larrydag

Tue, 09 Dec 2014 14:18:58 GMT

Another journal paper covering the same thing. http://arxiv.org/abs/1312.6199. And the article I got that references it. http://www.i-programmer.info/news/105-artificial-intelligenc...

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by hyperbovine

Sun, 02 Nov 2014 03:40:40 GMT

Bad things: http://arxiv.org/abs/1312.6199

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by abeppu

Sun, 02 Nov 2014 02:07:35 GMT

(a) The document linked by the OP is to a blog post that discusses and extensively quotes another blog post [1] which in turn discusses an actual paper [2]. Naturally the paper is where the good stuff is. (b) Both blog posts somewhat understate the problem. The adversarial examples given in the original paper aren't just classified differently than their parent image -- they're created to receive a specific classification. In the figure 5 of the arxiv version, for example, they show clear images of a school bus, temple, praying mantis, dog, etc, which all received the label "ostrich, Struthio…

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by sotte

Wed, 03 Sep 2014 12:43:39 GMT

If you're interested in the generalization ability of neural networks I can recommend the following paper: "Intriguing properties of neural networks" http://arxiv.org/pdf/1312.6199v4.pdf TLDR: The authors create adversarial examples, i.e., they slightly modify the original images, which look exactly the same to humans but neural networks can't classify the images correctly anymore. What does that imply on the ability to generalize? :) On a more general note: NN are always treated as something magic. I think a "sober view" is that NN are a special way to parameterize non-liner functions. This…

Paper and discussion history. A mention is not an endorsement.

Intriguing properties of neural networks — mentioned by ot

Wed, 09 Apr 2014 09:26:59 GMT

This is a very interesting perspective on neural networks, is it novel? I've never seen a geometric interpretation of NNs. In particular this: > (Apparently determining if knots are trivial is NP. This doesn’t bode well for neural networks.) Is there any theoretical research on obstructions to NN learning? Not that it would change much the practice (for instance, MLE learning of gaussian mixtures is NP-hard, but everybody does it anyway), but it could shed some light on the geometry of the "hard" instances. EDIT: for example, to get a sense of how ill-conditioned is deep NN learning, a recent…

Paper and discussion history. A mention is not an endorsement.