Intriguing properties of neural networks

With the exception of counterfactual regret minimisation, which has nothing to do with deep learning, all those are applications of existing approaches that have become possible because of the increased expenditure of resources- processing power and data. There have been no…

YeGoblynQueenneJan 7, 2022View on HNParent context

It isn't "science fiction" though, we have a mountain of examples that demonstrate how probabilistic AI fails when the training dataset doesn't perfectly align with the eventual application. It isn't restricted to AI either, there have been numerous studies showing that…

sennightMar 19, 2021View on HNParent context

Original paper: https://arxiv.org/abs/1312.6199

verdvermMar 8, 2020View on HNParent context

the Uber incident investigation showed how a continuous input generated wildly varying output in practice. in theory the existence of adversarial images and adversarial training has been studied for a while, but the state of the art can still produce singularities around…

LoSboccaccDec 22, 2019View on HNParent context

Verification of machine learning models, especially w.r.t. robustness to so-called "adversarial examples" [1], is a major research topic at the moment. There are certainly people worrying about it in the machine learning community, but it appears to be a very difficult problem.…

bagnallaDec 9, 2017View on HNParent context

Hi. Thanks for the great article. Can you elaborate on what you mean by "directions in activation space" ? If I understand it right: You take a few neurons in a layer, and you follow some linear combination of their weights; you are then walking along a random direction. You…

pinouchonNov 8, 2017View on HNParent context

It's probably referencing Szegedy et al. 2013 (https://arxiv.org/abs/1312.6199).

anishathalyeNov 2, 2017View on HNParent context

Not necessarily. Adversarial examples have been shown to, for instance, be transferable across different networks with different hyperparameters (e.g., number of layers) trained on disjoint subsets of a training set [0, section 4.2]. There are more references from the paper…

haraldurtNov 2, 2017View on HNParent context

Actually, no [1]. From the abstract: We can cause the network to misclassify an image by applying a certain imperceptible perturbation, which is found by maximizing the network's prediction error. In addition, the specific nature of these perturbations is not a random artifact…

yorwbaOct 27, 2017View on HNParent context

Yes: > Adversarial training seeks to improve the generalization of a model when presented with adversarial examples at test time by proactively generating adversarial examples as part of the training procedure. This idea was first introduced by Szegedy et al. [SZS13] but was not…

moyixJul 26, 2017View on HNParent context

This is called "adversarial training" and is ineffective because of transferability. Also the space of adversarial examples is thought to be large (especially as dimensionality increases), making robust adversarial training intractable in practice. See…

mholtApr 15, 2017View on HNParent context

Ensembles are also vulnerable to adversarial images. From one of the canonical papers on adversarial images: > In addition, the specific nature of these perturbations is not a random artifact of learning: the same perturbation can cause a different network, that was trained on a…

BickNowstromNov 2, 2016View on HNParent context

AI/ML algos cannot magically guarantee safety, for example, deep nets are known to be very un-robust: https://arxiv.org/abs/1312.6199. We are not yet at the point where AI/ML based algos can be guaranteed to be safe.

obastaniOct 20, 2016View on HNParent context

It's not at all the same -- self-driving cars can risk the lives not only of the passengers, but of innocent bystanders. Several commenters have pointed out problems such as rain, snow, darkness, etc., but there are more fundamental problems. For example, self-driving cars based…

obastaniOct 20, 2016View on HNParent context

1. If that's the case perhaps another kind of blurring? "Intriguing properties of neural networks" (https://arxiv.org/pdf/1312.6199.pdf page 6) has examples where you get radically different classifications that I don't think would occur naturally or survive a blur with some…

jkldotioMay 10, 2016View on HNParent context

Failure to generalize is not always caused by overfitting. Even if there is no overfitting, deep neural networks seem to learn a surprisingly discontinuous function. Consequently, in rare cases, they can misclassify things with great confidence. [0] From the cited paper, >…

dlluMar 13, 2016View on HNParent context

To be clear I meant this paper (http://arxiv.org/abs/1312.6199) as the original paper for adversarial images. I think they did try transferring them between very different NNs: >In addition, the specific nature of these perturbations is not a random artifact of learning: the…

HoushalterMar 23, 2015View on HNParent context

Algorithmic biases often reflect human biases I've been doing this for a while and I've literally never met a human who told an algorithm to overweight x[23] ("good looking"), x[48] ("is white") and x[873] ("is wealthy"), for x a 1,100-dimensional feature vector. Algorithms do…

yummyfajitasMar 16, 2015View on HNParent context

You make some pretty strong attacks on the state of the art approach to computer vision. Could you please provide citations? As someone working on neural networks, although not really on computer vision, I do not believe several of the things you said are true. > > Pornographic…

colah3Jan 22, 2015View on HNParent context

Don't forget "Intriguing properties of neural networks", otherwise known as "Does Deep Learning have deep flaws?". In sum, you can teach a network to say "that's a dog" when presented with a picture of a dog, but you'll also be able to (1) find an imperceptibly modified version…

AndrewOMartinDec 30, 2014View on HNParent context

Another journal paper covering the same thing. http://arxiv.org/abs/1312.6199. And the article I got that references it. http://www.i-programmer.info/news/105-artificial-intelligenc...

larrydagDec 9, 2014View on HNParent context

(a) The document linked by the OP is to a blog post that discusses and extensively quotes another blog post [1] which in turn discusses an actual paper [2]. Naturally the paper is where the good stuff is. (b) Both blog posts somewhat understate the problem. The adversarial…

abeppuNov 2, 2014View on HNParent context

Bad things: http://arxiv.org/abs/1312.6199

hyperbovineNov 2, 2014View on HNParent context

If you're interested in the generalization ability of neural networks I can recommend the following paper: "Intriguing properties of neural networks" http://arxiv.org/pdf/1312.6199v4.pdf TLDR: The authors create adversarial examples, i.e., they slightly modify the original…

sotteSep 3, 2014View on HNParent context

This is a very interesting perspective on neural networks, is it novel? I've never seen a geometric interpretation of NNs. In particular this: > (Apparently determining if knots are trivial is NP. This doesn’t bode well for neural networks.) Is there any theoretical research on…

otApr 9, 2014View on HNParent context