HackerPapers

arXiv:1508.06576cs.CV

A Neural Algorithm of Artistic Style

Leon A. Gatys, Alexander S. Ecker, Matthias Bethge

Abstract

In fine art, especially painting, humans have mastered the skill to create unique visual experiences through composing a complex interplay between the content and style of an image. Thus far the algorithmic basis of this process is unknown and there exists no artificial system with similar capabilities. However, in other key areas of visual perception such as object and face recognition near-human performance was recently demonstrated by a class of biologically inspired vision models called Deep Neural Networks. Here we introduce an artificial system based on a Deep Neural Network that creates artistic images of high perceptual quality. The system uses neural representations to separate and recombine content and style of arbitrary images, providing a neural algorithm for the creation of artistic images. Moreover, in light of the striking similarities between performance-optimised artificial neural networks and biological vision, our work offers a path forward to an algorithmic understanding of how humans create and perceive artistic imagery.

arXiv comments
Journal reference
DOI
Subjects
cs.CVComputer Vision and Pattern Recognition cs.NENeural and Evolutionary Computing q-bio.NCNeurons and Cognition

On Hacker News

Linked by 24 identifiable commenters in 28 comments across 28 threads, from Aug 2015 to Apr 2023. Found in 5 HN submissions from Aug 2015 to Oct 2015. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments28 threads

    • > In September 2019 this must have been a GAN face. Well, yes,¹ but actually no. StyleGAN1's public release was February 2019, and it's capable of far more than just faces. > Three years ago there was nothing remotely as impressive as modern GPT style I predicted that! Albeit…

      wizzwizz4View on HNParent context

  1. Ask HN: ML Papers to Implement 1 comment
    • A lot depends on what you're interested in. Some papers that are runnable on a laptop CPU (so long as you stick to small image sizes/tasks): 1) Generative Adversarial Networks (https://arxiv.org/abs/1406.2661). Good practice to have a custom training loops, different optimisers…

      phonebucketView on HNParent context

    • That is one of the two concerns. The other concern, is that will be a paid product. It's users decide to pay or not, based on the quality of the code suggestions. The quality of the code suggestions are tied to the quality of the training corpus. The manufacturer of the product…

      belterView on HNParent context

    • > I really don't see how machines could distill the ideas out of a text and reinterpret them in a nice way. Well, machines are already doing it with paintings: https://arxiv.org/pdf/1508.06576v2.pdf. There's no technical reason for not being able to achieve this sort of 'machine…

      bpizziView on HNParent context

  2. FastPhotoStyle from Nvidia 1 comment
    • Ah okay right this is the answer. Previous approaches [1] are deep generative models that you have to optimize for each input, whereas here you run just a forward evaluation on a model that you've trained beforehand. I would still argue the term closed-form is misleading here…

      dbranesView on HNParent context

    • Absolutely you can do 'AI Research' without a degree in maths. Sure you need a grounding in linear algebra, stats, probability and calculus, but not much more than a CS or physics degree will teach you. That stuff is indeed learned easily, and it's not what the Reddit user is…

      joshvmView on HNParent context

    • Ahh I see, these are some resources that should help you. The original Neural Style Transfer paper should help you understand the loss functions involved: https://arxiv.org/abs/1508.06576 The paper introducing Real-Time Style Transfer is basically the algorithm used here…

      reiinakanoView on HNParent context

    • > If you succed in manufacturing a stigma against arXiv papers That's not exactly my intention, and your statement goes in line with what I said.. > sure I've cited software in the past, and having an article to go with it is even better where "software" could also include blog…

      radarsat1View on HNParent context

    • I guess you already have read about Prisma App. Here is the research paper it is based on.. -> https://arxiv.org/pdf/1508.06576.pdf

      haloboy777View on HNParent context

    • Hi savanaly, When we talk about "what makes Bach sound like Bach," the technical concept we have in mind is the recent work in computer vision on style transfer. For example, https://arxiv.org/abs/1508.06576 We are excited to work on adapting these models to the musical domain!…

      jthickstunView on HNParent context

    • Maybe this?: https://www.youtube.com/watch?v=Rfhxy2w3Qi4 Using a gram matrix across several layers to hold the style, and more conventional activation optimization for the content. Trained on VGG. It combines the two loss functions of style and content to optimise noise towards…

      n17r4mView on HNParent context

  3. Google Noto Fonts 1 comment
    • I wonder whether they used neural style transfer to automatically generate them. https://arxiv.org/pdf/1508.06576.pdf (See p. 5 for examples.)

      AlexCoventryView on HNParent context

    • They say they use a variation on the neural art transfer algorithm: "We apply an algorithm developed by Leon Gatys, Alexander Ecker and Matthias Bethge. " https://deepart.io/page/about/ https://arxiv.org/abs/1508.06576 There are various tweaks for how to do it and some hacks to…

      andreykView on HNParent context

    • The demo uses WebGL, and if you can't get the demo to work, you can here(https://github.com/Erkaman/regl-cnn) find a recorded gif that shows what it is supposed to look like. This demo does handwritten digit recognition by evaluating a Convolutional Neural Network on the GPU…

      erkamanView on HNParent context

    • The demo uses WebGL, and if you can't get the demo to work, you can here(https://github.com/Erkaman/regl-cnn) find a recorded gif that shows what it is supposed to look like. This demo does handwritten digit recognition by evaluating a Convolutional Neural Network on the GPU…

      erkamanView on HNParent context

    • Take a look at these: * Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis (http://arxiv.org/abs/1601.04589) * A Neural Algorithm of Artistic Style (http://arxiv.org/abs/1508.06576) They do much more than photoshopping. It's actual synthesis of…

      visargaView on HNParent context

  4. [dead] 1 comment
    • According to their about page, that site is using this algorithm http://arxiv.org/pdf/1508.06576.pdf whereas neural-image-analogies is a combination of http://www.mrl.nyu.edu/projects/image-analogies/index.html and http://arxiv.org/pdf/1601.04589v1.pdf Basically you can do a few…

      awentzonlineView on HNParent context

    • Here's the backend it's based on: https://github.com/jcjohnson/neural-style and the paper: http://arxiv.org/pdf/1508.06576v2.pdf

      narratorView on HNParent context

    • Original paper this is based on http://arxiv.org/abs/1508.06576 Neat to see a web application implementation of it, though looking through the code suggests the actual algorithm implementation is based on https://github.com/jcjohnson/neural-style

      andreykView on HNParent context

  5. Visual Turing Test 1 comment
    • Note that all of the computer generated pieces were created by this impressive deep net algorithm (paper prepublished in 8/2015): http://arxiv.org/abs/1508.06576 Somehow I cannot believe how fast they got the idea to create profit from it: http://deepart.io/

      DanielleMolloyView on HNParent context

    • One weird trick for parallelizing ConvNets: http://arxiv.org/abs/1404.5997 Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks: http://arxiv.org/abs/1511.06434 A Neural Network of Artistic Style: http://arxiv.org/abs/1508.06576

      temuzeView on HNParent context

    • Not really sure why this page made it to HN. This group's most interesting work was the paper that outlines "style transfer"[1], which is what all those photos-painted-in-the-style-of-van-gogh-etc pictures[2] that went around a few months ago were using. [1]…

      nlView on HNParent context

    • This article has a clickbait headline and no technical details. Here's the actual (pre-print) paper on this technique: http://arxiv.org/abs/1508.06576

      panicView on HNParent context

  6. [dead] 1 comment
    • More Information can be found here: https://www.youtube.com/watch?v=-R9bJGNHltQ&feature=youtu.be The paper is here: http://arxiv.org/abs/1508.06576v1

      VozzeView on HNParent context

    • A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, Matthias Bethge http://arxiv.org/abs/1508.06576

      deepnetView on HNParent context

    • Paper: http://arxiv.org/abs/1508.06576 Related discussion on the paper: https://news.ycombinator.com/item?id=10141516 Additional example: http://imgur.com/a/jeJB6

      myth_busterView on HNParent context

  7. [dead] 1 comment
    • URLs Examples [0]: https://i.imgur.com/sb8dHcY.png [1]: http://imgur.com/a/jeJB6 Paper: http://arxiv.org/abs/1508.06576 Related discussion on the paper: https://news.ycombinator.com/item?id=10141516

      myth_busterView on HNParent context

    • http://arxiv.org/abs/1508.06576 [non - pdf] Here we introduce an artificial system based on a Deep Neural Network that creates artistic images of high perceptual quality. The system uses neural representations to separate and recombine content and style of arbitrary images…

      deepnetView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.