HackerPapers

arXiv:1603.08155cs.CV

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

Justin Johnson, Alexandre Alahi, Li Fei-Fei

Abstract

We consider image transformation problems, where an input image is transformed into an output image. Recent methods for such problems typically train feed-forward convolutional neural networks using a \emph{per-pixel} loss between the output and ground-truth images. Parallel work has shown that high-quality images can be generated by defining and optimizing \emph{perceptual} loss functions based on high-level features extracted from pretrained networks. We combine the benefits of both approaches, and propose the use of perceptual loss functions for training feed-forward networks for image transformation tasks. We show results on image style transfer, where a feed-forward network is trained to solve the optimization problem proposed by Gatys et al in real-time. Compared to the optimization-based method, our network gives similar qualitative results but is three orders of magnitude faster. We also experiment with single-image super-resolution, where replacing a per-pixel loss with a perceptual loss gives visually pleasing results.

arXiv comments
Journal reference
DOI
Subjects
cs.CVComputer Vision and Pattern Recognition cs.LGMachine Learning

On Hacker News

Linked by 5 identifiable commenters in 5 comments across 5 threads, from Jun 2016 to Oct 2017.

Comments linking this paper, per year.

Discussed in comments5 threads

  1. Show HN: Paint yourself in the style of classic art, in your browser 1 comment

    • Ahh I see, these are some resources that should help you. The original Neural Style Transfer paper should help you understand the loss functions involved: https://arxiv.org/abs/1508.06576 The paper introducing Real-Time Style Transfer is basically the algorithm used here…

      reiinakanoView on HNParent context

  2. Prisma’s next AI project is a fun selfie sticker maker called Sticky 1 comment

    • Slightly off-topic, but there's some pretty fascinating work that's been done recently on using neural networks for style transfer: https://arxiv.org/pdf/1603.08155v1.pdf http://www.anishathalye.com/2015/12/19/an-ai-that-can-mimic-...

      nilknView on HNParent context

  3. Deep Learning enables hearing aid wearers to pick out a voice in a crowded room 1 comment

    • This approach surprised me. Why are they doing feature extraction and then feeding that into a DNN? It seems much more straightforward to have the input of the network be noisy samples and the output be clean samples a la super resolution[0] in images. They probably wouldn't…

      chasView on HNParent context

  4. Prisma: neural-style instagram filters 1 comment

    • IIRC you only get to choose from a fixed set of styles to process your image, right? Then they're probably using this algorithm to precompute the style model and seriously speed up rendering: http://arxiv.org/abs/1603.08155

      j2kunView on HNParent context

  5. DeepArt.io introduces style transfer for videos 1 comment

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.