HackerPapers

arXiv:1609.04802cs.CV

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, Wenzhe Shi

Abstract

Despite the breakthroughs in accuracy and speed of single image super-resolution using faster and deeper convolutional neural networks, one central problem remains largely unsolved: how do we recover the finer texture details when we super-resolve at large upscaling factors? The behavior of optimization-based super-resolution methods is principally driven by the choice of the objective function. Recent work has largely focused on minimizing the mean squared reconstruction error. The resulting estimates have high peak signal-to-noise ratios, but they are often lacking high-frequency details and are perceptually unsatisfying in the sense that they fail to match the fidelity expected at the higher resolution. In this paper, we present SRGAN, a generative adversarial network (GAN) for image super-resolution (SR). To our knowledge, it is the first framework capable of inferring photo-realistic natural images for 4x upscaling factors. To achieve this, we propose a perceptual loss function which consists of an adversarial loss and a content loss. The adversarial loss pushes our solution to the natural image manifold using a discriminator network that is trained to differentiate between the super-resolved images and original photo-realistic images. In addition, we use a content loss motivated by perceptual similarity instead of similarity in pixel space. Our deep residual network is able to recover photo-realistic textures from heavily downsampled images on public benchmarks. An extensive mean-opinion-score (MOS) test shows hugely significant gains in perceptual quality using SRGAN. The MOS scores obtained with SRGAN are closer to those of the original high-resolution images than to those obtained with any state-of-the-art method.

arXiv comments
19 pages, 15 figures, 2 tables, accepted for oral presentation at CVPR, main paper + some supplementary material
Journal reference
DOI
Subjects
cs.CVComputer Vision and Pattern Recognition stat.MLMachine Learning (Statistics)

On Hacker News

Linked by 8 identifiable commenters in 10 comments across 7 threads, from Sep 2016 to Jul 2020. Found in 2 HN submissions from Sep 2016 to Feb 2017. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments7 threads

  1. Ask HN: What weird or hard problems are you trying to solve? 1 comment

    • Upscaling videos. This is a hard problem. It may not even be perfectly solvable at any time in the future. When you increase the size of any kind of raster image, you're creating new information from the old. There's been some pretty good approaches out there, like this [1][2].…

      shaknaView on HNParent context

  2. With Google's RAISR, images can be up to 75% smaller without losing detail 1 comment

    • Here's a list of various image interpolation techniques: - http://www.ifp.illinois.edu/~jyang29/papers/chap1.pdf * Image super-resolution: Historical overview and future challenges - http://www.robots.ox.ac.uk/~vgg/publications/papers/pickup08... * Machine Learning in…

      ingenterView on HNParent context

  3. Major advancements in Deep Learning in 2016 1 comment

    • Because that isn't true? Looking at the first thing (GANs): developed by Ian Goodfellow, who is now at OpenAI. The first linked paper (https://arxiv.org/pdf/1605.05396v2.pdf) is out of University of Michigan and Max Planck Institute. The second…

      nlView on HNParent context

  4. RAISR Sharp Images with Machine Learning 1 comment

    • Some of the deeper - and wider - convolutional neural network-based SR methods do this, implicitly. We have a few papers, like SRGAN (https://arxiv.org/abs/1609.04802) or AffGAN (https://arxiv.org/abs/1610.04490) that generate pretty interesting context-aware textures.…

      fhuszarView on HNParent context

  5. Neural Enhance – Super Resolution for images using deep learning 3 comments

    • Agreed, it would have been nice to show other upscaling algorithms. But neural net super resolution generators can still have significantly more detail at 4-8x, as shown here http://arxiv.org/abs/1609.04802

      dharma1View on HNParent context

    • Here's a list of various image interpolation techniques, with similar goal: - http://www.wisdom.weizmann.ac.il/~vision/SingleImageSR.html - http://chiranjivi.tripod.com/EDITut.html - http://www.tecnick.com/pagefiles/appunti/iNEDI_tesi_Nicola_A...…

      ingenterView on HNParent context

    • (Author here.) My biggest insight from this project is that super-resolution with neural networks benefits significantly from being domain specific. If you train on broader datasets, it does pretty well but has to make compromises. Many recent papers do a comparison in terms of…

      nuclaiView on HNParent context

  6. Subpixel: A subpixel convolutional neural network implementation with Tensorflow 2 comments

    • If you are interested in how it compares to bicubic or the original. Check these papers using the sub pixel convolutional layer: https://arxiv.org/abs/1609.05158 https://arxiv.org/abs/1609.04802.

      trustswzView on HNParent context

    • from trustswz' comment: https://arxiv.org/abs/1609.04802 The pic with the boat on page 13 is interesting. In the SRGAN version I would take the shore for some sort of cliff, while the original shows separated boulders.

      anotheryouView on HNParent context

  7. Neural Photo Editor 1 comment

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.