HackerPapers

arXiv:1502.01852cs.CV

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun

Abstract

Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on our PReLU networks (PReLU-nets), we achieve 4.94% top-5 test error on the ImageNet 2012 classification dataset. This is a 26% relative improvement over the ILSVRC 2014 winner (GoogLeNet, 6.66%). To our knowledge, our result is the first to surpass human-level performance (5.1%, Russakovsky et al.) on this visual recognition challenge.

arXiv comments
Journal reference
DOI
Subjects
cs.CVComputer Vision and Pattern Recognition cs.AIArtificial Intelligence cs.LGMachine Learning

On Hacker News

Linked by 6 identifiable commenters in 6 comments across 6 threads, from Feb 2015 to Dec 2023. Found in 2 HN submissions from Feb 2015 to Feb 2015. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments6 threads

  1. Direct initialization of transformers using larger pretrained ones 1 comment

    • Not necessarily. It turns out to really matter how the weights are initialized (randomly, but from which distribution?) for ordinary feed-forward neural nets that only uses standard neurons -- different activation functions have different optimal initialization distributions.…

      peterfireflyView on HNParent context

  2. Explainable Deep Learning: A Field Guide for the Uninitiated 1 comment

    • Words matter, concretely define "open ended"? Did you just add that phrase to preemptively nullify evidence to the contrary? Deep learning has surpassed human level performance on many tasks [1][2]... (could add more you get the point). [1]…

      kahnjwView on HNParent context

  3. China is catching up to the USA, while Japan is being left behind 1 comment

    • https://arxiv.org/abs/1512.03385 Deep Residual Learning for Image Recognition Citation 4000+. Milestone of image recognition. https://arxiv.org/abs/1502.01852 Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. Also known as PreLU…

      tanilamaView on HNParent context

  4. All Tesla Cars Being Produced Now Have Full Self-Driving Hardware 1 comment

    • Researchers already passed human level image classification performance in 2015. https://arxiv.org/pdf/1502.01852.pdf

      xiphiasView on HNParent context

  5. Microsoft Research wins image recognition competition 1 comment
  6. Batch Normalization: Accelerating Deep Network Training [pdf] 1 comment

    • Since this paper along with [1] both beat human level performance on ImageNet, using seemingly unrelated techniques, it seems like there's a clear path to achieve the next state of the art result: Implement and train a network that uses both Batch Normalization and PReLU's at…

      larsView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.