HackerPapers

arXiv:1312.6082cs.CV

Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks

Ian J. Goodfellow, Yaroslav Bulatov, Julian Ibarz, Sacha Arnoud, Vinay Shet

Abstract

Recognizing arbitrary multi-character text in unconstrained natural photographs is a hard problem. In this paper, we address an equally hard sub-problem in this domain viz. recognizing arbitrary multi-digit numbers from Street View imagery. Traditional approaches to solve this problem typically separate out the localization, segmentation, and recognition steps. In this paper we propose a unified approach that integrates these three steps via the use of a deep convolutional neural network that operates directly on the image pixels. We employ the DistBelief implementation of deep neural networks in order to train large, distributed neural networks on high quality images. We find that the performance of this approach increases with the depth of the convolutional network, with the best performance occurring in the deepest architecture we trained, with eleven hidden layers. We evaluate this approach on the publicly available SVHN dataset and achieve over $96\%$ accuracy in recognizing complete street numbers. We show that on a per-digit recognition task, we improve upon the state-of-the-art, achieving $97.84\%$ accuracy. We also evaluate this approach on an even more challenging dataset generated from Street View imagery containing several tens of millions of street number annotations and achieve over $90\%$ accuracy. To further explore the applicability of the proposed system to broader text recognition tasks, we apply it to synthetic distorted text from reCAPTCHA. reCAPTCHA is one of the most secure reverse turing tests that uses distorted text to distinguish humans from bots. We report a $99.8\%$ accuracy on the hardest category of reCAPTCHA. Our evaluations on both tasks indicate that at specific operating thresholds, the performance of the proposed system is comparable to, and in some cases exceeds, that of human operators.

arXiv comments
Journal reference
DOI
Subjects
cs.CVComputer Vision and Pattern Recognition

On Hacker News

Linked by 8 identifiable commenters in 9 comments across 7 threads, from Jan 2014 to Mar 2017.

Comments linking this paper, per year.

Discussed in comments7 threads

  1. ReBreakCaptcha: Breaking Google’s ReCaptcha v2 Using Google 1 comment

    • The state of the art has improved so much that for the older tasks, computers are arguably better than humans at them: https://arxiv.org/abs/1312.6082 So humans have to clear higher and higher hurdles to prove themselves human while the computers use the answers from the…

      cbhlView on HNParent context

  2. Intro to Hidden Markov Models (2010) [pdf] 1 comment

    • Sure, there are a lot of potential things to do - either by enumerating towards the full "brute force" loss partway as I think you are hinting toward, or treating as a subsequences, or something else [0, 1, 2]. The point is, IMO RNNs do need some kind of structured loss (more…

      kastnerkyleView on HNParent context

  3. Judge tosses proposed class action accusing Google of CAPTCHA fraud 1 comment

    • Distorted text isn't even used in reCAPTCHA all that much anymore -- Google used a Deep Convolutional Neural Network and got over 90% accuracy in reading street numbers in Street View, and 99.8% accuracy in solving distorted text reCAPTCHA (http://arxiv.org/abs/1312.6082). Most…

      cbhlView on HNParent context

  4. Inside Google Brain 1 comment
  5. Captchas are Becoming Ridiculous 1 comment

    • We found that neural networks can solve CAPTCHAS much better than humans, 99.8% on the "hard" ReCAPTCHA instances: http://arxiv.org/pdf/1312.6082.pdf This is why visual recognition is just one of the signals you need to use to tell humans and computers apart…

      yaroslavvbView on HNParent context

  6. Google's Street View computer vision can beat reCAPTCHA with 99% accuracy 3 comments

    • This is essentially the technology that was discussed here 3 months ago [1], and it links to the exact same article on arxiv, titled: "Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks". [2] They new addition to the article is that…

      pavelrubView on HNParent context

    • http://arxiv.org/abs/1312.6082

      dwielView on HNParent context

    • It's a new success for Deep Learning. It seems to be actually 99.8% accuracy according to their paper: http://arxiv.org/abs/1312.6082 That's one order of magnitude higher.

      spcollView on HNParent context

  7. How Google Cracked House Number Identification in Street View 1 comment

    • I think the actual paper (linked at the end) is much more informative and interesting than this coverage: http://arxiv.org/abs/1312.6082 (I wouldn't call it blogspam, because it looks like they interviewed the researchers, but the summary leaves something to be desired)

      magicalistView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.