HackerPapers

arXiv:1610.05256cs.CL

Achieving Human Parity in Conversational Speech Recognition

W. Xiong, J. Droppo, X. Huang, F. Seide, M. Seltzer, A. Stolcke, D. Yu, G. Zweig

Abstract

Conversational speech recognition has served as a flagship speech recognition task since the release of the Switchboard corpus in the 1990s. In this paper, we measure the human error rate on the widely used NIST 2000 test set, and find that our latest automated system has reached human parity. The error rate of professional transcribers is 5.9% for the Switchboard portion of the data, in which newly acquainted pairs of people discuss an assigned topic, and 11.3% for the CallHome portion where friends and family members have open-ended conversations. In both cases, our automated system establishes a new state of the art, and edges past the human benchmark, achieving error rates of 5.8% and 11.0%, respectively. The key to our system's performance is the use of various convolutional and LSTM acoustic model architectures, combined with a novel spatial smoothing method and lattice-free MMI acoustic training, multiple recurrent neural network language modeling approaches, and a systematic use of system combination.

arXiv comments
Revised for publication, updated results
Journal reference
DOI
Subjects
cs.CLComputation and Language eess.ASAudio and Speech Processing

On Hacker News

Linked by 5 identifiable commenters in 5 comments across 4 threads, from Oct 2016 to Oct 2020. Found in 1 HN submission from Oct 2016 to Oct 2016. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments4 threads

    • This (https://arxiv.org/abs/2010.03449) is by far not the first publication which claims super human speech recognition, or human parity. For example https://arxiv.org/abs/1610.05256 or https://arxiv.org/abs/1703.02136. And since then, the WER has dropped much more on the same…

      albertzeyerView on HNParent context

    • Ok, we'll change to that. Thanks! From https://arxiv.org/abs/1610.05256 via https://blogs.microsoft.com/ai/2016/10/18/historic-achieveme....

      dangView on HNParent context

    • The actual paper has a section on error analysis that is particularly enlightening: https://arxiv.org/abs/1610.05256 On the CallHome dataset humans confuse words 4.1% of the time, but delete 6.5% of words, most commonly deleting the word "I". Their ASR system confuses 6.5% of…

      EridrusView on HNParent context

    • Section 9 in the paper[1] is all about comparing these mistakes between the system and humans. The most common mistakes for humans and the system are in tables 9—11. We find that the artificial errors are substantially the same as human ones with one large exception confusions…

      szupieView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.