HackerPapers

arXiv:2010.10504eess.AS

Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition

Yu Zhang, James Qin, Daniel S. Park, Wei Han, Chung-Cheng Chiu, Ruoming Pang, Quoc V. Le, Yonghui Wu

Abstract

We employ a combination of recent developments in semi-supervised learning for automatic speech recognition to obtain state-of-the-art results on LibriSpeech utilizing the unlabeled audio of the Libri-Light dataset. More precisely, we carry out noisy student training with SpecAugment using giant Conformer models pre-trained using wav2vec 2.0 pre-training. By doing so, we are able to achieve word-error-rates (WERs) 1.4%/2.6% on the LibriSpeech test/test-other sets against the current state-of-the-art WERs 1.7%/3.3%.

arXiv comments
11 pages, 3 figures, 5 tables. Accepted to NeurIPS SAS 2020 Workshop; v2: minor errors corrected
Journal reference
DOI
Subjects
eess.ASAudio and Speech Processing cs.LGMachine Learning cs.SDSound

On Hacker News

Linked by 2 identifiable commenters in 2 comments across 2 threads, from Jul 2021 to Aug 2021.

Comments linking this paper, per year.

Discussed in comments2 threads

  1. Program Synthesis with Large Language Models 1 comment

    • First, congratulations to you and the other authors - this is a very informative and accessible paper with impressive results. > On both datasets, we find that synthesis performance scales log-linearly with model size This "scaling law" observation is starting to become a major…

      criticaltinkerView on HNParent context

  2. Wav2vec Overview: Semi and Unsupervised Speech Recognition 1 comment

    • One addendum to the linked post's notes: > SoTa in low-resource setting Libri-light by a lot on WER clean test 100h labeled: others ~4 vs theirs ~2.5 > SoTa on high-resource noisy data (3.3 vs 3.4) close to SoTa on clean data This note isn't super specific, but it's outdated if…

      lunixbochsView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.