Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition · HackerPapers discussions

Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition — mentioned by criticaltinker

Wed, 18 Aug 2021 03:41:24 GMT

First, congratulations to you and the other authors - this is a very informative and accessible paper with impressive results. > On both datasets, we find that synthesis performance scales log-linearly with model size This "scaling law" observation is starting to become a major trend in NLP [1] and other other modalities such as speech recognition [2] and protein structure/function prediction [3]. Do you have any insight or commentary to offer regarding the next step in improving program synthesis? For example, will techniques to scale up model size continue to be a primary focus (eg as in…

Paper and discussion history. A mention is not an endorsement.

Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition — mentioned by lunixbochs

Sat, 03 Jul 2021 16:03:00 GMT

One addendum to the linked post's notes: > SoTa in low-resource setting Libri-light by a lot on WER clean test 100h labeled: others ~4 vs theirs ~2.5 > SoTa on high-resource noisy data (3.3 vs 3.4) close to SoTa on clean data This note isn't super specific, but it's outdated if I'm understanding it correctly. To my understanding, the SOTA on this data is held by Conformer 1B (a 1 billion parameter model), at 1.4 clean, 2.6 noisy. Conformer 1B is something like wav2vec 2.0 pretraining + conformer + noisy student + specaugment. https://arxiv.org/pdf/2010.10504.pdf -- Wav2vec 2.0 is very cool…

Paper and discussion history. A mention is not an endorsement.