Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis · HackerPapers discussions

Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis — mentioned by okamiueru

Sat, 25 Sep 2021 08:59:48 GMT

Having played around with flite a decade ago, and at that time feeling that it was already then nowhere close to the fidelity of other speech synthesis examples. I find it surprising that there still isn't anything better than festival/flite? It sounded then like a clunky robot, and still does today. Surely some of the many research projects have released their work as open source? Work like, say https://arxiv.org/abs/1806.04558 [paper] https://github.com/CorentinJ/Real-Time-Voice-Cloning [repo]

Paper and discussion history. A mention is not an endorsement.

Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis — mentioned by mandiantBob

Tue, 05 Jan 2021 15:43:14 GMT

This is not correct. The security page only shows that they use Google for Speech to Text. Instead, Text to Speech is done using technology developed by Lyrebird.ai, which Descript has bought over. Descript rebranded it as Overdub. Note that style transfer learning of voices is a hard problem and Overdub seems to have nailed it perfectly. I speculate that the underlying technology of Overdub is based on sv2tts (https://arxiv.org/abs/1806.04558). The closest comparision to Overdub would be https://www.resemble.ai/.

Paper and discussion history. A mention is not an endorsement.

Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis — mentioned by nighthawk454

Thu, 11 Jun 2020 20:09:59 GMT

This can be trained using only 5 Seconds of reference audio: https://google.github.io/tacotron/publications/speaker_adapt... https://arxiv.org/pdf/1806.04558.pdf It's been mentioned a bit already, but thought it was worth calling out. This may be one of the lowest-overhead ways to start experimenting, at least in terms of data collection.

Paper and discussion history. A mention is not an endorsement.

Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis — mentioned by audiohermit

Thu, 11 Jun 2020 16:17:50 GMT

Hey, speech ML researcher here. Make sure you have different recordings of different contexts. fifteen.ai's best TTS voices use ~90 min of utterances, some separated by emotion. If you're having her read a text, make sure it's engaging--we do a lot of unconscious voicing when reading aloud. Tbh, if she has a non-Anglophone accent, you're going to need more because the training data is biased towards UK/US speakers. If you want to read up on the basics, check out the SV2TTS paper: https://arxiv.org/pdf/1806.04558.pdf Basically you use a speaker encoding to condition the TTS output. This…

Paper and discussion history. A mention is not an endorsement.

Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis — mentioned by fxtentacle

Thu, 11 Jun 2020 16:09:49 GMT

The paper "Generalization Of Audio Deepfake Detection" gives an overview. The paper https://arxiv.org/abs/1904.05441 has a list of spoofing methods. Here's one method as paper https://arxiv.org/pdf/1806.04558.pdf And here on GitHub https://github.com/CorentinJ/Real-Time-Voice-Cloning

Paper and discussion history. A mention is not an endorsement.