HackerPapers

arXiv:1609.03499cs.SD

WaveNet: A Generative Model for Raw Audio

Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, Koray Kavukcuoglu

Abstract

This paper introduces WaveNet, a deep neural network for generating raw audio waveforms. The model is fully probabilistic and autoregressive, with the predictive distribution for each audio sample conditioned on all previous ones; nonetheless we show that it can be efficiently trained on data with tens of thousands of samples per second of audio. When applied to text-to-speech, it yields state-of-the-art performance, with human listeners rating it as significantly more natural sounding than the best parametric and concatenative systems for both English and Mandarin. A single WaveNet can capture the characteristics of many different speakers with equal fidelity, and can switch between them by conditioning on the speaker identity. When trained to model music, we find that it generates novel and often highly realistic musical fragments. We also show that it can be employed as a discriminative model, returning promising results for phoneme recognition.

arXiv comments
Journal reference
DOI
Subjects
cs.SDSound cs.LGMachine Learning

On Hacker News

Linked by 10 identifiable commenters in 10 comments across 10 threads, from Nov 2016 to Jan 2026.

Comments linking this paper, per year.

Discussed in comments10 threads

  1. Pocket TTS: A high quality TTS that gives your CPU a voice 1 comment

    • Using neural nets (machine learning) to train TTS voices has been around a long time. [1] (2016 https://arxiv.org/abs/1609.03499) WaveNet: A Generative Model for Raw Audio [2] (2017 https://arxiv.org/abs/1711.10433) Parallel WaveNet: Fast High-Fidelity Speech Synthesis [3] (2021…

      rhdunnView on HNParent context

  2. Show HN: Nevermind XOR – Deep Learning Has an Issue with Sin 1 comment

    • You're using a pointwise loss, which means you are treating each timepoint as conditionally independent. That's a deeply flawed assumption, probabilistically. Try an autoregressive model of the joint probability distribution of the sine wave timepoints -- like WaveNet [1]. It…

      rryanView on HNParent context

  3. Ask HN: What is the state of the art in text to speech? 1 comment

    • I don't know what the current state of the art is, but you should take a look at WaveNet if you haven't already - https://arxiv.org/pdf/1609.03499.pdf While trying to search for an answer, I came across this list of papers, which has a section on speech synthesis…

      zip6View on HNParent context

  4. A Tour of the Top Algorithms for Machine Learning Newbies 1 comment

    • The simplest way would be downloading something like MaryTTS [1], read the documentation, and train your own voice model. It won't be perfect, but shouldn't be too hard. The best results would probably be achieved by implementing DeepMind's WaveNet paper [2], but it might be too…

      probably_wrongView on HNParent context

  5. Markov Chains Explained Visually (2014) 1 comment

    • Isn't this like an "auto-regressive feed forward neural net"[0]? > Instead of making predictions from a state that depends on the entire history, an autoregressive model directly predicts yt using only the k most recent inputs, xt−k+1,…,xt. This corresponds to a strong…

      tree_of_itemView on HNParent context

  6. Today’s dominant approach to A.I. has not worked out 1 comment

    • Duplex's underlying text-to-speech technology research (WaveNet) has produced several papers and is now in public beta. It represents a huge advance in text-to-speech fidelity, using a remarkably straightforward algorithm. https://arxiv.org/pdf/1609.03499.pdf…

      jdietrichView on HNParent context

  7. Finding Waldo Using Semantic Segmentation and Tiramisu 1 comment

    • I think an aspect of deep learning that is often overlooked is that it is still not clear how much of current algorithm performance is defined by local "obsession to detail" vs global "understanding" of the subject matter. The "tiramisu" layers here are an interesting example of…

      ml_thoughtsView on HNParent context

  8. A novel approach to neural machine translation 1 comment
  9. Baidu Deep Voice Explained: Part 1 – the Inference Pipeline 1 comment

    • Showing one of the top two samples from their blog (full prediction) along with the one you link (only acoustic model) would more clearly show what you are getting at in the explanation, since how the text inference works is most of the complexity in this model (given baseline…

      kastnerkyleView on HNParent context

  10. Speech-to-Text-WaveNet: End-to-end sentence level English speech recognition 1 comment

    • > Did the original WaveNet text to speech demo come with a paper or source code? Paper, yes. [1] Source code, no. [1] https://arxiv.org/pdf/1609.03499.pdf

      bmc7505View on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.