A Recurrent Latent Variable Model for Sequential Data · HackerPapers discussions

A Recurrent Latent Variable Model for Sequential Data — mentioned by kastnerkyle

Sun, 09 Aug 2015 01:51:45 GMT

This is really cool stuff - the network structure reminds me a lot of Graves' MDRNN[1] and Grid LSTM[2], as well as some work I helped with (ReNet [3]) I wonder if the structure over frequency/time is too "regular" - in general for sound the frequency correlation and the time correlation are on wildly different scales. Also if you are looking to go farther you might reconsider adding NADE or RBM [4] on top, or latent variables in the hiddens[5][6] to add more stochasticity. There was some alternate work by Kratarth Goel extending RNN-RBM to LSTM and DBN, it might give you some ideas to look…

Paper and discussion history. A mention is not an endorsement.

A Recurrent Latent Variable Model for Sequential Data — mentioned by kastnerkyle

Thu, 06 Aug 2015 21:31:02 GMT

The literature is mostly image based, but as other users have said voice, text, and video have also seen substantial gains. It has also been used to great effect for taxi destination prediction (heterogenous information) [1], drug discovery [2], and general operations on graphs [3]. There are also lots of examples beyond classification - structured prediction such as image segmentation [4], predicting depth, surface normals, and pixels from only RGB [5], text to text translation [6], image captioning [7], Turing machines [8], speech synthesis [9], question answering (Q/A) [10][11]…

Paper and discussion history. A mention is not an endorsement.

A Recurrent Latent Variable Model for Sequential Data — mentioned by kastnerkyle

Thu, 11 Jun 2015 15:45:17 GMT

This is pretty hard - we use raw data for speech [1, talked about in comment above] but it still needs some work to do really good synthesis. FFT is not really the way to go either - then you still need to deal with the problems of complex data which is very, very unpleasant. Most people use FFT -> IDCT (cepstrum) or a filtered version (mel-frequency cepstral coefficients, MFCC). This can work but it is a lot of domain knowledge. One thing we tried in early testing, but did not pursue farther was vector quantized X (where X is MFCCs, LPC, LSF, FFT, cepstrum). Basically you use K-means to find…

Paper and discussion history. A mention is not an endorsement.

A Recurrent Latent Variable Model for Sequential Data — mentioned by kastnerkyle

Thu, 11 Jun 2015 11:48:00 GMT

If you are into this kind of stuff (time series density/probability modeling), there are several places to go beyond this. The simplest example is the RNN-RBM (restricted boltzmann machine) from the Theano deep learning tutorials [1]. There are follow-up papers by Kratarth Goel extending this to LSTM-DBN, and RNN-NADE (neural autoregressive density estimator) has also been explored in this space. The results for MIDI are pretty good. Ales Graves [2] has done a huge amount of work in sequence modeling for classification and generation. His work with LSTM-GMM for online handwriting modeling can…

Paper and discussion history. A mention is not an endorsement.