Taylorformer: Probabilistic Modelling for Random Processes including Time Series — mentioned by omernivro

Sat, 08 Jun 2024 21:47:11 GMT

This is an interesting study. A similar permutation approach appears already in the Taylorformer paper (https://arxiv.org/pdf/2305.19141v1). The authors use a Transformer decoder for continuous processes, like time series. During training, each sequence is shuffled randomly. Each sequence element has a positional encoding. Then, they use log-likelihood on the shuffled sequence. There, the permutation helps with predictions for interpolation, extrapolation and irregularly sampled data. Also, they show it helps with 'consistency', i.e., roughly the MSE is the same regardless of the generated…

Paper and discussion history. A mention is not an endorsement.

Taylorformer: Probabilistic Modelling for Random Processes including Time Series · HackerPapers discussions

Taylorformer: Probabilistic Modelling for Random Processes including Time Series — mentioned by omernivro