Simple and Controllable Music Generation · HackerPapers discussions

Simple and Controllable Music Generation — mentioned by efskap

Tue, 28 Oct 2025 03:17:45 GMT

The current music generators use next token prediction, like LLMs, not image denoising. [0] https://arxiv.org/abs/2503.08638 (grep for "audio token") [1] https://arxiv.org/abs/2306.05284

Paper and discussion history. A mention is not an endorsement.

Simple and Controllable Music Generation — mentioned by zaptrem

Mon, 03 Mar 2025 20:26:45 GMT

There are! Audio models are actually quite similar to image models, but there are a few key differences. First, is the autoencoder needs to be designed much more carefully as human hearing is insanely good and music requires orders of magnitude more spatial compression (image AEs do 8X8 downsampling, audio AEs need to do thousands of times downsampling). Second the model itself needs to be really good at placing lyrics/beats (similar to placing text in image diffusion): a sixth finger in an image model is fine, but a missed beat can ruin a song. That's why language model approaches (which…

Paper and discussion history. A mention is not an endorsement.

Simple and Controllable Music Generation — mentioned by fagerhult

Mon, 19 Jun 2023 10:09:18 GMT

Here is the MusicGen paper from Facebook research: https://arxiv.org/abs/2306.05284 MusicGen is an LLM on top of EnCodec tokens, instead of working directly with audio. EnCodec is neural audio compression algorithm that encodes audio as tokens from a codebook. It's a really clever trick!

Paper and discussion history. A mention is not an endorsement.