Moshi: a speech-text foundation model for real-time dialogue · HackerPapers discussions

Moshi: a speech-text foundation model for real-time dialogue — mentioned by quotemstr

Wed, 08 Jul 2026 18:04:06 GMT

> GPT-Live-1 is the first version of a new generation of models, and we believe the full-duplex architecture + delegation enables entirely new ways of human-AI interaction. Awesome. Are you guys able to share anything about the model architecture? I've been interested lately in split-transformer RVQ-based conversational agents, e.g. via stuff like https://arxiv.org/abs/2412.10208 (ResGen) and https://arxiv.org/abs/2603.18090 (MOSS-ITT) and of course Moshi (https://arxiv.org/pdf/2410.00037). Intuitively, decoupling semantic and audio-timeslice-space generations with coupled but distinct…

Paper and discussion history. A mention is not an endorsement.

Moshi: a speech-text foundation model for real-time dialogue — mentioned by rockwotj

Tue, 03 Mar 2026 03:01:49 GMT

Fundamentally, the "guessing when its your turn thing" needs to be baked into the model. I think the full duplex mode that Moshi pioneered is probably where the puck is going to end up: https://arxiv.org/abs/2410.00037

Paper and discussion history. A mention is not an endorsement.

Moshi: a speech-text foundation model for real-time dialogue — mentioned by navanchauhan

Mon, 02 Mar 2026 18:22:50 GMT

Not affiliated with Sesame, but this is what the realtime models are trying to solve. If you look at NVIDIA’s PersonaPlex release [0], it uses a duplex architecture. It’s based on Moshi [1], which aims to address this problem by allowing the model to listen and generate audio at the same time. [0] https://github.com/NVIDIA/personaplex [1] https://arxiv.org/abs/2410.00037

Paper and discussion history. A mention is not an endorsement.

Moshi: a speech-text foundation model for real-time dialogue — mentioned by vvolhejn

Tue, 21 Oct 2025 16:02:47 GMT

Author here. Speech-to-text is more or less solved, it's easy to automatically get captions including precise timestamps. For training Moshi, Kyutai's audio LLM, my colleagues used whisper-timestamped to transcribe 7 million hours of audio. See Section 4.2 in the Moshi paper: https://arxiv.org/pdf/2410.00037

Paper and discussion history. A mention is not an endorsement.

Moshi: a speech-text foundation model for real-time dialogue — mentioned by tmzt

Mon, 06 Jan 2025 16:53:06 GMT

I would say Gemini Live is getting there. It's lacking integration with NotebookLM and Keep. It would be amazing if I started a project conceptually and wanted to move to code it could fire up VS Code and let me get to work. Gemini's home automation works nicely and it can understand comments like it's too dark in here or it's cold inside and act appropriately. This is using the Android app as an assistant, not live mode. OpenAI's implementation is apparently similar but I haven't tried the voice mode as a free user. I haven't tried Apple Intelligence yet on my M1 and don't have an iPhone, so…

Paper and discussion history. A mention is not an endorsement.

Moshi: a speech-text foundation model for real-time dialogue — mentioned by benob

Thu, 02 Jan 2025 07:39:52 GMT

A relatively detailed description of such systems: https://arxiv.org/abs/2410.00037

Paper and discussion history. A mention is not an endorsement.