Muon is Scalable for LLM Training — mentioned by in-silico

Tue, 27 Jan 2026 19:51:09 GMT

A lot of people point to the Muon optimizer that Moonshot (the creators of Kimi) pioneered. Compared to the standard optimizer AdamW, Muon amplifies low-magnitude gradient directions which makes the model learn faster (and maybe gives Kimi its unique qualities). Muon paper: https://arxiv.org/abs/2502.16982

Paper and discussion history. A mention is not an endorsement.

Muon is Scalable for LLM Training — mentioned by godelski

Fri, 26 Sep 2025 22:46:45 GMT

> Apologies if this came across the wrong way I was uncertain but your other statements made me think that sentiment was unintentional. I just want to push back against it because it is too common and misused even with good intentions. I hope you don't see this as me saying anything about your character. Honestly, impressions are that you do care. > It sounds like the generalisation of projected gradient decent to "Muon" I'm not a niche expert here, but do have knowledge in adjacent/overlapping domains. It sounds like you're in a similar boat? I ask because this pulls back to what I was…

Paper and discussion history. A mention is not an endorsement.

Muon is Scalable for LLM Training · HackerPapers discussions

Muon is Scalable for LLM Training — mentioned by in-silico

Muon is Scalable for LLM Training — mentioned by godelski