Half the Experts, All the Code: One-Shot Domain Pruning of Mixture-of-Experts LLMs for Coding — mentioned by Tade0

Tue, 06 Oct 2026 16:49:02 GMT

> Does a system need the knowledge of the full works of Shakespeare to be able to output Javascript? Turns out that actually - no. Researchers have managed to prune half the Experts in a MoE model that had a low probability of getting activated during coding tasks, resulting in a more focused model: https://arxiv.org/abs/2607.16721 Main benefit is that it greatly reduces the amount of RAM required to run these models. Of course you could just cache those unused experts on disk instead, but the main point here is that you know which ones matter. But aside from that recent models, like…

Paper and discussion history. A mention is not an endorsement.

Half the Experts, All the Code: One-Shot Domain Pruning of Mixture-of-Experts LLMs for Coding · HackerPapers discussions

Half the Experts, All the Code: One-Shot Domain Pruning of Mixture-of-Experts LLMs for Coding — mentioned by Tade0