MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts · HackerPapers discussions

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts — mentioned by intalentive

Fri, 23 Feb 2024 22:41:23 GMT

https://arxiv.org/abs/2401.04081 https://github.com/jzhang38/LongMamba

Paper and discussion history. A mention is not an endorsement.

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts — mentioned by f_devd

Thu, 18 Jan 2024 21:33:58 GMT

You should check out MoE-Mamba (https://arxiv.org/abs/2401.04081), it's faster and more accurate than Transformer-MoE. Of course only time will tell if it's better when scaled up further than the paper goes.

Paper and discussion history. A mention is not an endorsement.

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts — HN submission

Tue, 09 Jan 2024 21:28:11 GMT

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

129 points; 39 discussion replies on HN. Paper and source history. A submission is not an endorsement.

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts — HN submission

Tue, 09 Jan 2024 03:16:02 GMT

Moe-Mamba: Efficient Selective State Space Models with Mixture of Experts

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.