Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent · HackerPapers discussions

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent — HN submission

Mon, 11 Nov 2024 21:20:29 GMT

Hunyuan-Large: An Open-Source Moe Model with 52B Activated Parameters

7 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent — mentioned by dplavery92

Tue, 05 Nov 2024 21:43:57 GMT

The title of Tencent's paper [0] as well as their homepage for the model [1] each use the term "Open-Source" in the title, so I think they are making the claim. [0] https://arxiv.org/pdf/2411.02265 [1] https://llm.hunyuan.tencent.com/

Paper and discussion history. A mention is not an endorsement.

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent — mentioned by 1R053

Tue, 05 Nov 2024 20:31:36 GMT

the paper with details: https://arxiv.org/pdf/2411.02265 They use - 16 experts, of which one is activated per token - 1 shared expert that is always active in summary that makes around 52B active parameters per token instead of the 405B of LLama3.1.

Paper and discussion history. A mention is not an endorsement.

Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent — HN submission

Tue, 05 Nov 2024 16:27:55 GMT

Hunyuan-Large: An Open-Source Moe Model with 52B Activated Parameters

5 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.