FLM-101B: An Open LLM and How to Train It with $100K Budget — mentioned by sapphire42

Sat, 02 Nov 2024 22:19:35 GMT

They do actually make several claims as to the efficiency of the architecture compared to the Transformer, as you can see by the many graphs throughout the document. Their claim that their architecture is the only one that allows for gradually increasing the number of weights is a prominent one too, though, so I'll explain why I don't find that claim credible. The idea of gradually increasing the size of a Transformer to save on training costs is not a novel one, and researchers have explored ideas to this effect almost since the Transformer's inception. There are many ways to do it. We can…

Paper and discussion history. A mention is not an endorsement.

FLM-101B: An Open LLM and How to Train It with $100K Budget — HN submission

Fri, 08 Sep 2023 05:32:33 GMT

FLM-101B: An Open LLM and How to Train It with $100K Budget

2 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

FLM-101B: An Open LLM and How to Train It with $100K Budget · HackerPapers discussions

FLM-101B: An Open LLM and How to Train It with $100K Budget — mentioned by sapphire42

FLM-101B: An Open LLM and How to Train It with $100K Budget — HN submission