Using the Output Embedding to Improve Language Models — mentioned by woadwarrior01

Tue, 09 Dec 2025 15:51:54 GMT

It's just a long winded way of saying "tied embeddings"[1]. IIRC, GPT-2, BERT, Gemma 2, Gemma 3, some of the smaller Qwen models and many more architectures use weight tied input/output embeddings. [1]: https://arxiv.org/abs/1608.05859

Paper and discussion history. A mention is not an endorsement.

Using the Output Embedding to Improve Language Models · HackerPapers discussions

Using the Output Embedding to Improve Language Models — mentioned by woadwarrior01