NeoBERT: A Next-Generation BERT — mentioned by dust42

Mon, 28 Jul 2025 10:50:16 GMT

Minor correction, BERT is an encoder (not encoder-decoder), ChatGPT is a decoder. Encoders like BERT produce better results for embeddings because they look at the whole sentence, while GPTs look from left to right: Imagine you're trying to understand the meaning of a word in a sentence, and you can read the entire sentence before deciding what that word means. For example, in "The bank was steep and muddy," you can see "steep and muddy" at the end, which tells you "bank" means the side of a river (aka riverbank), not a financial institution. BERT works this way - it looks at all the words…

Paper and discussion history. A mention is not an endorsement.

NeoBERT: A Next-Generation BERT — HN submission

Sat, 01 Mar 2025 18:20:31 GMT

NeoBERT: A Next-Generation Bert

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

NeoBERT: A Next-Generation BERT · HackerPapers discussions

NeoBERT: A Next-Generation BERT — mentioned by dust42

NeoBERT: A Next-Generation BERT — HN submission