https://hackerpaper.kaipfister.de/p/2010.10499/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMThttps://news.ycombinator.com/item?id=26534367urn:hackerpapers:mention:2010.10499:26534367Sun, 21 Mar 2021 21:42:39 GMT<p>Whilst obviously BERT is not the same as GPT-3 in architecture, Amazons recent paper discussing architecture optimizations for BERT seems pretty relevant here (https://arxiv.org/pdf/2010.10499.pdf) given the chance to improve upon GPT-3s architecture (because it surely isn&#x27;t the best we can get). Have the Eleuther.ai team been exploring this?</p><p><a href="https://hackerpaper.kaipfister.de/p/2010.10499/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.CL