Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings — mentioned by in-silico

Wed, 14 Oct 2026 05:33:40 GMT

I like the enthusiasm but no they actually work fine without positional encodings: https://arxiv.org/abs/2512.12167

Paper and discussion history. A mention is not an endorsement.

Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings · HackerPapers discussions