SuperBPE: Space Travel for Language Models · HackerPapers discussions

SuperBPE: Space Travel for Language Models — mentioned by cschmidt

Tue, 24 Jun 2025 18:45:00 GMT

This paper has a good solution: https://arxiv.org/abs/2402.14903 You right to left tokenize in groups of 3, so 1234567 becomes 1 234 567 rather than the default 123 456 7. And if you ensure all 1-3 digits groups are in the vocab, it does much better. Both https://arxiv.org/abs/2503.13423 and https://arxiv.org/abs/2504.00178 (co-author) both independently noted that you can do this with just by modifying the pre-tokenization regex, without having to explicitly add commas.

Paper and discussion history. A mention is not an endorsement.

SuperBPE: Space Travel for Language Models — mentioned by woodson

Sat, 31 May 2025 16:10:58 GMT

How does this differ from SuperBPE, which seems to pursue a similar goal? https://arxiv.org/abs/2503.13423 Looks like parallel invention. (I’m not associated with the paper or its authors.)

Paper and discussion history. A mention is not an endorsement.

SuperBPE: Space Travel for Language Models — mentioned by woodson

Fri, 30 May 2025 20:01:31 GMT

There has been some innovation, e.g. SuperBPE (https://arxiv.org/abs/2503.13423) claims substantial gains on 8B-sized models.

Paper and discussion history. A mention is not an endorsement.

SuperBPE: Space Travel for Language Models — HN submission

Sat, 29 Mar 2025 14:39:27 GMT

SuperBPE: Space Travel for Language Models

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.