FoNE: Precise Single-Token Number Embeddings via Fourier Features — mentioned by search_facility

Sat, 25 Oct 2025 08:13:18 GMT

Interesting research, but it is still fascinates me why AI devs of current SOTAs ignore possibility to add numbers as first-grade citizens to AI. like for example suggested here: https://huggingface.co/papers/2502.09741 clean separation matter, it’s really strange to force models to mimic numbers and math via incredibly unfit token-mangling stuff, imho

Paper and discussion history. A mention is not an endorsement.

FoNE: Precise Single-Token Number Embeddings via Fourier Features — mentioned by search_facility

Wed, 25 Jun 2025 21:15:10 GMT

turns out - no, by intuition they should do this for sure - but no. UPD: Found the paper: - https://huggingface.co/papers/2502.09741 - https://fouriernumber.github.io/ in paper mentioned “number” is a single sort-of “token” with numeric value, so network dealing with numbers like real numbers, separately from char representation. All the math happens directly on “number value”. In majority of current models numbers are handled like sequences of chars

Paper and discussion history. A mention is not an endorsement.

FoNE: Precise Single-Token Number Embeddings via Fourier Features · HackerPapers discussions

FoNE: Precise Single-Token Number Embeddings via Fourier Features — mentioned by search_facility

FoNE: Precise Single-Token Number Embeddings via Fourier Features — mentioned by search_facility