HackerPapers

arXiv:2312.10794cs.LG

A mathematical perspective on Transformers

Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, Philippe Rigollet

HN submission. Found in a submission link or body. Submitters do not count as commenters in the main ranking; a submission is not an endorsement.

Abstract

Transformers play a central role in the inner workings of large language models. We develop a mathematical framework for analyzing Transformers based on their interpretation as interacting particle systems, which reveals that clusters emerge in long time. Our study explores the underlying theory and offers new perspectives for mathematicians as well as computer scientists.

arXiv comments
Journal reference
Bull. Amer. Math. Soc. 62 (2025), 427-479
DOI
Subjects
cs.LGMachine Learning math.APAnalysis of PDEs math.DSDynamical Systems

On Hacker News

No comments linking this paper were found in the collected sources. Found in 1 HN submission from Dec 2023 to Dec 2023. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments0 threads

No comments linking this paper were found in the collected sources. Open submission discussions.

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.