2D-TPE: Two-Dimensional Positional Encoding Enhances Table Understanding for Large Language Models — mentioned by fi-le

Sat, 05 Jul 2025 11:43:56 GMT

Good point. The architectural solution that would come to mind is 2D text embeddings, i.e. we add 2 sines and cosines to each token embedding instead of 1. Apparently people have done it before: https://arxiv.org/abs/2409.19700v2

Paper and discussion history. A mention is not an endorsement.

2D-TPE: Two-Dimensional Positional Encoding Enhances Table Understanding for Large Language Models · HackerPapers discussions

2D-TPE: Two-Dimensional Positional Encoding Enhances Table Understanding for Large Language Models — mentioned by fi-le