Making Pre-trained Language Models Great on Tabular Prediction — mentioned by scottyak

Tue, 05 Mar 2024 15:56:19 GMT

One promising approach is to encode each feature key and feature value as embedding vectors, concatenate them into "feature tokens", then feed them into a Transformer (without positional encodings). This takes advantage of column-order invariance. See: https://arxiv.org/abs/2403.01841 (ICLR 2024 spotlight)

Paper and discussion history. A mention is not an endorsement.

Making Pre-trained Language Models Great on Tabular Prediction · HackerPapers discussions

Making Pre-trained Language Models Great on Tabular Prediction — mentioned by scottyak