Logical Languages Accepted by Transformer Encoders with Hard Attention — mentioned by yorwba

Fri, 13 Mar 2026 17:56:02 GMT

Treating attention as a lookup operation is popular among computational complexity theorists (e.g. https://arxiv.org/abs/2310.03817 ) because it's easier to work with when you're explicitly constructing a transformer to perform a particular computation, just to demonstrate that tranformers can, in theory, perform it. That's also why there are no training details: the weights are computed directly and not trained.

Paper and discussion history. A mention is not an endorsement.

Logical Languages Accepted by Transformer Encoders with Hard Attention · HackerPapers discussions

Logical Languages Accepted by Transformer Encoders with Hard Attention — mentioned by yorwba