A Topological Version of Schaefer's Dichotomy Theorem — mentioned by nyrikki

Thu, 22 Feb 2024 18:42:37 GMT

The known lower bound for self attention is n^(2-epislon) PH, or the polynomial hierarchy is the collapse I was talking about, but note I was talking about global and not individual or families of instances. Schaefer's Dichotomy Theorem shows that HORNSAT is solvable, given those constraints. I found this paper useful as it reframes it as topology which is helpful to build it intuitions, at least me. https://arxiv.org/abs/2307.03446 Of the proposed linear self attention like solutions I have seen, they tend to approach it more from what can be reduced the clustering problem. A lens for that…

Paper and discussion history. A mention is not an endorsement.

A Topological Version of Schaefer's Dichotomy Theorem · HackerPapers discussions

A Topological Version of Schaefer's Dichotomy Theorem — mentioned by nyrikki