The Descriptive Complexity of Graph Neural Networks — mentioned by bmc7505

Sat, 17 Jun 2023 15:44:53 GMT

The current generation of LLMs will never generalize on problems whose expected complexity is significantly harder than the training set. One way to think of why this is the case is because transformers are learning shortcuts to multistep reasoning tasks [1, 2], but do not learn to reason systematically and unsurprisingly fail to generalize when presented with more challenging tasks. The proper way to evaluate systematic reasoning is by training on a curriculum of problems of increasing average-case complexity. Think random k-SAT in the critical region, bisimulation games, or interpolation in…

Paper and discussion history. A mention is not an endorsement.

The Descriptive Complexity of Graph Neural Networks · HackerPapers discussions

The Descriptive Complexity of Graph Neural Networks — mentioned by bmc7505