Graph Neural Networks are Dynamic Programmers · HackerPapers discussions

Graph Neural Networks are Dynamic Programmers — mentioned by stefanpie

Tue, 19 Sep 2023 16:33:24 GMT

I've work with GNNs. I have not explicitly looked into this, but in general, the dynamics of GNNs are really interesting and there are many different lenses in which one can study the behavior of GNNs and find interesting takeaways. - Explicit Message Passing: https://arxiv.org/abs/2202.11097 - Algorithmic Reasoning: https://arxiv.org/abs/2203.15544 - Diffusion: https://proceedings.mlr.press/v139/chamberlain21a.html - Spectral: https://arxiv.org/abs/2010.02863 - Sparsity: https://arxiv.org/abs/2211.15335 - Training Tricks: https://arxiv.org/abs/2108.10521 - Interaction Networks / Physical…

Paper and discussion history. A mention is not an endorsement.

Graph Neural Networks are Dynamic Programmers — mentioned by chaxor

Thu, 25 May 2023 01:05:15 GMT

Perhaps, but taken with other works in the area, a better picture does emerge regarding this claim. Substantial works already showing reasoning capabilities in GPT-4, which show that these models do reason very well - near human performance for many causal reasoning tasks. (1) Additionally, there is a mathematical proof that these systems align with dynamic programming, and therefore can perform algorithmic reasoning. (2,3) 1) https://arxiv.org/abs/2305.00050.pdf 2) https://arxiv.org/pdf/1905.13211.pdf 3) https://arxiv.org/pdf/2203.15544.pdf

Paper and discussion history. A mention is not an endorsement.

Graph Neural Networks are Dynamic Programmers — mentioned by chaxor

Thu, 25 May 2023 01:01:18 GMT

There are substantial works already showing reasoning capabilities in GPT-4, which show that these models do reason extremely well - near human performance for many causal reasoning tasks. (1) Additionally, there is a mathematical proof that these systems align with dynamic programming, and therefore can do algorithmic reasoning. (2,3) 1) https://arxiv.org/abs/2305.00050.pdf 2) https://arxiv.org/pdf/1905.13211.pdf 3) https://arxiv.org/pdf/2203.15544.pdf

Paper and discussion history. A mention is not an endorsement.

Graph Neural Networks are Dynamic Programmers — mentioned by chaxor

Tue, 16 May 2023 15:06:41 GMT

You should read some of the papers referred to in the above comments before making that assertion. It may take a while to realize the overall structure of the argument, how the category theory is used, and how this is directly applicable to LLMs, but if you are in ML it should be obvious. https://arxiv.org/abs/2203.15544

Paper and discussion history. A mention is not an endorsement.

Graph Neural Networks are Dynamic Programmers — mentioned by chaxor

Sun, 14 May 2023 04:39:33 GMT

It's literally the title. Peter Velockovic is probably the most prolific author of this era regarding LLMs, and does not yet realize it. https://arxiv.org/pdf/2203.15544.pdf

Paper and discussion history. A mention is not an endorsement.