Explaining grokking through circuit efficiency — mentioned by naasking

Tue, 12 Nov 2024 23:22:17 GMT

Not sure what's so controversial, it's well known that LLMs can trivially be viewed as operating in higher dimensional space: https://gcptips.medium.com/a-geometric-perspective-on-large-... As for generalizing to algorithms, LLMs don't yet do this as well as humans, but they do do it: https://arxiv.org/abs/2309.02390 Finally, there's no intrinsic reason why an AI that can reliably solve deductive problems like ARC would be limited to two dimensions.

Paper and discussion history. A mention is not an endorsement.

Explaining grokking through circuit efficiency — HN submission

Thu, 14 Sep 2023 16:51:26 GMT

Explaining Neural Network grokking through circuit efficiency

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Explaining grokking through circuit efficiency · HackerPapers discussions

Explaining grokking through circuit efficiency — mentioned by naasking

Explaining grokking through circuit efficiency — HN submission