SETOL: A Semi-Empirical Theory of (Deep) Learning · HackerPapers discussions

SETOL: A Semi-Empirical Theory of (Deep) Learning — mentioned by riemannzeta

Wed, 04 Feb 2026 17:15:35 GMT

Neat result. The symmetry exploitation here reminds me of recent work connecting neural network training dynamics to renormalization group theory. Charles Martin's SETOL paper https://arxiv.org/abs/2507.17912 shows that well-trained layers converge to something like an RG fixed point—the eigenvalue spectrum of the weight matrix develops power-law tails with exponent α ≈ 2, which is the signature of scale invariance. At this fixed point, the "effective correlation space" is low-dimensional: you can truncate the SVD aggressively and recover nearly identical test accuracy. I wonder if there's a…

Paper and discussion history. A mention is not an endorsement.

SETOL: A Semi-Empirical Theory of (Deep) Learning — mentioned by riemannzeta

Tue, 03 Feb 2026 23:51:50 GMT

Not an expert in neurobiology, but... heck this is Hacker News: Let's assume that the brain implements a renormalization group flow in a way similar to transformers, as described here: https://arxiv.org/abs/2507.17912 In biological neural networks, synaptic weight distributions presumably have some effective α. Learning perturbs α away from optimal — overfitting would correspond to α drifting toward values that capture training noise rather than generalizable structure. The question becomes: what mechanism pushes α back toward 2? Astrocytes are positioned to do exactly this. They don't modify…

Paper and discussion history. A mention is not an endorsement.

SETOL: A Semi-Empirical Theory of (Deep) Learning — HN submission

Fri, 25 Jul 2025 15:31:56 GMT

Setol: SemiEmpirical Theory of (Deep) Learning

6 points; 3 discussion replies on HN. Paper and source history. A submission is not an endorsement.