Discovering Symbolic Models from Deep Learning with Inductive Biases · HackerPapers discussions

Discovering Symbolic Models from Deep Learning with Inductive Biases — mentioned by goodmachine

Thu, 09 May 2024 06:15:34 GMT

In order for that not to happen (uninterpretable ML models) some research on symbolic distillation, aka symbolic regression https://arxiv.org/abs/2006.11287 https://www.science.org/doi/10.1126/sciadv.aay2631

Paper and discussion history. A mention is not an endorsement.

Discovering Symbolic Models from Deep Learning with Inductive Biases — mentioned by tbalsam

Tue, 19 Sep 2023 16:26:39 GMT

I'm curious how this relates to MDL-tied regularization. I don't see any explicit reference to the L2 norm here, and I'm not too smart about graph neural networks. But it seems like overfitting is almost necessary unless some kind of obscure variational technique is used? They did have one paper where they used a progressively compressed GNN to isolate an approximation for a cosmological equation smaller and more accurate than had been previously achieved. I'll update here if I find it. Edit: Ah, here we are: https://arxiv.org/abs/2006.11287 I believe there was some very good followup work…

Paper and discussion history. A mention is not an endorsement.

Discovering Symbolic Models from Deep Learning with Inductive Biases — mentioned by KhoomeiK

Tue, 08 Dec 2020 21:24:03 GMT

Discovering Symbolic Models from Deep Learning with Inductive Biases [1] trains graph neural nets on astrophysical phenomena and then performs symbolic regression to generate algebraic formulae to elegantly model the phenomena in a classical physics framework. It's largely gone under the radar but has pretty interesting implications for NLP and language theory in my opinion. Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures [2] applies DFA, an approach to training neural nets without backprop, to modern architectures like the Transformer. It does surprisingly…

Paper and discussion history. A mention is not an endorsement.

Discovering Symbolic Models from Deep Learning with Inductive Biases — HN submission

Mon, 20 Jul 2020 19:31:32 GMT

Discovering Symbolic Models from Deep Learning with Inductive Biases

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Discovering Symbolic Models from Deep Learning with Inductive Biases — HN submission

Mon, 29 Jun 2020 19:09:37 GMT

Discovering Symbolic Models from Deep Learning with Inductive Biases

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Discovering Symbolic Models from Deep Learning with Inductive Biases — HN submission

Sun, 28 Jun 2020 12:14:40 GMT

Discovering Symbolic Models from Deep Learning with Inductive Biases

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.