Polynomial Regression As an Alternative to Neural Nets · HackerPapers discussions

Polynomial Regression As an Alternative to Neural Nets — mentioned by nickpsecurity

Fri, 08 May 2026 15:53:02 GMT

There's also work in polynomial regression as an alternative to regular, neural networks. Polynomial Regression As an Alternative to Neural Nets (2018) https://arxiv.org/abs/1806.06850 Π-nets: Deep Polynomial Neural Networks (2020) https://arxiv.org/abs/2003.03828

Paper and discussion history. A mention is not an endorsement.

Polynomial Regression As an Alternative to Neural Nets — mentioned by bigbillheck

Mon, 24 May 2021 23:27:23 GMT

The central conceit of the paper (https://arxiv.org/abs/1806.06850) is the "NN<->PR correspondence" from section 6, which argues that since nice nonlinear activation functions like tanh can be arbitrarily well-approximated by polynomials, then the whole stack of convolutional layers can be arbitrarily well-approximated by polynomials, and that means the whole network is just a polynomial. And sure, ```technically''' that's true but there are good reasons that numerics aren't just all polynomials all the time (check out Lloyd Trefethen's 'Approximation Theory and Approximation Practice', for…

Paper and discussion history. A mention is not an endorsement.

Polynomial Regression As an Alternative to Neural Nets — mentioned by tbenst

Tue, 20 Oct 2020 23:42:02 GMT

That's a strong unsubstantiated claim. On the other hand, there's been some nice theoretical work arguing that deep learning is a form of polynomial regression. https://arxiv.org/abs/1806.06850

Paper and discussion history. A mention is not an endorsement.

Polynomial Regression As an Alternative to Neural Nets — mentioned by pidtuner

Thu, 27 Aug 2020 09:08:56 GMT

"The real promise of these methods is to use the universal approximator power of NNs...", still if one is to use a grey-box non-linear model dx/dt = F(x, u, t), why use NNs to characterize F? I would be more comfortable using a polynomial to characterize non-linearity than a "deep" black-box. Polynomials are much easier to "train" because it is just one linear regression with no iteration. It has also been hinted that NN are in essence polynomial regressions [0]. Furthermore, most activation functions are base on e^x where the actual implementation of e^x in a computer is again a polynomial!…

Paper and discussion history. A mention is not an endorsement.

Polynomial Regression As an Alternative to Neural Nets — HN submission

Tue, 17 Sep 2019 10:55:50 GMT

NNs are in fact essentially polynomial regression models

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Polynomial Regression As an Alternative to Neural Nets — mentioned by carapace

Fri, 30 Aug 2019 17:59:25 GMT

They revised the paper in response to earlier, uh, response. https://matloff.wordpress.com/2018/07/01/update-on-polynomia... https://arxiv.org/abs/1806.06850

Paper and discussion history. A mention is not an endorsement.

Polynomial Regression As an Alternative to Neural Nets — HN submission

Sun, 24 Jun 2018 07:09:37 GMT

Polynomial Regression as an Alternative to Neural Nets

137 points; 46 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Polynomial Regression As an Alternative to Neural Nets — HN submission

Thu, 21 Jun 2018 13:31:02 GMT

Polynomial Regression as an Alternative to Neural Nets

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.