KAN: Kolmogorov-Arnold Networks · HackerPapers discussions

KAN: Kolmogorov-Arnold Networks — mentioned by thesz

Tue, 02 Jul 2024 13:37:26 GMT

The original paper [1] used LBFGS [2], it is quasi-second-order optimization algorithm. [1] https://arxiv.org/pdf/2404.19756 - "Both MLPs and KANs are trained with LBFGS for 1800 steps in total." [2] https://en.wikipedia.org/wiki/Limited-memory_BFGS (Quasi-)Newton methods approximate learning rate using local curvature which gradient-based methods do not do. The post relies on Tinygrad because it is familiar to author and author tinkers with batch size and learning rate, but not with optimizer itself. I think that even line search for minimum on the direction of the batch gradient can provide…

Paper and discussion history. A mention is not an endorsement.

KAN: Kolmogorov-Arnold Networks — HN submission

Tue, 07 May 2024 02:00:18 GMT

Kan: Kolmogorov-Arnold Networks

28 points; 4 discussion replies on HN. Paper and source history. A submission is not an endorsement.

KAN: Kolmogorov-Arnold Networks — HN submission

Mon, 06 May 2024 07:51:54 GMT

Kan: Kolmogorov–Arnold Networks

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

KAN: Kolmogorov-Arnold Networks — HN submission

Sat, 04 May 2024 23:40:02 GMT

Kan: Kolmogorov-Arnold Networks

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

KAN: Kolmogorov-Arnold Networks — HN submission

Fri, 03 May 2024 17:37:45 GMT

Kolmogorov-Arnold Networks, an alternative to Multi-Layer Perceptrons

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

KAN: Kolmogorov-Arnold Networks — mentioned by hansvm

Fri, 03 May 2024 12:47:36 GMT

The paper [0] is pretty good for handling questions like those. > doesn't KA representation require continuous univariate functions? All multivariate continuous functions (on a bounded domain) can be represented as compositions of addition and univariate continuous functions. Much like an MLP, you can also approximate discontinuous functions well on most of the domain (learning a nearby continuous function instead). > do B-splines actually cover the space of all continuous functions Much like an MLP, you can hit your favorite accuracy bound with more control points. > wouldn't... MLPs be…

Paper and discussion history. A mention is not an endorsement.

KAN: Kolmogorov-Arnold Networks — HN submission

Fri, 03 May 2024 09:16:21 GMT

Kolmogorov–Arnold Networks: Alternative to Multilayer Perceptrons.

2 points; 3 discussion replies on HN. Paper and source history. A submission is not an endorsement.

KAN: Kolmogorov-Arnold Networks — HN submission

Thu, 02 May 2024 09:42:36 GMT

Kan: Kolmogorov-Arnold Networks

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

KAN: Kolmogorov-Arnold Networks — mentioned by nyrikki

Thu, 02 May 2024 00:38:56 GMT

Not for discontinuous functions, as your paper explains, we know that g exists for discontinuous bounded, but nothing to find it with. > A practical construction of g in cases with discontinuous bounded and un- bounded functions is not yet known. For such cases Theorem 2.1 gives only a theoretical understanding of the representation problem. This is because for the representation of discontinuous bounded functions we have derived (2.1) from the fact that the range of the operator Z∗ is the whole space of bounded functions B(Id). This fact directly gives us a formula (2.1) but does not tell…

Paper and discussion history. A mention is not an endorsement.

KAN: Kolmogorov-Arnold Networks — HN submission

Wed, 01 May 2024 17:40:03 GMT

Kan: Kolmogorov-Arnold Networks

3 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

KAN: Kolmogorov-Arnold Networks — mentioned by nextaccountic

Wed, 01 May 2024 17:10:14 GMT

At page 6 it explains how they did back propagation https://arxiv.org/pdf/2404.19756 (and in page 2 it says that previous efforts to leverage Kolmogorov-Arnold representation failed to use backpropagation), so maybe using backpropagation to train multilayer networks with this architecture is their main contribution? > Unsurprisingly, the possibility of using Kolmogorov-Arnold representation theorem to build neuralnetworks has been studied [8, 9, 10, 11, 12, 13]. However, most work has stuck with the original depth-2 width-(2n + 1) representation, and did not have the chance to leverage more…

Paper and discussion history. A mention is not an endorsement.