Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land · HackerPapers discussions

Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land — HN submission

Mon, 30 Jun 2025 18:02:06 GMT

Alice's Adventures in a Differentiable Wonderland

157 points; 26 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land — mentioned by zackmorris

Wed, 27 Nov 2024 18:26:20 GMT

LLMs and neural nets from first principles: https://arxiv.org/pdf/2404.17625 (pdf) https://news.ycombinator.com/item?id=40408880 (llama3 implementation) https://news.ycombinator.com/item?id=40417568 (my comment on llama3 with breadcrumbs) Admittedly, I'm way behind on how this translates to software on the newest video cards. Part of that is that I don't like the emphasis on GPUs. We're only seeing the SIMD side of deep learning with large matrices and tensors. But there are at least a dozen machine learning approaches that are being neglected, mainly genetic algorithms. Which means that…

Paper and discussion history. A mention is not an endorsement.

Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land — HN submission

Mon, 04 Nov 2024 16:42:54 GMT

Alice's Adventures in a Differentiable Wonderland (PDF)

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land — mentioned by seanhunter

Sat, 12 Oct 2024 13:27:21 GMT

Not exactly an answer to your question, but in the ballpark and hopefully you might find interesting https://arxiv.org/pdf/2404.17625 In the section on "Automatic differentiation" he considers the question of differentiable computation graphs.

Paper and discussion history. A mention is not an endorsement.

Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land — HN submission

Wed, 24 Jul 2024 09:29:59 GMT

Alice's Adventures in a Differentiable Wonderland (updated and in print)

3 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land — mentioned by zackmorris

Mon, 20 May 2024 17:04:31 GMT

Ya there are concepts in programming and math that are mostly self-teachable from first principles, but then there's what looks like gibberish because it's too new to have been distilled down into something tractable yet. I would say that arrays and matrices are straightforward to understand, while tensors are not. So I'm disappointed that so much literature currently revolves around tensors. Same for saying embedding instead of just vector representation, etc. It helps me to think in terms of levels of abstraction rather than complexity. My education stopped at a 4 year degree, but AI is…

Paper and discussion history. A mention is not an endorsement.

Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land — HN submission

Sat, 04 May 2024 20:02:38 GMT

Alice's Adventures in a Differentiable Wonderland

4 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land — HN submission

Thu, 02 May 2024 16:59:38 GMT

Alice's Adventures in a Differentiable Wonderland

1 point; 2 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land — mentioned by zackmorris

Tue, 30 Apr 2024 19:59:20 GMT

Ya so far this is the best introduction to neural networks from first principles that I've seen. Quickly skimming the draft pdf at https://arxiv.org/pdf/2404.17625 I can grok it instantly, because it's written in familiar academic language instead of gobbledygook. Anyone with an undergrad math education in engineering, computer science, etc or a self-taught equivalent understanding of differential equations should be able to read it easily. It does a really good job of connecting esoteric terms like tensors with arrays, gradients with partial derivatives, Jacobians with gradients and…

Paper and discussion history. A mention is not an endorsement.