Deep Learning in Neural Networks: An Overview · HackerPapers discussions

Deep Learning in Neural Networks: An Overview — mentioned by jll29

Thu, 26 Sep 2024 18:33:21 GMT

> but it would be impractical to make it book-sized Not really: History: https://arxiv.org/abs/2212.11279 (75 pp.) Survey: https://arxiv.org/abs/1404.7828 (88 pp.) Conveniently skim-read over the course of the four weekends on one month.

Paper and discussion history. A mention is not an endorsement.

Deep Learning in Neural Networks: An Overview — HN submission

Mon, 18 Dec 2023 21:28:05 GMT

Deep Learning in Neural Networks: An Overview

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Deep Learning in Neural Networks: An Overview — mentioned by albertzeyer

Thu, 16 Nov 2023 22:44:21 GMT

Ah good question. I think I have read that statement by some other people. But the limit is kind of arbitrary. And of course, this limit will be higher and higher over time, that's why I put the year. I think the limit should also not be much lower. We already have language models three order of magnitude larger (>1T params), and we also call them "large", so in this context, all those single-digit billion parameter models feel quite small. Similarly, when is a network "deep"? It used to mean more than 2 or 3 layers. And then there was a definition for "very deep", starting with more than 10…

Paper and discussion history. A mention is not an endorsement.

Deep Learning in Neural Networks: An Overview — mentioned by mjn

Sat, 05 Oct 2019 20:00:42 GMT

I mean it's self-promotional yes, but I read this as more of a blog post about advances specifically in his own group. For the Schmidhuberian take on the broader history of deep learning, this other one's the go-to article (though it's much longer): https://arxiv.org/abs/1404.7828 Not everyone likes that article either, but it does at least extensively cite prior work, i.e. accounts for "how his contributions were (like everyone else's) evolutions of existing methods". In particular, sections 5.1–5.4 credit a large amount of work from the 1960s-80s that he considers foundational.

Paper and discussion history. A mention is not an endorsement.

Deep Learning in Neural Networks: An Overview — HN submission

Fri, 14 Jun 2019 03:40:25 GMT

Ask HN: Please suggest review papers in CS topics

The one paper I have read is - Deep Learning in Neural Networks: An Overview - Juergen Schmidhuber - https://arxiv.org/abs/1404.7828, I would like to explore more review papers on various topics, just to broaden my horizons.

3 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Deep Learning in Neural Networks: An Overview — mentioned by YorkshireSeason

Tue, 24 Jul 2018 09:54:19 GMT

It's not widely known, but Alan Turing tinkered with neural nets in 1948, see [1], with his boss at the time calling Turing's work a "schoolboy essay". For a history of the subject, see Section 5.1 of [2]. Note that some parts of [2] are rather controversial. For a historical account of back-progagation, which itself was 'invented' many times, see [3, 4]. It is also not widely known that regular expressions, one of programmers' favourite tools, were introduced by Kleene in [5]: he was interested in characterising the behaviour of McCulloch-Pitts "nerve nets" (early versions of neural nets)…

Paper and discussion history. A mention is not an endorsement.

Deep Learning in Neural Networks: An Overview — mentioned by iluvmylife

Fri, 22 Sep 2017 13:47:50 GMT

If you want a more nuanced research on the history on deep learning in neural networks, here is an excellent historical survey paper: https://arxiv.org/abs/1404.7828

Paper and discussion history. A mention is not an endorsement.

Deep Learning in Neural Networks: An Overview — mentioned by cosmoharrigan

Mon, 16 Jan 2017 21:17:34 GMT

For more details, refer to Schmidhuber's own website: http://people.idsia.ch/~juergen/ and his excellent review paper, "Deep Learning in Neural Networks: An Overview": https://arxiv.org/pdf/1404.7828.pdf

Paper and discussion history. A mention is not an endorsement.

Deep Learning in Neural Networks: An Overview — mentioned by jabl

Mon, 19 Sep 2016 07:17:43 GMT

https://arxiv.org/abs/1404.7828 (Disclaimer: I don't know that much about ML/Deep Learning, the above was recommended by deeplearning.net and is what I'm slogging through ATM. I do have some background in "classical" statistics & optimization theory, and the above article is not that math-heavy)

Paper and discussion history. A mention is not an endorsement.

Deep Learning in Neural Networks: An Overview — mentioned by speechduh

Tue, 30 Jun 2015 20:01:28 GMT

So, for anyone not aware: Schmidhuber is _obsessed_ with this. He wrote an enormous literature review of deep learning [0] basically because he felt that people weren't crediting ideas enough. This isn't a one-off essay, for him, he's been banging this drum for quite a while. Not saying he's wrong, just FYI. [0] http://arxiv.org/abs/1404.7828

Paper and discussion history. A mention is not an endorsement.

Deep Learning in Neural Networks: An Overview — mentioned by jcr

Sat, 20 Dec 2014 01:48:08 GMT

Machine Learning is a fast moving field, and many papers are incremental improvements, so an always up-to-date list will be unlikely. None the less, there are quite a few lists out there if you look. Here are some I've seen posted recently: https://news.ycombinator.com/item?id=8728112 http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Recomm... http://www.cs.toronto.edu/~hinton/deeprefs.html http://arxiv.org/abs/1404.7828

Paper and discussion history. A mention is not an endorsement.

Deep Learning in Neural Networks: An Overview — mentioned by Houshalter

Thu, 11 Dec 2014 02:25:06 GMT

There are some survey papers like this: http://arxiv.org/abs/1404.7828 The point isn't to read every paper it references, but to just get a summary of what the current state of research is. If you are interested in an area then you can go read the papers there.

Paper and discussion history. A mention is not an endorsement.

Deep Learning in Neural Networks: An Overview — HN submission

Fri, 02 May 2014 14:29:02 GMT

Deep learning in neural networks: An overview

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.