Deep Residual Learning for Image Recognition · HackerPapers discussions

Deep Residual Learning for Image Recognition — mentioned by rhdunn

Fri, 30 Oct 2026 10:13:50 GMT

AFAICT there are several things that affect the size of the model: 1. the number of transformer layers -- e.g. each layer adds the ability to provide more information to the residual stream [1], [2]; each layer is `tokens * features * active context` 2. the number of tokens -- the more tokens there are the more word fragments and Unicode characters you can represent 3. the number of features assigned for a token -- these are used to associate trained semantic information for the token; with the active context this can encode contextual information for that token (the idea being that the model…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by cute_boi

Tue, 07 Jul 2026 19:03:23 GMT

No need stupid moving texts. CS231n: Convolutional Neural Networks for Visual Recognition - https://cs231n.github.io/ The Unreasonable Effectiveness of Recurrent Neural Networks - https://karpathy.github.io/2015/05/21/rnn-effectiveness/ Understanding LSTM Networks - https://colah.github.io/posts/2015-08-Understanding-LSTMs/ ImageNet Classification with Deep Convolutional Neural Networks - https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436... Deep Residual Learning for Image Recognition - https://arxiv.org/abs/1512.03385 Multi-Scale Context Aggregation by Dilated Convolutions…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by gooob

Tue, 07 Jul 2026 18:54:32 GMT

https://colah.github.io/posts/2015-08-Understanding-LSTMs/ https://papers.nips.cc/paper/2012/hash/c399862d3b9d6b76c8436... https://papers.nips.cc/paper_files/paper/2012/file/c399862d3... https://arxiv.org/pdf/1512.03385 https://arxiv.org/pdf/1511.07122 https://arxiv.org/pdf/1603.05027 https://arxiv.org/pdf/1409.2329 https://arxiv.org/pdf/1512.02595 https://arxiv.org/pdf/1409.0473 https://arxiv.org/pdf/1506.03134 https://arxiv.org/pdf/1706.03762 https://nlp.seas.harvard.edu/annotated-transformer/ https://arxiv.org/pdf/1410.5401 https://arxiv.org/pdf/1706.01427 https://arxiv.org/pdf/1806.01822…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by grumbelbart2

Thu, 22 May 2025 12:55:23 GMT

> What contributed more towards success in my opinion are "shortcut connections" through layers which enable more influence on early layers during learning. For those who don't know, that is the idea behind ResNet (He et al., Deep Residual Learning for Image Recognition, https://arxiv.org/abs/1512.03385), one of the most influential papers in deep learning of all time. Residual connections make it possible to train networks that are arbitrarily deep. Before ResNet, networks that were too deep were essentially not trainable due to vanishing or exploding gradients.

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — HN submission

Mon, 09 Sep 2024 18:23:51 GMT

Show HN: Turn Any ArXiv Paper into a 200-Page Prerequisite Reading Book

I created this tool over the weekend because, as someone interested in AI and technology, I find many research papers on arXiv fascinating but often incredibly dense and difficult to understand due to the heavy jargon and technical language. I wanted to make these complex topics more accessible to laypeople like myself by converting the topics described in these papers into full-length books that break down key concepts, making cutting-edge research easier to grasp. You can think of it as generating prerequisite reading materials before being able to read the actual paper. Here are some…

6 points; 3 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by is_true

Sat, 18 May 2024 12:26:29 GMT

a backup just in case https://nlp.seas.harvard.edu/annotated-transformer/ https://scottaaronson.blog/?p=762 https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://colah.github.io/posts/2015-08-Understanding-LSTMs/ https://arxiv.org/pdf/1409.2329.pdf https://www.cs.toronto.edu/~hinton/absps/colt93.pdf https://arxiv.org/pdf/1506.03134.pdf https://proceedings.neurips.cc/paper_files/paper/2012/file/c... https://arxiv.org/pdf/1511.06391.pdf https://arxiv.org/pdf/1811.06965.pdf https://arxiv.org/pdf/1512.03385.pdf https://arxiv.org/pdf/1511.07122.pdf https://arxiv.org/pdf/1704.01212.pdf…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by godelski

Tue, 28 Nov 2023 18:39:13 GMT

> The field of ML research is moving so fast that people don’t even take time anymore to explain the design decisions behind their architectures. Has it ever? I've read a lot of "old" papers and idk if there's ever been a strong theoretical framework. I'd say it is strongest today but not in main papers and those papers are getting rejected because of lack of experiments (wtf is going on with reviewing and why are our solutions "authors have to show reviewers used LLMs"[-1] not "reviewers have to provide meaningful review"? What a joke!). Here's a more recent Universal Approximation paper[0].…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by radq

Mon, 19 Jun 2023 07:21:26 GMT

We were missing two architecture patterns that were needed to get deeper nets to converge: residual nets [1] which solved gradient propagation, and batch normalization [2] which solved initialization. [1] Residual nets (2015): https://arxiv.org/abs/1512.03385 [2] Batch normalization (2015): https://arxiv.org/abs/1502.03167

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by godelski

Wed, 02 Jun 2021 20:05:40 GMT

For more context to people, we have the universal approximation theorem for neural nets that basically says if a network is wide enough it can approximate anything (with at least 2 layers). So a lot of stuff was really wide. Then VGG[0] came out and showed that deep networks were very effective (along with other papers, things happen in unison. Leibniz and Newton). Then you get ResNets[1] with skip connections and move forward to today. Today we've started looking more at what networks are doing and where their biases lie. This is because we're running into some roadblocks with CNNs vs…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by andersource

Wed, 02 Jun 2021 18:23:59 GMT

Haven't read thoroughly but it seems they are investigating ResNet models [0] trained for image classification. > We apply CKA to a family of ResNets of varying depths and widths, trained on common benchmark datasets (CIFAR-10, CIFAR-100 and ImageNet) [0] https://arxiv.org/abs/1512.03385

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by laplacesdemon48

Tue, 06 Apr 2021 02:25:07 GMT

1) How similar is this to creating a resnet? [1]. What are some key differences and similarities? 2) Has a CNN version of this been implemented in PyTorch? [1] https://arxiv.org/pdf/1512.03385.pdf (Figure 2)

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by alquemist

Tue, 10 Nov 2020 13:30:47 GMT

Intelligent trial and error. 1. Transformers are an extension of the attention mechanism, a well known LSTM extension that worked well for machine translation (2014, https://arxiv.org/abs/1409.0473). A transformer model is essentially building a multi headed attention module, analogue to CNNs, then stacking several layers of them, analogue to CNNs / stacked LSTMs. (1998, http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf) 2. Transfomers use residual blocks, which were introduced by the ResNet CNN arhitecture (2015, https://arxiv.org/abs/1512.03385). At the time, ResNet was topping the…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by yorwba

Wed, 12 Aug 2020 13:19:07 GMT

ResNet https://arxiv.org/abs/1512.03385 came out of Microsoft Research Asia. I think that's a pretty good thing they produced. Evidently they managed to hire the top talent behind that work at some point. Looks like Kaiming He works at Facebook now, though. http://kaiminghe.com/

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by spaced-out

Sat, 15 Dec 2018 03:23:18 GMT

This paper explains the reasoning begins that https://arxiv.org/abs/1512.03385

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by Q6T46nT668w6i3m

Sat, 16 Jun 2018 15:08:12 GMT

Conceptually, if you’re trying to classify artwork by a category like artist or period, the problem is just ordinary image classification and won’t require anything unique so I’d recommend just reading the current SOA papers for that task. The foundation for many of the current SOA papers is He, et. al: https://arxiv.org/abs/1512.03385 (I wrote and maintain Keras-ResNet and I’m always looking for interesting examples. Let me know if you cook up something interesting!)

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by tanilama

Tue, 14 Nov 2017 07:25:52 GMT

https://arxiv.org/abs/1512.03385 Deep Residual Learning for Image Recognition Citation 4000+. Milestone of image recognition. https://arxiv.org/abs/1502.01852 Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. Also known as PreLU, citation 1600+. The initialization scheme is widely adapted by a lot of following works. Those are the most influential ones, all come from Kaiming He. He is now in FAIR, and keeps very good publication record, with works like MASK-RCNN, but that doesn't really count as purely Chinese contribution, so we skipped here. Some…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by nl

Sat, 03 Sep 2016 05:16:58 GMT

In natural language processing tasks you see a lot of non-CNN architectures. These usually are designed to be able to deal with sequential data, so some kind of "memory" is needed. Sometimes you see this combined with a CNN. There has been a few question answering systems that have one or more CNN layers. In don't entirely understand these designs, but presumably the convultional layers are an attempt to understand the different orders of words. There are lots of techniques that people use to try to make deep networks work well. Mostly theses are about making errors backprog better. One of…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by Kip9000

Sun, 05 Jun 2016 17:49:47 GMT

Deep Residual Learning for Image Recognition https://arxiv.org/abs/1512.03385 Abstract: Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by walrus

Wed, 13 Apr 2016 05:38:14 GMT

I'm not qualified to answer this, but I will anyway. To "operate" neural networks (as opposed to writing a framework for them), you need to know the building blocks. There are basic blocks like fully connected layers, convolutions, and nonlinear activations. Beyond those, there are higher level building blocks like LSTMs[1], gated recurrent units[2], highway layers[3], batch normalization[4], and residual blocks[5] that are made up of simpler blocks. Learning what these do and when it's appropriate to use them requires following current literature. Operating neural networks requires some…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by radarsat1

Mon, 04 Apr 2016 12:43:32 GMT

Having not read the paper, something I find unclear: is it only the feedback path that is skipped, or is the feedforward path also skipped? The abstract mentions replacing the layer with an identity function. I'm not sure how this would work, wouldn't it change the result (i.e the encoding used by the following layer would be corrupted) if you just multiply the inputs by 1 and add them? Otherwise, how precisely do you "skip" a layer without corrupting the training of lower layers? Edit: the answer is in the definition of "skip layers", introduced in a previous paper…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by argonaut

Wed, 09 Mar 2016 01:06:06 GMT

Yes. The general task of looking at arbitrary images and labeling objects (from a set of known categories) in those images is called "detection." In fact the problem you described is easier, because there's only one category (people), and the system only needs to provide a count, rather than provide bounding box rectangles around each object (which is what the standard "detection" task entails). Convolutional neural nets are the state of the art for this, specifically deep residual learning (http://arxiv.org/abs/1512.03385). It requires a good deal of background to understand what's going on…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by nrmn

Wed, 23 Dec 2015 19:44:54 GMT

I've been trying to read a paper a day since midsummer. These are a few of the interesting, for me personally, since then: Generating Sequences With Recurrent Neural Networks - http://arxiv.org/abs/1308.0850 Older one, but important to understand deeply since other recent ideas have come from this! Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks - http://arxiv.org/abs/1511.06434 Unitary Evolution Recurrent Neural Networks - http://arxiv.org/abs/1511.06464 State of the Art Control of Atari Games Using Shallow Reinforcement Learning…

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by racoonear

Wed, 23 Dec 2015 17:56:12 GMT

Deep Residual Learning for Image Recognition http://arxiv.org/abs/1512.03385 Batch Normalization http://jmlr.org/proceedings/papers/v37/ioffe15.pdf Deep Neural Decision Forests http://research.microsoft.com/pubs/255952/ICCV15_DeepNDF_mai... Spatial Transformer Networks https://papers.nips.cc/paper/5854-spatial-transformer-networ...

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — mentioned by dplarson

Mon, 14 Dec 2015 01:26:42 GMT

It's mentioned in the article, but here's a direct link to the paper on arXiv: http://arxiv.org/abs/1512.03385 And direct link to the PDF: http://arxiv.org/pdf/1512.03385v1.pdf

Paper and discussion history. A mention is not an endorsement.

Deep Residual Learning for Image Recognition — HN submission

Fri, 11 Dec 2015 15:50:37 GMT

Deep Residual Learning for Image Recognition

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Deep Residual Learning for Image Recognition — HN submission

Fri, 11 Dec 2015 05:42:20 GMT

Deep Residual Learning for Image Recognition

47 points; 6 discussion replies on HN. Paper and source history. A submission is not an endorsement.