Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth — mentioned by sova

Wed, 02 Jun 2021 16:47:32 GMT

At first I thought this had something to do with the classic "breadth vs. depth" notion on learning stuff -- if you're preparing for the MCAT it is better to have breadth that covers all the topics than depth in one or two particulars for the exam, but this is actually just about the dimensions of the neural network used to create representations. Naturally, one would expect a "sweet spot" or series of "sweet spots." From the paper at https://arxiv.org/pdf/2010.15327.pdf > As the model gets wider or deeper, we seethe emergence of a distinctive block structure— a considerable range of hidden…

Paper and discussion history. A mention is not an endorsement.

Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth · HackerPapers discussions

Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth — mentioned by sova