Accelerating Deep Learning by Focusing on the Biggest Losers — mentioned by dgacmu

Wed, 18 May 2022 19:01:57 GMT

You have to stash more information from the forward pass in order to calculate the gradients during backprop. You can't just naively use an inference accelerator as part of training - inference-only gets to discard intermediate activations immediately. (Also, many inference accelerators use lower precision than you do when training) There are tricks you can do to use inference to accelerate training, such as one we developed to focus on likely-poorly-performing examples: https://arxiv.org/abs/1910.00762

Paper and discussion history. A mention is not an endorsement.

Accelerating Deep Learning by Focusing on the Biggest Losers — HN submission

Mon, 07 Oct 2019 23:20:14 GMT

Accelerating Deep Learning by Focusing on the Biggest Losers

5 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Accelerating Deep Learning by Focusing on the Biggest Losers · HackerPapers discussions

Accelerating Deep Learning by Focusing on the Biggest Losers — mentioned by dgacmu

Accelerating Deep Learning by Focusing on the Biggest Losers — HN submission