A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes — mentioned by Mr_P

Wed, 01 Jun 2022 10:07:11 GMT

I think they should have let him publish the paper. We've seen time and again that various trends in ML turn out to have actually been dead-ends. A few examples: https://arxiv.org/abs/2102.06356 https://arxiv.org/abs/1905.03678 That said, I've also seen plenty of competitive drama in FAANG research labs, so this story is not hard to believe. More senior engineers often will use their seniority to power-grab control of projects. It sounds like Google execs did the right thing in the end.

Paper and discussion history. A mention is not an endorsement.

A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes · HackerPapers discussions

A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes — mentioned by Mr_P