arXiv:2003.08505cs.CV
Deep metric learning papers from the past four years have consistently claimed great advances in accuracy, often more than doubling the performance of decade-old methods. In this paper, we take a closer look at the field to see if this is actually true. We find flaws in the experimental methodology of numerous metric learning papers, and show that the actual improvements over time have been marginal at best.
Linked by 3 identifiable commenters in 3 comments across 3 threads, from May 2020 to Dec 2022. Found in 3 HN submissions from May 2020 to May 2020. Open submission discussions.
Another paper on the related topic of metric learning arguing that metric learning hasn't actually made any progress and is piggybacking on progress elsewhere: https://arxiv.org/abs/2003.08505
AI is having some reproducibility issues, especially reinforcement learning: https://www.wired.com/story/artificial-intelligence-confront... In other areas of machine learning, things have gotten a lot better. A decade ago people rarely released their code or trained models. I…
Connected through shared HN threads, not inferred similarity or a recommendation.
Ben Recht actually has a line of work showing that we aren't over fitting the validation/test set for now (amazingly...). What I mean is, by chasing higher and higher SotA with more and more money and compute, whole fields can go "improving" only for papers like…
igorkrawView on HNParent context