Standardization of Post-Publication Code Verification is Possible with the Support of the Community

Abstract

Reproducibility remains a challenge in machine learning research. While code and data availability requirements have become increasingly common, post-publication verification is still limited and unformalized. This position paper argues that journals and conference proceedings should all implement a post-publication verification submission system. We propose a modification to ACM and IEEE Access verification badges that allows independent researchers to submit post-publication code replications directly to the journal or proceedings, leading to visible verification badges included in the article metadata. Each article may earn up to two badges, each linked to verified code in its corresponding public repository. This framework would be a step forward toward a standardized reproducibility policy and would provide young researchers with an opportunity to improve their curriculum. We describe the motivation, related initiatives, a formal framework, the potential impact, possible limitations, and alternative views.