arXiv:2302.13246math.OC
PDFO: A Cross-Platform Package for Powell's Derivative-Free Optimization Solvers
Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.
Abstract
The late Professor M. J. D. Powell devised five trust-region methods for derivative-free optimization, namely COBYLA, UOBYQA, NEWUOA, BOBYQA, and LINCOA. He carefully implemented them into publicly available solvers, renowned for their robustness and efficiency. However, the solvers were implemented in Fortran 77 and hence may not be easily accessible to some users. We introduce the PDFO package, which provides user-friendly Python and MATLAB interfaces to Powell's code. With PDFO, users of such languages can call Powell's Fortran solvers easily without dealing with the Fortran code. Moreover, PDFO includes bug fixes and improvements, which are particularly important for handling problems that suffer from ill-conditioning or failures of function evaluations. In addition to the PDFO package, we provide an overview of Powell's methods, sketching them from a uniform perspective, summarizing their main features, and highlighting the similarities and interconnections among them. We also present experiments on PDFO to demonstrate its stability under noise, tolerance of failures in function evaluations, and potential to solve certain hyperparameter optimization problems.
Good coffee is needed to understand Powell’s papers on these algorithms. If you don’t have, section 3 of the following paper is a good place to start with: https://arxiv.org/pdf/2302.13246.pdf
domcoltdView on HNParent context
> DFOs tend to have issues past a few dozens variables. It is interesting to check the results in section 5.1 of the following paper: https://arxiv.org/pdf/2302.13246.pdf where these algorithms are compared with finite-difference CG and BFGS included in SciPy.
domcoltdView on HNParent context