HackerPapers

arXiv:1605.06170cs.LG

Evaluation System for a Bayesian Optimization Service

Ian Dewancker, Michael McCourt, Scott Clark, Patrick Hayes, Alexandra Johnson, George Ke

Early discovery. Linked by one identifiable commenter. This paper is outside the main ranking; a mention is not an endorsement.

Abstract

Bayesian optimization is an elegant solution to the hyperparameter optimization problem in machine learning. Building a reliable and robust Bayesian optimization service requires careful testing methodology and sound statistical analysis. In this talk we will outline our development of an evaluation framework to rigorously test and measure the impact of changes to the SigOpt optimization service. We present an overview of our evaluation system and discuss how this framework empowers our research engineers to confidently and quickly make changes to our core optimization engine

arXiv comments
Journal reference
DOI
Subjects
cs.LGMachine Learning

On Hacker News

Linked by 1 identifiable commenter in 2 comments across 2 threads, from Aug 2016 to May 2018.

Comments linking this paper, per year.

Discussed in comments2 threads

  1. Circulant Binary Embeddings 1 comment

    • We built an evaluation suite [1] and have some benchmarks [2] that we presented at some ICML workshops in 2016 comparing against some standard and open source methods. Members of our research team built or contributed to many of the popular open source libraries in Bayesian…

      Zephyr314View on HNParent context

  2. Bayesian Optimization for Collaborative Filtering with MLlib 1 comment

    • We don't explicitly share information at the experiment level like that for privacy reasons. All user data is explicitly isolated from other users, on top of the fact that due to the black box nature of our optimizer we often do not know what the underlying method or system…

      Zephyr314View on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.