Scalable Bayesian Optimization Using Deep Neural Networks · HackerPapers discussions

Scalable Bayesian Optimization Using Deep Neural Networks — mentioned by abhgh

Thu, 03 Jan 2019 09:32:50 GMT

While this is mostly true there is has been some research to use non-GP models for Bayesian Optimization or the general Sequential Model Based Optimization (SMBO) problem. Some examples: * SMAC (using Random forests) - [1] * Using Deep Neural Networks - [2] * Tree Structured Parzen Estimators - [3] [1] https://www.cs.ubc.ca/~hutter/papers/10-TR-SMAC.pdf [2] https://arxiv.org/pdf/1502.05700.pdf [3] https://papers.nips.cc/paper/4443-algorithms-for-hyper-param...

Paper and discussion history. A mention is not an endorsement.

Scalable Bayesian Optimization Using Deep Neural Networks — mentioned by pedrosorio

Thu, 09 Jun 2016 13:28:26 GMT

It has been done already: http://arxiv.org/abs/1502.05700 Edit: Although what they seem to describe is replacing GPs with neural networks in Bayesian optimization which is supposedly more efficient. Since the point of Bayesian optimization is to limit the number of times you have to evaluate a new set of hyperparameters, I am not sure how useful it is to "be able to scale" (i.e. even if maintaining the GP is O(n^3) with the number of evaluations, the costly part should be to evaluate the hyperparameters in the first place) but I haven't read the paper so they may show some high dimensional…

Paper and discussion history. A mention is not an endorsement.

Scalable Bayesian Optimization Using Deep Neural Networks — mentioned by mccourt

Fri, 11 Dec 2015 03:00:34 GMT

That may be. And I don't think kernels will ever be a hot topic. But part of that is that they are a very old topic (you can see Gauss referring to them in slide 22 of http://math.iit.edu/~fass/590/notes/Notes590_Ch1.pdf). I think that kinda prevents funding agencies from pushing too hard for research into them, since there is a more likely quick return on investment into a new topic. And I don't begrudge them that - funding should be given to people with new ideas which have the potential to revolutionize things. I'd also mention that there are people trying to bridge the gap between how GPs…

Paper and discussion history. A mention is not an endorsement.

Scalable Bayesian Optimization Using Deep Neural Networks — HN submission

Tue, 01 Sep 2015 14:11:21 GMT

Scalable Bayesian Optimization Using Deep Neural Networks

63 points; 3 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Scalable Bayesian Optimization Using Deep Neural Networks — HN submission

Fri, 20 Feb 2015 21:38:46 GMT

Scalable Bayesian Optimization Using Deep Neural Networks

4 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.