Discontinuous (integer) search spaces in MOE

[dwinston]

I am linking to @HGeerlings issue raised as Yelp/MOE#438. I think flooring is possible, as suggested in Yelp/MOE#369. It appears that SigOpt (the company founded by the creator of MOE) claims to support categorical variables in its service. I don't know how SigOpt does it / whether it's effective.

FYI I set up a small example (via a82497f) to evaluate SigOpt for the water-splitting problem.

[dwinston]

Hmm, SigOpt's "premium" plan (free for 30 days) only allows 150 observations per experiment. The best GA algorithm took ~3000 calculations on average to find the top 20 water-splitting materials (better than ~18,000 for random search!), so unless SigOpt's algorithm is really good, we might have to lean on MOE.

[HGeerlings]

Yeah Neeka and I took a peak at SigOpt's trial deal, and it seems the
limitations on it may definitely be too extensive to see if it actually
works well. I received an email from Scott, co-founder of SigOpt, that
basically summarized a couple of ways (like rounding) too outfit Moe for
integers that I can forward to you all.

Does the code you posted 'perov_sigopt' require only a trial of SigOpt?

Thanks,
Henry

On Tue, Jul 14, 2015 at 3:09 PM, Donny Winston [email protected]
wrote:

Hmm, SigOpt's "premium" plan (free for 30 days) only allows 150
observations per experiment. The best GA algorithm took ~3000 calculations
on average to find the top 20 water-splitting materials (better than
~18,000 for random search!), so unless SigOpt's algorithm is really
good, we might have to lean on MOE.

—
Reply to this email directly or view it on GitHub
#15 (comment)
.

[dwinston]

Yes, the code requires only a trial. I actually invited you and @nmashouf to the trial I started, but you can also start your own.

[dwinston]

Yes, please post the email content here if that makes sense (i.e. no sensitive info), o/w just fwd to us.

[HGeerlings]

From Scott:

Unfortunately, I am no longer actively maintaining the public version of
MOE. I'll outline a few options though.

1. You can use SigOpt (sigopt.com http://sigopt.com/), which handles
integers (and categories) natively by extending MOE and other methods.

2. You can use the rounding method described above. This may cause repeats
as you point out though. You could try to extend MOE to discount these
though, or tell MOE that those points are already being sampled, which will
also discount their EI. Artificially setting the length scales of those
variables to be "integer length" could also help.

3. You can extend MOE to handle these types of parameters natively using a
continuous relaxation or other method.

I hope this helps. Let me know if you'd like to try our SigOpt and I can
help get you set up.

Thanks Donny,

On Tue, Jul 14, 2015 at 3:21 PM, Donny Winston [email protected]
wrote:

Yes, please post the email content here if that makes sense (i.e. no
sensitive info), o/w just fwd to us.

—
Reply to this email directly or view it on GitHub
#15 (comment)
.