Practical bayesian optimization using Goptuna

  • Tuning Video Streaming Algorithms
  • Tuning Machine Learning Models
  • AR/VR Hardware Design
  • Tuning Just-In-Time compiler (for HHVM)

Introduction of Bayesian Optimization

First, I will explain what bayesian optimization algorithms are doing. When we have a something system and its performance depends on something continuous parameters in the range of 0 to 10. I would like to specify an optimal parameter that gives the best performance of our system.

Investigating existing Bayesian Optimization library for Go

I googled existing libraries for bayesian optimization in Go, and I found go-bayesopt, gaussian process based bayesian optimization library. I prepared the minimization problem like following.

Optimization example for a quadratic function using go-bayesopt.
$ go run _examples/main.go
...
2019/07/25 15:23:23 x: map[bayesopt.Param]float64{bayesopt.UniformParam{Name:"X1", Max:10, Min:-10}:1.0221672603083967, bayesopt.UniformParam{Name:"X2", Max:10, Min:-10}:1.8540991095989217}
2019/07/25 15:23:23 y: 0.021778
Values of X1 searched by go-bayesopt.
Values of x2 searched by go-bayesopt

TPE bayesian optimization using Goptuna

TPE(Tree-structured Parzen Estimator) requires O(t), t is an iteration count. So I googled exiting TPE implementation in Go, but it’s not found. So I implemented it and published Goptuna yesterday.

An example of Goptuna
See the paper of Optuna at KDD 2019 for more details about Define-by-Run interface. Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In The 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’19), August 4–8, 2019.
$ go run _examples/simple_random_search/main.go
...
Result:
- best value 2.5904889254208054
- best param map[x1:0.4405659427118902 x2:0.7883530201915825]
$ go run _examples/simple_tpe/main.go
...
Result:
- best value 0.6195459685895217
- best param map[x1:0.7770961438621743 x2:1.2451093857329996]
Goptuna (TPE) works extremely fast!

Conclusion

Goptuna seems to be practical optimization library in Go. I hope everyone will use Goptuna and feedback me!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Masashi SHIBATA

Masashi SHIBATA

Creator of go-prompt and kube-prompt. Optuna core-dev. Kubeflow/Katib reviewer. GitHub: c-bata