Practical bayesian optimization using Goptuna

  • Tuning Video Streaming Algorithms
  • Tuning Machine Learning Models
  • AR/VR Hardware Design
  • Tuning Just-In-Time compiler (for HHVM)

Introduction of Bayesian Optimization

Investigating existing Bayesian Optimization library for Go

Optimization example for a quadratic function using go-bayesopt.
$ go run _examples/main.go
...
2019/07/25 15:23:23 x: map[bayesopt.Param]float64{bayesopt.UniformParam{Name:"X1", Max:10, Min:-10}:1.0221672603083967, bayesopt.UniformParam{Name:"X2", Max:10, Min:-10}:1.8540991095989217}
2019/07/25 15:23:23 y: 0.021778
Values of X1 searched by go-bayesopt.
Values of x2 searched by go-bayesopt

TPE bayesian optimization using Goptuna

An example of Goptuna
See the paper of Optuna at KDD 2019 for more details about Define-by-Run interface. Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Framework. In The 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’19), August 4–8, 2019.
$ go run _examples/simple_random_search/main.go
...
Result:
- best value 2.5904889254208054
- best param map[x1:0.4405659427118902 x2:0.7883530201915825]
$ go run _examples/simple_tpe/main.go
...
Result:
- best value 0.6195459685895217
- best param map[x1:0.7770961438621743 x2:1.2451093857329996]
Goptuna (TPE) works extremely fast!

Conclusion

--

--

--

Creator of go-prompt and kube-prompt. Optuna committer. Kubeflow/Katib reviewer. GitHub: c-bata

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Bayes by Back prop (BBB), From Robust Neural Networks to Unified theory of Brain?

Can you Teach a Computer to See Your Pain?

Faster Notes with Python and Deep Learning

Machine learning packages used at Google

The AI & Analytics Engine vs. Amazon SageMaker Canvas

Dewarping Documents with AI

Bias & Variance in Machine Learning

My Machine Learning Daiary: Day 32

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Masashi SHIBATA

Masashi SHIBATA

Creator of go-prompt and kube-prompt. Optuna committer. Kubeflow/Katib reviewer. GitHub: c-bata

More from Medium

Optimize Deep Learning Inference on GPU — Part 1

Making an optimisation algorithm 10k times faster 🏎

HOLA Optimization: A Lightweight Hyperparameter Optimization Software Package

Tutorial — Basic Kubeflow Pipeline From Scratch