How bayesopt find kernel parameters

Hello all,

I know that bayesopt uses fitrgp to create gaussian process in each iteration. But how bayesopt find the optimize kernel parameters of the Gaussian process regression in each step? Does it optimize kernel parameters at all? If not, what are the kernel paramters being used in each iteration?

I want to know the default configurations of bayesopt for the items above, I was not able to find my answer in the documentation.

Thank you in advance

[Aep]

Any help on this is greatly appreciated.

Answer 1

[Mohith Kulkarni]

By default the optimize parameter is set to 0 for the fitrgp KernelFunction and KernelScale hyperparmeters. Refer to the below code to change the parameter:

params = hyperparameters('fitrgp',X,y);
params(3).Optimize = true;              %set KernelFunction optimize to true
params(4).Optimize = true;              %set KernelScale optimize to true

In case of "fitrgp" fit function, check Hyperparameter Optimization section of fitrgp arguments for more information. You can check the default Kernel Function and Kernel Parameters of fitrgp fit function here:

KernelFunction and KernelParameters

you can then use the fit function in the objective function.

For more information on performing Bayesian Optimization using bayesopt refer to:

Bayesian Optimization using bayesopt

[Aep]

Thank you for your explanations Mohith. As mentioned here , MATLAB uses matern 5/2 kernel function for bayesopt. So my main question is how the kernela paraemeters are found for this Kernel function in Bayesopt. As you mentioned the initial values of kernel parameters for matern 5/2 is mentioned here. So does the bayesopt uses these default values? If yes, how does these "initial" kernel parameters change in each iteration of bayesopt? or do they change at all or they remain the same in each iteration?

In other words: are the hyperparameters found by maximizing their marginal likelihood in the default configurations of bayesopt?

[Mohith Kulkarni]

Regarding the second question, yes it does change. In each iteration, fitrgp is called with the default initial values for its own hyperparameters (covariance and the kernel parameters), and it's fitted from scratch every iteration. To avoid getting stuck with potentially poor parameters, BayesOpt does not start from the fitted values from the previous iteration.

[Aep]

Thank you very much for your answer Mohith. So as you mentioned the inital values are different in each iteration. But considering just one single iteration. We have some initial values. What happens to these values in that specific iteration in bayesopt? Do they change in that iteration? If yes, how?

I apologize if it is a basic question in Gaussian process. I am new to this and I want to make it clear for myself about how Bayesopt does this.