Gradient Descent Implementation on a function, as opposed to an equation

11 views (last 30 days)
I have built a function
function [RMS] = ErrorFunction(ui,vi,wi,cx,cy,cz)
which outputs a certain error based on the six intial conditions I input to the model. Now, my model is iterative and the error depends on some intermediate parameters hence it is not possible to define a relationship between the error and six inputs. My aim is to minimize the error to zero using Gradient/Steepest Descent Method and I'm hoping somone would guide me in implementing it on a function, as opposed to a straightforward explicit relationship like, f(x,y) = 4x^2-4xy+2y^2

Accepted Answer

Bjorn Gustavsson
Bjorn Gustavsson on 7 Sep 2022
You could do something like this:
1, rewrite the function to take an array as input-parameter instead of a number of scalar parameters
function [RMS] = ErrorFunction([ui_vi_wi_cx_cy_cz])
2, calculate a numerical gradient (central difference):
curr_pars = [ui_vi_wi_cx_cy_cz]; % current point
h = 0.01; % you have to decide what a suitably small step is, and if you need different sized steps for the different input variables...
for i_v = numel(curr_pars):-1:1
cp = curr_pars;
cp(i_v) = cp(i_v) + h;
cm = curr_pars;
cm(i_v) = cm(i_v) - h;
gradErrorFunction(i_v) = (ErrorFunction(cp) - ErrorFunction(cm))/(2*h);
end
HTH
  1 Comment
PASUNURU SAI VINEETH
PASUNURU SAI VINEETH on 10 Sep 2022
Thank you very much, that was brilliant! This approach suits my purpose.
Apologize for the late response, I've been trying out different combinations of the stepsize h and learning rate gamma before getting back to you.
clc
clear all
gamma = 2;
curr_pars = [1 1 1 1 1 1]; % current point
tic
for k = 1:100
h = [1 1 1 1 1 1];
for i_v = numel(curr_pars):-1:1
cp = curr_pars;
cp(i_v) = cp(i_v) + h(i_v);
cm = curr_pars;
cm(i_v) = cm(i_v) - h(i_v);
gradErrorFunction(i_v) = (ErrorFunction(cp) - ErrorFunction(cm))/(2*h(i_v));
end
curr_pars = curr_pars - gamma*gradErrorFunction
[err] = ErrorFunction(curr_pars)
end
toc
I was wondering if setting gamma = 2*h makes things simpler as they cancel out each other in the line
curr_pars = curr_pars - gamma*gradErrorFunction
which makes the updated parameter array solely depend on h (from the difference of error functions).
Do you think the current stepsize and learning rate are reasonable or are they too high? I observed that towards the end of 100 iterations, the error stopped reducing and started increasing. But if I reduce those two variables, the error didn't even tend to decrease in the first place. I just played around with various combinations but didn't find a way to optimize them.
Again, thanks a lot for taking the time to answer my query.

Sign in to comment.

More Answers (1)

Torsten
Torsten on 7 Sep 2022
Use
fun = @(x)ErrorFunction(x(1),x(2),x(3),x(4),x(5),x(6))
as the function handle you work on in the steepest decent.

Categories

Find more on Conditional Mean Models in Help Center and File Exchange

Products


Release

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!