fminunc initial point is local minimum, but fminsearch returns reasonable estaimtes

Hi,

I have a dataset for individual i and time t. I try to find 3 parameters w,a,b to minimize an objective function. Given the parameter values and data, the objective function first compute an optimal decision , then compute the sum of squared difference between and the observed choice X.

That is, I try to do:

When I use fminsearch, it does return reasonable estiamtes around different starting values. The final points also have lower objective values.

However, fminunc always say "Initial point is a local minimum" and the Hessian is all 0s. I've tried (1) other starting values, (2) change the optimality tolerance to 10e-12, but the first-order optimality is 0 at starting values.

Since fminsearch does go to other points with lower objectives, does this mean my objective function isn't actually flat but fminunc just doesn't work well?

I do want to use fminunc to get the Hessian matrix... How can I debug/fix this?

Answer 1

[Matt J]

It could happen if your objective function is piece-wise flat (and hence non-differentiable). fminsearch is a derivative-free solver, so it is less vulnerable to this, but a piece-wise flat objective is best avoided.

t=linspace(-5,5,1e6);
fun=@(x) interp1(t,t.^2,x,'nearest'); %piece-wise constant


tmin=fminunc(fun,3.5)

Initial point is a local minimum.

Optimization completed because the size of the gradient at the initial point 
is less than the value of the optimality tolerance.

tmin = 3.5000

tmin=fminsearch(fun,3.5)

tmin = -1.3323e-15

However, now let's make the objective smooth:

fun=@(x) interp1(t,t.^2,x,'cubic'); %smoothed version of previous objective

tmin=fminunc(fun,3.5)

Local minimum found.

Optimization completed because the size of the gradient is less than
the value of the optimality tolerance.

tmin = -8.2595e-08

tmin=fminsearch(fun,3.5)

tmin = -1.3323e-15

[Tian]

Thanks a lot! Ahh that would be unfortunate... Is there a way to check whether the objective is piece-wise flat?

[Tian]

I see. Thank you for the detailed example!