what is the difference between 5 different algorithm in fmincon? which one would you suggest for my case which is explained in the body?
6 views (last 30 days)
I used fmincon to find x which minimized my function and my object function looks like
so at the end fmincon needed to provide me x which is a vector 2*1 .
when I changed my x0 I've got different answers, then I thought may my function doesn't have global minimum. I plot it and I see it has a valley but that is not an exact point but also it's a surface.
John D'Errico on 6 Jan 2018
If you have no clue about your objective? Then you should spend some time in learning about it, thinking about it, plot it, etc. If you know absolutely nothing about what you are trying to solve, are unwilling to do the necessary work in advance, then you are probably the wrong person to be doing this optimization.
You should never just throw a function at some optimizer and expect something meaningful to come out, with no thought invested. Well, if you do, expect less than useful results. So plot your function. Does it appear to have simple behavior? Is it continuous? Differentiable? Smoothly so? Are there bumps in the function, multiple dips? Should there be? Verify that your plots make sense in context of what you know or expect. Do there appear to be local minima on the boundary?
Yes, if your function is highly multi-dimensional, then all of this gets harder. Things can get nasty in high dimensions.
In general, an objective can be arbitrarily complex. There is no simple scheme that will always work to come up with a good set of starting values. If there were, then optimization tools would use it!
In the case of your plotted function, it appears to live on a triangular domain. (If not, then why is that what you have shown us?) So I assume the region arises from your constraints. The function appears to be relatively well posed, with a probable minimum somewhere in the interior.
If the function has some global minimum that lies outside of your constraint boundary, that is irrelevant. All you care about are minima that lie inside, and it appears to be quite a simple shape. So what is the problem?
If you are worried and still have no clue, then pick multiple points inside the boundary. Use the best of the lot as your start point.
Thomas Steffen on 5 Jan 2018
If your objective function has only one local minimum, it does not matter which x0 you specify.
If there are several local minima, fmincon (depending on the solver) may find the closest local minimum, which may not be the global minimum.
Since you know little about your objective function, the best approach is to try different values for x0. You can either take random values, or regular spaces values. See if they lead to the same minimum - if they do, it is likely to be the global optimum. If not, x0 needs to be carefully chosen.
Also try the trust-region-reflective algorithm, it can be much better at finding the global minimum than the default interior point algorithm. https://uk.mathworks.com/help/optim/ug/fmincon.html#busoq0w-1
Walter Roberson on 6 Jan 2018
fmincon is not a global minimizer. Depending on the algorithm you choose, you might not even get the "closest" minima even if your function is not especially steep: the algorithms used to decide on the next point to examine can overproject into a different "catch basin", especially early on.
There is a global optimization toolbox. Even the algorithms there are not promised to find the global minima (and often do not, even when they do end up in the right catch basin). The different algorithms have different strategies to attempt to find the global minima, but it can be proved that there cannot be any algorithm that is certain to find the global minima in finite time.
It is often useful to try fminsearch for functions with multiple minima.
To say much more we would need to see the code for your function, preferably with data file to allow us to test.
John D'Errico on 7 Jan 2018
Edited: John D'Errico on 7 Jan 2018
To answer your new question, which is now what is the difference between the 5 algorithms in fmincon, just read the
Follow the link under "Algorithms" for "Choosing the algorithm". It has a fairly detailed explanation of the differences, and there seems to be no purpose in copying that explanation when it is already well written there.
Most of those differences may seem somewhat subtle to someone who is not into optimization methods. One method requires the user to supply a gradient. Others are able to survive NaN or inf results, something that can blow some codes out of the water. Some are better at truly large problems than others. Some might be a bit more efficient on one problem or another.
But all of those methods will be able to solve your problem, finding a local minimum based on the start point, if used properly. Ok, not trust-region-reflective, since it does not allow inequality constraints. But then the optimizer will tell you there is a problem and force you to switch methods.
NONE of the algorithms in fmincon are global methods. They will all converge to a point based on the start point. (Not even a method described as a global method is assured to always converge to the true global solution for a complete black box objective. For that to happen, you need to have some way to limit the derivatives of your function.)
So the real issue here seems to be you need to understand a concept that I've always called a basin of attraction. Any local minimum will be a point of convergence for a given set of start points. So the set of all start points that converge to a given solution are the basin of attraction for that solution. That basin may well be different for different methods, but all methods in fmincon are only expected to converge to a local minimizer at best, IF they converge at all. Sufficiently nasty problems will exist to confuse any method that works on black box objectives.