fmincon active-set bounds only

I have a collection of similar objective functions for each of which the computation time is about 12 seconds and analytic gradients are not available. There are 8 parameters in each objective function. There are also only bound constraints, but consequently fmincon is needed for these bounds.

I have determined that all three of the interior-point, sqp, and active-set algorithms lead to the same solution from any variety of initial points. However, active-set is significantly the fastest of the three for these particular objective functions. So I have chosen active-set for this problem.

The Matlab literature that I have looked at is not quite clear as to what active-set is doing in these circumstances. However, my understanding is the following:

Provided my initial point is in bounds, and my descent sequence stays in bounds, then fmincon active-set with bound constraints only will effectively execute a quasi-Newton line-search with BFGS Hessian update as if the problem were unconstrained (provided the descent sequence remains in bounds). That is, all computed Lagrange multipliers will always be zero (provided the descent sequence remains in bounds).

Is my description of fmincon active-set with bounds only accurate in this scenario? Can I say that "I am essentially running a quasi-Newton line-search with BFGS Hessian update when the descent sequence remains in bounds"?

Answer 1

[Ghada Saleh]

Hi David,

I assume you are using 'fmincon' since you have a nonlinear constrained optimization. The 'active-set' algorithm is based on the 'Sequential Geometric Programming (SQP)' method. The 'active-set' can take large steps, which adds speed. The algorithm is effective on some problems with nonsmooth constraints. However, it is not a large-scale algorithm.

The SQP method can be viewed as a generalization of Newton's method for unconstrained optimization in that it finds a step away from the current point by minimizing a quadratic model of the problem. The SQP algorithm replaces the objective function with the quadratic approximation and replaces the constraint functions by linear approximations.

I hope this information helps.

Ghada

[Shiryaev]

The above answer does not say anything more than the very basic information offered in the Matlab documentation. Also SQP stands for "Sequential Quadratic Programming" and not "Sequential Geometric Programming". SQP is a method that applies to problems with equality and/or inequality constraints. I am talking about a problem that only has bound constraints, moreover bounds that are not violated during the descent sequence of a well conditioned nonlinear program. The active-set algorithm does not always use SQP. This answer is rejected.