Is there a better function to minimize than fminsearch ?

Hello, I would like to know if it exits a better function to minimize a function than fminsearch ? I have this line :

[X, fval, exitflag, output] = fminsearch(@func, X0, options, params)

I precise I have the optimization toolbox. Thank you for your help !

[Walter Roberson]

Do you need a global optimization or a local optimization? Is the function differentiable? Is its Jacobian known? Is its Hessian known? If you were to pass symbolic variables into the function would it be able to return a symbolic formula in response ?

Answer 1

[Birdman]

There is a user-written function which contains Hooke-Jeeves algorithm. Maybe this will help you. The inputs and the outputs are clearly defined.

    function [X,BestF,Iters] = hookejeeves(N, X, StepSize, MinStepSize, Eps_Fx, MaxIter, myFx)
    % Function HOOKEJEEVS performs multivariate optimization using the
    % Hooke-Jeeves search method.
    %
    % Input
    %
    % N - number of variables
    % X - array of initial guesses
    % StepSize - array of search step sizes
    % MinStepSize - array of minimum step sizes
    % Eps_Fx - tolerance for difference in successive function values
    % MaxIter - maximum number of iterations
    % myFx - name of the optimized function
    %
    % Output
    %
    % X - array of optimized variables
    % BestF - function value at optimum
    % Iters - number of iterations
    %

    Xnew = X;
    BestF = feval(myFx, Xnew, N);
    LastBestF = 100 * BestF + 100;

    bGoOn = true;
    Iters = 0;

    while bGoOn

      Iters = Iters + 1;
      if Iters > MaxIter
        break;
      end

      X = Xnew;

      for i=1:N
        bMoved(i) = 0;
        bGoOn2 = true;
        while bGoOn2
          xx = Xnew(i);
          Xnew(i) = xx + StepSize(i);
          F = feval(myFx, Xnew, N);
          if F < BestF
            BestF = F;
            bMoved(i) = 1;
          else
            Xnew(i) = xx - StepSize(i);
            F = feval(myFx, Xnew, N);
            if F < BestF
              BestF = F;
              bMoved(i) = 1;
            else
              Xnew(i) = xx;
              bGoOn2 = false;
            end
          end
        end
      end

      bMadeAnyMove = sum(bMoved);

      if bMadeAnyMove > 0
        DeltaX = Xnew - X;
        lambda = 0.5;
        lambda = linsearch(X, N, lambda, DeltaX, myFx);
        Xnew = X + lambda * DeltaX;
      end

      BestF = feval(myFx, Xnew, N);

      % reduce the step size for the dimensions that had no moves
      for i=1:N
        if bMoved(i) == 0
          StepSize(i) = StepSize(i) / 2;
        end
      end

      if abs(BestF - LastBestF) < Eps_Fx
        break
      end

      LastBest = BestF;
      bStop = true;
      for i=1:N
        if StepSize(i) >= MinStepSize(i)
          bStop = false;
        end
      end

      bGoOn = ~bStop;

    end

    function y = myFxEx(N, X, DeltaX, lambda, myFx)

      X = X + lambda * DeltaX;
      y = feval(myFx, X, N);

    % end

    function lambda = linsearch(X, N, lambda, D, myFx)

      MaxIt = 100;
      Toler = 0.000001;

      iter = 0;
      bGoOn = true;
      while bGoOn
        iter = iter + 1;
        if iter > MaxIt
          lambda = 0;
          break
        end

        h = 0.01 * (1 + abs(lambda));
        f0 = myFxEx(N, X, D, lambda, myFx);
        fp = myFxEx(N, X, D, lambda+h, myFx);
        fm = myFxEx(N, X, D, lambda-h, myFx);
        deriv1 = (fp - fm) / 2 / h;
        deriv2 = (fp - 2 * f0 + fm) / h ^ 2;
        diff = deriv1 / deriv2;
        lambda = lambda - diff;
        if abs(diff) < Toler
          bGoOn = false;
        end
      end

    % end

[abc abc]

Thank you but how can I adjust your code with mine ?

[Walter Roberson]

Pass in your @func in the myFx argument position.

[Birdman]

Just decide how many variables you have(N) and initial condition(X). You can pass your function from myFx and can easily find the solution. Hope this helps.

[abc abc]

I try to write like this [xu,BestF,Iters] = hookejeeves(11, x0u, StepSize, MinStepSize, Eps_Fx, MaxIter, @intrafun) but Matlab answered me 'Undefined function or variable 'StepSize'.' But I don't know the value of StepSize :/

[Birdman]

You define StepSize, MinStepSize, Eps_Fx as

[]

[abc abc]

Thank you and for MaxIter ?

[Birdman]

Enter it as 100. You can increase or decrease it.

[abc abc]

It does not work I get Struct contents reference from a non-struct array object. :/

[Birdman]

How did you define X and N? Write here please.

[abc abc]

X = [a1 a2 a3 a4 a5 a6 a7 a8 a9 a10 a11]; and N = 11

[Birdman]

X is the initial condition array and you have to give numerical values instead of a1, a2, etc...

[abc abc]

Yes but a1, a2,... are numerical values

[Birdman]

Try this:

Enter stepsize=[0.5 ..... 0.5] Minstepsize=[0.01 .... 0.01] Eps_Fx=[1e-6 ..... 1e-6]

[abc abc]

I try but I still get Struct contents reference from a non-struct array object. :/

[abc abc]

I think it is due to params beause params contains a lot of thing I have params.args = varargin; params.LB = LB; params.UB = UB; params.fun = fun; params.n = n; params.OutputFcn = [];

[Birdman]

I will deal with code and respond to you later.

[Birdman]

 Firstly, enter the following informations for the *hookejeeves* function.

N=..;
X=[a1 .. a11];
StepSize=[0.5 .. 0.5];
MinStepSize=[0.01 .. 0.01];
Eps_Fx=[];%let it be empty

Then, save the following function with the name hookejeeves

function [X,BestF,Iters] = hookejeeves(N, X, StepSize, MinStepSize, MaxIter, myFx)
%Başlangıç atamalarının yapılması. BestF=x(k+1), LastBestF=x(k) gibi
%düşünülebilir.
Xnew = X;
BestF = feval(myFx, Xnew, N);
LastBestF = 100 * BestF + 100;
%bGoOn değişkenine bağlı while döngüsü, maksimum iterasyon sayısına veya
%verilen toleransa ulaşılınca biter.
bGoOn = true;
Iters = 0;
%Civar aramasının gerçekleştiği while döngüsüdür.
while bGoOn
    Iters = Iters + 1;
    if Iters > MaxIter
      break;
    end
    X = Xnew;
  %N=2 değişken için arama yapılmaktadır.
    for i=1:N
      bMoved(i) = 0;
      bGoOn2 = true;
      while bGoOn2
        xx = Xnew(i);
        Xnew(i) = xx + StepSize(i);
        F = feval(myFx, Xnew, N);
        if F < BestF
          BestF = F;
          bMoved(i) = 1;
        else
          Xnew(i) = xx - StepSize(i);
          F = feval(myFx, Xnew, N);
          if F < BestF
            BestF = F;
            bMoved(i) = 1;
          else
            Xnew(i) = xx;
            bGoOn2 = false;
          end
        end
      end
    end
    for i=1:N
    bMadeAnyMove(i) = sum(bMoved(i));%Civar araması başarılıysa, bMadeAnyMove(i) değişkeni 0'dan farklı olur.
    if bMadeAnyMove(i) > 0
      Xnew1(i) = 2*Xnew(i) - X(i);%if bloğunda yeni x(k+1) değeri yukarıda kullanılmak üzere elde edilir.
    end
    end
    BestF = feval(myFx, Xnew1, N);
    LastBestF = feval(myFx, Xnew, N);
    %Fonksiyonun değeri bir öncekinden daha küçükse, bir önceki değerlerin
    %yeni x değerini bulurken kullanılması.
    for i=1:N
    if BestF < LastBestF 
        Xnew(i)=Xnew1(i);
        X(i)=Xnew(i);
      Xnew1(i) = 2*Xnew(i) - X(i);
    end
    end
    %Civar araması başarısızsa, adım sayısı yarıya düşürülür.
  for i=1:N
      if bMoved(i) == 0
        StepSize(i) = StepSize(i) / 2;
      end
    end
    %Adım sayısı, verilen adım sayısından daha küçük olursa iterasyon sonlanır.
    bStop = true;
    for i=1:N
      if StepSize(i) >= MinStepSize(i)
        bStop = false;
      end
    end
    bGoOn = ~bStop;
  end

Don't worry about the comment lines, they are in turkish. Then enter the following code:

hookejeeves(N,X,StepSize,MinStepSize,Eps_Fx,@intrafunc)

This one should work. Let me know the outcome. If you give 11 variables, it might take a little longer to calculate but do not worry and wait for it to get it done.

Answer 2

[Alan Weiss]

You might be interested in the Optimization Decision Table, which exists to help you choose the most appropriate solver.

Alan Weiss

MATLAB mathematical toolbox documentation