How to save data from Genetic Algorithm in case MATLAB crashes?

Hi there,

I'm using the GA to find input parameters for my model. My fitness function is computationally expensive and takes about 2 days. In situation if MATLAB unexpectedly crashes, is there a way to save the data?

I have used the 'OutputFcn' option however this seems to save the data once my MATLAB finish the simulation. I purposely stop MATLAB and re-open it but can't seem to access the saved data.

If anyone have any useful tips to save simulation data while the simulation is running so I can access in case it crashes, I would be very grateful.

Many thanks,

How

[Stephen23]

"My fitness function is computationally expensive and takes about 2 days"

Do you mean that your fitness function takes two day to calculate, or the GA does?

"In situation if MATLAB unexpectedly crashes, is there a way to save the data?"

Once it has crashed, not really... but you could regularly save the GA population (e.g. using OutputFcn), and then start the GA again using the latest population to set the InitialPopulationMatrix.

[Syahmeer How]

Cheers Stephen for the quick reply. Yup I have read about using the latest population and restarting the simulation again from that population if it stops.

Just wondering though if you could share how to regularly save the population while the GA runs?

Actually I'm using a PSO function which I found from the file exchange but the way it's written was very much like the ga function on matlab, so I wrote ga here for better understanding.

It's the PSO aka (ga) that takes 2 days to calculate. I have written an OutputFcn which saves the population when I can only retrieve the data once the function finished calculating. Just not sure how to write the code in a such a way that if I stop Matlab on purpose, it would give me the output, without waiting for the function to finish calculating. Hope that makes sense.

Cheers

How

Here's my OutputFcn code if it helps and a script which I run the PSO function

OutputFcn code: @gaoutfun

function [state,options,optchanged] = gaoutfun(options,state,flag)

persistent state_record

if isempty(state_record)

state_record = struct('Population', {}, 'xGlobalBest', {}, 'Score', {},'Iteration',{},'SS',{});

end

if nargin == 0

state = state_record;

options = [];

optchanged = [];

else

state_record(end+1) = struct('Population', state.Population, 'xGlobalBest', state.xGlobalBest, 'Score', state.Score,'Iteration',state.Generation,'SS',state.fGlobalBest);

optchanged = false;

end

end

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

% Script I used to run the pso

tic

clear gaoutfun

options=psooptimset('Generations',200,'PopulationSize',40,'PlotFcns',{@psoplotbestf,@psoplotswarmsurf},'OutputFcns',@gaoutfun);

[ypso,fval,exitflag,output,population,scores]=pso(fitnessfcn,nvars,[],[],[],[],lbound,ubound,[],options);

record = gaoutfun();

toc

gapopulationhistory = vertcat(record.Population);

gabesthistory = vertcat(record.xGlobalBest);

gascorehistory = vertcat(record.Score);

gaiterhistory = vertcat(record.Iteration);

gasshistory = vertcat(record.SS);

save lale.mat

gahistory = [gascorehistory, gabesthistory, gapopulationhistory];

[Stephen23]

Define the outputFcn following the documentation:

https://www.mathworks.com/help/gads/genetic-algorithm-options.html#f17837

https://www.mathworks.com/help/gads/custom-output-function-for-genetic-algorithm.html

Pay careful attention to the flag values, the input arguments, the output arguments. You would need to save the data on every 'init' call, or if that slows down your code too much, then use persistent and a simple counter to save the data every N calls (for an N of your choice).

[Syahmeer How]

This was the OutputFcn I defined, but will probably need some thinking about calling it on every N calls to save the data..

function [state,options,optchanged] = gaoutfun(options,state,flag)
    persistent state_record 
    if isempty(state_record)
      state_record = struct('Population', {}, 'xGlobalBest', {}, 'Score', {},'Iteration',{},'SS',{});
    end
    if nargin == 0
      state = state_record;
      options = [];
      optchanged = [];
    else
      state_record(end+1) = struct('Population', state.Population, 'xGlobalBest', state.xGlobalBest, 'Score', state.Score,'Iteration',state.Generation,'SS',state.fGlobalBest);
      optchanged = false;
    end
end

[Star Strider]

It would likely be worth experimenting to see of OutputFcn can include a way of using save to append the Best score for the current generation to a .mat file or text file, likely opening it and closing it each time so that nothing is lost. The documentation states that Best is part of the state structure, so it should be available. The documentation on Custom Output Function for Genetic Algorithm offers guidance.

I have never attempted this with ga (my optimisations rarely take more than a few hours at most), so I have no personal experience with it. (In the past, I wrote my own genetic algorithm codes, and saved the best individual to a text file in each generation.)

[Stephen23]

@Syahmeer How: I strongly recommend that you use the flag input to initilalize your function, and select when to save the data, and clean up when complete.

When you wrote save I presumes that you actually meant save, as in "export the data to file". If you expect MATLAB to crash or hang, there is little point in your approach of just storing that data in some variable/s. Appending to a .mat file is probably the simplest.

[Syahmeer How]

@Star Strider: Thanks for your reply, that's exactly what I've been thinking but just not sure how to write it with little examples on the web. I will have to perform some more trial and error to see if I could save the data in a mat.file with each generation as the simulation is running. But would appreciate if you could share some examples about doing that though :)

[Syahmeer How]

@ Stephen Cobeldick. Cheers for your recommendation Stephen. Yup I see what you mean here. Will have to do a bit more digging to append the data into a .mat file as the simulation is running. Still a bit clueless on how to do that though.

[Star Strider]

@Syahmeer How — My pleasure. I have some code that specifically uses ga, so I will experiment with that and will post back here if I have a successful result.

[Syahmeer How]

@starstrider Thank you so much :) I look forward to hearing back from you

Answer 1

[Star Strider]

It took a bit of trial and error, however this appears to work:

    function [state,options,changed] = SaveOut(options,state,flag)
        file_name = 'SaveBest.mat';                                     % Name File
        if strcmp(flag,'iter')
            ibest = state.Best(end);
            ibest = find(state.Score == ibest,1,'last');
            bestx = state.Population(ibest,:);
            previous = load('SaveBest.mat');
            var = [previous.var; bestx];                                % Read Previous Results, Append New Value
            save(file_name, 'var')                                      % Write ‘Best Individual’ To File
        end
        changed = true;                                                 % Necessary For Code, Use  Appropriate Value
    end

It reads the existing saved values of ‘var’, appends the new value to the end of the matrix, then writes the new matrix to the .mat file.

It never occurred to me that this was even possible! I will use it in my longer optimisations from now on.

[Syahmeer How]

Cheers for the code StarStrider!! I seem to have an error of

Reference to non-existent field 'var'.

Do I have to change var to something else?

[Syahmeer How]

I guess it's probably that the simulation I run was a wee bit short so 'var' may not exist at that time. I would run it longer and stop the simulation to see if the works this time.

Thanks so much for your help StarStrider!!

[Star Strider]

I ran that code so many times to test it that I did not encounter that problem.

My apologies.

I deleted the file first each time during this test, then ran my optimisation again without problems with slightly edited code.

This works:

    function [state,options,changed,str] = SaveOut(options,state,flag)
        file_name = 'SaveBest.mat';                                     % Name File
        if strcmp(flag,'init')
            Var = state.Population;
            save(file_name, 'Var')                                      % Write ‘Best Individual’ To File
        elseif strcmp(flag,'iter')
            ibest = state.Best(end);
            ibest = find(state.Score == ibest,1,'last');
            bestx = state.Population(ibest,:);
            previous = load('SaveBest.mat');
            Var = [previous.Var; bestx];                                % Read Previous Results, Append New Value
            save(file_name, 'Var')                                      % Write ‘Best Individual’ To File
        end
        changed = true;                                                 % Necessary For Cide, Use  App
    end

If you want to see the last five rows in the .mat file (as I named it):

BestInGen = load('SaveBest.mat')
LastFive = BestInGen.var(end-4:end,:)

EDIT — (18 Dec 2019 at 23:34)

Changed ‘var’ to ‘Var’ to avoid overshadowing the var (variance) function if the file is loaded without output arguments. Code otherwise unchanged.

[Syahmeer How]

Brilliant!!! The code works! Thanks heaps for your help! Now I don't have to worry about running my simulations for a very long time fearing it might crash in any second haha.

Thanks again and have a very good Christmas break :D

[Star Strider]

I very much appreciate your compliment!

As always, my pleasure!

I wish you the best the Christmas Holidays can bring!

[Syahmeer How]

No worries, and thanks for your wish :)

Actually I have one last question if that's okay. So if in unexpected circumstances my simulation crashes and I managed to save the 'Best scores' of my last generation. Is this the only variable I need to initialise to complete the rest of my simulation?

[Star Strider]

As always, my pleasure!

The ‘var’ field in my code would be the InitialPopulationMatrix (or at least part of it). The rest of your optimoptions structure would be as it was previously. The ga funciton will reconstitute everything else it need in the first iteration.

[Manjur Basnet]

Hi Star Strider, I have been trying to use the Saveout function to save best value from each generation but I keep getting this error after 1 generation

"Unrecognized field name "Best".

Error in SaveOut (line 7)

ibest = state.Best(end);

Error in gaoutput (line 51)

[state,optnew,changed] = feval(functions{i},options.OutputPlotFcnOptions,state,flag,args{i}{:});

Error in gamultiobjsolve (line 86)

[state,options] = gaoutput(FitnessFcn,options,state,currentState);

Error in gamultiobj (line 338)

[x,fval,exitFlag,output,population,scores,residuals] = gamultiobjsolve(FitnessFcn,nvars, ...

Error in pareto (line 28)

[x,fval,exitFlag,output] = gamultiobj(fun,5,A,B, Aeq, beq, lb,ub,nlcon, intcon, options);"

Can you help me with this? I tried searching for this kind of error but could find nothing.

Thank you so much.

[Star Strider]

@Manjur Basnet — The correct thread for your problem is: Saving population individauls for each generation of GA Multiobjective optimisation.

Unfortunately, Shahrbanoo Shamekhi-Amiri didn’t accept it in spite of my posting a working solution.

See if that Answer solves your problem.