What is an efficient way to use a sum of objective functions for fmincon?

I'm preparing to use fmincon with an objective function that is a sum of (weighted) functions, such that the user of the program can select desired objective functions and specify the weight factors. The first idea that comes to mind is defining the objective function as a sum of functions defined in different .m files. This way, I can later on easily add objective functions by simply creating a new function. However, I am aware (rather, I strongly suspect) that it's costly to often call upon a function in another file.

My question is: is the approach that I sketch above reasonably efficient? If not, what other approach could I use which is both simple to use for me and efficient?

Other approaches that come to mind are

1. writing all code in one function using if-statements, but that would not be so nice to use for me;
2. using local/nested functions; not bad, but not as nice to use for me as having multiple functions in .m files.

Answer 1

[Matt J]

However, I am aware (rather, I strongly suspect) that it's costly to often call upon a function in another file.

No, not really. If you are making calls repeatedly within a long for-loop, that might be the case, but here where the users are manually selecting a handful of desired functions, it presumably means the loop will be fairly short.

[Steven H]

Well, the loop would be done by fmincon in the form of iterations, right? Some preliminary results show around 150-250 iterations. Or am I imagining fmincon the wrong way and those iterations do not imply that all those objective functions are called 150-250 times?

[Walter Roberson]

250 calls is negligible overhead, less than 1/1000 of a second.

[Matt J]

fmincon does call your objective function iteratively, but within each iteration, I wouldn't expect the time to call a handful of mfunctions to be a big fraction of the time for the other operations that are done.

Are your constituent objective function terms (the ones you are weighting together) very simple (like adding two numbers together)? How many unknown variables are there?

[Steven H]

Alright, you're claiming that that approach should be reasonably efficient? Then I feel more confident to give it a try.

The objective functions are expected to become somewhat complex, hence creating mfunctions is the more ideal approach, and the solution will probably be e.g. a 50x1 double vector, although for some applications that size could increase a lot. The number of objective functions will indeed be a handful, probably around 5 at most.

[Steven H]

Also thanks for your input @Walter Roberson. I don't really have a feeling for how much time it could take.

[Walter Roberson]

Function calls have some overhead.

I posted some timing of function calls yesterday; see https://www.mathworks.com/matlabcentral/answers/898187-what-is-the-alternative-function-for-evalin#comment_1689662

On the MATLAB Online system, it took about 0.0025 seconds to make 500*5 = 2500 function calls passing a single parameter, so 1e-6 seconds per function call. If you look at some of the earlier timings in the thread, you will see that it was over 100 times as expensive to just square the elements of a large vector as it was to make a function call.

Anonymous functions with captured variables are measureably less efficient than plain function calls.

[Steven H]

Very interesting @Walter Roberson! So nested functions performs worse than local functions, although this difference only really shows in case of passing data through a series of functions, but not so much if it's about the number of function calls.

What about creating multiple mfunctions?

I appended your code in your other post a little, the result does not differ much from local functions, but it appears multiple mfunctions perform even better. Not really what I expected.

To obtain this result, I put your local functions called via_mfuncx into individual mfunctions and added to test_case the line

tic; for K = 1:N; via_mfunc1(A); end; t(3) = toc;

NOTE: interesting to note is that test_case should be called at least twice, because the first run shows a different result compared to all subsequent results.

[Walter Roberson]

@Steven

To confirm, the via_mfunc* series were each individual .m files?

If so, did you happen to test timing for the case where they have an end statement matching the function statement, compared to no end statement?

[Steven H]

@Walter Roberson

Five individual .m files, yes.

I just tried and there's no difference in results. What were you expecting? Is there an advantage/disadvantage?

[Walter Roberson]

When you have an end statement then a "static workspace" is geneated. Static workspaces have a "contract" that called functions cannot (are not permitted to) "poof" variables into existence -- so functions with static workspaces have more opportunities for optimization.

For example at the time that via_mfunc1 is being processed by itself, it does not know that via_mfunc2 will not assignin('caller') and mess something up in the workspace. Static workspaces say any called function will be blocked if it tries. Not just called functions: if a script is invoked in a static workspace, it is not permitted to create new variables.

... and when you can start relying on assignments you can see, and not having to worry about assignments you cannot see, then you can make tighter code that does not have to insert checks in case variables suddenly have new values or new datatypes.

So, on the whole, functions with static workspaces can be more efficient. But that might not show up for such simple functions as are being used in this test.

You might have noticed the functions in scripts have to have the matching end statements... which forces them to have static workspaces. Only functions with static workspaces can have nested functions and shared variables.