Why eval seems to be faster than other alternatives in this example?!

Asked by Hesham on 17 Sep 2019 at 4:50

Latest activity Commented on by Guillaume on 17 Sep 2019 at 9:57

38 views (last 30 days)

I know that using eval is highly discouraged, but for the below test case it seems to be faster than other options. Can you help me explain this and let me know which alternative you prefer?

struct1.data = 2*ones(1e3);
struct2.data = 4*ones(1e3);
struct(1).data = 2*ones(1e3);
struct(2).data = 4*ones(1e3);
selected_struct_str = 'struct1';
selected_struct_num = 1;
num_iter = 10;
tic
for i = 1:num_iter
    test1 = doWork1(struct1, struct2, selected_struct_str);
end
toc
tic
for i = 1:num_iter
    test2 = doWork2(struct1, struct2, selected_struct_num);
end
toc
tic
for i = 1:num_iter
    test3 = doWork3(struct, selected_struct_num);
end
toc
function data = doWork1(struct1, struct2, selected_struct)
    data = eval([selected_struct '.data']);
end
function data = doWork2(struct1, struct2, selected_struct_num)
    switch selected_struct_num
        case 1
            data = struct1.data;
        case 2
            data = struct2.data;
    end
end
function data = doWork3(struct, selected_struct_num)
    data = struct(selected_struct_num).data;
end

I am getting the following output:

>> eval_test
Elapsed time is 0.000418 seconds.
Elapsed time is 0.001516 seconds.
Elapsed time is 0.001594 seconds.
>> eval_test
Elapsed time is 0.000384 seconds.
Elapsed time is 0.001027 seconds.
Elapsed time is 0.001186 seconds.
>> eval_test
Elapsed time is 0.000558 seconds.
Elapsed time is 0.001280 seconds.
Elapsed time is 0.001304 seconds.
>> eval_test
Elapsed time is 0.000641 seconds.
Elapsed time is 0.001754 seconds.
Elapsed time is 0.001802 seconds.
>> eval_test
Elapsed time is 0.000357 seconds.
Elapsed time is 0.001117 seconds.
Elapsed time is 0.001430 seconds.
>> eval_test
Elapsed time is 0.000533 seconds.
Elapsed time is 0.001231 seconds.
Elapsed time is 0.001234 seconds.

2 Comments

Walter Roberson on 17 Sep 2019 at 5:57

Possibly the JIT is optimizing away calls???

Bruno Luong on 17 Sep 2019 at 8:09

I put your script in function to make sure JIT optimization works correctly (in script JIT might not work), and increase the num_iter to 1000 for better average. Here is my result (R2019b, Windows PC).

>> test
Elapsed time is 0.015345 seconds. % eval
Elapsed time is 0.000555 seconds. % struct switch/case
Elapsed time is 0.000928 seconds. % switch array

So EVAL is 28 times slower than struct switch/case and 16 times slower than switch array.

function test
struct1.data = 2*ones(1e3);
struct2.data = 4*ones(1e3);
struct(1).data = 2*ones(1e3);
struct(2).data = 4*ones(1e3);
selected_struct_str = 'struct1';
selected_struct_num = 1;
num_iter = 1000;
tic
for i = 1:num_iter
    test1 = doWork1(struct1, struct2, selected_struct_str);
end
toc
tic
for i = 1:num_iter
    test2 = doWork2(struct1, struct2, selected_struct_num);
end
toc
tic
for i = 1:num_iter
    test3 = doWork3(struct, selected_struct_num);
end
toc
end
function data = doWork1(struct1, struct2, selected_struct)
data = eval([selected_struct '.data']);
end
function data = doWork2(struct1, struct2, selected_struct_num)
switch selected_struct_num
    case 1
        data = struct1.data;
    case 2
        data = struct2.data;
end
end
function data = doWork3(struct, selected_struct_num)
data = struct(selected_struct_num).data;
end

Products

MATLAB

Release

R2017a

1 Answer

Link

Answer by per isakson on 17 Sep 2019 at 6:21

Edited by per isakson on 17 Sep 2019 at 8:37

Accepted Answer

Testing is tricky

>> Untitled5
Elapsed time is 0.167926 seconds.
Elapsed time is 0.007457 seconds.
Elapsed time is 0.010043 seconds.

where Untitled5 is

%%
struct1.data = 2*ones(2);
struct2.data = 4*ones(2);
struct(1).data = 2*ones(2);
struct(2).data = 4*ones(2);
selected_struct_str = 'struct1';
selected_struct_num = 1;
num_iter = 1e4;     %    <<<<<<<<<<<<<<<<<<
tic
for i = 1:num_iter
    test1 = doWork1(struct1, struct2, selected_struct_str);
end
toc
tic
for i = 1:num_iter
    test2 = doWork2(struct1, struct2, selected_struct_num);
end
toc
tic
for i = 1:num_iter
    test3 = doWork3(struct, selected_struct_num);
end
toc
function data = doWork1(struct1, struct2, selected_struct)
    data = eval([selected_struct '.data']);
    data = rand()*data;            %    <<<<<<<<<<<<<<<<<<
end
function data = doWork2(struct1, struct2, selected_struct_num)
    switch selected_struct_num
        case 1
            data = struct1.data;
            data = rand()*data;    %    <<<<<<<<<<<<<<<<<<
        case 2
            data = struct2.data;
            data = rand()*data;    %    <<<<<<<<<<<<<<<<<<
    end
end
function data = doWork3(struct, selected_struct_num)
    data = struct(selected_struct_num).data;
    data = rand()*data;            %    <<<<<<<<<<<<<<<<<<
end

ADDENDUM

I coverted the script to a function and left doWork-functions as subfunction. The elapse times didn't change much

>> Untitled5
Elapsed time is 0.171473 seconds.
Elapsed time is 0.005879 seconds.
Elapsed time is 0.010672 seconds.
>> Untitled5
Elapsed time is 0.172317 seconds.
Elapsed time is 0.006429 seconds.
Elapsed time is 0.010051 seconds.
>> 

/R2018b

5 Comments

Show 2 older comments

per isakson on 17 Sep 2019 at 7:52

I increased num_iter to improve the precision of the timing.

The real difference in our tests is the line

data = rand()*data;

which forces Matlab to do some work, not only copy data, which the JIT (just in time compiler) is smart enough to avoid. I decreased the size of data to avoid letting the multiplication dominate. Mathworks have given the JIT a fancy name, which I forgotten.

Mathworks keeps the inner working of the JIT secret, hence no answer to "Why?". However, see Documentation on JIT compiler?

Bruno Luong on 17 Sep 2019 at 8:16

"Can we conclude that eval is faster for small num_iter, but it is slower for larger num_iter?"

No we rather conclude that the test on small num_iter is not reliable.

Guillaume on 17 Sep 2019 at 9:57

In any case, in a real world case the eval test case would involve constructing the structure name with something like sprintf('test%d', idx) which will absolutely kill performance compared to the much simpler test(idx).

Products

MATLAB

Release

R2017a

You are now following this question

Hesham (view profile)