MATLAB Answers

Matlab script is very very slow! Help me please!

5 views (last 30 days)
Rocco De Luca
Rocco De Luca on 26 Dec 2019
Commented: Rocco De Luca on 29 Dec 2019
Hello! I am a student and I created a Matlab code that allows me to open many txt files in succession and perform these sums and other operations. The work that the code has to do is a lot but it is really too slow. I calculated that it would take two days to finish. I think there is some conflict. Please ask me to help me otherwise I will have to do all those operations by hand and it will be a disaster for me. Thanks to those who will help me
I enclose the code I wrote in a text file

  3 Comments

Image Analyst
Image Analyst on 26 Dec 2019
Can you also attach 2 or 3 of your text data files so we can at least run the code ourselves? Make it easy for us to help you, not hard.
Rocco De Luca
Rocco De Luca on 26 Dec 2019
You're absolutely right, sorry. I put a mega link where you can download the files. I can't attach them here because they are more than 7 mega and it doesn't accept rar files. Thanks so much.
Rocco De Luca
Rocco De Luca on 26 Dec 2019
These are hourly powers related to a simulated hybrid system.

Sign in to comment.

Answers (2)

per isakson
per isakson on 26 Dec 2019
Edited: per isakson on 26 Dec 2019
Why not just let it run over a weekend?
I've downloaded your files and invoked your script. I interrupted the execution during the fourth data file. The script consists of three nested loops. One statement in the innermost loop uses 90% of the execution time.
for ii = 1 : 512
...
for kk = 1 : 8760 % What about leap years?
...
for jj = 1 : 8760 % What about leap years?
R(ii,[1:5,8:27])=R(ii,[1:5,8:27])+intermedia(jj,[1:5,8,9,11:21,29:35]); % 90%
...
end
end
end
The innermost loop does a number of summations. Replacing the innermost loop by
R(ii,[1:5,8:27]) = sum( intermedia(1:8760,[1:5,8,9,11:21,29:35]), 1 );
is_positive = intermedia(1:8760,7)>0;
R(ii,6) = sum( intermedia(is_positive,7) );
R(ii,7) = sum( intermedia(not(is_positive),7) );
improves the speed of the script significantly. (You need to confirm that it returns the correct result.)

  3 Comments

Rocco De Luca
Rocco De Luca on 26 Dec 2019
Thank you infinitely! I ran the code with your replacement, running it for the first file "COMPLETOA20KM1" (so without the most external for) the results seem correct and the execution time is about 25.5 seconds, so for all 512 files it will take 3 and a half hours a lot better than the 48 before! So the problem is nested for loops. Thank you again, if there are other tricks to speed up this code I would like to know them!
per isakson
per isakson on 26 Dec 2019
Did you try the profiler?
I don't think there are more low hanging fruits.
Rocco De Luca
Rocco De Luca on 26 Dec 2019
Yes, I used the profiler. I attach the profiler and the code that I used since I slightly modified it.

Sign in to comment.


Rocco De Luca
Rocco De Luca on 27 Dec 2019
Hello and sorry if I still disturb you. To speed up even more I thought of replacing the second for with a method similar to the one you used. I am attaching the code, what do you think?

  2 Comments

per isakson
per isakson on 27 Dec 2019
Assume that your original code with the three loops produce the expected result. Use that result as a reference. Each new version shall produce exactly (a floating point error is ok) the same result. That can be tested systematically with the Unit Testing Framework. If the results are identical and if there is enough variation in the data files, COMPLETOA20KM*, the new code is most likely correct. Are the results identical?
Rocco De Luca
Rocco De Luca on 29 Dec 2019
Thanks, the results are identical, but now the code is extremely faster. I thank you very much for your help.

Sign in to comment.

Sign in to answer this question.

Tags