Clear Filters
Clear Filters

Fastest way to replace values in an array if they equal a certain value?

21 views (last 30 days)
Hello,
I have a very large array (4 billion x 2). I frequently need to update the values in column two based on a value from another list. This is how I currently have it:
for i = 1:size(net,1)
imcoords(imcoords(:,2)==net{i,7},2) = i;
end
Where net is a cell array and imcoords is the Nx2 array. Just as it is written, I need to replace all values of the second column of imcoords that equal the value in the 7th column of the cell array with the index of the current row of the cell array we are in. With the method I have right now this takes way too long and is the current bottleneck of my code since I need to do this frequently.
Does anyone have any ideas to make this quicker?
Thanks you!
Eric
  2 Comments
John D'Errico
John D'Errico on 26 May 2019
Don't forget, to NEVER test for exct equality of floating point numbers. If you do not learn to use a tolerance, then your next plaintive question will be why does my code not work some of the time?
Eric Chadwick
Eric Chadwick on 26 May 2019
Thank you for your comment. The numbers in question are always integers so this will not be an issue for me.

Sign in to comment.

Accepted Answer

John D'Errico
John D'Errico on 26 May 2019
Since the numbers are all integers, just do it as a lookup table. That makes the replacement a simple index into a vector. Done in milliseconds.
  3 Comments
Walter Roberson
Walter Roberson on 26 May 2019
Your net{i,7} appear to be scalar . If so then
net7 = [net{:,7}];
look7(net7) = 1:length(net7);
imcoords(:,2) = look7(imcoords(:,2));
This depends upon:
  • that the net{:,7} are scalars
  • that there are no duplicate values (if there are then the replacement is order dependent)
  • that the existing values are all positive integers
  • that all of the existing values are being replaced
  • that the existing values all exceed the number of rows in net (otherwise you could get multiple replacements with your code
If not all of the existing values are being replaced, then you can initialize
look7 = 1 : maximum_expected_value
look7(net7) = 1:length(net7);
Eric Chadwick
Eric Chadwick on 28 May 2019
Hi Walter,
Thank you for clarifying! This works great now! The next bottleneck in my code is the step before this where I populate imcoords using knnsearch. Basically imcoords(:,1) are my indices used in knnsearch (in subscript form of course) and imcoords(:,2) are the closest point's indices retrieved by knnsearch.
Do you know of any faster methods to knnsearch? I found this link that John apparently helped with, but this is slower than matlab's knnsearch for large datasets as the reviews echo.
I should note that the data I am working with is clusters of voxels that are all directly connected to eachother (i.e. they are technically one cluster), but I have labelled them with IDs to differentiate them. If this is too off-topic I can open a new question.
Thanks again!
Cheers,
Eric

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!