Delete duplicate rows from a large cell array

8 views (last 30 days)
mashtine
mashtine on 5 Mar 2014
Commented: mashtine on 5 Mar 2014
I asked this before but it seems a bit tricky!
I have a cell array (attached) of the structure 1 x 2 then 1 x 8. This hold data for 2 years and the 8 arrays of the 1 x 8 array hold data in various formats.
However, the data has come with duplicates like this:
'2000-01-10 1:00' 'HCM' '268' '20' '260' '2345' '0' '90'
'2000-01-10 1:00' 'HCM' '268' '20' '260' '2345' '0' '90'
'2000-01-10 2:00' 'HCM' '268' '14' '210' '645' '3' '12'
'2000-01-10 2:00' 'HCM' '268' '14' '210' '645' '3' '12'
'2000-01-10 3:00' 'HCM' '268' '02' '230' '345' '2' '40'
'2000-01-10 3:00' 'HCM' '268' '02' '230' '345' '2' '40'
Where each column shown above actually represents a XXXX x 1 cell array (cannot remember the row count)
As chronological order is very important, I would like to remove corresponding duplicate rows (based on the first time array, from each array without changing the order to get something like this:
'2000-01-10 1:00' 'HCM' '268' '20' '260' '2345' '0' '90'
'2000-01-10 2:00' 'HCM' '268' '14' '210' '645' '3' '12'
'2000-01-10 3:00' 'HCM' '268' '02' '230' '345' '2' '40'
Big help if you can help me. I have tried many too many options!
  2 Comments
mashtine
mashtine on 5 Mar 2014
Correct, the most tricky part is describing this as I am not used to it. Star really showed patience in helping me and I think his final comment about reshaping (In my case, making the 1x8 cell array into a 2767793 x 8) and then running his above code. However, as the data is in varying formats, I cannot use cell2mat.
I attached the file above to better show my data. Again, thanks for the help!

Sign in to comment.

Accepted Answer

Thomas
Thomas on 5 Mar 2014
Edited: Thomas on 5 Mar 2014
wit_dup={'2000-01-10 1:00' 'HCM' '268' '20' '260' '2345' '0' '90'
'2000-01-10 1:00' 'HCM' '268' '20' '260' '2345' '0' '90'
'2000-01-10 2:00' 'HCM' '268' '14' '210' '645' '3' '12'
'2000-01-10 2:00' 'HCM' '268' '14' '210' '645' '3' '12'
'2000-01-10 3:00' 'HCM' '268' '02' '230' '345' '2' '40'
'2000-01-10 3:00' 'HCM' '268' '02' '230' '345' '2' '40'
}
wd=wit_dup;
[~,idx]=unique(strcat(wd(:,1),wd(:,2),wd(:,3),wd(:,4),wd(:,5),wd(:,6),wd(:,7),wd(:,8)) );
withoutduplicates=wd(idx,:)
  1 Comment
mashtine
mashtine on 5 Mar 2014
Thanks Thomas, I am pretty sure that would work but my data is a bit different. Did you have a look at the file I uploaded? The first two arrays of a 1x8 are cell and the rest are double so that formatting throws it off. It would most certainly work however if I could make all of them double and then perform your code or the ones provided before.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!