How to quickly find the indecis of an array in another array?

Dear All,

I have arrays A and B. I want to quickly find out the repeat entries of A in B without using intersect. For example, A = [21 32 33 41 28 91 30], and B = [32 67 41 91 100].

I want to find out the indecies of the entries in B also showing in array A. The solution is C =[1 3 4] whose correspoding entries are 32, 41 and 91 which are also entries in A.

I donot want to use intersect because it is slow in computation.

Thanks.

Benson

Answer 1

[Jan]

Do you have such tiny inputs in all cases? Then a linear search is faster than the smart sorting of intersect and the binary search:

X = [21 32 33 41 28 91 30];
Y = [32 67 41 91 100];
tic; for k = 1:1e5; [~, ind] = intersect(Y, X); end; toc
tic; for k = 1:1e5; ind = AinBidxLin(Y, X); end; toc
tic; for k = 1:1e5; ind = AinBidx(Y, X); end; toc

% Matlab R2018b:
% Elapsed time is 7.531878 seconds.  % Intersect
% Elapsed time is 0.296715 seconds.  % Linear search with loops
% Elapsed time is 0.323204 seconds.  % ismembc-Mex

function Ai = AinBidxLin(A, B)
% INPUT:  A, B: Vectors, numerical or CHAR. No NaNs.
% OUTPUT: Ai: Indices of elements of A which appear in B.
% Linear search, efficient for small inputs, e.g. < 20 elements.
% AUTHOR: Jan, Heidelberg, CC BY-SA 3.0
nA = numel(A);
M  = false(1, nA);
on = true;
B  = B(:);
for iA = 1:nA
   if ~isempty(find(A(iA) == B, 1))
      M(iA) = on;
   end
end
Ai = find(M);
end

function Ai = AinBidx(A, B)
% INPUT:  A, B: Vectors, numerical or CHAR. No NaNs.
% OUTPUT: Ai: Indices of elements of A which appear in B.
Ai = find(ismembc(A(:), sort(B(:))));
end

If the inputs are not tiny, use the ismembc appraoch. This is the core of intersect also, but unfortunately it is not documented. Maybe this compiled C-Mex function will be removed in future Matlab versions.

[EDITED] Some further tests let me prefer the ismembc approach for all inputs, even very small ones. Please try this by your own.

The set functions intersect, setdiff, union has been accelerated in modern Matlab versions. But at least at Matlab online, ismembc is still 10 times faster.

[dpb]

I've never tried spelunking, wonder if builtins aren't smart enough to swap algorithms at some size input breakpoints???

[Jan]

@dpb: The functions for comparisons of sets are optimzed for large input and to handle all types of inputs. They decide for the lagacy style of sorted outputs, tolerances and thr 'rows'-method. There are different algorithms depending on the sizes of the inputs and NaNs must be caught explicitely. But the overhead is remarkable according to the powers of the functions.

I'm using some replacements for these methods, when the inputs are tiny, to avoid this overhead. Especially for cell strings, some C-mex functions can be 10 times faster than ISMEMBER etc. But even a linear search can be much faster as M function, when the inputs are "small". In my productive code different algorithms are used depending on which of the two inputs is larger.

[Benson Gou]

Hi, Jan,

I tested your code. I used intersect 3 times in my code. I compared the speed between your code and intersect. I found 2 of them intersect is fsater, but 1 of them your code is much faster. I do not know why.

It seems to me that the 3 times using intersect looks no different.

Thanks a lot.

Benson

[Benson Gou]

Dear Jan,

I tested your code again. I found your code performs very well and saved me about 10 seconds. All three times using your code is faster than intersect.

Thanks a lot.

Benson

Answer 2

[dpb]

>> [~,ia]=find(ismember(B,A))
ia =
     1     3     4
>> 

whether it is any faster I doubt, but it doesn't use intersect.

Whatever has to do the work, not sure there is magic bullet here.

[Benson Gou]

Hi, dpb,

Your code generated a ia = [ 1 1 1 1 1 1 1 1 1]. It is faster but the indecies are not correct.

Thanks a lot.

Benson