I think what you're asking is how to remove rows of A that are nearly equal to rows of B. Removal is different from subtraction and setxor does the former, not the latter.
The question is, what do you define as "near"?
I used pdist2() to compute the pairwise distances between all points in A and B and there is no obvious cluster of distances around 0 (note, pdist2 will be slow due to large variable sizes!).
A = readmatrix('A.xlsx');
B = readmatrix('B.xlsx');
When we look at distances around 0:.2, there is still no clear cluster that would support an objective definition of "near".
title('Points at distances less than 0.2')
The minimum distance is
If there were a clear definition of "near" you could use that as a threshold to detect and remove "near" coordinates from the pd pair-wise distance matrix. But that isn't clear from the data.
If, on the other hand, you're looking for coordinates that are nearly 1 unit apart in distanced, then the data above do suggest a clear, more-objective way to detect and eliminate paired coordinates at nearly 1-unit distance, though you'll still need to subjectively select a threshold distance from 1.
For example, any pairwise distance within 0.05 of 1 identifies a row of A to be removed.