There might be a better mehod, at least more geometric, than linear system solving.
I understand you want to find m (3 x 1) such that
m must be perpendicular to all columns of B, and B must be on a 2D plane.
If you make an SVD of B the smallest singular vector is then // to m, the two others form a basis of the plane where B live in.
You can project x and B on this plane, call the projection xp and Bp.
If you take their cross product
you must find it equal to m*|xp|^2 (from triplet cross product properties).
So we can to estimate m simply as
m = mean( cross(xp,Bp) / |xp|^2).
There might be something similar with quaternion, all those are related somehow.
B = cross(repmat(m,1,size(x,2)),x)
mrecover = mean(cross(xp,Bp,1)./sum(xp.^2,1),2)