MATLAB Answers

How does MATLAB deal with wanted linear dependencies?

3 views (last 30 days)
Christian
Christian on 3 Oct 2019
Commented: Christian on 4 Oct 2019
Dear Community,
I am using the function fitlm for solving a regression problem. I am using the 'modelspec'-option 'quadratic' which includes all products of pairs of distinct predictors like:
My Question is about the last term.
I guess if I want to use these Interaction MATLAB adds a extra column to X, but this makes X linear dependend.
So how does MATLAB deal with these Interactions, which I want in my model but which make my Designmatrix linear dependend?
I mean it gets rid of unwanted linear dependencies automatically, but I am asking about a wanted Interaction .
Thank you in advance!
Chris

Answers (1)

Bjorn Gustavsson
Bjorn Gustavsson on 3 Oct 2019
That's not the problem. What you should look at is how "wellconditioned" your model-matrix X is. Look at this example:
[x,y] = meshgrid(-3.7:0.31:2.7,-4.1:0.29:2);
M = [ones(size(x(:))),x(:),y(:),x(:).^2,y(:).^2,x(:).*y(:)];
cond(M)
%
% ans =
%
% 13.9913
So that model matrix M is not that poorly conditioned. What you really should look at is the singular values of M:
[U,S,V] = svd(M);
diag(S)
%
%ans =
%
% 169.9684
% 98.1878
% 82.1093
% 28.9911
% 23.1045
% 12.1482
However if we shift the x and y-points problems arise:
x = x + 321;
y = y + 567;
M = [ones(size(x(:))),x(:),y(:),x(:).^2,y(:).^2,x(:).*y(:)];
cond(M)
% ans =
%
% 4.6934e+10
[U,S,V] = svd(M);
log10(diag(S))
%
%ans =
%
% 6.9145
% 4.5290
% 1.8519
% 1.5992
% -1.0903
% -3.7570
So here you see that the matrix M becomes rather poorly conditioned and the smallest components of the eigenvalue-spectra is now smaller and much smaller than 1 (this means trouble, in the general/typical this leads to noise amplification)
HTH
  3 Comments
Christian
Christian on 4 Oct 2019
Thank you a lot for your detailed answer. I got it now and I didn't get a problem with a I'll conditioned matrix since I used sigmoid-standardizaion already :)
Thank you again for your help! Best regards
Chris

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!