3 views (last 30 days)

Dear Community,

I am using the function fitlm for solving a regression problem. I am using the 'modelspec'-option 'quadratic' which includes all products of pairs of distinct predictors like:

My Question is about the last term.

I guess if I want to use these Interaction MATLAB adds a extra column to X, but this makes X linear dependend.

So how does MATLAB deal with these Interactions, which I want in my model but which make my Designmatrix linear dependend?

I mean it gets rid of unwanted linear dependencies automatically, but I am asking about a wanted Interaction .

Thank you in advance!

Chris

Bjorn Gustavsson
on 3 Oct 2019

That's not the problem. What you should look at is how "wellconditioned" your model-matrix X is. Look at this example:

[x,y] = meshgrid(-3.7:0.31:2.7,-4.1:0.29:2);

M = [ones(size(x(:))),x(:),y(:),x(:).^2,y(:).^2,x(:).*y(:)];

cond(M)

%

% ans =

%

% 13.9913

So that model matrix M is not that poorly conditioned. What you really should look at is the singular values of M:

[U,S,V] = svd(M);

diag(S)

%

%ans =

%

% 169.9684

% 98.1878

% 82.1093

% 28.9911

% 23.1045

% 12.1482

However if we shift the x and y-points problems arise:

x = x + 321;

y = y + 567;

M = [ones(size(x(:))),x(:),y(:),x(:).^2,y(:).^2,x(:).*y(:)];

cond(M)

% ans =

%

% 4.6934e+10

[U,S,V] = svd(M);

log10(diag(S))

%

%ans =

%

% 6.9145

% 4.5290

% 1.8519

% 1.5992

% -1.0903

% -3.7570

So here you see that the matrix M becomes rather poorly conditioned and the smallest components of the eigenvalue-spectra is now smaller and much smaller than 1 (this means trouble, in the general/typical this leads to noise amplification)

HTH

Bjorn Gustavsson
on 3 Oct 2019

As I showed in my little example, the matrix M (or X in your notation) does not become that poorly conditioned when you add the x(:).*y(:) column to it - that's not a linear combination of x and y. If you try this scary example instead:

M = [ones(size(x(:))),x(:),y(:),x(:).^2,y(:).^2,x(:).*y(:)];

M2 = [ones(size(x(:))),x(:),y(:),x(:).^2,y(:).^2,x(:)+y(:)];

cond(M)

%

% ans =

%

% 13.9913

%

%

cond(M2)

%

%ans =

%

% 2.3730e+16

There you get a poorly conditioned problem.

Remember that the main point of interest is whether your is ill-conditioned or not, check that, if it isn't you're not in trouble yet. If it is ill-conditioned then you have to start working. First thing is to centre and scale your x and y data:

x_p = (x-mean(x(:)))/std(x(:));

y_p = (y-mean(y(:)))/std(y(:));

M3 = [ones(size(x_p(:))),x_p(:),y_p(:),x_p(:).^2,y_p(:).^2,x_p(:).*y_p(:)];

cond(M3)

%

% ans =

%

% 4.0024

This centering is helpful even for the nice-case example. For this you obviously also have to keep track of the variable transformation since your fitted model will be fitted as a 2-D polynomial in x_p and y_p and not your original x and y.

If the centering doesn't help then you might fit for too many parameters, and you have to accept that your data doesn't allow you to estimate that much. You might turn to the idea that you could try to iteratively search for what parameters to fit for (maybe it is enough to fit for a subset of all 6). There are tools for that as well, but that might be for later.

Opportunities for recent engineering grads.

Apply Today
## 0 Comments

Sign in to comment.