Asked by HAAAAH
on 12 Sep 2019

I have a data set and will divide it into " training set" and " validation set" . I choose that divide 70% training and 30% validation. I have calculated the linear regression from " training set" , and then calculate data of validation set by using linear regression from "training set". My result is pretty small than the correct one. Explaination can be the concept of our linear regression which do not apply for the new data (in this case is validation data) . Or is there another reason?

"And a model’s ability to predict future outcomes in light of what you discovered here". What does the question mean ?

The formula for fitting error where y1 = y * e^(t) is the estimated value of y

E(error) = (1/n )* sqrt ( ( sum(y1 -y)^2) (i= 0 to n )

Answer by Image Analyst
on 12 Sep 2019

You can't overfit a linear regression. Overfitting is basically where you have your model go through, or mostly through, your data points. For example if you had 4 data points and fit a cubic, that would be overfitting. If you have N data points and fit a polynomial with, oh I don't know, say, N/2 or something, then you might have over fitting. But that won't happen with a line since it won't go through all your data points unless you had only 2 points.

It could be that your data does not fit the model that was determined using the 70% of other points, but I doubt it since you probably chose them randomly. I'd think the 30% error was much worse than the other 70% just due to randomness. In general (usually) testing/validation will be worse error than the training points, but it could be better.

Bruno Luong
on 12 Sep 2019

"You can't overfit a linear regression."

Of course you can

Linear regression means the model is linear wrt the parameters, it does not mean fitting necessary a line/plane.

You can fit

yj = sum on i ( A_i * f_i(x_j)) to some data and it is easy to overfit.

Spline fitting is in this formulation and poeple can overfit. Waba and al are notorious for studying regularization method to make a "right" fit.

Image Analyst
on 13 Sep 2019

Sign in to comment.

Opportunities for recent engineering grads.

Apply Today
## 1 Comment

## Torsten (view profile)

## Direct link to this comment

https://www.mathworks.com/matlabcentral/answers/480071-overfitting-and-what-is-it#comment_745138

Sign in to comment.