# Least Absolute Value based regression

50 views (last 30 days)
abc on 9 Nov 2013
Edited: Matt J on 7 Apr 2014
Hi ,
I want to use linear regression based on least absolute value deviation to find the coefficients of my model with the help of measured data and 3 independent variables. The number of measurements I have are much greater than 3( number of cofficients). I have been trying to implement LAV method for this, but with no success. I need urgent help. Can someone please guide me in this. If someone already has the code for LAV, I would be grateful if you could share it with me.
Thank you!

Matt J on 10 Nov 2013
With only 3 unknowns, fminsearch should work fairly well
fminsearch(@(x) norm(A*x-b,1), x0 )

Matt J on 7 Apr 2014
Hi Puneet,
No, it's not clear to me that the auxiliary variables can be interpreted as slack. Slack variables are used to convert inequalities to equalities, whereas my formulation contains only inequalities.
As far as I can see, the only change you've made is to flip the sign of speye(m), which is equivalent to changing the sign of the auxiliary variables in the solution. It would be equivalent to redefining the objective weight vector as
f=[zeros(1,n), -ones(1,m)];
But this, in turn, is the same as flipping the sign of f. So, ultimately, it looks like you are maximizing the same objective function that I was minimizing. Not sure how maximizing instead of minimizing led to better results...
Puneet on 7 Apr 2014
Hello Matt,
Thanks for your reply. I have not changed the sign of f in your formulation. Can you tell me the reference as how you came up with your formulation with the auxiliary variables? I am not sure on the use of auxiliary variables. I just flipped the sign of speye(m) assuming it to be slack variables. Since I tested on a small system and it gave me very good results but before I apply this to very large formulations I need to be sure if I am fundamentally correct in formulation.
Matt J on 7 Apr 2014
The problem
min_x norm(A*x-b,1)
is equivalent to the constrained problem,
min_xz sum(z)
subject to
-z <= A*x-b <= z
Here z is a vector of auxiliary variables. The string of inequalities -z <= A*x-b <= z when re-arranged leads to
Ac*[x;z]<=bc
where Ac and bc are defined as earlier and sum(z) can be implemented as dot(f,[x;z]) using the f I also defined earlier.