How to apply PSO FOR REGRESSION?.

If i have number of predictors and output in excel sheet file. Can i apply PSO for prediction of output?.

Can anyone help me to implement it?.

Thanks in advance

Answer 1

[Sam Chak]

Hi @Ahmed Eltantawi

it is technically possible to use PSO for predicting the output, although not by directly applying PSO. The regression problem can be formulated as a least-squares problem, and an objective function can be constructed from it, which can then be minimized using PSO. Here is an example, but please note that it can be somewhat tedious, as MATLAB's particleswarm() is designed for single-objective optimization.

%% Data
x    = 0:5;                             % input  vector
y    = [2.1 7.7 13.6 27.2 40.9 61.1];   % output vector

%% Data processing
n    = length(x);
Sx   = sum(x);
Sx2  = sum(x.^2);
Sx3  = sum(x.^3);
Sx4  = sum(x.^4);
Sy   = sum(y);
Sxy  = sum(x.*y);
Sx2y = sum((x.^2).*y);

%% Least-square Regression model: lsy(x) = p1·x² + p2·x + p3;
% Sx2*p1 +  Sx*p2 +   n*p3 = Sy     ... Eq.(1)
% Sx3*p1 + Sx2*p2 +  Sx*p3 = Sxy    ... Eq.(2)
% Sx4*p1 + Sx3*p2 + Sx2*p3 = Sx2y   ... Eq.(3)

% p3 = (Sy - (Sx2*p1 + Sx*p2))/n                             ... from Eq.(1)
% Sx3*p1 + Sx2*p2 +  Sx*((Sy - (Sx2*p1 + Sx*p2))/n) = Sxy    ... Eq.(4)
% Sx4*p1 + Sx3*p2 + Sx2*((Sy - (Sx2*p1 + Sx*p2))/n) = Sx2y   ... Eq.(5)

% p2 = (Sxy - Sx*Sy/n - Sx3*p1 + Sx*Sx2/n*p1)/(Sx2 - Sx*Sx/n)                                                                                                                   ... from Eq.(4)
% Sx4*p1 + Sx3*((Sxy - Sx*Sy/n - Sx3*p1 + Sx*Sx2/n*p1)/(Sx2 - Sx*Sx/n)) + Sx2*((Sy - (Sx2*p1 + Sx*((Sxy - Sx*Sy/n - Sx3*p1 + Sx*Sx2/n*p1)/(Sx2 - Sx*Sx/n))))/n) - Sx2y = 0      ... Eq.(6)

%% Make Eq.(6) a convex function so that PSO can be used
fun  = @(p1) (Sx4*p1 + Sx3*((Sxy - Sx*Sy/n - Sx3*p1 + Sx*Sx2/n*p1)/(Sx2 - Sx*Sx/n)) + Sx2*((Sy - (Sx2*p1 + Sx*((Sxy - Sx*Sy/n - Sx3*p1 + Sx*Sx2/n*p1)/(Sx2 - Sx*Sx/n))))/n) - Sx2y).^2;
nvar = 1;
p1   = particleswarm(fun, nvar)

Optimization ended: relative change in the objective value 
over the last OPTIONS.MaxStallIterations iterations is less than OPTIONS.FunctionTolerance.

p1 = 1.8607

p2   = (Sxy - Sx*Sy/n - Sx3*p1 + Sx*Sx2/n*p1)/(Sx2 - Sx*Sx/n)

p2 = 2.3593

p3   = (Sy - (Sx2*p1 + Sx*p2))/n

p3 = 2.4786

%% Find the coefficient of determination, R²
xbar = mean(x);
ybar = mean(y);
dev  = y - ybar;
Sdev = sum(dev.^2);
lsy  = @(x) p1*x.^2 + p2*x + p3;
err  = y - lsy(x);
Serr = sum(err.^2);
Rsq  = (Sdev - Serr)/Sdev   % R-square

Rsq = 0.9985

%% Plot result
xx   = 0:0.01:5;
plot(x, y, 'o', 'markersize', 12, 'linewidth', 2), hold on
plot(xx, lsy(xx)), grid on
xlabel('x'), ylabel('y')
title('Polynomial Regression using PSO')