Perform Incremental Learning Using IncrementalRegressionLinear Fit and Predict Blocks

Since R2023b

This example shows how to use the IncrementalRegressionLinear Predict and IncrementalRegressionLinear Fit blocks for incremental learning and response prediction in Simulink®. The IncrementalRegressionLinear Fit block fits a chunk of observations (predictor data) using a configured incremental model for linear regression (incrementalRegressionLinear) and outputs updated incremental learning model parameters as a bus signal. The IncrementalRegressionLinear Predict block accepts an IncrementalRegressionLinear model and a chunk of predictor data, and outputs the predicted response for the observations.

Create Incremental Learner Model

Load the robot arm data set.

load robotarm.mat
n = numel(ytrain);

For details on the data set, enter Description at the command line.

Create an incremental linear model for regression. Specify that the data has 32 predictors and to standardize the data using an estimation period of 500 observations. Create a workspace variable linearMdl to store the initial incremental learning model.

rng(0,"twister") % For reproducibility
Mdl = incrementalRegressionLinear(NumPredictors=32, ...
    Standardize=true,EstimationPeriod=500);
linearMdl = Mdl;

Create Input Data for Simulink

Simulate streaming data by dividing the training data into chunks of 50 observations. For each chunk, select a single observation from the test set to import into the IncrementalRegressionLinear Predict block.

numObsPerChunk = 50;
nchunk = floor(n/numObsPerChunk);
for j = 1:nchunk
    ibegin = min(n,numObsPerChunk*(j-1) + 1);
    iend = min(n,numObsPerChunk*j);
    idx = ibegin:iend;   
    Xin(:,:,j) = Xtrain(idx,:);
    Yin(:,j) = ytrain(idx);
    Xtestset(1,:,j) = Xtest(j,:);
end

Convert the training and test set chunks into time series objects.

t = 0:size(Xin,3)-1;
Xtrain_ts = timeseries(Xin,t,InterpretSingleRowDataAs3D=true);
Ytrain_ts = timeseries(Yin',t,InterpretSingleRowDataAs3D=true);
Xtest_ts = timeseries(Xtestset,t,InterpretSingleRowDataAs3D=true);

Open Provided Simulink Model

This example provides the Simulink model slexIncRegressionLinearPredictExample.slx, which includes the IncrementalRegressionLinear Predict and IncrementalRegressionLinear Fit blocks. The Simulink model is configured to use linearMdl as the initial model for incremental learning and prediction.

Open the Simulink model slexIncRegressionLinearPredictExample.slx.

slName = "slexIncRegressionLinearPredictExample";
open_system(slName);

Simulate Model

Simulate the Simulink model to perform incremental learning and predict responses to the test set observations. Export the simulation outputs to the workspace. You can use the Simulation Data Inspector (Simulink) to view the logged data of an Outport block.

simOut = sim(slName,"StopTime",num2str(numel(t)-1));

% Extract responses
yfit_sig = simOut.yout.getElement(1);
yfit_sl = squeeze(yfit_sig.Values.Data);

% Extract CanPredict values
iswarm_sig = simOut.yout.getElement(2);
iswarm_sl = squeeze(iswarm_sig.Values.Data);

% Extract beta values
beta_sig = simOut.yout.getElement(3);
beta_sl = squeeze(beta_sig.Values.Data);

% Extract bias values
bias_sig = simOut.yout.getElement(4);
bias_sl = squeeze(bias_sig.Values.Data);

At each iteration, the IncrementalRegressionLinear Fit block trains the model and updates the model parameters. The IncrementalRegressionLinear Predict block calculates the predicted response for the test set observation.

Analyze Model During Training

To see how the model parameters and response values evolve during training, plot them on separate tiles.

figure
tiledlayout(4,1);
nexttile
plot(iswarm_sl,".-")
ylabel("CanPredict")
xlabel("Iteration")
xlim([0 nchunk])
nexttile
plot(yfit_sl,".-")
ylabel("Response")
xlabel("Iteration")
xlim([0 nchunk])
nexttile
plot(beta_sl(1,:),".-")
ylabel("\beta_1")
xlabel("Iteration")
xlim([0 nchunk])
nexttile
plot(bias_sl,".-")
ylabel("Bias")
xlabel("Iteration")
xlim([0 nchunk])

During the estimation period, the IncrementalRegressionLinear Fit block estimates hyperparameters but does not fit the initial model (see incrementalRegressionLinear). Therefore, the CanPredict and yfit outputs of the IncrementalRegressionLinear Predict block, model beta coefficients, and model bias all equal zero during the first 10 iterations. At the end of the estimation period, the IncrementalRegressionLinear Fit block updates the model parameters and calculates predicted responses. After the first iteration following the estimation period, the response values vary between approximately –0.1 and 0.1. The first beta coefficient varies significantly during the first 15 iterations following the estimation period, and shows only small variations between –0.001 and 0.001 thereafter. The bias (intercept) term has an initial value of 0.08 and gradually approaches zero.