Per Observation Loss

Per observation regression or classification error of incremental model

Since R2025a

expand all in page
  • Per Observation Loss Block Icon

Libraries:
Statistics and Machine Learning Toolbox / Incremental Learning / Drift Detection

Description

The Per Observation Loss block outputs the per observation regression or classification error of a configured incremental model, given predictor data x and ground-truth responses (labels) y.

Import a trained incremental learning or incremental drift-aware model object into the block by specifying the name of a workspace variable that contains the object. The input port mdl receives a bus signal that represents an incremental learning model fit to streaming data. The input port x receives a chunk of predictor data (observations), and the input port y receives a chunk of responses or labels for measuring the model loss. The output port IsWarm returns a value indicating if the model is warm, which means that it is computing the loss. The output port loss returns the computed loss.

Examples

expand all

This example uses:

Open Live Script

Load the human activity data set and randomly shuffle the observations. For details on the data set, enter Description at the command line.

load humanactivity
n = numel(actid);
rng(0,"twister") % For reproducibility
idx = randsample(n,n);
X = feat(idx,:);
Y = actid(idx);

Responses can be one of five classes: Sitting, Standing, Walking, Running, or Dancing. Dichotomize the response by identifying whether the subject is moving (actid > 2).

Y = Y > 2;

Create an incremental linear model for binary classification. The data is standardized during the estimation period and the Per Observation Loss block does not output the performance metrics for the next 500 observations. 

linearMdl = incrementalClassificationLinear(ClassNames=[false,true], ...
    NumPredictors=60, Standardize=true, EstimationPeriod=500, MetricsWarmupPeriod=500)
linearMdl = 
  incrementalClassificationLinear

            IsWarm: 0
           Metrics: [1×2 table]
        ClassNames: [0 1]
    ScoreTransform: 'none'
              Beta: [60×1 double]
              Bias: 0
           Learner: 'svm'


  Properties, Methods

Create a copy of linearMdl to be used as the initial model in the Per Observation Loss block.

initMdl = linearMdl;

Select the first 10,000 observations as the training set, and the next 10,000 observations to track the model performance metrics. 

n = 10000;
Xtrain = X(1:n,:);
Ytrain = Y(1:n,:);
Xmetrics = X(n+1:2*n,:);
Ymetrics = Y(n+1:2*n,:);

Simulate streaming data by dividing the training data into chunks of 50 observations.

numObsPerChunk = 50;
nchunk = floor(n/numObsPerChunk);
for j = 1:nchunk
    ibegin = min(n,numObsPerChunk*(j-1) + 1);
    iend = min(n,numObsPerChunk*j);
    idx = ibegin:iend;   
    Xin(:,:,j) = X(idx,:);
    Yin(:,j) = Y(idx);
    Xm_in(:,:,j) = Xmetrics(idx,:);
    Ym_in(:,j) = Ymetrics(idx);
 end

Next, convert the training and metric chunks into time series objects to load into the Simulink model.

t = 0:size(Xin,3)-1;
X_ts = timeseries(Xin,t,InterpretSingleRowDataAs3D=true);
Y_ts = timeseries(Yin',t,InterpretSingleRowDataAs3D=true);
Xm_ts = timeseries(Xm_in,t,InterpretSingleRowDataAs3D=true);
Ym_ts = timeseries(Ym_in',t,InterpretSingleRowDataAs3D=true);

This example provides a Simulink model, slexPerObsLoss.slx, shown in the figure below. The model is configured to use linearMdl as the initial model for the IncrementalClassificationLinear Fit block and initMdl with the logit loss function for the Per Observation Loss block. Double-click the block to access the Block Parameters dialog box.

slName = "slexPerObsLoss";
open_system(slName);

Simulate the Simulink model to perform incremental learning and compute the per observation loss metrics. Export the simulation outputs to the workspace. You can use the Simulation Data Inspector (Simulink) (Simulink) to view the logged data of an Outport block.

simOut = sim(slName,"StopTime",num2str(numel(t)-1));
% Extract IsWarm values
IsWarm_sig = simOut.yout.getElement(1);
IsWarm_sl = squeeze(IsWarm_sig.Values.Data);

% Extract Metrics values
loss_sig = simOut.yout.getElement(2);
loss_sl = squeeze(loss_sig.Values.Data);

At each iteration, the IncrementalClassificationLinear Fit block trains the linear model and updates the model parameters. The Per Observation Loss block calculates the logistic error per observation.

Plot the IsWarm status indicator value and logistic error logit for the observations in the last chunk on separate tiles.

figure
tiledlayout(2,1);
nexttile
plot(IsWarm_sl,".")
ylabel("IsWarm")
xlabel("Iteration")
xlim([0 nchunk])
nexttile
plot(loss_sl(end,:),".")
ylabel("Loss (logit)")
xlabel("Observations")
xlim([0 numObsPerChunk])
ylim([0 0.5])

Figure contains 2 axes objects. Axes object 1 with xlabel Iteration, ylabel IsWarm contains a line object which displays its values using only markers. Axes object 2 with xlabel Observations, ylabel Loss (logit) contains a line object which displays its values using only markers.

Ports

Input

expand all

Incremental learning model fit to streaming data, specified as a bus signal (see Composite Signals (Simulink)).

Data Types: single | double | half | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | Boolean | fixed point

Chunk of predictor data, specified as a numeric matrix. The orientation of the variables and observations is specified by Predictor data observation dimension. The default orientation is rows, which indicates that observations in the predictor data are oriented along the rows of x.

The length of the observation responses y and the number of observations in x must be equal; y(j) is the response of observation j (row or column) in x.

The block supports only numeric input predictor data. If your input data includes categorical data, you must prepare an encoded version of the categorical data. Use dummyvar to convert each categorical variable to a numeric matrix of dummy variables. Then, concatenate all dummy variable matrices and any other numeric predictors. For more details, see Dummy Variables.

Data Types: single | double | half | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | Boolean | fixed point

Chunk of responses (labels) for measuring the model performance, specified as a numeric, logical, or enumerated vector.

The length of the observation responses y and the number of observations in x must be equal; y(j) is the response of observation j (row or column) in x.

For classification problems:

  • Each label must correspond to one row of the array.

  • If y contains a label that is not in mdl.ClassNames, the block issues an error.

Data Types: single | double | half | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64 | Boolean | fixed point | enumerated

Output

expand all

Flag indicating whether the incremental model tracks performance metrics, returned as logical 0 (false) or 1 (true).

ValueDescription
1 (true)The incremental model mdl is warm. Consequently, the block tracks performance metrics in the bus signal output.
0 (false)The block does not track performance metrics.

Per observation regression or classification error of the model predictions using the predictors in x and true observed values in y, returned as a numeric matrix. The matrix is of size nchunk-by-nobs where nchunk is the total number of chunks and nobs is the number of observations in a chunk of x. The loss is -1 if IsWarm is false.

Specify the loss function to compute the error using Loss function.

Parameters

expand all

To edit block parameters interactively, use the Property Inspector. From the Simulink® Toolstrip, on the Simulation tab, in the Prepare gallery, select Property Inspector.

Specify the name of a workspace variable that contains the configured model object. You can use the following objects:

If a drift-aware model object is provided, the block will use the incremental learning model inside it.

The following constraints apply:

  • The predictor data cannot include categorical predictors (logical, categorical, char, string, or cell). If you supply training data in a table, the predictors must be numeric (double or single). To include categorical predictors in a model, preprocess them by using dummyvar before fitting the model.

  • The NumPredictors property of the initial model must be a positive integer scalar, and must be equal to the number of predictors in x.

  • The ScoreTransform property of the initial model (classification only) cannot be "invlogit" or an anonymous function.

Programmatic Use

Block Parameter: InitialLearner
Type: character vector or string
Values: workspace model object name
Default: "initMdl"

Specify the loss function to compute the per observation loss during incremental learning. For more details on loss functions, see loss.

The following table lists the built-in loss function names:

  • For Regression Models:

    NameDescription
    "squarederror"Squared error
    "epsiloninsensitive"Epsilon-insensitive error

    Default is "squarederror" for regression models.

  • For Classification Models:

    NameDescription
    "binodeviance"Binomial deviance
    "classiferror"Misclassification error rate
    "exponential"Exponential
    "hinge"Hinge
    "logit"Logistic
    "quadratic"Quadratic

    Note

    You can only specify "classiferror" for incrementalClassificationECOC models.

The Per Observation Loss block ignores the metrics specified by the Metric property of mdl.

Programmatic Use

Block Parameter: LossFun
Type: character vector or string
Values: "classiferror" | "binodeviance" | "exponential" | "hinge" | "logit" | "quadratic" | "squarederror" | "epsiloninsensitive"
Default: "classiferror" for classification models and "squarederror" for regression models

Specify the observation dimension of the predictor data. The default value is rows, which indicates that observations in the predictor data are oriented along the rows of x.

Programmatic Use

Block Parameter: ObservationsIn
Type: character vector
Values: "rows" | "columns"
Default: "rows"

Specify the discrete interval between sample time hits or specify another type of sample time, such as continuous (0) or inherited (–1). For more options, see Types of Sample Time (Simulink).

By default, the Per Observation Loss block inherits sample time based on the context of the block within the model.

Programmatic Use

Block Parameter: SystemSampleTime
Type: string scalar or character vector
Values: scalar
Default: "–1"

Data Types

Fixed-Point Operational Parameters

Specify the rounding mode for fixed-point operations. For more information, see Rounding Modes (Fixed-Point Designer).

Block parameters always round to the nearest representable value. To control the rounding of a block parameter, enter an expression into the mask field using a MATLAB® rounding function.

Programmatic Use

Block Parameter: RndMeth
Type: character vector
Values: "Ceiling" | "Convergent" | "Floor" | "Nearest" | "Round" | "Simplest" | "Zero"
Default: "Floor"

Specify whether overflows saturate or wrap.

ActionRationaleImpact on OverflowsExample

Select this check box (on).

Your model has possible overflow, and you want explicit saturation protection in the generated code.

Overflows saturate to either the minimum or maximum value that the data type can represent.

The maximum value that the int8 (signed 8-bit integer) data type can represent is 127. Any block operation result greater than this maximum value causes overflow of the 8-bit integer. With the check box selected, the block output saturates at 127. Similarly, the block output saturates at a minimum output value of –128.

Clear this check box (off).

You want to optimize the efficiency of your generated code.

You want to avoid overspecifying how a block handles out-of-range signals. For more information, see Troubleshoot Signal Range Errors (Simulink).

Overflows wrap to the appropriate value that the data type can represent.

The maximum value that the int8 (signed 8-bit integer) data type can represent is 127. Any block operation result greater than this maximum value causes overflow of the 8-bit integer. With the check box cleared, the software interprets the value causing the overflow as int8, which can produce an unintended result. For example, a block result of 130 (binary 1000 0010) expressed as int8 is –126.

Programmatic Use

Block Parameter: SaturateOnIntegerOverflow
Type: character vector
Values: "off" | "on"
Default: "off"

Select this parameter to prevent the fixed-point tools from overriding the data type you specify for the block. For more information, see Use Lock Output Data Type Setting (Fixed-Point Designer).

Programmatic Use

Block Parameter: LockScale
Type: character vector
Values: "off" | "on"
Default: "off"

Data Type

Specify the data type for the loss output. The type can be specified directly or expressed as a data type object such as Simulink.NumericType.

For more information about data types, see Control Data Types of Signals (Simulink).

Click the Show data type assistant button to display the Data Type Assistant, which helps you set the data type attributes. For more information, see Specify Data Types Using Data Type Assistant (Simulink).

Programmatic Use

Block Parameter: LossDataTypeStr
Type: character vector or string
Values: | "double" | "single" | "half" | "int8" | "uint8" | "int16" | "uint16" | "int32" | "uint32" | "int64" | "uint64" | "boolean" | "fixdt(1,16,0)" | "fixdt(1,16,2^0,0)" | "<data type expression>"
Default: "double"

Specify the lower value of the loss output range that Simulink checks.

Simulink uses the minimum value to perform:

The Loss data type Minimum parameter does not saturate or clip the actual loss output. To do so, use the Saturation (Simulink) block instead.

Programmatic Use

Block Parameter: LossOutMin
Type: string or numeric scalar
Values: "[]" | scalar
Default: "[]"

Specify the upper value of the loss output range that Simulink checks.

Simulink uses the maximum value to perform:

The Loss data type Maximum parameter does not saturate or clip the actual loss output. To do so, use the Saturation (Simulink) block instead.

Programmatic Use

Block Parameter: LossOutMax
Type: string or numeric scalar
Values: "[]" | scalar
Default: "[]"

Specify the data type for the response output from the predict block (IncrementalRegressionLinear Predict, IncrementalClassificationLinear Predict, IncrementalClassificationECOC Predict etc.) that is internal to the Per Observation Loss block. The type can be inherited, specified directly, or expressed as a data type object such as Simulink.NumericType.

When you select Inherit: auto, the block uses a rule that inherits a data type.

For more information about data types, see Control Data Types of Signals (Simulink).

Click the Show data type assistant button to display the Data Type Assistant, which helps you set the data type attributes. For more information, see Specify Data Types Using Data Type Assistant (Simulink).

Programmatic Use

Block Parameter: PredictResponseDataTypeStr
Type: character vector or string
Values: "Inherit: auto" | "double" | "single" | "half" | "int8" | "uint8" | "int16" | "uint16" | "int32" | "uint32" | "int64" | "uint64" | "boolean" | "fixdt(1,16,0)" | "fixdt(1,16,2^0,0)" | "<data type expression>"
Default: "Inherit: auto"

Specify the lower value of the internal predict response output range that Simulink checks.

Simulink uses the minimum value to perform:

The Predict response data type Minimum parameter does not saturate or clip the actual predict response output. To do so, use the Saturation (Simulink) block instead.

Programmatic Use

Block Parameter: PredictResponseOutMin
Type: string or numeric scalar
Values: "[]" | scalar
Default: "[]"

Specify the upper value of the internal predict response output range that Simulink checks.

Simulink uses the maximum value to perform:

The Predict response data type Maximum parameter does not saturate or clip the actual predict response output. To do so, use the Saturation (Simulink) block instead.

Programmatic Use

Block Parameter: PredictResponseOutMax
Type: string or numeric scalar
Values: "[]" | scalar
Default: "[]"

Additional Data Types

Specify the data type for the additional response output from the predict block (IncrementalRegressionLinear Predict, IncrementalClassificationLinear Predict, or IncrementalClassificationECOC Predict) that is internal to the Per Observation Loss block. The type can be inherited, specified directly, or expressed as a data type object such as Simulink.NumericType.

When you select Inherit: Inherit via internal rule, the block uses an internal rule to determine the additional predict data type. The internal rule chooses a data type that optimizes numerical accuracy, performance, and generated code size, while taking into account the properties of the embedded target hardware. The software cannot always optimize efficiency and numerical accuracy at the same time.

For more information about data types, see Control Data Types of Signals (Simulink).

Click the Show data type assistant button to display the Data Type Assistant, which helps you set the data type attributes. For more information, see Specify Data Types Using Data Type Assistant (Simulink).

Programmatic Use

Block Parameter: AdditionalPredictDataTypeStr
Type: character vector
Values: "Inherit: Inherit via internal rule" | "double" | "single" | "half" | "int8" | "uint8" | "int16" | "uint16" | "int32" | "uint32" | "int64" | "uint64" | "boolean" | "fixdt(1,16,0)" | "fixdt(1,16,2^0,0)" | "<data type expression>"
Default: "Inherit: Inherit via internal rule"

Specify the lower value of the additional predict output range that Simulink checks.

Simulink uses the minimum value to perform:

The Additional predict data type Minimum parameter does not saturate or clip the actual additional predict. To do so, use the Saturation (Simulink) block instead.

Programmatic Use

Block Parameter: AdditionalPredictOutMin
Type: character vector
Values: "[]" | scalar
Default: "[]"

Specify the upper value of the additional predict output range that Simulink checks.

Simulink uses the maximum value to perform:

The Additional predict data type Maximum parameter does not saturate or clip the actual additional predict. To do so, use the Saturation (Simulink) block instead.

Programmatic Use

Block Parameter: AdditionalPredictOutMax
Type: string or numeric scalar
Values: "[]" | scalar
Default: "[]"

Specify the data type for the internal linear sum. The type can be inherited, specified directly, or expressed as a data type object such as Simulink.NumericType.

For more information about data types, see Control Data Types of Signals (Simulink).

Click the Show data type assistant button to display the Data Type Assistant, which helps you set the data type attributes. For more information, see Specify Data Types Using Data Type Assistant (Simulink).

Programmatic Use

Block Parameter: LinearSumDataTypeStr
Type: character vector or string
Values: Inherit: Inherit via internal rule | "double" | "single" | "half" | "int8" | "uint8" | "int16" | "uint16" | "int32" | "uint32" | "int64" | "uint64" | "boolean" | "fixdt(1,16,0)" | "fixdt(1,16,2^0,0)" | "<data type expression>"
Default: Inherit: Inherit via internal rule

Specify the lower value of the linear sum range that Simulink checks.

Simulink uses the minimum value to perform:

The Linear sum data type Minimum parameter does not saturate or clip the actual linear sum. To do so, use the Saturation (Simulink) block instead.

Programmatic Use

Block Parameter: LinearSumOutMin
Type: string or numeric scalar
Values: "[]" | scalar
Default: "[]"

Specify the upper value of the linear sum range that Simulink checks.

Simulink uses the maximum value to perform:

The Linear sum data type Maximum parameter does not saturate or clip the actual linear sum. To do so, use the Saturation (Simulink) block instead.

Programmatic Use

Block Parameter: LinearSumOutMax
Type: string or numeric scalar
Values: "[]" | scalar
Default: "[]"

Block Characteristics

Data Types

Boolean | double | enumerated | fixed point | half | integer | single

Direct Feedthrough

yes

Multidimensional Signals

no

Variable-Size Signals

no

Zero-Crossing Detection

no

Extended Capabilities

expand all

C/C++ Code Generation
Generate C and C++ code using Simulink® Coder™.

Fixed-Point Conversion
Design and simulate fixed-point systems using Fixed-Point Designer™.

Version History

Introduced in R2025a

See Also

Blocks

Objects

Functions