trackErrorMetrics

Track error and NEES

Description

The trackErrorMetrics System object™ compares tracks from a multi-object tracking system against known truth by automatic assignment of tracks to known truth at each track update.

To generate track assignment metrics:

  1. Create the trackErrorMetrics object and set its properties.

  2. Call the object with arguments, as if it were a function.

To learn more about how System objects work, see What Are System Objects? (MATLAB).

Creation

Syntax

errorMetrics = trackErrorMetrics
errorMetrics = trackErrorMetrics(Name,Value)

Description

errorMetrics = trackErrorMetrics creates a trackErrorMetrics System object with default property values.

example

errorMetrics = trackErrorMetrics(Name,Value) sets properties for the trackErrorMetrics object using one or more name-value pairs. For example, metrics = trackErrorMetrics('MotionModel','constvel') creates a trackErrorMetrics object with a constant velocity motion model. Enclose property names in quotes.

Properties

expand all

Unless otherwise indicated, properties are nontunable, which means you cannot change their values after calling the object. Objects lock when you call them, and the release function unlocks them.

If a property is tunable, you can change its value at any time.

For more information on changing property values, see System Design in MATLAB Using System Objects (MATLAB).

Error function format specified as 'built-in' or 'custom'.

  • 'built-in' – Enable the MotionModel property.

    This property is a convenient interface when using tracks reported by any built-in multi-object tracker, and truths reported by the platformPoses object function of a trackingScenario object. The default estimation error function assumes tracks and truths are arrays of structures or arrays of objects.

  • 'custom' – Enable custom properties: EstimationErrorLabels, EstimationErrorFcn, TruthIdentifierFcn, and TrackIdentifierFcns. These properties can be used to construct error functions for arbitrary tracks and truths input arrays.

Labels for outputs of error estimation function, specified as an array of strings or cell array of character vectors. The number of labels must correspond to the number of outputs of the error estimation function. Specify the error estimation functions using the EstimationErrorFcn property.

Example: {'posMSE','velMSE'}

Dependencies

To enable this property, set the ErrorFunctionFormat property to 'custom'.

Data Types: char | string

Error estimation function, specified as a function handle. The function determines estimation errors of truths to tracks.

The error estimation function can have multiple scalar outputs and must have the following syntax.

 [out1,out2, ...,outN] = estimationerror(onetrack,onetruth)
The number of outputs must match the number of entries in the labels array specified in the EstimationErrorLabels property.

onetrack is an element of the tracks array passed in as input trackErrorMetric at object updates. onetruth is an element of the truths array passed in at object updates. The trackErrorMetrics object averages each output arithmetically when reporting across tracks or truths.

Example: @errorFunction

Dependencies

To enable this property, set the ErrorFunctionFormat property to 'custom'.

Data Types: function_handle

Track identifier function, specified as a function handle. Specifies the track identifiers for the tracks input at object update. The track identifiers are unique string or numeric values.

The track identifier function must have the following syntax:

 trackID = trackIDentifier(tracks)

tracks is the same as the tracks array passed as input for trackErrorMetric at object update. trackID is the same size as tracks. The default identification function handle, @defaultTrackIdentifier, assumes tracks is an array of structures or objects with a 'TrackID' field name or property.

Dependencies

To enable this property, set the ErrorFunctionFormat property to 'custom'.

Data Types: function_handle

Truth identifier function, specified as a function handle. Specifies the truth identifiers for the truths input at object update. The truth identifiers are unique string or numeric values.

The truth identifier function must have the following syntax:

 truthID = truthIDentifier(truths)

truths is the same as the truths array passed as input for trackErrorMetric updates. truthID must have the same size as truths. The default identification function handle, @defaultTruthIdentifier, assumes truths is an array or structures or objects with a 'PlatformID' field name or property.

Dependencies

To enable this property, set the ErrorFunctionFormat property to 'custom'.

Data Types: function_handle

Usage

To estimate errors, call the track error metrics object with arguments, as if it were a function (described here).

Syntax

[posRMSE,velRMSE,posANEES,velANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)
[posRMSE,velRMSE,accRMSE,posANEES,velANEES,accANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)
[posRMSE,velRMSE,yawRateRMSE,posANEES,velANEES,yawRateANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)
[out1,out2, ... ,outN] = errorMetrics(tracks,trackIDs,truths,truthIDs)

Description

example

[posRMSE,velRMSE,posANEES,velANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs) returns the metrics

  • posRMSE – Position root mean squared error

  • velRMSE – Velocity root mean squared error

  • posANEES – Position average normalized-estimation error squared

  • velANEES – Velocity average normalized-estimation error squared.

for constant velocity motion. trackIDs is the set of track identifiers for all tracks. truthIDs is the set of truth identifiers. tracks are the set of tracks, and truths are the set of truths. trackIDs and truthIDs are each a vector whose corresponding elements match the track and truth identifiers found in tracks and truths, respectively.

To enable this syntax, set the ErrorFunctionFormat property to 'built-in' and the MotionModel property to 'constvel'.

[posRMSE,velRMSE,accRMSE,posANEES,velANEES,accANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs) also returns the metrics

  • accRMS – Acceleration root mean squared error

  • accANEES – acceleration average NEES

for constant acceleration motion.

To enable this syntax, set the ErrorFunctionFormat property to 'built-in' and the MotionModel property to 'constacc'.

[posRMSE,velRMSE,yawRateRMSE,posANEES,velANEES,yawRateANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs) also returns the metrics

  • yawRateRMSE – yaw rate root mean squared error.

  • yawRateANEES – yaw rate average normalized-estimation error squared.

for constant turn rate motion.

To enable this syntax, set the ErrorFunctionFormat property to 'built-in' and the MotionModel property to 'constturn'.

[out1,out2, ... ,outN] = errorMetrics(tracks,trackIDs,truths,truthIDs) returns the user-defined metrics out1, out2, ... , outN.

To enable this syntax, set the ErrorFunctionFormat property to 'custom'. The number of outputs corresponds to the number of elements listed in the EstimationErrorLabels property, and must match the number of outputs in the EstimationErrorFcn. The results of the estimation errors are averaged arithmetically over all track-to-truth assignments.

Input Arguments

expand all

Track information, specified as a structure or array of structures. For built-in trackers such as trackerGNN or trackerTOMHT, the structure contains 'State', 'StateCovariance', and 'TrackID' information.

Data Types: struct

Track identifiers, specified as a real-valued vector. trackIDs elements match the tracks found in tracks.

Truth information, specified as a structure or array of structures. When using a trackingScenario, truth information can be obtained from the platformPoses object function.

Data Types: struct

Truth identifiers, specified as a real-valued vector. truthIDs elements match the truths found in truths.

Output Arguments

expand all

Position root mean squared error, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Velocity root mean squared error, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Acceleration root mean squared error, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Yaw rate root mean squared error, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Position average normalized estimation error squared, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Velocity average normalized estimation error squared, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Acceleration average normalized estimation error squared, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Yaw rate average normalized estimation error squared, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

Custom error metric outputs, returned as scalars. These errors are the output of the error estimation function specified in the EstmationErrorFcn property.

Dependencies

To enable these arguments, set the ErrorFunctionFormat property to 'custom'.

Object Functions

To use an object function, specify the System object as the first input argument. For example, to release system resources of a System object named obj, use this syntax:

release(obj)

expand all

cumulativeTrackMetricsCumulative metrics for recent tracks
cumulativeTruthMetricsCumulative metrics for recent truths
currentTrackMetricsMetrics for recent tracks
currentTruthMetricsMetrics for recent truths
releaseRelease resources and allow changes to System object property values and input characteristics
resetReset internal states of System object
isLockedDetermine if System object is in use
cloneCreate duplicate System object

Examples

expand all

Examine the assignments and errors for a system tracking two targets.

First, load the stored track data.

load trackmetricex tracklog truthlog

Create objects to analyze assignment and error metrics.

tam = trackAssignmentMetrics;
tem = trackErrorMetrics;

Create the output variables.

posRMSE = zeros(numel(tracklog),1);
velRMSE = zeros(numel(tracklog),1);
posANEES = zeros(numel(tracklog),1);
velANEES = zeros(numel(tracklog),1);

Loop over all tracks to:

  • Extract the tracks and ground truth at the i th tracker update.

  • Analyze and retrieve the current track-to-truth assignment.

  • Analyze instantaneous error metrics over all tracks and truths.

for i=1:numel(tracklog)
    tracks = tracklog{i};
    truths = truthlog{i};
    [trackAM,truthAM] = tam(tracks, truths);
    [trackIDs,truthIDs] = currentAssignment(tam);
    [posRMSE(i),velRMSE(i),posANEES(i),velANEES(i)] = ...
        tem(tracks,trackIDs,truths,truthIDs);
end

Show the track metrics table.

trackMetricsTable(tam)
ans =

  4x15 table

    TrackID    AssignedTruthID    Surviving    TotalLength    DeletionStatus    DeletionLength    DivergenceStatus    DivergenceCount    DivergenceLength    RedundancyStatus    RedundancyCount    RedundancyLength    FalseTrackStatus    FalseTrackLength    SwapCount
    _______    _______________    _________    ___________    ______________    ______________    ________________    _______________    ________________    ________________    _______________    ________________    ________________    ________________    _________

       1             NaN            false         1120            false               0                false                 3                   3                false                 0                   0                false                 0                0    
       2             NaN            false         1736            false               0                false                 8                  88                false                 0                   0                false                 0                3    
       6               3            true          1138            false               0                false                 4                 314                false                 1                  28                false                 0                2    
       8               2            true           662            false               0                false                 2                  29                false                 1                 169                false                 0                0    

Show the truth metrics table.

truthMetricsTable(tam)
ans =

  2x10 table

    TruthID    AssociatedTrackID    DeletionStatus    TotalLength    BreakStatus    BreakCount    BreakLength    InCoverageArea    EstablishmentStatus    EstablishmentLength
    _______    _________________    ______________    ___________    ___________    __________    ___________    ______________    ___________________    ___________________

       2               8                false            2678           false           4             168            true                 true                    56         
       3               6                false            2678           false           3             645            true                 true                    84         

Plot the RMSE and ANEES error metrics.

subplot(2,2,1)
plot(posRMSE)
title('Position Error')
xlabel('tracker update')
ylabel('RMSE (m)')

subplot(2,2,2)
plot(velRMSE)
title('Velocity Error')
xlabel('tracker update')
ylabel('RMSE (m/s)')

subplot(2,2,3)
plot(posANEES)
title('Position Error')
xlabel('tracker update')
ylabel('ANEES')

subplot(2,2,4)
plot(velANEES)
title('Velocity Error')
xlabel('tracker update')
ylabel('ANEES')

Show the current error metrics for each individual recorded track.

currentTrackMetrics(tem)
ans =

  2x5 table

    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       6       44.712    20.988    0.05974     0.31325 
       8       129.26    12.739     1.6745      0.2453 

Show the current error metrics for each individual recorded truth object.

currentTruthMetrics(tem)
ans =

  2x5 table

    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       129.26    12.739     1.6745      0.2453 
       3       44.712    20.988    0.05974     0.31325 

Show the cumulative error metrics for each individual recorded track.

cumulativeTrackMetrics(tem)
ans =

  4x5 table

    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       1       117.69    43.951    0.58338     0.44127 
       2        129.7      42.8    0.81094     0.42509 
       6       371.35    87.083     4.5208      1.6952 
       8       130.45    53.914     1.0448     0.44813 

Show the cumulative error metrics for each individual recorded truth object.

cumulativeTruthMetrics(tem)
ans =

  2x5 table

    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       258.21    65.078     2.2514     0.93359 
       3       134.41    48.253    0.96314     0.49183 

Introduced in R2018b