cumulativeTruthMetrics

Cumulative metrics for recent truths

Syntax

metricsTable = cumulativeTruthMetrics(errorMetrics)

Description

metricsTable = cumulativeTruthMetrics(errorMetrics) returns a table of cumulative metrics, metricsTable, for every truth identifier provided in the most recent update.

Examples

Assignment and Error Metrics for Two Tracked Targets

Examine the assignments and errors for a system tracking two targets.

First, load the stored track data.

load trackmetricex tracklog truthlog

Create objects to analyze assignment and error metrics.

tam = trackAssignmentMetrics;
tem = trackErrorMetrics;

Create the output variables.

posRMSE = zeros(numel(tracklog),1);
velRMSE = zeros(numel(tracklog),1);
posANEES = zeros(numel(tracklog),1);
velANEES = zeros(numel(tracklog),1);

Loop over all tracks to:

• Extract the tracks and ground truth at the i th tracker update.

• Analyze and retrieve the current track-to-truth assignment.

• Analyze instantaneous error metrics over all tracks and truths.

for i=1:numel(tracklog)
    tracks = tracklog{i};
    truths = truthlog{i};
    [trackAM,truthAM] = tam(tracks, truths);
    [trackIDs,truthIDs] = currentAssignment(tam);
    [posRMSE(i),velRMSE(i),posANEES(i),velANEES(i)] = ...
        tem(tracks,trackIDs,truths,truthIDs);
end

Show the track metrics table.

trackMetricsTable(tam)

ans=4×16 table
    TrackID    AssignedTruthID    Surviving    TotalLength    UnreportedStatus    DeletionStatus    DeletionLength    DivergenceStatus    DivergenceCount    DivergenceLength    RedundancyStatus    RedundancyCount    RedundancyLength    FalseTrackStatus    FalseTrackLength    SwapCount
    _______    _______________    _________    ___________    ________________    ______________    ______________    ________________    _______________    ________________    ________________    _______________    ________________    ________________    ________________    _________

       1             NaN            false         1120             false              false               0                false                 3                   3                false                 0                   0                false                  0               0    
       2             NaN            false         1736             false              false               0                false                 8                  88                false                 0                   0                false                 28               3    
       6               3            true          1138             false              false               0                false                 4                 314                false                 1                  28                false                  0               2    
       8               2            true           662             false              false               0                false                 2                  29                false                 1                 169                false                 28               0    

Show the truth metrics table.

truthMetricsTable(tam)

ans=2×11 table
    TruthID    AssociatedTrackID    DeletionStatus    TotalLength    UnreportedStatus    BreakStatus    BreakCount    BreakLength    InCoverageArea    EstablishmentStatus    EstablishmentLength
    _______    _________________    ______________    ___________    ________________    ___________    __________    ___________    ______________    ___________________    ___________________

       2               8                false            2678             false             false           4             168            true                 true                    56         
       3               6                false            2678             false             false           3             645            true                 true                    84         

Plot the RMSE and ANEES error metrics.

subplot(2,2,1)
plot(posRMSE)
title('Position Error')
xlabel('tracker update')
ylabel('RMSE (m)')

subplot(2,2,2)
plot(velRMSE)
title('Velocity Error')
xlabel('tracker update')
ylabel('RMSE (m/s)')

subplot(2,2,3)
plot(posANEES)
title('Position Error')
xlabel('tracker update')
ylabel('ANEES')

subplot(2,2,4)
plot(velANEES)
title('Velocity Error')
xlabel('tracker update')
ylabel('ANEES')

Show the current error metrics for each individual recorded track.

currentTrackMetrics(tem)

ans=2×5 table
    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       6       44.712    20.988    0.05974     0.31325 
       8       129.26    12.739     1.6745      0.2453 

Show the current error metrics for each individual recorded truth object.

currentTruthMetrics(tem)

ans=2×5 table
    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       129.26    12.739     1.6745      0.2453 
       3       44.712    20.988    0.05974     0.31325 

Show the cumulative error metrics for each individual recorded track.

cumulativeTrackMetrics(tem)

ans=4×5 table
    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       1       117.69    43.951    0.58338     0.44127 
       2        129.7      42.8    0.81094     0.42509 
       6       371.35    87.083     4.5208      1.6952 
       8       130.45    53.914     1.0448     0.44813 

Show the cumulative error metrics for each individual recorded truth object.

cumulativeTruthMetrics(tem)

ans=2×5 table
    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       258.21    65.078     2.2514     0.93359 
       3       134.41    48.253    0.96314     0.49183 

Input Arguments

errorMetrics — Error metrics object

trackErrorMetrics System object

Error metrics object, specified as a trackErrorMetrics System object™.

Output Arguments

metricsTable — Truth error metrics

table

Truth error metrics, returned as a table.

• When you set the ErrorFunctionFormat property of the input error metrics object to 'built-in', the table columns depend on the setting of the MotionModel property.

  Motion Model	Table Columns
  'constvel'	posRMSE, velRMSE, posANEES, velANEES
  'constacc'	posRMSE, velRMSE, accRMSE, posANEES, velANEES, accANEES
  'constturn'	posRMSE, velRMSE, yawRateRMSE, posANEES, velANEES, yawRateANEES

  RMSE and ANEES denote root mean squared error and average normalized estimation error squared of a truth trajectory for the entire tracking scenario time history. Since a truth trajectory can associate with multiple tracks at a time step, the calculation of cumulative RMSE and ANEES values is each separated into two steps. For example, in the first step of the position RMSE value calculation, the function first calculates the RMSE value at a given time step t as:

  $$S_t={\displaystyle \sum _{k=1}^{K_t}\Vert \Delta p_{t,k}{\Vert }^2}$$

  where K_t is the number of tracks associated with the truth at time step t, and

  $$\Delta p_{t,k}=p_{track,t,k}-p_{truth,t}$$

  is the position difference between the position of kth associated track and the position of the truth at time step t. In the second step, the S_t values of all the time steps (t = 1,2,…,N) are summed and averaged over the total number of associated tracks (denoted by R) to obtained the cumulative position RMSE value as:

  $$\text{posRMSE}=\sqrt{\frac{1}{{\displaystyle \sum _{t=1}^N{K}_t}}{\displaystyle \sum _{t=1}^N{\displaystyle \sum _{k=1}^{K_t}\Vert \Delta p_{t,k}{\Vert }^2}}}$$

  where the total number of associated tracks, R, is given by

  $$R={\displaystyle \sum _{t=1}^N{K}_t}.$$

  The cumulative RMSE values for other states (vel, pos, acc, and yawRate) are defined similarly.

  The calculation of the cumulative position ANEES value, posANEES, for a truth trajectory can also be separated into two steps. In the first step, the function calculates the ANEES value at a given time step t as:

  $$Q_t={\displaystyle \sum _{k=1}^{K_t}\Delta p_{t,k}{}^T{C}_{p,t,k}^{-1}\Delta p_{t,k}}$$

  where C_p,t,k is the covariance corresponding to the position of the kth associated track at time step t. In the second step, the Q_t values for all the time steps (t = 1,2,…,N) are summed and averaged over the total number of associated tracks (denoted by R) to obtained the cumulative position ANEES value as:

  $$posANEES=\frac{1}{{\displaystyle \sum _{t=1}^N{K}_t}}{\displaystyle \sum _{t=1}^N{\displaystyle \sum _{k=1}^{K_t}\Delta p_{t,k}{}^T{C}_{p,t,k}^{-1}\Delta p_{t,k}}}$$

  The cumulative ANEES values for other states (vel, pos, acc, and yawRate) are defined similarly.

• When you set the ErrorFunctionFormat property to 'custom', the table contains the arithmetically averaged values of the custom metrics output from the error function.

Version History

Introduced in R2018b