Inconsistent ROC confidence intervals: rocmetrics plot vs. manually extracted intervals
Show older comments
I am currently working with the rocmetrics class and its bootstrap-based confidence intervals. While the built-in plotting functionality works well, I observe a discrepancy when I try to extract and visualise the confidence intervals manually from the Metrics property.
Specifically, the confidence intervals shown by
plot(RO,"ShowConfidenceIntervals",true);
in the plot in red do not match the intervals I obtain when directly plotting the corresponding columns from RO.Metrics in blue, but the ROC itself matches.
The manually reconstructed intervals appear:
- narrower, and
- slightly shifted along the x-axis
compared to the intervals shown by the built-in plot.


Minimal Working Example
close all
rng(1)
nData = 500;
scores = rand(nData,1);
class = (scores+(rand(nData,1)*2-1)*0.4)>0.5;
RO = rocmetrics(class,scores,1,NumBootstraps=100);
plotObj = plot(RO.Metrics.FalsePositiveRate(:,1),RO.Metrics.TruePositiveRate(:,1));
hold on
fill([RO.Metrics.FalsePositiveRate(:,2);flipud(RO.Metrics.FalsePositiveRate(:,3))],...
[RO.Metrics.TruePositiveRate(:,2);flipud(RO.Metrics.TruePositiveRate(:,3))],...
plotObj.Color,'FaceAlpha',0.3,'EdgeAlpha',0);
plot(RO,"ShowConfidenceIntervals",true);
Observations
- The central ROC curve ((:,1)) aligns well between both approaches.
- However, the confidence intervals differ noticeably:
- The manually plotted intervals are smaller.
- They also appear misaligned in FPR (x-direction).
- There seems to be no direct documentation on the Metrics data using bootstrapping
Question
I would like to understand:
- Why do the values in RO.Metrics.FalsePositiveRate(:,2:3) and TruePositiveRate(:,2:3) not reproduce the plotted intervals?
- Are these quantities defined on a different grid than the plotted ROC?
- Does the plot function perform additional processing (e.g., interpolation onto a common FPR grid)?
- What is the correct way to extract and reproduce the confidence intervals programmatically?
Environment
MATLAB Version: 25.2.0.3150157 (R2025b) Update 4
Operating System: Windows 11
Statistics and Machine Learning Toolbox: Version 25.2
Additional Notes
My assumption is that the discrepancy might stem from how bootstrap ROC curves are aligned (e.g. interpolation to a common FPR grid before computing percentiles), whereas the raw Metrics values may correspond to threshold-based sampling. However, I have not found clear documentation on this.
Answers (0)
Categories
Find more on Detection in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!