How to use the 'perfcurve' of Matlab with specific inputs?
1 view (last 30 days)
Show older comments
Hi Smart guys,
I wrote following codes to get a plot of 'classification accuracy' vs. 'threshold':
(The datasets has the ground truth contains two classes labelled 'Good' or 'Bad')
LDAClassifierObject = ClassificationDiscriminant.fit(featureSelcted, groundTruthGroup, 'DiscrimType', 'linear');
[LDALabel, LDAScore] = resubPredict(LDAClassifierObject);
[~, AccuracyLDA, Thr] = perfcurve(groundTruthNumericalLable(:,1), LDAScore(:,1), 1,'yCrit','accu');
figure,
plot(Thr,AccuracyLDA,'r-');
hold on;
plot(Thr,AccuracyLDA,'bo');
xlabel('Threshold for ''good'' Returns');
ylabel('Classification Accuracy');
grid on;
[maxVal, maxInd] = max(AccuracyLDA)
maxVal =
0.8696
maxInd =
15
Thr(15)
ans =
0.7711
Also, I run the ROC analysis for the same datasets that the ground truth contains two classes labelled 'Good' or 'Bad'
[FPR, TPR, Thr, AUC, OPTROCPT] = perfcurve(groundTruthGroup(:,1), LDAScore(:,1), 'Good');
OPTROCPT =
0.1250 0.8667
Why Thr(15)=0.7711 is different from OPTROCPT(2)=0.8667 ?
Is the best cut-off point (ie, the best threshold OPTROCPT) obtained by ROC is the one has maximum accuracy of LDA?
Or maybe I am wrong, then what exactly `perfcurve(groundTruthNumericalLable(:,1), LDAScore(:,1), 1,'yCrit','accu')` tell us?
Thanks a lot.
A.
1 Comment
Ilya
on 19 Mar 2013
Why would any value in Thr be equal to any value in OPTROCPT? Could you copy and paste the part in the doc that made you believe these two should be equal?
Answers (1)
Neil Caithness
on 23 Oct 2013
OPTROCPT(2) is the TPR value of the optimal cut-point, not the threshold value itself.
In your example, try
a = find(TPR==OPTROCPT(2))
One of these should be your index value 15, then
Thr(a)
should be your optimal threshold value 0.7711
0 Comments
See Also
Categories
Find more on ROC - AUC in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!