perfcurve and ROC curve

Dear friends,

I have a confusion about ROC curve and hopefully you can help me!

To plot ROC, i was naively using a simple command as plot(False_alarm_rate,Hit_rate,'-'). But, it is not exactly the same as perfcurve plot. To use this function, i wrote the following script

Q=reshape([Hit_rate False_alarm_rate],[],1);
    Labels=[]; Labels = ones(size(Q,1),1);
    Labels(end/2+1:end) = 0; 
    PosClass = 1;
    X=[];Y=[];
    [X Y T,AUC] = perfcurve(Labels,Q,PosClass);
    figure, plot(X,Y,'r') % ROC

could you please tell me , what i am missing here?

- BTW, can we calculate d-prime from output of perfcurve?

thanks in advance, Karlo

Answer 1

[Ilya]

Q must be classification scores. What you put in Q sounds more like what perfcurve should return as output. Take a classifier from the Statistics and Machine Learning Toolbox such as decision tree, discriminant etc and look at the predict method. The second output from the predict method is classification score.

[karlo gonzales]

Thanks Ilya for your reply.

Indeed, Q represent the Hit rate and the False alarm rate for different subjects who performed a memory test. would you please tell me how you would plot ROC for this case?

[Ilya]

The standard ROC curve is a plot of TPR vs FPR. The doc for perfcurve defines TPR and FPR (as well as other criteria) in the name-value pair section. You could write down definitions of false alarm rate etc and see if you can transform those into TPR and FPR. I am sure you are at least as good as I am at algebra, and, unlike you, I do not know what hit rate and false alarm rate are.