How to use ROC FUNCTION in MATLAB

Dear Matlab user, I have difficulties in applying the ROC FUNCTION. Let have the discussion by having a simple example.

Assume, we have the true condition and predicted condition represent by the vector actual and predicted, respectively.

actual      = [1;1;1;1;1;1;1;0;0;0]; % targets
predicted   = [1;1;1;1;1;0;0;1;0;0]; % outputs.

Manual calculate the True positive rate (FPR) and false positive rate (FPR) yield 5/7 and 1/3, respectively. The figure below show the confusion matrix and the calculation

However, by using the ROC FUNCTION, MATLAB give 1x10 and 1x10 instead of a single value for each of the TPR and FPR (i.e., 5/7 & 1/3).

[tpr,fpr,th] = roc(actual,predicted);

In this case, I thing I have misunderstood the usage of ROC function. I really appreciate if someone can show what and how to use the ROC function, correctly

Thanks in advance.

Answer 1

[Star Strider]

They apparently need to be row vectors. (I found the documentation a bit confusing, but then I do not use these functions frequently.)

[tpr,fpr,th] = roc(actual',predicted')

and using format rat, produces:

tpr =
       0              5/7     

fpr =
       0              1/3     

th =
       1              0

[balandong]

Hi SS, Thanks for the quick reply. Two follow-up questions 1) Do you know what is Threshold, in the above, you abbreviate it as th. 2) May I know what other Function you use as alternative to ROC, I may want to give it a try.

[Star Strider]

As always, my pleasure.

I cannot figure out from the documentation (and some online and personal textbook searches) what the significance of the ‘threshold’ is, or how it is calculated.

I usually use the crosstab (and sometimes the confusionmat) functions, since I am usually interested in the chi-square statistic, then do my own calculations on the matrix itself.

You might also be interested in the plotconfusion function if you want a graphical representation. (Note that the matrices produced by confusionmat and plotconfusion are transposes of each other.)

[balandong]

HI SS

1) Hopefully the documentation will be improve later.

2) Thanks for introducing the 3 new functions. Really appreciate it as I dont need to manually construct the Conf Mat as i did in the first thread.

Thanks for the advice and for your contribution to the MATLAB community

[Star Strider]

I very much appreciate your acknowledgement! I do my best!

As always, my pleasure.

[balandong]

Hi SS

I just realize something

From the documentation, there are going to be two TPR and two FPR if we have two predicted vector. For example,

actual      = [1;1;1;1;1;1;1;0;0;0]; % targets
P1=[1 1 1 1 1 0 0 1 0 0];
P2=[1 0 1 0 1 0 0 1 1 0];
predicted (:,1:2)   = [P1' P2']; % outputs
[tpr,fpr,th] = roc(actual',predicted');

However, executing the code above only give the TPR and FPR for P1.

So, to mitigate this problem, a FOR loop is introduce. However, I dont know whether this is the correct way of using ROC FUNC.

actual      = [1;1;1;1;1;1;1;0;0;0]; % targets
P1=[1 1 1 1 1 0 0 1 0 0];
P2=[1 0 1 0 1 0 0 1 1 0];
predicted (:,1:2)   = [P1' P2']; % outputT
c=1;
for i=1:2 
[tpr,fpr,th] = roc(actual',predicted(:,i)');
r_tpr(c)=tpr (2);
r_fpr(c)=fpr (2);
c=c+1;
end

[Star Strider]

I am having a very difficult time reconciling the information you are giving the roc function with the documentation for it. The ‘actual’ and ‘predicted’ matrices need to be the same size. This may be the reason you are having problems with it.

Please run the ‘iris data’ demo code in the roc dicumentation to see what the function expects.

[balandong]

Hi SS, Thanks for the effort and time taken to understand about the ROC function.

1) In your comment where you wrote " matrices need to be the same size", you are implying about something like

actualDouble = repmat (actual, 2);

However, executing the full code as below, still produce single value of TPR and FPR

actual      = [1;1;1;1;1;1;1;0;0;0]; % targets
actualDouble = repmat (actual, 2);
P1=[1 1 1 1 1 0 0 1 0 0];
P2=[1 0 1 0 1 0 0 1 1 0];
predicted (:,1:2)   = [P1' P2']; % outputs
[tpr,fpr,th] = roc(actual', predicted');

2) ‘iris data’

I had run and go through the output variable one by one. However, it is very hard to understand what exactly the output are. Compared with the example provided above where

actual      = [1;1;1;1;1;1;1;0;0;0]; % targets
predicted   = [1;1;1;1;1;0;0;1;0;0]; % outputs.

the value generated from the iris-data is not easily comprehend. For example

irisTargets = [1,1,1 ; ...
               0,0,0 ; ...
               0,0,0];
irisOutputs = [0.999,0.999,0.999;         ...  %  value generated from the Neural Net training.
               0.0002,0.0007,0.0002;      ...
               2.93e-07,1.010e-06,3.97e-07];

Yet, I think doing as below should agree with the iris-data example and thus answering your comment "T he ‘actual’ and ‘predicted’ matrices need to be the same size. "

actualDouble = repmat (actual, 2);

I will do some homework and update the finding here.