Leave-One-Out with ROC

Hi smart guys,

May I ask how to create ROC for the leave-one-out cross-validation?

I create partition of the data as:

    cvPartition = cvpartition(dataSize, 'leaveout');

However, when I take

    model = ClassificationDiscriminant.fit(X, Y, 'DiscrimType', 'linear');

    [predictL, predictS] = model.predict(X_Test);
    performanceVec = classperf(Y_Teset, predictL);
    [FPR, TPR ,~, AUC] = perfcurve(Y_Test, predictS(:,2), 1);

in which, `X, Y, X_Test, Y_Test` are obtained from `cvPartition`. Then there are errors:

    Error using classperf (line 205)
    Ground truth must have at least two classes.

    Error using perfcurve (line 368)
    Less than two classes are found in the array of true class labels.

Anyone can give an example of using leave one out cross validation and ROC analysis using Matlab? Thanks a lot.

A.

Answer 1

[Ilya]

Here is an example for cross-validating a discriminant model: http://www.mathworks.com/help/stats/discriminant-analysis.html#bs2r8ue. Here is a description of the crossval method: http://www.mathworks.com/help/stats/classificationdiscriminant.crossval.html. You can just replace 'kfold',5 with 'leaveout','on' in that example.