Clear Filters
Clear Filters

How can I compute statistical significance with cross-validation?

2 views (last 30 days)
I've done nested cross-validation with 10 outer runs and 10 inner runs to evaluate different classifiers. In the end I average the accuracies of the 10 outer cross-validation runs to get a final accuracy for one classifier.
Now, I would like to do some statistical test among the classifiers to see if one classifier performs better than another.
How can this be done in Matlab (or in general) based on the cross-validation accuracies?
  1 Comment
Esmeralda Ruiz Pujadas
Esmeralda Ruiz Pujadas on 11 Aug 2022
Edited: Esmeralda Ruiz Pujadas on 11 Aug 2022
Hello, I am having the same issue. It seems no solution. According to a paper, the problem of nested cross-validation, you cannot apply statistical measurements afterwards because you have different parameters in each iteration. Were you able to implement an alternative?.

Sign in to comment.

Answers (0)


Find more on Statistics and Machine Learning Toolbox in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!