How can I compute statistical significance with cross-validation?
1 view (last 30 days)
Show older comments
Hello
I've done nested cross-validation with 10 outer runs and 10 inner runs to evaluate different classifiers. In the end I average the accuracies of the 10 outer cross-validation runs to get a final accuracy for one classifier.
Now, I would like to do some statistical test among the classifiers to see if one classifier performs better than another.
How can this be done in Matlab (or in general) based on the cross-validation accuracies?
1 Comment
Esmeralda Ruiz Pujadas
on 11 Aug 2022
Edited: Esmeralda Ruiz Pujadas
on 11 Aug 2022
Hello, I am having the same issue. It seems no solution. According to a paper, the problem of nested cross-validation, you cannot apply statistical measurements afterwards because you have different parameters in each iteration. Were you able to implement an alternative?.
Answers (0)
See Also
Categories
Find more on Statistics and Machine Learning Toolbox in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!