unit test in image segmentation

3 views (last 30 days)
Maia
Maia on 27 Feb 2018
Commented: Maia on 16 Apr 2018
Hi,
I am working on medical image segmentation (images acquired with several machines, several patients) and I have questions on how to do unit testing in Matlab: So far, I found this https://www.mathworks.com/help/matlab/class-based-unit-tests.html
but it is very general and i'd like to understand the procedure for image segmentation.
I searched for examples and I found this response:
who says that: "TDD in image processing only makes sense for deterministic problems like:
image arithmetic histogram generation and so on.. However TDD is not suitable for feature extraction algorithms like:
edge detection segmentation corner detection ... since no algorithm can solve this kind of problems for all images perfectly."
Do you agree with the above statement ?
I'd like to know how do you perform unit testing or TDD in your image segmentation.
Thanks a lot, Maia

Accepted Answer

Ashish Uthama
Ashish Uthama on 1 Mar 2018
Edited: Ashish Uthama on 1 Mar 2018
Maia, The stackoverflow link you found has a lot of relevant detail, especially the one from user "jilles de wit".
I would create some manual segmentation (for e.g using the image segmenter app) and then use that mask as the 'golden data' to compare against whatever new automated approach I am trying to develop. You could use one of these functions: jaccard, bfscore or dice metric to compare the results of the ground truth/golden data against the algorithm you are developing with some allowance/tolerance to account for variability.
  2 Comments
Maia
Maia on 2 Mar 2018
Thank you very much.
Maia
Maia on 16 Apr 2018
Hi,
Thank you again for your answers.
I'd like to know how would you perform unit tests with several image samples, i.e., testing a single function on several images, which have different expected values. I tried these codes, but I am not sure if this is the way to go (especially the for ... end). Also, I want to track/know for which image the test didn't pass, but verifyEqual does not return a value to test that.
So far, I created a subfolder for each image test where I put data like expected value for that image.
===
for iCase = 1:length(testCase.TestData.data)
data = testCase.TestData.data(iCase);
actSol = functionToBeTested(); expSol = data.value; verifyEqual(testCase, actSol, expSol);
end

Sign in to comment.

More Answers (1)

CJ
CJ on 1 Mar 2018
I second Ashish's comment. It sounds to me that the main concern here is the ever-changing test set and the non-deterministic nature of the algorithm.
TDD and unit test can mean a lot of things, but in general we want things to be fixed and deterministic so we can test them. In that case one needs to try to eliminate variables from the system. The ground truth approach suggested by Ashish is one way of doing that. Another example would be incorporating statistics into your expected results. Let's say your algorithm may produce variable results on a single/small set of images, then perhaps increase the sample size and obtain a fixed lower/higher bound according to your requirement.

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!