Detecting circular shaped fruit in 2D images using circular Hough transform or alternative algorithm

I am trying to detect circular fruits using the circular Hough transform after applying edge detection. However, the circular Hough transform parameters seem to be very sensitive, i.e. the ''Sensitivity'' and ''EdgeTreshold''. A small change in one of these parameters has huge impact on the amount of circles that are detected. How can I avoid this or is there a better algorithm to detect the circular shaped fruits? Moreover, I need to use the Rmax of 85 for optimal results, while the absolute maximum Rmax is 55, after measuring the distance with imdistline.

I am using now the following parameters:

[centers,radii, metric] = imfindcircles(BW_Canny,[30 85],'Sensitivity',0.95,'EdgeThreshold',0.15);
viscircles(centers, radii,'EdgeColor','b');

I have added the original RGB image, the images after color thresholding and edge detection, and beneath my own results.

Answer 1

[Image Analyst]

You don't need to find circular shapes. From your other posts, you're trying to find oranges on a tree. The oranges are almost always overlapping with other oranges so the combined blob is not round, or don't have a circular shape because there is shine on the rind, or leaves are obscuring the shape, making it not round. If you limit your segmentation to only round blobs, you're going to miss the majority of blobs.

That said, there is a 'circularity' property in regionprops that gives you the circularity of blobs. You can also use bwpropfilt() to remove blobs outside the acceptable range of circularity. Again, I recommend not doing that.

[S.]

Thanks for your response. Actually I am not missing the majority of the blobs. The circular Hough transform gives a quite good approximation in my opinion even if the fruit is occluded by another fruit or a leaf, but tuning is hard.

Moreover, you mention 'circularity' property in regionprops and then not recommending to do that. Or is the comment refering to not use bwpropfilt()? What makes 'circularity' property better compared to the circular Hough transform.

Can you also have a look at my other topic of evaluating the performance (your a knowledgable person)? I am quite stuck there and don't know what to do.

[Image Analyst]

I don't know what you mean by "tuning". What are your plans if there is a semicircle there? Depending on the settings to imfindcircle(), it might be found or not found.

[S.]

By ''tuning'' I mean that imfindcircle() has the parameters ''Sensitivity'' and ''EdgeTreshold'', which can be ''tuned'' to get better results.

Well, if there are semicircles, I think imfindcircle() is still able to guess, based on the edge of the semicircle that is a full circle and it will detect the circle. What would you suggest then as an alternative for better results? The 'circularity' property in regionprops?

[Image Analyst]

My code gives all orange blobs regardless of what shape they are. It seems weird to call a function that finds circles and then complain that it needs to be tuned to find non-circular shapes. I'm not sure I understand your rationale for using a circle-finding function. Sure the oranges are round in reality, but they don't always appear round in the image so we can't look only for round blobs or else we'll miss some of the oranges. Why not just use color segmentation to find the color(s) you want regardless of their shape? You can then filter them based on size (area), circularity, or whatever you want, for example to get rid of blobs that you consider too small to be of concern with the bwareaopen() function or bwareafilt() function.

[S.]

Which code are you referring to? Do you mean the 'circularity' property in regionprops? The reason I use a circle-finding function is in that way the clearly observed oranges are recognized. Moreover, circle-finding function gives the centerpoint and radius as output, which is very convenient.

I used color segmentation and I used the functions bwareaopen() and bwarefilt() to get the color(s) I want regardless of their shape and remove most noise. The results of the three images, you can see above. Do you think with regionprops I will get better results, as also recognizes not-round blobs which might be oranges?

About: Sure the oranges are round in reality, but they don't always appear round in the image so we can't look only for round blobs or else we'll miss some of the oranges --> Of course, but how will the centerpoint of this half orange or not-circular shape be determined by regionprops? Moreover, not all orange blobs are oranges, but could also be the environment. If a orange is occluded by a leaf, regionprops will recognize it as two different blobs, so two oranges and not one, while in the first image in my post the two seperated blobs are seen as one orange, which is correct as it was occluded by a leaf. So the counting will be different. If you please can explain if and how regionprops takes the two bold mentioned problems into account.

[Image Analyst]

regionprops() can return the centroid of the blob. It can also return the circularity which is (4*pi*area)/(perimeter squared), which is 1 for perfect circles and lower numbers for non-circles.

There is not a reliable way to tell if two orange blobs separated by green are the result of a single orange obscured by a leaf, or two oranges. That's why I think counting in this situation is never going to be perfect and why I recommend you just go with are fraction, or maybe a model composed of area fraction and blob count to get a model. Train the model by going out and actually physically counting the oranges and then make up a model that gives an estimate of the number of oranges there based on the area fraction and blob count. This will tell you when there are enough oranges to do your harvest.

[S.]

I tried regionprops(). However, it gives unfortunately even worser results (completely wrong answers).

stats = regionprops(BW_Canny,'Centroid', 'Area','Perimeter','Circularity');
Threshold = 0.7;

for i = 1:length(stats)
    if (stats(i).Circularity > Threshold && stats(i).Circularity <= 1)
    Detected_center = stats(i).Centroid;
    hold on
    scatter(Detected_center(1),Detected_center(2),'red','+','LineWidth',2)
    end
end

The blue crosses are the ''true'' centers and the ''red'' centers are the predicted centers.

I have also added the figures in the attachement, to see if you know what went wrong.

[Image Analyst]

You should not use regionprops on an edge image. I don't know why you did Canny on your original mask. I cannot say that the centroid of a perimeter image would be the same place as if the outline were filled solid. It probably won't be. Just use the original binary image if you have it. If you only have the original image, then call imfill(cannyImage, 'holes') to fill in those blobs, then call regionprops().

[S.]

I did Canny on my original mask, as it performs better with the Circular Hough Transform if you only see the edges. However, after using the original binary image, regionprops() indeed performs better than initially (i.e.with Canny on the original mask). Do you know why?

However, I see also the following differences:

When I use regionprops():

• Every blob is recognized as an orange and therefore it performs good when an orange is for a large part occluded by a leaf/trunk. However, if an orange is splitted in two different blobs, it is also recognized as two different oranges.
• Two overlaying blobs (two overlaying oranges) are recognized as one area and therefore one orange.

When I use the CHT:

• Two overlaying blobs (two overlaying oranges) are recognized as two different oranges, due to two different edge patterns.
• If an orange is splitted in two different blobs, it is recognized as one orange due to the circular shape.
• It doesn't perform good when an orange is for a large part occluded by a leaf/trunk due to the weird shape.

However, as I mentioned earlier, the CHT is very sensitive, changing the ''Sensitivity'' or the ''Edge threshold'' parameter slightly changes the performance drastically. Moreover, I still don't know why I need to set the minimum radius to 20 and the maximum radius to 60 for best results, while the maximum radius in reality is 50.

I don't know whether there are other techniques than regionprops() and CHT that I could try.

''Train the model by going out and actually physically counting the oranges and then make up a model that gives an estimate of the number of oranges there based on the area fraction and blob count." --> The goal is to detect the center of an orange as accurate as possible to be clear. I did indeed physically count the oranges to determine the true oranges. Do you think its necessary to make a train and test set? Maybe I can make a train set to train the algorithm for different settings and see which settings give the best performance.

[Image Analyst]

I can see that we're never going to see eye-to-eye on this. For another example you want to measure things that I say don't matter, like the location of the center of an orange, which of course varies tremendously depending on where the photographer was standing and how he was aiming the camera, and literally how the wind was blowing. Things like finding only round blobs, counting blobs, etc. will always be problematic and that's why, based on my 40+ years of image processing, you should look for other things like area fraction for determining when to harvest your crop or how much the oranges have grown.

So good luck to you with your approach.