Issues training Mask RCNN on custom data

Hello,

I have been following the Mask RCNN tutorial provided by Matlab, however I have ran into some issues trying to train the model on my own dataset. I have labeled my data using the ImageLabeler app, and then followed the example Export Ground Truth Object to Custom and COCO JSON Files to put the labels in the proper COCO format. I am able to read the new JSON file into the workspace just fine, but it seems to always have issues when I try to unpack the annotations to .mat files using the helper.unpackAnnotations() command. It does not give me an error, but it simply does not unpack the annotations into the 'annotations_unpacked' directory like it says it should.

I have found another post on github discussing the issue but I have not found any suggestions on exactly how to fix this. It seems that my primary problem is that my images only contain a single class per image (but often times multiple instances of that class). From the github post, I see the issue stems from there only being a single class labeled in my images instead of multiple classes like from the example.

I would like to know whether I can edit the unpackAnnotations.m file or the CocoApi.m file to properly unpack the annotations? If so, I would also like to know what needs to be edited in order to achieve this.

I greatly appreciate any help regarding this issue.

Thanks,

Connor

Answer 1

[Aneela]

Hi @Connor Seavers

The helper function, "unpackAnnotations", only retrieve annotations of those images that contains all the training categories. Thus, relevant training data from images that have single category will be lost.

You can modify the code to retrieve image IDs for each category separately and then combine the results. Refer to the below code snippet for retrieving the image IDs for each category individually:

combined_img_ids = [];
% Loop through each category
for i = 1:length(trainCats)
    cat_id = coco.getCatIds('catNms', trainCats{i});
    img_ids = coco.getImgIds('catIds', cat_id);
    % Combine image IDs using union
    combined_img_ids = union(combined_img_ids, img_ids);
end

[Connor Seavers]

Hello @Aneela,

Thank you so much for the response. I was able to use your code snippet to alter the unpackAnnotations.m file and successfully unpacked my annotations in the directory for unpacked annotations. I am not exactly sure if it fully worked as I had 38 images labeled but it only created 9 individual .mat files in the unpacked annotations directory, but regardless it certainly worked.

I am now running into issues using the detectMaskRCNN function in the subsequent lines of code in the Mask RCNN example. I assume that it is due to not having the 'background' class label in my images. I shall make another post if I decide to try and overcome that issue as well. At this point, I am considering other networks since my project is time sensetive.

Once again, I greatly appreciate your prompt and insightful response to the matter.

Best