Automated image labelling with Attributes
4 views (last 30 days)
Show older comments
Dear all,
I'm trying to automate image labelling. I went throught several available examples (Examples), and I think I'm nearly done, but I can't automate the attributes.
My problem is the following, I want to detect goats in images and then, classify the goats depending on their color (white, red, brown and black).
I created one rectangle label, with one attribute, the color, taking value into {'white','red','brown','black'}
I manually labelled more than 3000 images and know I have trained CNNs able to do, approximately, the job for me. I want to label more images to possibly improve my CNN accuracy, but it's a shame I have to label new images manually, when I have a method that is correct at 90%
Using the example liste above, for automatic car and distance labelling, I was able to design a code that automatically detect the goats. To be more precise, if I comment everything that is related to the attributes, my code run and I have bounding boxes around the detected animals.
But when the codes about the attributes is un-comment, I have error message: "Expected automated labels to be a valid label from among the existing label definitions".
I can understand the error message, but I can't find why the definition of my label is not correct in my code ...
I hope somebody will have any idea. Best would be that MAtlab provides more examples on automated labelling :)
Below is my code:
% This class defines a template for creating a custom label automation
% algorithm, to be used in labeling apps (Image Labeler, Video Labeler and
% Ground Truth Labeler). To access help for this class, enter the following
% at the MATLAB command prompt:
%
% >> doc vision.labeler.AutomationAlgorithm
%
% For reference, see the following AutomationAlgorithm class:
%
% >> edit vision.labeler.PeopleDetectorACF
%
%
% To use this algorithm from within the Image Labeler, Video Labeler or
% Ground Truth Labeler App, follow the steps outlined below and complete
% the class definition. Then, save this file as follows.
%
% Create a +vision/+labeler folder within a folder that is already
% on the MATLAB path. For example, if the folder /local/MyProject is on
% the MATLAB path, then create a +vision/+labeler folder hierarchy as
% follows:
%
% projectFolder = fullfile('local','MyProject');
% automationFolder = fullfile('+vision','+labeler');
%
% mkdir(projectFolder, automationFolder)
%
% Saving the file to the package directory is required to use your custom
% algorithm from within the labeling apps. You can add a folder to the
% path using the ADDPATH function.
%
% Save your algorithm class in the folder created in step 1. Refresh the
% algorithm list from within the app to start using your custom algorithm.
classdef CabrisCouleurs < vision.labeler.AutomationAlgorithm
%----------------------------------------------------------------------
% Step 1: Define required properties describing the algorithm. This
% includes Name, Description and UserDirections.
properties(Constant)
% Name: Give a name for your algorithm.
Name = 'Detection des cabris et de leurs couleurs';
% Description: Provide a one-line description for your algorithm.
Description = 'Use it to detect cabris with Brinno time lapse cameras';
% UserDirections: Provide a set of directions that are displayed
% when this algorithm is invoked. The directions
% are to be provided as a cell array of character
% vectors, with each element of the cell array
% representing a step in the list of directions.
UserDirections = {...
['Automation algorithms are a way to automate manual labeling ' ...
'tasks. This AutomationAlgorithm is a template for creating ' ...
'user-defined automation algorithms. Below are typical steps' ...
'involved in running an automation algorithm.'], ...
['Run: Press RUN to run the automation algorithm. '], ...
['Review and Modify: Review automated labels over the interval ', ...
'using playback controls. Modify/delete/add ROIs that were not ' ...
'satisfactorily automated at this stage. If the results are ' ...
'satisfactory, click Accept to accept the automated labels.'], ...
['Change Settings and Rerun: If automated results are not ' ...
'satisfactory, you can try to re-run the algorithm with ' ...
'different settings. In order to do so, click Undo Run to undo ' ...
'current automation run, click Settings and make changes to ' ...
'Settings, and press Run again.'], ...
['Accept/Cancel: If results of automation are satisfactory, ' ...
'click Accept to accept all automated labels and return to ' ...
'manual labeling. If results of automation are not ' ...
'satisfactory, click Cancel to return to manual labeling ' ...
'without saving automated labels.']};
end
%---------------------------------------------------------------------
% Step 2: Define properties to be used during the algorithm. These are
% user-defined properties that can be defined to manage algorithm
% execution.
properties
%% Couleur du cabris
couleur
%% Detector
detector
%% Couleur classification
net
end
properties (Constant, Access = private)
% Flag to enable Distance attribute estimation automation
AutomatecouleurAttribute = true;
% Supported Distance attribute name.
% The label must have an attribute with the name specified.
SupportedcouleurAttribName = 'couleur';
end
properties (Access = private)
% Actual attribute name for distance
couleurAttributeName;
% Flag to check if attribute specified is a valid distance
% attribute
HasValidcouleurAttribute = false;
end
%----------------------------------------------------------------------
% Step 3: Define methods used for setting up the algorithm.
methods
% a) Use the checkLabelDefinition method to specify whether a label
% definition is valid for the algorithm. This method is invoked
% on each ROI and Scene label definition to determine whether it
% is valid for the specified algorithm.
%
% For more help,
% >> doc vision.labeler.AutomationAlgorithm.checkLabelDefinition
%
function isValid = checkLabelDefinition(~, labelDef)
isValid = labelDef.Type == labelType.Rectangle;
end
% b) Use the checkSetup method to specify whether the algorithm is
% ready and all required set up is complete. If your algorithm
% requires no setup from the user, remove this method.
%
% For more help,
% >> doc vision.labeler.AutomationAlgorithm.checkSetup
%
function isReady = checkSetup(algObj)
isReady = ~isempty(algObj.SelectedLabelDefinitions);
end
% c) Optionally, specify what settings the algorithm requires by
% implementing the settingsDialog method. This method is invoked
% when the user clicks the Settings button. If your algorithm
% requires no settings, remove this method.
%
% For more help,
% >> doc vision.labeler.AutomationAlgorithm.settingsDialog
%
function settingsDialog(algObj)
disp('Executing settingsDialog')
%--------------------------------------------------------------
% Place your code here
%--------------------------------------------------------------
end
end
%----------------------------------------------------------------------
% Step 4: Specify algorithm execution. This controls what happens when
% the user presses RUN. Algorithm execution proceeds by first
% executing initialize on the first frame, followed by run on
% every frame, and terminate on the last frame.
methods
% a) Specify the initialize method to initialize the state of your
% algorithm. If your algorithm requires no initialization,
% remove this method.
%
% For more help,
% >> doc vision.labeler.AutomationAlgorithm.initialize
%
function initialize(algObj, ~)
disp('Executing initialize on the first image frame')
% Couleur du cabris
algObj.couleur = algObj.SelectedLabelDefinitions.Name;
% Detector
load Detector_27_08.mat
algObj.detector = detectorc;
% Couleur classification
load ResNet_September2021.mat
algObj.net = net;
% Initialize parameters to compute color
if algObj.AutomatecouleurAttribute
initializeAttributeParams(algObj);
end
%-------------------------------------------------------------
% Place your code here
%--------------------------------------------------------------
end
function initializeAttributeParams(algObj)
% Initialize properties relevant to attribute automation.
% The label must have an attribute with name Distance and type
% Numeric Value.
hasAttribute = isfield(algObj.ValidLabelDefinitions, 'Attributes') && ...
isstruct(algObj.ValidLabelDefinitions.Attributes);
if hasAttribute
attributeNames = fieldnames(algObj.ValidLabelDefinitions.Attributes);
idx = find(contains(attributeNames, algObj.SupportedcouleurAttribName));
if ~isempty(idx)
algObj.couleurAttributeName = attributeNames{idx};
algObj.HasValidcouleurAttribute = validatecouleurType(algObj);
end
end
end
function tf = validatecouleurType(algObj)
% Validate the attribute type.
tf = isfield(algObj.ValidLabelDefinitions.Attributes, algObj.couleurAttributeName);% && ...
%isfield(algObj.ValidLabelDefinitions.Attributes.(algObj.couleurAttributeName), 'DefaultValue') && ...
%isnumeric(algObj.ValidLabelDefinitions.Attributes.(algObj.couleurAttributeName).DefaultValue);
end
% b) Specify the run method to process an image frame and execute
% the algorithm. Algorithm execution begins at the first image
% frame and is invoked on all image frames selected for
% automation. Algorithm execution can produce a set of labels
% which are to be returned in autoLabels.
%
% For more help,
% >> doc vision.labeler.AutomationAlgorithm.run
%
function autoLabels = run(algObj, I)
img = I;
disp('Executing run on image frame')
ordre = cell(4,2);
ordre{1,1} = 1:4;
ordre{1,2} = transpose(1:4);
for n = 2:4
col = [];
C = nchoosek(1:4,n);
for i = 1:size(C,1)
col = [col ; perms(C(i,:))];
end
ordre{n,2} = col;
lig = repmat(1:n,[size(col,1) , 1]);
ind = zeros(size(lig,1),n);
for i = 1:n
ind(:,i) = sub2ind([n , 4],lig(:,i),col(:,i));
end
ordre{n,1} = ind;
end
autoLabels = [];
[bbox , probad , class] = algObj.detector.detect(img);
[bbox , probad ] = selectStrongestBbox(bbox,probad,'NumStrongest',4);
% bbox = algObj.detector.detect(I);
if ~isempty(probad)
proba = [];
for j = 1:numel(probad)
[~ , p] = algObj.net.classify(imresize(imcrop(img,bbox(j,:)),[224 224]));
% proba = [proba ; probad(j).*(p')];
proba = [proba ; p];
end
n = numel(probad);
ind = ordre{n,1};
c = ordre{n,2};
s = 0;
for j = 1:n
s = s + proba(j,c(:,j))';
end
[~ , m] = max(s);
couleurnum = c(m,:);
autoLabels.Name = 'goat';
autoLabels.Type = labelType.Rectangle;
autoLabels.Position = bbox;
if (algObj.AutomatecouleurAttribute && algObj.HasValidcouleurAttribute)
attribName = algObj.couleurAttributeName;
% Attribute value is of type 'Numeric Value'
for z = 1:size(bbox,1)
autoLabels(z).(attribName) = algObj.net.Layers(end).Classes(couleurnum(z));
end
end
end
end
% c) Specify the terminate method to clean up state of the executed
% algorithm. If your method requires no clean up, remove this
% method.
%
% For more help,
% >> doc vision.labeler.AutomationAlgorithm.terminate
%
function terminate(algObj)
disp('Executing terminate')
%--------------------------------------------------------------
% Place your code here
%--------------------------------------------------------------
end
end
end
0 Comments
Answers (0)
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!