detect the longest edge and calculate the slope

I have a RGB image which is an output from another matlab script that deletes the green chroma key background. https://picasaweb.google.com/117741942742967044268/MatlabTests#5720160787578435266

As you can see, it is slightly tilted. I have hundreds of such images. What I'm trying to do is, find the angle of tilt (which is absolutely random but a small value) and rotate the image so the bottom edge is perfectly horizontal.

Here is what i have tried so far. Applied canny edge detection which gives something like this - https://picasaweb.google.com/117741942742967044268/MatlabTests#5720162301142246066

Now I want to detect the longest line and find its angle with horizontal axis. If it is a large value, i can assume that the lien doesn't belong to the edge of the box. but if it is a fair value (<20) then I can assume that the box is titled by that angle. I can rotate the image by that value then.

Is this a good way to resolve this problem? Thanks in advance.

Answer 1

[Image Analyst]

Atul: Copy and paste this demo. I just thresholded the red channel, measured the angle, and called imrotate to straighten it.

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
imtool close all;  % Close all imtool figures.
clear;  % Erase all existing variables.
format compact;
workspace;  % Make sure the workspace panel is showing.
fontSize = 20;

% Change the current folder to the folder of this m-file.
if(~isdeployed)
  cd(fileparts(which(mfilename)));
end
  
% Check that user has the Image Processing Toolbox installed.
hasIPT = license('test', 'image_toolbox');
if ~hasIPT
  % User does not have the toolbox installed.
  message = sprintf('Sorry, but you do not seem to have the Image Processing Toolbox.\nDo you want to try to continue anyway?');
  reply = questdlg(message, 'Toolbox missing', 'Yes', 'No', 'Yes');
  if strcmpi(reply, 'No')
    % User said No, so exit.
    return;
  end
end

% Read in a standard MATLAB color demo image.
folder = 'C:\Users\user\Documents\Temporary';
baseFileName = 'cag00089_x0_y270.jpg';
fullFileName = fullfile(folder, baseFileName);
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
if ~exist(fullFileName, 'file')
  % Didn't find it there.  Check the search path for it.
  fullFileName = baseFileName; % No path this time.
  if ~exist(fullFileName, 'file')
    % Still didn't find it.  Alert user.
    errorMessage = sprintf('Error: %s does not exist.', fullFileName);
    uiwait(warndlg(errorMessage));
    return;
  end
end
rgbImage = imread(fullFileName);
% Get the dimensions of the image.  numberOfColorBands should be = 3.
[rows columns numberOfColorBands] = size(rgbImage);
% Display the original color image.
subplot(3, 2, 1);
imshow(rgbImage, []);
title('Original Color Image', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]); % Maximize figure.

% Extract the individual red, green, and blue color channels.
redChannel = rgbImage(:, :, 1);
% greenChannel = rgbImage(:, :, 2);
% blueChannel = rgbImage(:, :, 3);
% Display the individual color channels.
subplot(3, 2, 2);
imshow(redChannel, []);
title('Red Channel', 'FontSize', fontSize);

% Threshold the red channel.
binaryImage = redChannel > 117;
subplot(3, 2, 3);
imshow(binaryImage, []);
title('Binarized Red Channel', 'FontSize', fontSize);

% Get the convex hull to fill in all the nooks and crannies and holes.
chImage = bwconvhull(binaryImage);
subplot(3, 2, 4);
imshow(chImage, []);
title('Convex Hull', 'FontSize', fontSize);

% Label it
cc = bwconncomp(chImage);

% Measure the angle (orientation).
measurements = regionprops(cc, 'orientation');

% Tell user what the angle is.
angle = measurements(1).Orientation;
message = sprintf('The angle is %.3f degrees\nNow we will rotate original image by %.3f degrees', ...
  angle, -angle);
uiwait(msgbox(message));

% Rotate it the image by minus that angle to straighten it.
rotatedImage = imrotate(rgbImage, -angle);
% Display final result.
subplot(3, 2, 5);
imshow(rotatedImage, []);
caption = sprintf('Original Image Rotated by %.3f degrees', -angle);
title(caption, 'FontSize', fontSize);

msgbox('Done with demo');

[Atul Morey]

Image Analyst. you are the man! this worked like a charm. Thanks so much. Now its time to convert it into a batch script and run on hundreds of images I have. I will post my feedback here in couple of days. Thanks so much.

[Image Analyst]

Well, I do this for a living. You may have to make adjustments to the thresholds for other images. It may be different depending on the image.

Answer 2

[Jeff E]

How about answering it in a slightly different way? Consider using regionprops to fit an ellipse to your box region, and use the orientation of the major axis to obtain the angle of tilt? To get the appropriate result from regionprops, you'll want to invert your binary mask (make black pixels white, and vice versa), and fill in holes using imfill.

[Atul Morey]

Jeff EA, thanks for your smart feedback. This almost worked. But the issue is noise. Here is how imfill result looks like

https://picasaweb.google.com/117741942742967044268/MatlabTests#5720196918979957474

and i am not sure how to get a crisp bounding values for the box. Any suggestions?

[Atul Morey]

Hey Jeff EA,

i actually found a way to get the angle of tilt from your method but unfortunately it works only with the image i attached earlier. This logic fails on some of the other image. The most prominent issue is imfill doesn't provide a perfect box region. The graphics inside the box cause some black areas which imfill fails to handle.

So this problem goes all the way back. Is there a way to get crisp bounding box without human interaction? If I can get a perfect white rectangle, the ellipse method will able to handle any case.

Please advise. Thanks so much.

[Jeff E]

From the images you posted, it really looks like you are not inverting the mask before calling imfill. My apologies if that is not the case, but you might try posting what you're doing to make sure. It should look something like the following:

BWimg = edge(img);

BWimg_inv = ~BWimg;

BWimg_inv_fill = imfill(BWimg_inv, 'holes');

stats = regionprops(BWimg_inv_fill, 'Orientation');

rotated_img = imrotate(img, -stats.Orientation);

Failing that, if the corners of the box are robustly detected, you can use bwconvhull to get a good outline. The only thing is this is a relatively new function introduced in R2011b, I think. Older versions of matlab won't have this.

If your background is consistently green,

[Atul Morey]

Jeff EA, thanks so much. It worked! I was not aware of bwconvhull. It solved the problem. Now I'm gonna try this to every image I have and see if it works in every situation. I really appreciate your time and efforts. Thanks again.

Answer 3

[Geoff]

What about using the Hough transform to detect lines in your edge image? You could search the resulting Hough-space for lines within an acceptable orientation and then choose the line with the strongest signal. This could be checked against potential vertical lines (offset by 90 degrees).

[Atul Morey]

Geoff, I tried this option as well. and it worked too. It was nice to know an alternative to solve a problem. Now I will run it on every image ans observe the results. Thanks so much for your time and efforts.