How to perform image registration on non-rectangular image

I'm interesting in peforming image registration using imregtform on a non-rectangular region of interest that would be the "moving" image. Is there any way to do this? I initially tried to turn all the pixels within the "moving" image matrix that I don't want to affect registration to NaNs, but the imregtform function won't accept that input.

[Biral Pradhan]

I understand, you have a non-rectangular region of interest in a rectangular image, and you want to use "imregtform" it. Could you try converting all the other pixels to "black" colour instead of NaN?

[Jeremy Eekhoff]

That would not work because the black pixels would still affect registration. My images do not have a black background.

Answer 1

[Matt J]

If you have the Optimization Toolbox, you could use lsqcurvefit (don't be misled by the name) to minimize your own reigstration cost function, one which accepts an ROI mask.

[Jeremy Eekhoff]

Thanks for your comment. I've been trying to implement something along these lines, with the goal of maximizing the mutual information between images similar to imregtform behaves. I've been trying to use fmincon and globalsearch with different settings but so far it's been failing to consistently find the global solution to the problem. Any advice on how to set up an algorithm that will behave similarly to the imregtform regular step gradient descent? I'm not sure how to implement this.

[Matt J]

Are you implementing a multi-resolution pyramid?

[Jeremy Eekhoff]

No, nor am I familiar with what that is.

[Matt J]

What is usually done to improve robustness is that you first smooth and downsample the images and perform an initial registration. Then you reduce the smoothing and repeat the registration with the previous registration result as your initial guess. This is repeated until the registration is done at full resolution.

[Jeremy Eekhoff]

Ok thanks, I'll try that out.

Answer 2

[Image Analyst]

Is it rotating and scaling while it translates? If not, you could use normalized cross correlation. See attached demo.

Or try one of the panoramic stitching algorithms. They find matching regions, which is necessary to construct the overall image.

Panorama and Image Registration - MATLAB & Simulink

Feature Based Panoramic Image Stitching - MATLAB & Simulink

Panoramic photograph stitching

Cris’ Image Analysis Blog | Panoramic photograph stitching

[Matt J]

Is it rotating and scaling while it translates?

For that matter, is the registration rigid or deformable?

[Jeremy Eekhoff]

Right, sorry, I should have included that in the initial question. The registration includes any sort of affine deformation, so correlation isn't an option. My application is registering a region of the initial frame to each successive frame of a video to track deformation and calculate mechanical strain in a material during a tensile test. I haven't had any success using feature-based registration with my images in the past - the best results I've had were using imregtform to maximize the mutual information between images. It works great when I have a rectangular region of the image to track, but when the region of the material I want to track is a different shape then I can't use imregtform.

[Image Analyst]

You might try SIFT or SURF

https://www.mathworks.com/products/computer-vision.html#feature-detection

and point set matching on the results.

https://en.wikipedia.org/wiki/Point-set_registration

I think the Computer Vision Toolbox has some points set matching functions with the LIDAR capability.

Good luck - it's not an easy problem.

[Jeremy Eekhoff]

Hey - quick following up on this. I was able to somewhat get this working using fmincon, but the results weren't quite as good as I get using imregtform and I would often have to change the optimizer options for different sets of images. I had the idea to try to go back and use imregtform and set the pixel intensities of all pixels outside of the polygonal region of interest to uniformly distributed random values, the idea being that the blocks of random pixels would have zero mutual information with the fixed image regardless of where it ended up and therefore wouldn't affect the registration. This ended up working better than my attemps to make my own registration optimizer. Does this seem like a reasonable solution or am I missing something that could be problematic?

[Matt J]

Does this seem like a reasonable solution or am I missing something that could be problematic?

I can't argue with success, but I vaguely wonder why setting the background pixels to some constant wouldn't do the same thing. An atomic distribution is independent of every distribution as well.