Find a Non-Cropped Template in a Larger Image

I want to be able to find knits and purls in a larger image of knitwear. I'm trying not to use a direct crop from the larger image, some sample images are attached. I'm not advanced in Matlab but I have done some research.

My issue is that nearly all solutions to finding a template image in another image requires the template to have come directly from the image or needs the object to appear more obviously on an image than any knit does.

I have looked into normxcorr2 and xcorr2 but I'm not sure they are suitable. I also don't believe the key point detectors, such as FAST, BRISK or SURF would work because of the colour difference between the samples and the image I'm trying to find them in. I have looked into papers that have tried what I have and found only Inverse Procedural Modeling of Knitwear which uses BBS from the paper Best-Buddies Similarity for Robust Template Matching however, they use a template image cropped from the image they wanted to find it in so I am not sure this would work for me either.

I am ideally looking for suggestions or links that I could follow to find a method that works.

I am using R2020b.

Answer 1

[Milan Bansal]

Hi Sarah Forster

I understand that you're interested in identifying knits and purls within a larger image, using a template image that may not be directly extracted from the larger image itself. For this purpose, Local Binary Patterns (LBP) can be quite effective. LBPs are known for their capability in texture classification, making them suitable for differentiating between various knitting patterns. By utilizing the extractLBPFeatures function, you can extract the LBP features from both the larger image and the template image.

To locate the desired pattern, you can slide a window across the larger image, matching the size of the template image, and calculate the distance between the LBP features of the window and those of the template image. The window that shows the lowest distance is likely to contain the pattern you're looking for.

Please refer to the code snippet given below to find the knit pattern from the larger image:

% Read and preprocess images
template = imread('template_image.jpg');
image = imread('large_image.jpg');
template_gray = rgb2gray(template);
image_gray = rgb2gray(image);

% Extract LBP features
template_lbp = extractLBPFeatures(template_gray);
image_lbp = extractLBPFeatures(image_gray, 'Upright', false);

% Use a sliding window to compare LBP features
windowSize = size(template_gray);
bestMatch = Inf;
bestPos = [0, 0];

for i = 1:(size(image_gray,1) - windowSize(1))
    for j = 1:(size(image_gray,2) - windowSize(2))
        window = image_gray(i:i+windowSize(1)-1, j:j+windowSize(2)-1);
        window_lbp = extractLBPFeatures(window);
        dist = norm(template_lbp - window_lbp);
        if dist < bestMatch
            bestMatch = dist;
            bestPos = [i, j];
        end
    end
end

% Display results
figure, imshow(image_gray);
hold on;
rectangle('Position', [bestPos(2), bestPos(1), windowSize(2), windowSize(1)], 'EdgeColor', 'r');
hold off;

Please refer to the following documentation link to learn more about extractLBPFeatures function.

https://www.mathworks.com/help/vision/ref/extractlbpfeatures.html

Hope this helps!