Ok so I have not solved my own problem, but I have realized that I dont think my displacement field is wrong per se. The problem seems to be that you do not get the same result when you transform images with imregister() and imwarp(im,tform) even though you use the same tform. However the transformed images you get when using imregister() and imwarp(im,D), where D is the displacement field I made, are the same! What is going on?
% import matlab standard images
fixed = imread('hands1.jpg');
moving = imread('hands2.jpg');
fixed = rgb2gray(fixed);
moving = rgb2gray(moving);
% get optimizer and metric
[optimizer, metric] = imregconfig('monomodal');
% get registered image and tform1 used in function imregister()
[moving_reg11,Rreg,tform1] = imregister2(moving,fixed,'rigid',optimizer,metric);% extension of IMREGISTER that also returns the TFORM that is applied
% get tform1 used via function imregtform()
tform2 = imregtform(moving,fixed,'rigid',optimizer,metric);
% use found tforms to warp moving image
moving_reg12 = imwarp(moving,tform1);
moving_reg13 = imwarp(moving,tform2);
% make a displacement field that corresponds to tform = tform1 = tform2
[X,Y] = meshgrid(1:size(moving,2),1:size(moving,1)); % make grids with x and y coordinates of pixels
[XT,YT] = transformPointsForward(tform1,X,Y); % Transform pixel coordinates using tform
D = zeros(size(X,1),size(X,2),2);
D(:,:,1) = X-XT; % displacement field should be the difference!?
D(:,:,2) = Y-YT;
moving_reg14 = imwarp(moving,D); % warp moving image with the new displacement field
% compare the three images...
figure
subplot(3,1,1)
imshowpair(moving_reg12,moving_reg13); title('(moving-reg12 and moving-reg13). tform1 and tform2 returned by imregister2() and imregtform() are the same')
subplot(3,1,2)
imshowpair(moving_reg11,moving_reg12); title('(moving-reg11 and moving-reg12). Images transformed using imregister2() and imwarp(im,tform) with the same tform are not the same!')
subplot(3,1,3)
imshowpair(moving_reg11,moving_reg14); title('(moving-reg11 and moving-reg14). Images transformed using imregister2() and imwarp(im,D) with my displacement field are the same...')