How can I inpaint/interpolate data from an area that was masked during an experiment?

I have included an image, so that you can get a feel for the data. I have tried a number of approaches, mainly using code from chatgpt as I'm not great at coding. The best I have found so far is to do column and row pass envelopes, but a more radial approach would be better. Every time I have tried this though I have just ended up with a strange sphere under the main peak.

I would really appreciate if anyone has any suggestions in MATLAB about functions that would be useful to either automatically mask the gridded sections or interpolate well using neighbouring info from the unmasked regions?

[Stephen23]

https://www.mathworks.com/help/matlab/math/interpolating-scattered-data.html

[William Rose]

@Alex,

I zoomed in on the figure you provided. You say you want "functions that would be useful to either automatically mask the gridded sections or interpolate well using neighbouring info from the unmasked regions?". hose sound like quiotre different requests and it would help to explain what your real goal is with this data. Please also share the data file you used to make the image above.

My guess, based on looking at the image, is that you have z-data at all the points on a finely spaced x-y grid. But something masks or limits the z-values on a set of intesecting "streets and avenues". So you want to replace the z-values on the streets and avenues with interpolated z-values. But you also said "automatically mask or interpolate...." So if we just eliminated the streets and avenues points from the dataset, would that satisfy your data analysis goals? If you explain the big picture more, you are likely to get more useful answers.

[William Rose]

@Alex ,

One of the reasons I am asking you to share the data is so that I can understand if

A. The x-y coordinates of the points where interpolation is needed is obvious from the data (which is the case if the points on the "streets and avenues" are simply not present in the data set)

or

B. You need to figure out which points should be replaced with interpolated values (which is the case if there are z-values at all the x-y point on a fine grid, but some z-values are incorrect, because they are shadowed or masked during measurement).

The A or B distinction will be important in finding a useful solution.

[William Rose]

@Alex,

If your ultimate goal is to characterize the location, height, and spread of the central peak, then it will probably be better to eliminate the points on the streets and avenues (rather than replace them with interopolated values), and fit the remaining "good points" with a 2D Gaussian or Lorentzian or some other function.

[Alex]

Hi thanks very much for the response. This data that I have attached relates to a surface that was physically masked with gridlines, which look like the streets and avenues. I wanted to implement a mask to turn what was the physical mask into NaNs and then interpolate with the remaining 'relevant' data. Just eliminating the values is helpful, but they would also have to be interpolated afterwards. It would definitely be case B. Currently I'm using the attached code to take a 3 x 3 grid of the most prominent squares to extrapolate the pattern and then using the function Stephen 23 suggested to fill in the NaNs. It's working okay, but some parts of the masked grid are creeping and causing some imperfections.

[William Rose]

@Alex,

The z-values in the data file range from -71 nm to +745 nM. Are thenegative values measurement errors?

[Alex]

They should be measurement errors, but I think they are negligible enough to be treated that way even if they aren't.

Answer 1

[Stephen23]

Try downloading this: https://www.mathworks.com/matlabcentral/fileexchange/4551-inpaint_nans

[Alex]

Thanks so much for this suggestion! It's pretty much exactly what I'm looking for and has got me much closer to a workable solution

Answer 2

[William Rose]

@Alex,

Examine the data graphically to get some insight into the issues.

x=((1:380)-0.5)/380;    % x values from Exampledata.zip
y=x;                    % y values from Exampledata.zip
z=load('zdata');        % z values from Exampledata.zip
z=z.z;                  % get vector from the structure variable
Z=reshape(z,[380,380]); % reshape vector to matrix
figure
subplot(211), surf(x,y,Z,'EdgeColor','none')
xlabel('X'); ylabel('Y'); zlabel('Z'); grid on  % default view
subplot(223), surf(x,y,Z,'EdgeColor','none')
xlabel('X'); ylabel('Y'); zlabel('Z'); grid on; view(0,0) % X-Z view
subplot(224), surf(x,y,Z,'EdgeColor','none')
xlabel('X'); ylabel('Y'); zlabel('Z'); grid on; view(90,0) % Y-Z view

Zoom in on the X-Z and Y-Z views.

figure;
subplot(211), surf(x,y,Z,'EdgeColor','none')
xlabel('X'); ylabel('Y'); zlabel('Z'); grid on; view(0,0); % X-Z view
xlim([0.3,0.7]); zlim([-1e-7,1e-7]) 
subplot(212), surf(x,y,Z,'EdgeColor','none')
xlabel('X'); ylabel('Y'); zlabel('Z'); grid on; view(90,0); % Y-Z view
ylim([0.3,0.7]); zlim([-1e-7,1e-7]) 

The plot shows that the surface profile at the edge of the masked region is sloped, and not a sharp discontinuity. This means it is not obvious which points should be replaced with NaNs, since the edge of the masked region is somewhat gradual.

The code below generates another view, to reveal the street and avenue locations. Clip the Z values to the range +-1e-8 before plotting.

Zclipped=max(min(Z,1e-8),-1e-8);
figure; surf(x,y,Zclipped,'EdgeColor','none')
xlabel('X'); ylabel('Y'); zlabel('Z'); 
axis equal; view(0,90); colorbar  % X-Y view

The figure above shows that the avenue and street widths are not the same everywhere.

You said in your reply to @Stephen23 that you used inpaint_nans by @John D'Errico and that "It's working okay, but some parts of the masked grid are creeping and causing some imperfections." Try removing more points on the edges of the streets and avenues, based on the plots above and your own judgement. You may also find it useful to convert all negative values to zeros, using min(), before using inpaint_nans.