MATLAB Answers

netcdf reading parameters "Error using double Out of memory. Type HELP MEMORY for your options"

12 views (last 30 days)
Silver on 12 Nov 2018
Commented: Kelly Kearney on 14 Nov 2018
Hi everyone ! I have an "out of memory" error while reading chlorophyl data from an netcdf file. The file contains :
  • latitude
  • longitude
  • time
  • CHL ( chl_a)
when I use ncread for latitude longitude and time , it works without any issue , but when I use the ame function for chlorophyl read, it shows the out of memory error.
I used this code :
ncdisp(''); % to display the nc file
lon2 = ncread('','lon') ; %to read longitude
lat2 = ncread('','lat') ; %to read latitude
time2 = ncread('','time') ; %to read time
chl = ncread('','CHL') ; %to read chlorophyl , the error shows in this line
the file size is 955 Mo.
any ideas ? Thks in advance!


Sign in to comment.

Accepted Answer

Kelly Kearney
Kelly Kearney on 12 Nov 2018
What does the ncdisp command tell you about the variable sizes? Just paste the output of the following:
This is probably a pretty straightforward error... you're trying to read a very big dataset on a computer without enough memory to hold it all. So you either need more memory, or you need to read the data in smaller chunks.


Show 1 older comment
Kelly Kearney
Kelly Kearney on 13 Nov 2018
Well, first off, you say that your data is a 818 x 839( x 120) array, but then are assuming lat/lon dimensions of 818 x 818 in your loop... which is it?
But the main error in your logic here is that you are assuming there will be an exact match in your satellite grid coordinates to each of the trackline coordinates. That is very, very unlikely to be the case, unless the ferrybox data was subsampled from the satellite data.
What you really want to do is find the closest grid cell in the satellite product to each coordinate in the spreadsheet. (Or better yet, the closest 5-10 points, if you want a more robust average).
There are a few approaches you could use:
  1. If your satellite data is higher-resolution than the observation dataset, and well away from the poles, such that the nearest-neighbor calculation will be approximately the same in either geographic or cartesian lon/lat space, you could try using interp2. Just be careful if you're close to the dateline or prime meridan that your longitudes are properly wrapped to either [0 360] or [-180 180] as appropriate.
  2. If you're dealing with data with lots of pole crossings, or where the gridded dataset has a coarser resolution that the trackline data such that the nonlinearity of latitude and longitude becomes important, then you may want to look at pdist2 with a custom geographic-distance-based metric to build your own custom interpolant.
Silver on 13 Nov 2018
  1. So you mean that I am going to find verry few points from the satellite data that I am going to interpolate so I could fine a line of satellite data that could be superposed to the FerryBox line ? is that what you mean ? If it is the case , I have to first compare both data and extract the coincidance and that's what I didnt find how to do it. could you be more exlicit in a code please ?
  2. PS : while you are weeling with coordinate system , I want to ask you another question if you dont mind :) , in the photo I attached normally the line of the Ferrybox must be in the red line (as bellow) and not where it was located while the plot. The coordinate system of the satellite data is WGS 84 / Plate Carree (EPSG 32662). What can I do for the ferrybox coordinate to make it in the right location?
Thanks again!
Kelly Kearney
Kelly Kearney on 14 Nov 2018
Regarding reading, yes, you'll need to read a certain subset of the data. I'd start by seeing if you can read in just the grid that falls inside the bounding box of your line data. Take a look at the START and COUNT inputs for ncread.
As for the offset coordinates, that really depends on how the data is actually stored in each file. Porjection and reference ellipsoid shouldn't matter if you're dealing with lat/lon coordinates. Is one of your datasets actually stored using projected coordinates (i.e. X/Y)? If so, you'll need to find out all the details of that projection to back-calculate the appropriate lat/lon coordinates and compare the two datasets.

Sign in to comment.

More Answers (0)