The best method to increase memory is installing more RAM and using a 64 bit version of Matlab.
But in your case the error message is unequivocally an invalid file identifier. This is a completely other problem and not related to memory. It means that you have tried to open a not existing file. This should be checked in every case:
FID = fopen(FileName, 'r');
if FID == -1
error('Cannot open file for reading: [%s]', FileName);
The square brackets in the message helped me to recognize unexpected file names like control characters or spaces.
It is strongly recommended to include the path in the file name, because the current directory can change for unexpected reasons, e.g. GUI or TIMER callbacks.
Another problem can be that the closing of the files is not successful. Check this by:
The list of files, which can be open at the same time, is limited by the operating system. Usually the need to have more than 20 file opened is a clear argument for a bad program design.