JVM Out of memeory error when using multiple cluster profiles

1 view (last 30 days)
Hello,
I am working on a project which involves loading a large number of images, extracting certain features and saving these features to the disk.
I am using 64 bit Matlab R2013a on a linux server with 64 cores.
I created 4 copies of the local(default) cluster profile, so that I have 5 cluster profiles which can create 60 workers.
I am running into JVM memory problems when using all the cluster profiles parallely. I am able to run 2 cluster profiles simulaneously, but when I try to start a 3rd cluster profile parallely, matlab crashes with the follwoing error:
---------------------------------------- Error -------------------------------------------------------
Stage: SPMD job test (createCommunicatingJob)
Status: Failed
Description:The job errored or did not reach state finished.
Command Line Output:(none)
Error Report:(none)
Debug Log:
LOG FILE OUTPUT:
[7]Opening log file: /home/nchavali/java.log.21286
[2]Opening log file: /home/nchavali/java.log.21291
[10]Opening log file: /home/nchavali/java.log.21283
[9]#
[9]# A fatal error has been detected by the Java Runtime Environment:
[9]#
[9]# java.lang.OutOfMemoryError: Cannot create GC thread. Out of system resources.
[9]#
[9]# Internal Error (gcTaskThread.cpp:38), pid=21284, tid=139962136192768
[9]# Error: Cannot create GC thread. Out of system resources.
[9]#
[9]# JRE version: 6.0_17-b04
[9]# Java VM: Java HotSpot™ 64-Bit Server VM (14.3-b01 mixed mode linux-amd64 )
------------------------------------------------------------------
My code does not involve transfer of a large amount of data from client to workers.. My workers load image from a location specified in a variable, and save data into disk. And each worker saves data in different locations.
Can someone please help me is solving this problem.
This is a snippet of code that calls matlab workers:
clear;
load('foldernames.mat');
%loads folderNames variable
matlabpool open LocalProfile2;
parfor i=1:12
cffinal(folderNames(i,:));
end
exit;

Answers (1)

Elad
Elad on 5 Jul 2013
  1 Comment
Neelima
Neelima on 6 Jul 2013
Thanks you for your sugestion but my problem is not with the size of images as all the images I use are resized to 640x640 pixels. And I am not constrained memory on the computer. Matlab is not running out of memory, but JVM is. Can you tell me what causes this JVM error in matlab? I have seen in some answers, that this error can occur when there is a transfer of large amount of data between client and workers, but my code does not involve any such transfer.

Sign in to comment.

Categories

Find more on Startup and Shutdown in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!