JVM Out of memeory error when using multiple cluster profiles
1 view (last 30 days)
Show older comments
Hello,
I am working on a project which involves loading a large number of images, extracting certain features and saving these features to the disk.
I am using 64 bit Matlab R2013a on a linux server with 64 cores.
I created 4 copies of the local(default) cluster profile, so that I have 5 cluster profiles which can create 60 workers.
I am running into JVM memory problems when using all the cluster profiles parallely. I am able to run 2 cluster profiles simulaneously, but when I try to start a 3rd cluster profile parallely, matlab crashes with the follwoing error:
---------------------------------------- Error -------------------------------------------------------
Stage: SPMD job test (createCommunicatingJob)
Status: Failed
Description:The job errored or did not reach state finished.
Command Line Output:(none)
Error Report:(none)
Debug Log:
LOG FILE OUTPUT:
[7]Opening log file: /home/nchavali/java.log.21286
[2]Opening log file: /home/nchavali/java.log.21291
[10]Opening log file: /home/nchavali/java.log.21283
[9]#
[9]# A fatal error has been detected by the Java Runtime Environment:
[9]#
[9]# java.lang.OutOfMemoryError: Cannot create GC thread. Out of system resources.
[9]#
[9]# Internal Error (gcTaskThread.cpp:38), pid=21284, tid=139962136192768
[9]# Error: Cannot create GC thread. Out of system resources.
[9]#
[9]# JRE version: 6.0_17-b04
[9]# Java VM: Java HotSpot™ 64-Bit Server VM (14.3-b01 mixed mode linux-amd64 )
------------------------------------------------------------------
My code does not involve transfer of a large amount of data from client to workers.. My workers load image from a location specified in a variable, and save data into disk. And each worker saves data in different locations.
Can someone please help me is solving this problem.
This is a snippet of code that calls matlab workers:
clear;
load('foldernames.mat');
%loads folderNames variable
matlabpool open LocalProfile2;
parfor i=1:12
cffinal(folderNames(i,:));
end
exit;
0 Comments
Answers (1)
See Also
Categories
Find more on Startup and Shutdown in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!