MATLAB Answers

Is it possible to set a remote folder on HPC as a current folder?

72 views (last 30 days)
Mostafa on 13 Dec 2020
Commented: Mostafa on 19 Dec 2020
Hi everyone!
I would like to know if there is any way to set the current folder on a remote location. I am working using matlab parallel server and since my data is huge I put data and functions inside a folder DATA locatd in the cluster and send job by batch as follow:
c = parcluster('short');
job = batch(c, 'forHPC', 'Pool', 27, 'CurrentFolder', '~/DATA', 'AutoAddClientPath', false, 'AutoAttachFiles', false);
It perfectly works without transferring data between my computer and workers and finally it saves outputs in the folder DATA in the cluster.
I want to do the same thing in the interactive mode inside Matlab. Put differently, I would like to work with matlab interactively on my computer while my workers and current folder are remote to avoid trasferring data. Is it possible?
Any idea is highly appriciated.

Answers (1)

Raymond Norris
Raymond Norris on 15 Dec 2020
Can you provide a bit more info on how you'd like this to work. Given that you're using parcluster, batch, etc., I suspect you're also familiar with parpool -- is that what you mean by running MATLAb in interactive mode? Would the following work?
c = parcluster('short');
p = parpool(c,28);
pctRunOnAll cd ~/DATA
forHpc runs locally, except when you hit parfor, spmd, etc. That code (e.g. parfor) will be run on the cluster and the workers will run in ~/DATA. Any code in forHpc that is not embedded within a parfor or spmd block, distributed array, will be run on your local machine.
Does this address what you're asking? You didn't mention if your local machine and the cluster share a mount disk. Is ~/DATA reachable from your local machine?
Mostafa on 19 Dec 2020
The original forHpc uses several functions that I already created but I shortened it to the follwing code and still get the same error:
mdlSVMsmallOpt = fitcsvm(smallTT, 'o_member', 'OptimizeHyperparameters','auto',...
'expected-improvement-plus','ShowPlots', false ,'UseParallel', true)) ;
save('mdlSVMsmallOpt.mat', 'mdlSVMsmallOpt')

Sign in to comment.




Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!