Servicing Python Matlab API queries

Hello All,

I am dabbling into the python matlab api. I'd like to use the API to enable a python program that is spawned indirectly as a result of a system call from matlab, and allow that python program to interact with the parent matlab session using the API. Something like this:

function call_external_program
    matlab.engine.shareEngine; % share this engine so that we can connect to it from python script.
    
    %launch the program in the background
    if ispc
        %windows
        [status , result] = system('START wrapper_program.exe') %<-- note that wrapper_program.exe spawns a python program that connects back to this session...
    else
        %linux
        [status , result] = system('wrapper_program &') %<-- note that wrapper_program.exe spawns a python program that connects back to this session...
    end
    
    % wait until we detect termination of the wrapper_program.  during this time, we expect to be servicing calls to python_api_entrypoint coming from 
    % a python program that was spawned by wrapper_program...
    while 1
        %Hmm.. how can I pause here in such a way that Matlab can service the Python API?
        % It only works if I put a breakpoint here!
       pause(0.1)
       
       if detected_wrapper_program_completion || timeout %don't ask how we really do this...
           break;
       end
    end
    
end

function dataOut = python_api_entrypoint(dataIn)
    %example algorithm that we use matlab for...
    dataOut = sum(dataIn);
end

Unfortunately, the above scheme only works if I put a breakpoint at the 'pause(0.1)' line, presumably because matlab control returns to the command line. Otherwise, matlab will not service a call from the python program unless we quickly breakout of the parent function and return matlab control to the command line. This smells like something drawnow() could help with, but no such luck. Any ideas?

[Dario Ringach]

Hi Jesse,

Have you had any luck working this out? I am running into a similar situation, but pause() does not help either.

Thanks for any tips...

--

Dario

[Jesse Hopkins]

Hi Dario -

We do have quite a nasty hack in place that gives some functionality. In a nutshell, use a timer object to do some monitoring/processing of the external program while matlab control remains at the command prompt. Some more detail:

a. Data needed for the matlab function called from python is stored in a singleton matlab handle object. It is retrieved like:

function dataOut = python_api_entrypoint(dataIn)
    % get important state data from matlab
    stateDataObj = MyStateDataClass.getSingleton();
    
    %example algorithm that we use matlab for...
    dataOut = stateDataObj.dosomething(dataIn);
end

b. In order for the matlab/python API to service queries, seems that Matlab must be at an interactive command prompt. Never could get anything like pause/etc.. to work. So the matlab function that launches the external program must return back to the command prompt.

c. Note that (b) imposes a severe limitation. It is not possible in this scheme to call said matlab function from any other matlab script/function, as it must return back to the matlab command prompt in order for matlab to service the upcoming calls to the matlab/python API [ Have some ideas on addressing this limitation later..]

d. Needed to split the top-level matlab function mentioned in (b) into some parts:

1. Prepare to call the external program (i.e. create/update the singleton state object)
2. Setup a timer object to monitor the external program. This timer object will be started just before the top level function returns back to command prompt
3. Spawn the external program via system
4. Start timer object initialized by (2)
5. Top-level function returns to command line
6. External program continues to run
7. Timer object monitors execution of external program, deletes itself upon termination of external program.

Clear as mud?

I think that the limitation mentioned in (c) could be worked-around by spawning a completely seperate instance of matlab, which if setup with necessary state information could perhaps service the python API queries. It would need to be spawned and just sit and wait at the command line. Haven't gotten around to trying that.