When using Python's subprocess for running a compiled matlab programm, environment variables are not passed

I am using a Linux Docker container, and the code is managed by Python. I use Python code to generate a subprocess call that runs a compiled matlab program. This compile matlab program is calling another external program that requires to read an environmental variable.

my_env = os.environ.copy()

cmd = f'<abs path to my compiled program>/<compiled program "{path to a file it needs to be read}"'

log.info(f"Calling the matlab function with: \n {cmd}")

res = sp.run(cmd, shell=True, capture_output=True, text=True, env=my_env)

If I run the matlab program in the command line without Python's subprocess, the env variables are passed and the Matlab program works ok.

If I run the same command from Python, the env variables are not passed.

In Python, all the correct env variables are there (`os.getenv()` returns the proper variables, and my_env has the proper variables.

Any suggestions?

I don't want to edit the Matlab code, but if I would need to edit, would it be a solution to pass the env variables through path to a file it needs to be read and then do a setenv in Matlab would propagate the variables to all functions?

[Garikoitz Lerma Usabiaga]

it works using os.system(), but I need to write the output to a text file to be able to know what happened. It would be great to make it work with subprocess.call to have stderr and stdout

Answer 1

[Rupesh]

Hi Garikoitz ,

I understand that you are facing a challenge with passing environment variables to a compiled Matlab program when it is invoked from a Python script using the subprocess module. Your setup involves a Matlab program that calls an external program dependent on certain environment variables. While these variables are correctly set and accessible within your Python environment, they fail to be recognized by the Matlab program or its subprocesses when initiated through Python. Below are some system based approaches to address this issue, with concise code examples for each:

Explicitly Set Environment Variables in Command (Without Editing Matlab Code)

For Unix-like systems, embedding the environment variable directly in the command string offers a straightforward solution without the need to alter the Matlab code.

import subprocess as sp 
import os 

# Retrieve the value of the environment variable 
env_var_value = os.getenv("MY_ENV_VAR") 

# Define the path to the compiled Matlab program and the required file path 
compiled_matlab_prog_path = "/abs/path/to/compiled/program" 
file_path_needed = "path to a file it needs to be read" 

# Formulate the command with the environment variable 
cmd = f'export MY_ENV_VAR="{env_var_value}" && {compiled_matlab_prog_path} "{file_path_needed}"' 

# Execute the command 
res = sp.run(cmd, shell=True, capture_output=True, text=True) 

print(res.stdout)  

Pass Environment Variables as Command Line Arguments

If you're open to making minor adjustments to the Matlab code, passing the environment variables as command line arguments could be a viable strategy. This approach requires the Matlab program to interpret these arguments and set the environment variables internally.

import subprocess as sp 
import os 

# Acquire the environment variable's value 
env_var_value = os.getenv("MY_ENV_VAR") 

# Specify the compiled Matlab program and the necessary file path 
compiled_matlab_prog_path = "/abs/path/to/compiled/program" 
file_path_needed = "path to a file it needs to be read" 

# Construct the command, including the environment variables as arguments 
cmd = [compiled_matlab_prog_path, file_path_needed, "MY_ENV_VAR", env_var_value] 

# Run the command 
res = sp.run(cmd, capture_output=True, text=True) 

print(res.stdout) 

Adjustment in Matlab Code:

% Adapt this function to handle additional command line arguments 
function my_function(file_path, env_var_name, env_var_value) 
setenv(env_var_name, env_var_value); 
% Continue with the rest of your Matlab code 
End 

These methods enhance the interoperability between Python and Matlab, which ultimately results in seamless execution of your code. It is recommended to experiment with each approach to identify which best aligns with your specific requirements and system architecture, ultimately facilitating the successful passing of environment variables.

You can also refer to below document for more clear understanding of MATLAB-python interoperability .

https://in.mathworks.com/help/matlab/ref/matlab.pyclient.pythonenvironment.html

https://in.mathworks.com/help/matlab/matlab_external/install-supported-python-implementation.html

https://in.mathworks.com/matlabcentral/answers/1750425-python-virtual-environments-with-matlab

Hope this helps!

[Garikoitz Lerma Usabiaga]

Thanks for your answer, but the first one did not work, it is weird because the os.system() call works fine... I don't know if the problem is in Python's end or Matlab's end, but the env variables are not passed...

I don't want to use the second option for now, I will go on with the os.system call.

thanks!!