Matlab engine starting error in Python (docker)

Hi,

I am using Matlab with Python in a docker (cloud) environment. The installation was okay. But starting matlab engine (2022b) in Python (3.8) returns the following error:

>>> import matlab.engine
>>> eng = matlab.engine.start_matlab()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/matlab/.local/lib/python3.8/site-packages/matlab/engine/__init__.py", line 130, in start_matlab
    eng = future.result()
  File "/home/matlab/.local/lib/python3.8/site-packages/matlab/engine/futureresult.py", line 67, in result
    return self.__future.result(timeout)
  File "/home/matlab/.local/lib/python3.8/site-packages/matlab/engine/matlabfuture.py", line 87, in result
    handle = pythonengine.getMATLAB(self._future)
matlab.engine.EngineError: Connection to process with Exchange: " " was lost.

Using background = True didn't work as well:

>>> future = matlab.engine.start_matlab(background=True)
>>> eng = future.result()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/matlab/.local/lib/python3.8/site-packages/matlab/engine/futureresult.py", line 67, in result
    return self.__future.result(timeout)
  File "/home/matlab/.local/lib/python3.8/site-packages/matlab/engine/matlabfuture.py", line 87, in result
    handle = pythonengine.getMATLAB(self._future)
matlab.engine.EngineError: Connection to process with Exchange: " " was lost.

Can anyone help me identify the issue here? Thanks in advance!

Answer 1

[Rajeev]

The error message indicates that the connection to the MATLAB engine has been lost. This can happen for a number of reasons that include network issues, memory limitations, or other system constraints.

Here are some steps you can try to resolve this issue:

1. Increase the available memory for the Docker container. MATLAB can be memory-intensive, so if the container doesn't have enough memory allocated, it may cause the connection to be lost.
2. Check your network connection. Make sure the Docker container has access to the internet and that there are no network restrictions that may interfere with the MATLAB engine.
3. Try using a different version of MATLAB. Some versions of MATLAB may have compatibility issues with certain Python environments. You can refer to System Requirements for MATLAB Engine API for Python - MATLAB & Simulink (mathworks.com) and Versions of Python Compatible with MATLAB Products by Release - MATLAB & Simulink (mathworks.com)
4. Try starting the MATLAB engine with the -nojvm option. This will start the MATLAB engine without the Java Virtual Machine,that can help reduce memory usage and improve stability.
5. Try using a different Python version. Some users have reported issues with MATLAB engine in Python 3.8, so it may be worth trying a different version of Python, such as 3.7 or 3.9.

Have a nice day :)

[J.]

Hi @Rajeev thanks for the reply. I think I wil try 4 and 5. 2 and 3 should not be a problem in my case. Also, in 1, what does 'memory' mean here? It seems "By default, Docker doesn't set any resource constraints on a container. It allows the container to access all system resources." So it is not clear to me why it could be an issue. Can you give more explanation? Thanks a lot!

[Rajeev]

Hi @J.

You are correct that there is docker does not set any constraints on a container.

I meant the memory of your host system in this case.

You can refer to the docker documentation: https://docs.docker.com/config/containers/resource_constraints/#understand-the-risks-of-running-out-of-memory

and this stackoverflow answer: How to assign more memory to docker container - Stack Overflow for more info.

[J.]

This should not be a problem in my case. The host has 500+GB RAM.

Answer 2

[Rushikesh]

Hello @J.,

I understand that you are encountering the "EngineError Connection to process with Exchange was lost " while attempting to start MATLAB in a Docker cloud environment.

To clarify, when starting a new MATLAB session from Python within a Docker container, you typically need a license server to run MATLAB non-interactively. If you have an individual license, it requires you to start a MATLAB session interactively and then connect to this session from Python using the Engine API.

Assuming you have access to a license server, then mentioned ‘EngineError Connection to process with Exchange was lost’ can arise due to several factors such as licensing issues, network problems, or misconfigurations within the Docker environment. Here are some steps to address these issues:

1. Ensure that the server address and port for the Network License Manager are correctly specified. For example: [e-mail].

Use this information as an argument in the ‘start_matlab’ function.

2. Within the container, run the following command to unset an environment variable on the Linux prompt:

unset VARIANTmatlab

3. Override the default entrypoint in the Docker container to ensure the environment is correctly set up:

docker run -it --rm --entrypoint /bin/bash your_image_of_ matlab:r2022b

4. Execute the ‘start_matlab’ function with the port and server address or the full path to the ‘network.lic’ file in the command prompt from the container.

For example in python:

eng = matlab.engine.start_matlab("-c [e-mail]") 
%Or 
eng = matlab.engine.start_matlab("-c /full/path/to/network.lic") 

Ensure that the ‘network.lic’ file was volume mounted onto the container using the -v flag in the docker run command.

I hope these steps help you resolve the issue you are facing.