Issue in MATLAB Engine library for Python while using TensorFlow in Linux

Hello,

I am using MATLAB Engine for Python for calling MATLAB functions from Python Environment. I am currently working project that requires both MATLAB and tensorflow to work parallely. Unfortunately, as soon as I make a eval call to the MATLAB using the MATLAB engine for Python, I am getting a protobuf version mismatch error from Tensorflow. I am working on MATLAB 2019b, TensorFlow 1.15.0 in Ubuntu 18.04 platform. The error can be reproduced using the following code.

import matlab.engine as engine
eng = engine.connect_matlab(engine.find_matlab()[0])
eng.eval("a=5", nargout=0)
import tensorflow as tf 

ERROR
[libprotobuf FATAL This program requires version 3.8.0 of the Protocol Buffer runtime library, but the installed version is 3.6.1.  
    Please update your library.  If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library.  
    (Version verification failed in "bazel-out/k8-opt/genfiles/tensorflow/core/framework/tensor_shape.pb.cc".)]

I have explored a lot about this issue, MATLAB team gave a workaround in this question. But, it discuss about calling Python functions/ libraries from MATLAB using pyenv. The same workaround doesn't seem to work while calling MATLAB functions from Python. Is there any workaround avaliable for MATLAB Engine for Python?

Answer 1

[Priyank Pandey]

Hi Dinesh,

Python packages, like TensorFlow, often have dependencies on a number of third party libraries with specific version requirements. These may cause issues when MATLAB ships with the same libraries, but with different version requirements. Some of these conflicts may be avoided by running Python Interface in a separate process, but some conflicts may persist. One such conflict often occurs with the Protocol Buffer runtime library ('protobuf'), which is used by both TensorFlow and MATLAB, but with different versions. Other common examples include 'libexpat' and 'libcrypto'.

One workaround for this issue is to force TensorFlow to load the symbols from its own version of 'protobuf' instead of the version shipped with MATLAB. Taking the example of TensorFlow and 'protobuf', the code below demonstrates how to set the 'dlopen' flags and force TensorFlow to load its own version of 'protobuf'.

>> pyenv('Version', '<path to Python exectuable>', 'ExecutionMode', 'OutOfProcess');

>> RTLD_NOW=2;

>> RTLD_DEEPBIND=8;

>> flag=bitor(RTLD_NOW, RTLD_DEEPBIND);

>> py.sys.setdlopenflags(int32(flag));

>> c = py.tensorflow.constant(1);

The above example can be adapted for other packages where similar third-party library conflicts may be occurring.

Hope it works.