Failed to call python modules due to lack of 'core' in numba

I want to call python to caculate a high resolution infrared molecular spectra by radis.

First I run a test code by python, and test it in VS Code. It woked and printed right results with no errors.

But when I used to call python modules in matlab, An error happened: AttributeError: module 'numba' has no attribute 'core'.

Actually I can import core from numba in VS Code successfully.

Steps To Reproduce

Python Code:

# myfun.py
def test():
    print('Hello,Matlab!')

def add(a, b):
    c = a + b
    return c

def test2():
    from radis import calc_spectrum
    from numba import core
    s1 = calc_spectrum(
        1900 ,
        2000 ,
        molecule="CO2",
        isotope="1",
        pressure=1.01325 ,
        Tgas=500 ,
        mole_fraction=0.1,
        path_length=1 ,
        databank="HITEMP2010-CO2",  
        verbose=False,
    )
    a = s1.get('abscoeff')
    print(a)


if __name__ == '__main__':
    a=test2() 

Matlab Code:

[v, e, isloaded] = pyversion;
if isloaded
    disp('To change the Python version, restart MATLAB, then call pyversion.')
else
    pyversion('F:\Anaconda3\envs\radis-lab\python.exe') 
end

if count(py.sys.path,' ') == 0  %Add file path to Python environment
     insert(py.sys.path,int32(0),'');
end
 py.print('Hello, Python!')
% reload module
clear classes
obj = py.importlib.import_module('myfun');
py.importlib.reload(obj);
py.myfun.test2()

Answer 1

[Yongjian Feng]

I would recommand the following steps to narrow down the issue:

1. From the error message, it seems like matlab can find numba but can't find core. So first confirm this. Instead of importing core from numba, just import numba, and print its version. If successful, then it is indeed complaining about core, not numba.
2. If numba can be imported, print its path. See which numba matlab actually imported. Make sure it is the correct one that you want to use. If not, play with PYTHONPATH env var to point to the correct one
3. If numba can't be imported, play with PYTHONPATH env var to point to the correct one.

[JiYuan Lee]

I can run code below:

 import py.numba.*

I don't know how to print numba's path.

Actually, We can check the path by py.sys.path, and I think it's right.

 ['', 'F:\\Anaconda3\\envs\\radis-lab\\python36.zip', 'F:\\Anaconda3\\envs\\radis-lab\\DLLs', 'F:\\Anaconda3\\envs\\radis-lab\\lib', 'F:\\Anaconda3\\envs\\radis-lab', 'C:\\Users\\lijiyuan\\AppData\\Roaming\\Python\\Python36\\site-packages', 'F:\\Anaconda3\\envs\\radis-lab\\lib\\site-packages', 'F:\\Anaconda3\\envs\\radis-lab\\lib\\site-packages\\win32', 'F:\\Anaconda3\\envs\\radis-lab\\lib\\site-packages\\win32\\lib', 'F:\\Anaconda3\\envs\\radis-lab\\lib\\site-packages\\Pythonwin']

[Yongjian Feng]

Use inspect. Run the following code from python and from matlab (comment out the last two lines when running from matlab).

import inspect
import py
import py.numba
import py.numba.*

% this tells you where python import py
pypath = inspect.getfile(py)
print(pypath)

% this tells you where python import py.numba
numbapath = inspect.getfile(py.numba)
print(numbapath)

% this tells you where python import py.numba.core
% run this only from python, not from matlab
corepath = inspect.getfile(py.numba.core)
print(corepath)

Also there is this envs\radis-lab folder in your py.sys.path. Are you using virtualenv?

[JiYuan Lee]

Yes, I use Anaconda to manage my virtual environment.

[JiYuan Lee]

I run code from python

import inspect
import py
import numba
pypath = inspect.getfile(py)
print(pypath)
numbapath = inspect.getfile(numba)
print(numbapath)

print:

F:\Anaconda3\envs\radis-lab\lib\site-packages\py\__init__.py
F:\Anaconda3\envs\radis-lab\lib\site-packages\numba\__init__.py

And I run code from matlab

import inspect
import py
% import py.numba
import py.numba.*
% this tells you where python import py
pypath = inspect.getfile(py)
print(pypath)
% this tells you where python import py.numba
numbapath = inspect.getfile(py.numba)
print(numbapath)

Error:

The import parameter 'inspect' cannot be found or imported.

[Yongjian Feng]

virutalenv is always challenging.

Try this instead:

import py
import py.numba
import py.numba.*
% this tells you where python import py
% This shall work for both python and matlab
print(py.__file__)

% if this doesn't work, comment it out
print(py.numba.__file__)
% if this doesn't work, comment it out
print(py.numba.core.__file__)

[JiYuan Lee]

In matlab, print(py.__file__) doesn't work, '__' is an invalid text character.

Do we want to export path where python import py in matlab and compare it with in python? It seems like difference between them. Is it because of the virtual environment?

[Yongjian Feng]

Yes, as explained before, virtualenv is challenging. If it is possible, try without it.