Windows, Python 3.8 and TBB support

Numba Support: How do I do ...?
1

I want to use TBB threading layer with the Numba. My environment is

Windows 10
Python 3.8
numba 0.51.2
tbb 2020.3.254

But the following code give me an error:

import numpy as np
from numba import config, njit, threading_layer

config.THREADING_LAYER = 'tbb'

@njit(parallel=True)
def foo(a, b):
    return a + b

x = np.arange(10.)
y = x.copy()

foo(x, y)

print("Threading layer chosen: %s" % threading_layer())

Traceback (most recent call last):
  File "C:\Dev\Python\python-3.8.5\lib\site-packages\numba\core\errors.py", line 745, in new_error_context
    yield
  File "C:\Dev\Python\python-3.8.5\lib\site-packages\numba\core\lowering.py", line 230, in lower_block
    self.lower_inst(inst)
  File "C:\Dev\Python\python-3.8.5\lib\site-packages\numba\core\lowering.py", line 443, in lower_inst
    func(self, inst)
  File "C:\Dev\Python\python-3.8.5\lib\site-packages\numba\parfors\parfor_lowering.py", line 292, in _lower_parfor_parallel
    call_parallel_gufunc(
  File "C:\Dev\Python\python-3.8.5\lib\site-packages\numba\parfors\parfor_lowering.py", line 1419, in call_parallel_gufunc 
    _launch_threads()
  File "C:\Dev\Python\python-3.8.5\lib\site-packages\numba\np\ufunc\parallel.py", line 497, in _launch_threads
    raise_with_hint(requirements)
  File "C:\Dev\Python\python-3.8.5\lib\site-packages\numba\np\ufunc\parallel.py", line 442, in raise_with_hint
    raise ValueError(errmsg % hint)
ValueError: No threading layer could be loaded.
HINT:
Intel TBB is required, try:
$ conda/pip install tbb

The reason is the Python 3.8 DLL resolution procedure.

I found the workaround:

import importlib.metadata as metadata

def add_dll_directory(package_name, dll_name):
    dll_dirs = [item for item in metadata.files(package_name) if item.name == dll_name]
    if dll_dirs:
        path = dll_dirs.pop()
        dll_path = path.dist.locate_file(path.parent).resolve()
        os.add_dll_directory(dll_path)

add_dll_directory('tbb', 'tbb.dll')

After the patch everything works good.

My question is do I need manually add the dll directories for all numba dependencies? Or numba in future will collect all dependencies in accordance with the Python 3.8+ dll resolution process?

2

Hi @alkalinin,

I think you’ve diagnosed this correctly, not sure about what happens on windows in terms of resolving paths (presume you have a non-Anaconda distribution python?). On linux there’s been issues with pip for similar https://github.com/numba/numba/issues/6108, I suggest opening an issue on the issue tracker. Thanks.