The STUMPY library relies on Numba as a core dependency and we perform unit testing on Azure Pipelines (i.e., it’s a fresh installation every time). STUMPY only depends on NumPy, SciPy, and Numba and nothing else. We noticed that as of Numba 0.48.0+, we are seeing TBB warnings like:
##[warning]The TBB threading layer requires TBB version 2019.5 or later i.e., TBB_INTERFACE_VERSION >= 11005. Found TBB_INTERFACE_VERSION = 9002. The TBB threading layer is disabled.
On Azure Pipelines, we test on Windows, Mac OSX, and Linux (ubuntu-16.04) and this is repeated on Python 3.6, 3.7, and 3.8. This warning only appears on Linux.
Interestingly, on a local server running on Ubuntu 18.04, this warning does not appear. I want to ensure that my users aren’t seeing this warning when they are running their code and I would like to suppress this warning when testing on Azure Pipelines. What’s the best way to solve this problem?
The warning is because there’s probably an incompatible TBB library installed somewhere in the environment and the code is using Numba’s parallelisation capabilities without explicitly specifying a threading layer. As a result, Numba will scan the environment for threading libraries, and it’s then reporting that it found TBB but it’s too old to use. Options to “fix” include:
Thanks @stuartarchibald. Considering that this is a warning and it states at the end of it that “The TBB threading layer is disabled”. Is there a way to know what Numba is defaulting to since TBB is disabled? And is there a command to manually disable TBB (I don’t install it explicitly and so I suspect that it is coming natively with the Azure Pipelines image).
And is there a command to manually disable TBB (I don’t install it explicitly and so I suspect that it is coming natively with the Azure Pipelines image).
Not explicitly short of simply uninstalling it. You can elect to select a specific threading layer that is not TBB as described here.
@stuartarchibald In the case where numba is being downloaded as a package dependency, is there a good way to automate the selection/specification of the threading layer without user intervention? We also don’t know the platform a priori and so any automation to detect and handle this would be great. Or, maybe to ask the question differently, if numba is going to disable TBB anyways, is there a good way to silence that warning without requiring the user to explicitly specify the threading layer?
I have considered adding
import warnings
warnings.filterwarnings('ignore')
to the top of my modules but this seems reckless and would also hide other important warnings. I think that 99% of my users don’t care what threading layer is being used and would accept the best choice discovered by numba.
@seanlaw I guess there’s two options. In the upcoming Numba 0.55 release (currently at RC1) there’s a config setting that lets you adjust the threading layer priority: https://numba.readthedocs.io/en/0.55.0rc1/user/threading-layer.html#setting-the-threading-layer-selection-priority
with this you could make it so that TBB is tried only as a last resort, this won’t suppress the warning but would potentially make it very unlikely to appear (but also potentially prevents the use of TBB!).
Alright, it looks like this minimum example should solve the problem:
import numpy as np
import warnings
from numba import njit, prange
@njit(parallel=True)
def go_fast(a): # Function is compiled and runs in machine code
trace = 0.0
for i in prange(a.shape[0]):
trace += np.tanh(a[i, i])
return a + trace
x = np.arange(100).reshape(10, 10)
with warnings.catch_warnings():
warnings.filterwarnings("ignore", message="The TBB threading layer requires TBB version")
print(go_fast(x))