Prange is crashing jupyter lab with OMP: Error #15: Initializing libomp.dylib, but found libiomp5.dylib already initialized

Hi,

I recently updated to numba 0.52.0.

This code I was using before was running fine (in a jupyter notebook)

import numpy as np
from numba import njit, prange


def pi_montecarlo_python(n):
    in_circle = 0
    for i in prange(n):
        x, y = np.random.random(), np.random.random()
        if x ** 2 + y ** 2 <= 1.0:
            in_circle += 1
        
    return 4.0 * in_circle / n

Then in the next cell

pi_montecarlo_python_parallel = njit(parallel=True)(pi_montecarlo_python)

next cell

n = 100000
print('njit_python_parallel')
%time pi_montecarlo_python_parallel(n)

But it doesnt produce any time ie doesnt run. In my terminal I have this error

OMP: Error #15: Initializing libomp.dylib, but found libiomp5.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/

It doesn’t crash if it is just a normal range instead of prange - but then im not parallelising at all so…

As well as updating numba I then updated conda. As far as I know I didn’t change any thing else…?

Any insight would be greatly appreciated.

Fionnuala

How did you upgrade to Numba 0.52.0?

Can you provide the current output of numba -s please?

Hi,

Thanks for replying.

So I first did conda update numba and it looked like it did some updating but when I checked the version it was still 0.51.2 and so i then did conda install -c numba numba=0.52.0 and that worked. Then thought I needed to do conda install -c numba llvmlite=0.35.0 but that was update to date (from updating numba). Then did conda update conda

Ah yes meant to attach the output of numba -s

numba -s


System info:
--------------------------------------------------------------------------------
__Time Stamp__
Report started (local time)                   : 2021-03-04 20:06:50.038020
UTC start time                                : 2021-03-04 20:06:50.038027
Running time (s)                              : 3.189955

__Hardware Information__
Machine                                       : x86_64
CPU Name                                      : icelake-client
CPU Count                                     : 8
Number of accessible CPUs                     : ?
List of accessible CPUs cores                 : ?
CFS Restrictions (CPUs worth of runtime)      : None

CPU Features                                  : 64bit adx aes avx avx2
                                                avx512bitalg avx512bw avx512cd
                                                avx512dq avx512f avx512ifma
                                                avx512vbmi avx512vbmi2 avx512vl
                                                avx512vnni avx512vpopcntdq bmi
                                                bmi2 clflushopt cmov cx16 cx8 f16c
                                                fma fsgsbase fxsr gfni invpcid
                                                lzcnt mmx movbe pclmul popcnt
                                                prfchw rdpid rdrnd rdseed sahf sgx
                                                sha sse sse2 sse3 sse4.1 sse4.2
                                                ssse3 vaes vpclmulqdq xsave xsavec
                                                xsaveopt xsaves

Memory Total (MB)                             : 16384
Free Memory (MB)                              : 66

__OS Information__
Platform Name                                 : Darwin-19.6.0-x86_64-i386-64bit
Platform Release                              : 19.6.0
OS Name                                       : Darwin
OS Version                                    : Darwin Kernel Version 19.6.0: Thu Oct 29 22:56:45 PDT 2020; root:xnu-6153.141.2.2~1/RELEASE_X86_64
OS Specific Version                           : 10.15.7   x86_64
Libc Version                                  : ?

__Python Information__
Python Compiler                               : Clang 10.0.0 
Python Implementation                         : CPython
Python Version                                : 3.7.9
Python Locale                                 : en_IE.UTF-8

__LLVM Information__
LLVM Version                                  : 10.0.1

__CUDA Information__
CUDA Device Initialized                       : False
CUDA Driver Version                           : ?
CUDA Detect Output:
None
CUDA Librairies Test Output:
None

__ROC information__
ROC Available                                 : False
ROC Toolchains                                : None
HSA Agents Count                              : 0
HSA Agents:
None
HSA Discrete GPUs Count                       : 0
HSA Discrete GPUs                             : None

__SVML Information__
SVML State, config.USING_SVML                 : False
SVML Library Loaded                           : False
llvmlite Using SVML Patched LLVM              : True
SVML Operational                              : False

__Threading Layer Information__
TBB Threading Layer Available                 : False
+--> Disabled due to Unknown import problem.
OpenMP Threading Layer Available              : True
+-->Vendor: Intel
Workqueue Threading Layer Available           : True
+-->Workqueue imported successfully.

__Numba Environment Variable Information__
None found.

__Conda Information__
Conda Build                                   : not installed
Conda Env                                     : 4.9.2
Conda Platform                                : osx-64
Conda Python Version                          : 3.8.5.final.0
Conda Root Writable                           : True

__Installed Packages__
appnope                   0.1.2           py37hecd8cb5_1001  
argon2-cffi               20.1.0           py37h9ed2024_1  
async_generator           1.10             py37h28b3542_0  
attrs                     20.3.0             pyhd3eb1b0_0  
blas                      1.0                         mkl  
bleach                    3.3.0              pyhd3eb1b0_0  
brotlipy                  0.7.0           py37h9ed2024_1003  
ca-certificates           2021.1.19            hecd8cb5_0  
certifi                   2020.12.5        py37hecd8cb5_0  
cffi                      1.14.3           py37hed5b41f_0    anaconda
chardet                   4.0.0           py37hecd8cb5_1003  
cryptography              3.3.1            py37hbcfaee0_1  
cycler                    0.10.0                   py37_0  
cython                    0.29.21          py37hb1e8313_0    anaconda
decorator                 4.4.2              pyhd3eb1b0_0  
defusedxml                0.6.0              pyhd3eb1b0_0  
entrypoints               0.3                      py37_0  
freetype                  2.10.4               ha233b18_0  
idna                      2.10               pyhd3eb1b0_0  
importlib-metadata        2.0.0                      py_1  
importlib_metadata        2.0.0                         1  
intel-openmp              2019.4                      233  
ipykernel                 5.3.4            py37he01cfaa_1    conda-forge
ipython                   5.8.0                    py37_1    conda-forge
ipython_genutils          0.2.0              pyhd3eb1b0_1  
jinja2                    2.11.3             pyhd3eb1b0_0  
json5                     0.9.5                      py_0  
jsonschema                3.2.0                      py_2  
jupyter_client            6.1.7                      py_0  
jupyter_core              4.7.1            py37hecd8cb5_0  
jupyterlab                2.2.9                      py_0    conda-forge
jupyterlab_pygments       0.1.2                      py_0  
jupyterlab_server         1.2.0                      py_0  
kiwisolver                1.3.1            py37h23ab428_0  
libcxx                    10.0.0                        1  
libedit                   3.1.20191231         h1de35cc_1  
libffi                    3.3                  hb1e8313_2  
libgfortran               3.0.1                h93005f0_2  
libllvm10                 10.0.1               h76017ad_5  
libpng                    1.6.37               ha441bb4_0  
libsodium                 1.0.18               h1de35cc_0  
llvm-openmp               10.0.0               h28b9765_0  
llvmlite                  0.35.0                   py37_0    numba
markupsafe                1.1.1            py37h1de35cc_0  
matplotlib                2.2.5                h694c41f_3    conda-forge
matplotlib-base           2.2.5            py37h11da6c2_1    conda-forge
mistune                   0.8.4            py37h1de35cc_0  
mkl                       2019.4                      233  
mkl-service               2.3.0            py37h9ed2024_0  
mkl_fft                   1.3.0            py37ha059aab_0  
mkl_random                1.1.1            py37h959d312_0  
mpi                       1.0                       mpich  
mpi4py                    3.0.3            py37h77202c6_0    anaconda
mpich                     3.3.2                hc856adb_0  
nbclient                  0.5.3              pyhd3eb1b0_0  
nbconvert                 6.0.7                    py37_0  
nbformat                  5.1.2              pyhd3eb1b0_1  
ncurses                   6.2                  h0a44026_1  
nest-asyncio              1.5.1              pyhd3eb1b0_0  
notebook                  6.2.0            py37hecd8cb5_0  
numba                     0.52.0          np1.11py3.7h6440ff4_g1882505_0    numba
numexpr                   2.7.1            py37hce01a72_0    anaconda
numpy                     1.19.1           py37h3b9f5b6_0    anaconda
numpy-base                1.19.1           py37hcfb5961_0    anaconda
openssl                   1.1.1j               h9ed2024_0  
packaging                 20.9               pyhd3eb1b0_0  
pandoc                    2.11                 h0dc7051_0  
pandocfilters             1.4.3            py37hecd8cb5_1  
pexpect                   4.8.0              pyhd3eb1b0_3  
pickleshare               0.7.5           pyhd3eb1b0_1003  
pip                       21.0.1           py37hecd8cb5_0  
prometheus_client         0.9.0              pyhd3eb1b0_0  
prompt_toolkit            1.0.15                     py_1    conda-forge
ptyprocess                0.7.0              pyhd3eb1b0_2  
pycparser                 2.20                       py_2  
pygments                  2.8.0              pyhd3eb1b0_0  
pyopenssl                 20.0.1             pyhd3eb1b0_1  
pyparsing                 2.4.7              pyhd3eb1b0_0  
pyrsistent                0.17.3           py37haf1e3a3_0  
pysocks                   1.7.1            py37hecd8cb5_0  
python                    3.7.9                h26836e1_0  
python-dateutil           2.8.1              pyhd3eb1b0_0  
python_abi                3.7                     1_cp37m    conda-forge
pytz                      2021.1             pyhd3eb1b0_0  
pyzmq                     20.0.0           py37h23ab428_1  
readline                  8.1                  h9ed2024_0  
requests                  2.25.1             pyhd3eb1b0_0  
send2trash                1.5.0              pyhd3eb1b0_1  
setuptools                52.0.0           py37hecd8cb5_0  
simplegeneric             0.8.1                    py37_2  
six                       1.15.0           py37hecd8cb5_0  
sqlite                    3.33.0               hffcf06c_0  
terminado                 0.9.2            py37hecd8cb5_0  
testpath                  0.4.4              pyhd3eb1b0_0  
time                      1.8                  h01d97ff_0    conda-forge
tk                        8.6.10               hb0a8c7a_0  
tornado                   6.1              py37h9ed2024_0  
traitlets                 5.0.5              pyhd3eb1b0_0  
urllib3                   1.26.3             pyhd3eb1b0_0  
wcwidth                   0.2.5                      py_0  
webencodings              0.5.1                    py37_1  
wheel                     0.36.2             pyhd3eb1b0_0  
xz                        5.2.5                h1de35cc_0  
zeromq                    4.3.3                hb1e8313_3  
zipp                      3.4.0              pyhd3eb1b0_0  
zlib                      1.2.11               h1de35cc_3  

No errors reported.


__Warning log__
Warning (cuda): CUDA driver library cannot be found or no CUDA enabled devices are present.
Exception class: <class 'numba.cuda.cudadrv.error.CudaSupportError'>
Warning (roc): Error initialising ROC: No ROC toolchains found.
Warning (roc): No HSA Agents found, encountered exception when searching: Error at driver init: 

HSA is not currently supported on this platform (darwin).
:
Warning (psutil): psutil cannot be imported. For more accuracy, consider installing it.
--------------------------------------------------------------------------------
If requested, please copy and paste the information between
the dashed (----) lines, or from a given specific section as
appropriate.

=============================================================
IMPORTANT: Please ensure that you are happy with sharing the
contents of the information present, any information that you
wish to keep private you should remove before sharing.
=============================================================

Thanks,
Fionnuala

I’m wondering if this is due to the combination of both intel-openmp and llvm-openmp being installed in the environment. Can you try creating a fresh environment with and activating it with e.g.:

conda create -n numba-test numba::numba=0.52.0 python=3.7
conda activate numba-test

then see if running the code in that container works (as a regular Python script, rather than a notebook). If that’s OK, then try installing jupyter:

conda install jupyter

and check if the notebook works (the reason for suggesting doing it separately is that I’m not sure what the dependencies of jupyter are, and whether that would have had a hand in causing multiple openmp implementations to be installed).

If this doesn’t work out for whatever reason, could you please post back any error messages you get, and the output of numba -s in the new environment please?

2 Likes

Hi again,

Ok so I created the environment and ran it as a script, the script being

import numpy as np
from numba import njit, prange
import timeit

def pi_montecarlo_python(n):
    in_circle = 0
    for i in prange(n):
        x, y = np.random.random(), np.random.random()
        if x ** 2 + y ** 2 <= 1.0:
            in_circle += 1
        
    return 4.0 * in_circle / n


pi_montecarlo_python_parallel = njit(parallel=True)(pi_montecarlo_python)

n = 100000

print('njit_python_parallel run 1')
t0 = timeit.timeit()
pi_montecarlo_python_parallel(n)
print(timeit.timeit()-t0)

print('njit_python_parallel run 2')
t1 = timeit.timeit()
pi_montecarlo_python_parallel(n)
print(timeit.timeit()-t1)

It ran but this was the output

njit_python_parallel run 1
OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.
0.001960516999999995
njit_python_parallel run 2
-1.5994999999935366e-05

So there is still something it doesn’t like about OMP ? Don’t always get negative time, that doesnt really make sense… :sweat_smile:

numba -s output
System info: -------------------------------------------------------------------------------- __Time Stamp__ Report started (local time) : 2021-03-05 09:29:36.784206 UTC start time : 2021-03-05 09:29:36.784217 Running time (s) : 1.497689

Hardware Information
Machine : x86_64
CPU Name : icelake-client
CPU Count : 8
Number of accessible CPUs : ?
List of accessible CPUs cores : ?
CFS Restrictions (CPUs worth of runtime) : None

CPU Features : 64bit adx aes avx avx2
avx512bitalg avx512bw avx512cd
avx512dq avx512f avx512ifma
avx512vbmi avx512vbmi2 avx512vl
avx512vnni avx512vpopcntdq bmi
bmi2 clflushopt cmov cx16 cx8 f16c
fma fsgsbase fxsr gfni invpcid
lzcnt mmx movbe pclmul popcnt
prfchw rdpid rdrnd rdseed sahf sgx
sha sse sse2 sse3 sse4.1 sse4.2
ssse3 vaes vpclmulqdq xsave xsavec
xsaveopt xsaves

Memory Total (MB) : 16384
Free Memory (MB) : 244

OS Information
Platform Name : Darwin-19.6.0-x86_64-i386-64bit
Platform Release : 19.6.0
OS Name : Darwin
OS Version : Darwin Kernel Version 19.6.0: Thu Oct 29 22:56:45 PDT 2020; root:xnu-6153.141.2.2~1/RELEASE_X86_64
OS Specific Version : 10.15.7 x86_64
Libc Version : ?

Python Information
Python Compiler : Clang 10.0.0
Python Implementation : CPython
Python Version : 3.7.10
Python Locale : en_IE.UTF-8

LLVM Information
LLVM Version : 10.0.1

CUDA Information
CUDA Device Initialized : False
CUDA Driver Version : ?
CUDA Detect Output:
None
CUDA Librairies Test Output:
None

ROC information
ROC Available : False
ROC Toolchains : None
HSA Agents Count : 0
HSA Agents:
None
HSA Discrete GPUs Count : 0
HSA Discrete GPUs : None

SVML Information
SVML State, config.USING_SVML : False
SVML Library Loaded : False
llvmlite Using SVML Patched LLVM : True
SVML Operational : False

Threading Layer Information
TBB Threading Layer Available : False
±-> Disabled due to Unknown import problem.
OpenMP Threading Layer Available : True
±->Vendor: Intel
Workqueue Threading Layer Available : True
±->Workqueue imported successfully.

Numba Environment Variable Information
None found.

Conda Information
Conda Build : not installed
Conda Env : 4.9.2
Conda Platform : osx-64
Conda Python Version : 3.8.5.final.0
Conda Root Writable : True

Installed Packages
blas 1.0 mkl
ca-certificates 2021.1.19 hecd8cb5_0
certifi 2020.12.5 py37hecd8cb5_0
intel-openmp 2019.4 233
libcxx 10.0.0 1
libedit 3.1.20191231 h1de35cc_1
libffi 3.3 hb1e8313_2
libllvm10 10.0.1 h76017ad_5
llvm-openmp 10.0.0 h28b9765_0
llvmlite 0.35.0 py37he4411ff_4
mkl 2019.4 233
mkl-service 2.3.0 py37h9ed2024_0
mkl_fft 1.3.0 py37ha059aab_0
mkl_random 1.1.1 py37h959d312_0
ncurses 6.2 h0a44026_1
numba 0.52.0 np1.11py3.7h6440ff4_g1882505_0 numba
numpy 1.19.2 py37h456fd55_0
numpy-base 1.19.2 py37hcfb5961_0
openssl 1.1.1j h9ed2024_0
pip 21.0.1 py37hecd8cb5_0
python 3.7.10 h88f2d9e_0
readline 8.1 h9ed2024_0
setuptools 52.0.0 py37hecd8cb5_0
six 1.15.0 py37hecd8cb5_0
sqlite 3.33.0 hffcf06c_0
tk 8.6.10 hb0a8c7a_0
wheel 0.36.2 pyhd3eb1b0_0
xz 5.2.5 h1de35cc_0
zlib 1.2.11 h1de35cc_3

No errors reported.

Warning log
Warning (cuda): CUDA driver library cannot be found or no CUDA enabled devices are present.
Exception class: <class ‘numba.cuda.cudadrv.error.CudaSupportError’>
Warning (roc): Error initialising ROC: No ROC toolchains found.
Warning (roc): No HSA Agents found, encountered exception when searching: Error at driver init:

HSA is not currently supported on this platform (darwin).
:
Warning (psutil): psutil cannot be imported. For more accuracy, consider installing it.

If requested, please copy and paste the information between
the dashed (----) lines, or from a given specific section as
appropriate.

=============================================================
IMPORTANT: Please ensure that you are happy with sharing the
contents of the information present, any information that you
wish to keep private you should remove before sharing.

Thanks for looking into this, really appreciated!

Fionnuala

Forgot to add;

Then install jupyterlab and it ran perfectly in there - ie didnt say anything about OMP and no crashing

Glad it now runs! In your new environment I still see both intel-openmp and llvm-openmp, so I guess it was the combination of some other packages that was causing the issue.

The message

OMP: Info #270: omp_set_nested routine deprecated, please use omp_set_max_active_levels instead.

is normal and expected - see Issue #5520. Because Numba supports a range of OpenMP versions, it has to use the omp_set_nested function which is deprecated in newer versions, because the suggested replacement is not present in older versions.

The message was probably not visible in the Jupyter notebook because it was written to standard output or error directly outside of Python - it may be visible on the terminal in which Jupyter was launched (if you launched it from a terminal).

Regarding the timing in your most recent example: I would recommend using time.perf_counter() to get a timestamp - IIUC timeit.timeit() gives the execution time of pass: timeit — Measure execution time of small code snippets — Python 3.9.2 documentation - so you’re subtracting two independent measurements of the time taken to execute pass, rather than the timestamp before and after executing your Monte Carlo function.

2 Likes

Hi @gmarkall,

Thank you!

Yes both intel-openmp and llvm-openmp were installed with the initial installation of Numba 0.52.0.

There are a few of those packages I will need to run in conjunction so I will add them one by one, and check if they cause the error.

Ok thanks for the time tip too :smiley:

There are a few of those packages I will need to run in conjunction so I will add them one by one, and check if they cause the error.

Many thanks - please do report back whether you get the error - if you do, it might be that there’s an issue we need to look into.

Yup I will, thanks :+1: