How to install numba.openmp

I am trying to use multithreading in numba using PyOMP. This doesn’t seem to come with standard numbs installation.

from numba.openmp import openmp_context as openmp

`---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Input In [42], in
----> 1 from numba.openmp import openmp_context as openmp

ModuleNotFoundError: No module named ‘numba.openmp’`

This work appears to be based on a fork of Numba - it looks like the fork resides at GitHub - Python-for-HPC/PyOMP: OpenMP for Python in Numba

conda install numba cffi -c drtodd13 -c conda-forge --override-channels

I’m the author so please let me know if you have any problems. We are going to try to move this into mainline Numba after it is developed a bit more. In the meantime, you can use the command above.

Thank you. I have used the command, but still i get

from numba.openmp import openmp_context as openmp
Traceback (most recent call last):
File “”, line 1, in
ImportError: No module named ‘numba.openmp’

Did you create a new, clean conda environment to do the install in? Can you try that?

@DrTodd13 I have the same problem as the post author. I tried already your solution on a new Conda environment. Do you have any more tips? :frowning:

You get the exact same error? Did you activate the environment after creating it? Sorry for the simplistic question but I’ve had cases where that was the problem and it’s the first thing to check. Once you’re in the environment, would you please paste the result of “conda list”? Thanks! @roxaaams

I should have asked before but what OS and architectures are your machines? We only support Linux on x64 at the moment. The following tweaked command is better because it will force numba to come from my channel and won’t fall back to conda-forge and pick-up a Numba from there that matches your OS/arch pair. I suspect that is what happened in this case. We’re working on a Windows/x64 build at the moment so that should be coming soon.

conda install drtodd13::numba cffi -c conda-forge --override-channels

Hi.
I’ll try to install on linux x64 systeme with the following command :
conda install drtodd13::numba cffi -c conda-forge --override-channels
Unfortunately when I import njit from numba I obtain this error:
Traceback (most recent call last):
File “”, line 1, in
File “/soft/DL/conda_envs/omp/lib/python3.7/site-packages/numba/init.py”, line 19, in
from numba.core import config
File “/soft/DL/conda_envs/omp/lib/python3.7/site-packages/numba/core/config.py”, line 16, in
import llvmlite.binding as ll
File “/soft/DL/conda_envs/omp/lib/python3.7/site-packages/llvmlite/binding/init.py”, line 4, in
from .dylib import *
File “/soft/DL/conda_envs/omp/lib/python3.7/site-packages/llvmlite/binding/dylib.py”, line 3, in
from llvmlite.binding import ffi
File “/soft/DL/conda_envs/omp/lib/python3.7/site-packages/llvmlite/binding/ffi.py”, line 191, in
raise OSError(“Could not load shared object file: {}”.format(_lib_name))
OSError: Could not load shared object file: libllvmlite.so

Numba works well on the same system in another installation by conda.
Thanks for your help, pyOMP seems very useful

Hi.

When I tried to build PyOMP from the github project I get this error:
passmanagers.cpp:10:10: fatal error: llvm-c/Transforms/IntrinsicsOpenMP.h: No such file or directory
The IntrinsicsOpenMP.h head file not exist in the llvm-project source code

I get the same error as you when I tied to build llvmliteWithOpenmp from it’s own repository. I’m trying now to build the PyOMP project with all three git submodules checked out in the right locations.

Note that I made a pull request here to fix the git repo locations, since all except for the llvm-project had been moved ownership from @ggeorgakoudis to @Python-for-HPC.

Here are those git submodules needed. I’m not sure if that will help, but it’s where I’m at in the process of trying to build…

# .gitmodules
[submodule "llvm-project"]
path = llvm-project
url = https://github.com/ggeorgakoudis/llvm-project.git

[submodule "llvmliteWithOpenmp"]
path = llvmliteWithOpenmp
url = https://github.com/Python-for-HPC/llvmliteWithOpenmp.git

[submodule "NumbaWithOpenmp"]
path = NumbaWithOpenmp
url = https://github.com/Python-for-HPC/NumbaWithOpenmp.git

Hi I’m having the same issue as the author but I use VSCode instead of Anaconda for Python, is it possible for me to get this openmp subpackage from numba ? (I already installed numba with pip)

The best way to get Numba with OpenMP is to create a clean conda environment (I recommend Python 3.9 at this time) and run the following command:

conda install Python-for-HPC::numba Python-for-HPC::llvmlite -c conda-forge -–override-channels

If you aren’t or can’t use conda then at this point you’d need to build everything by hand which is very difficult.

Thank you very much, with a clean conda environment it seems to work well, I just have this warning at the execution of the example provided on the GitHub page :

UserWarning: llvmlite version format not recognized!
warnings.warn(“llvmlite version format not recognized!”)

However it does print the value of pi.
Do you have an idea of why I get this warning ?

You can disable the warning with

import logging;
logging.disable(logging.WARNING)

I’ll need to fix the source and create a new version to permanently get rid of the warning. It is just because the llvmlite version format string that we generate is not what Numba uses and so Numba can’t parse it to verify that it is compatible.

So it won’t work with WSL2?

I’m trying to make it work with WSL2, but I keep getting:

ModuleNotFoundError: No module named 'numba.openmp'

when trying to import numba.openmp.

So far i’ve tried to create clean conda environments with python 3.9 and 3.10. For both versions i runned combinations of these installations:

conda install Python-for-HPC::numba Python-for-HPC::llvmlite -c conda-forge -–override-channels
conda install numba cffi -c drtodd13 -c conda-forge --override-channels
conda install -c python-for-hpc -c conda-forge --override-channels pyomp

conda install -c python-for-hpc -c conda-forge --override-channels pyomp

The latest status is that using the above command, you can get pyomp for linux/x86, linux/powerpc, or mac/arm. We’ve put a lot of work into getting this packaged which is quite difficult when combined with the stuff you need for GPU overload support through openmp target directives. Unfortunately, because we use LLVM internally which does not support target offload on windows, we cannot provide that either. We’ve been focused on getting the linux GPU offload builds working but we do plan to go back and do a Windows build, which will have support for CPU only and should include 3.9 and 3.10 support. We’re giving a pyomp tutorial at supercomputing in November so expect to see us have these Windows builds before that point.

1 Like

@qedrohenrique We are not super familiar with WSL builds. If it appears as linux, then we would expect the last line to work. However, if the OS or architecture isn’t supported then the last line will fall back to the Numba in conda-forge and the problem you see is exactly what I would expect in this case. If you do the last line, can you report what “conda list” says because that should show where it got the Numba package. What happened when you tried the first line though because that should work as well and should give you a conda failure if we aren’t currently providing the os/arch combination that you have. Can you verify what happens with the first line? If it seems to succeed then again verify with “conda list” and let me know what it says. Can you see which OS and architecture conda is trying to find the package for?
thanks.

By executing:

conda create -n py-omp python=3.10
conda activate py-omp
conda install Python-for-HPC::numba Python-for-HPC::llvmlite -c conda-forge --override-channels

The output for the last command is:

Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/pedro/miniconda3/envs/py-omp

  added / updated specs:
    - Python-for-HPC::llvmlite
    - Python-for-HPC::numba


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    llvmlite-0.40_pyomp        |  py310h84b9e52_0        26.1 MB  Python-for-HPC
    numba-0.57_pyomp           |np1.22py3.9h84b9e52_0.57_pyomp       105.5 MB  Python-for-HPC
    ------------------------------------------------------------
                                           Total:       131.6 MB

The following NEW packages will be INSTALLED:

  cffi               conda-forge/linux-64::cffi-1.16.0-py310h2fee648_0
  icu                conda-forge/linux-64::icu-75.1-he02047a_0
  lark-parser        conda-forge/noarch::lark-parser-0.12.0-pyhd8ed1ab_0
  libblas            conda-forge/linux-64::libblas-3.9.0-23_linux64_openblas
  libcblas           conda-forge/linux-64::libcblas-3.9.0-23_linux64_openblas
  libgfortran-ng     conda-forge/linux-64::libgfortran-ng-14.1.0-h69a702a_0
  libgfortran5       conda-forge/linux-64::libgfortran5-14.1.0-hc5f4f2c_0
  libhwloc           conda-forge/linux-64::libhwloc-2.11.1-default_hecaa2ac_1000
  libiconv           conda-forge/linux-64::libiconv-1.17-hd590300_2
  liblapack          conda-forge/linux-64::liblapack-3.9.0-23_linux64_openblas
  libnsl             conda-forge/linux-64::libnsl-2.0.1-hd590300_0
  libopenblas        conda-forge/linux-64::libopenblas-0.3.27-pthreads_hac2b453_1
  libsqlite          conda-forge/linux-64::libsqlite-3.46.0-hde9e2c9_0
  libxcrypt          conda-forge/linux-64::libxcrypt-4.4.36-hd590300_1
  libxml2            conda-forge/linux-64::libxml2-2.12.7-he7c6b58_4
  libzlib            conda-forge/linux-64::libzlib-1.3.1-h4ab18f5_1
  llvm-openmp        conda-forge/linux-64::llvm-openmp-18.1.8-hf5423f3_0
  llvmlite           Python-for-HPC/linux-64::llvmlite-0.40_pyomp-py310h84b9e52_0
  numba              Python-for-HPC/linux-64::numba-0.57_pyomp-np1.22py3.9h84b9e52_0.57_pyomp
  numpy              conda-forge/linux-64::numpy-1.24.4-py310ha4c1d20_0
  pycparser          conda-forge/noarch::pycparser-2.22-pyhd8ed1ab_0
  python_abi         conda-forge/linux-64::python_abi-3.10-4_cp310
  tbb                conda-forge/linux-64::tbb-2021.12.0-h434a139_3
  zstd               conda-forge/linux-64::zstd-1.5.6-ha6fb4c9_0

The following packages will be REMOVED:

  libgomp-11.2.0-h1234567_1
  sqlite-3.45.3-h5eee18b_0
  zlib-1.2.13-h5eee18b_1

The following packages will be UPDATED:

  ca-certificates    pkgs/main::ca-certificates-2024.7.2-h~ --> conda-forge::ca-certificates-2024.7.4-hbcca054_0
  libgcc-ng          pkgs/main::libgcc-ng-11.2.0-h1234567_1 --> conda-forge::libgcc-ng-14.1.0-h77fa898_0
  libstdcxx-ng       pkgs/main::libstdcxx-ng-11.2.0-h12345~ --> conda-forge::libstdcxx-ng-14.1.0-hc0a3c3a_0
  libuuid              pkgs/main::libuuid-1.41.5-h5eee18b_0 --> conda-forge::libuuid-2.38.1-h0b41bf4_0
  ncurses                 pkgs/main::ncurses-6.4-h6a678d5_0 --> conda-forge::ncurses-6.5-h59595ed_0
  openssl              pkgs/main::openssl-3.0.14-h5eee18b_0 --> conda-forge::openssl-3.3.1-h4bc722e_2

The following packages will be SUPERSEDED by a higher-priority channel:

  _libgcc_mutex           pkgs/main::_libgcc_mutex-0.1-main --> conda-forge::_libgcc_mutex-0.1-conda_forge
  _openmp_mutex          pkgs/main::_openmp_mutex-5.1-1_gnu --> conda-forge::_openmp_mutex-4.5-2_kmp_llvm
  python               pkgs/main::python-3.10.14-h955ad1f_1 --> conda-forge::python-3.10.14-hd12c33a_0_cpython
  tk                        pkgs/main::tk-8.6.14-h39e8969_0 --> conda-forge::tk-8.6.13-noxft_h4845f30_101


Proceed ([y]/n)? y


Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

And finally when conda list:

# packages in environment at /home/pedro/miniconda3/envs/py-omp:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                  2_kmp_llvm    conda-forge
bzip2                     1.0.8                h5eee18b_6
ca-certificates           2024.7.4             hbcca054_0    conda-forge
cffi                      1.16.0          py310h2fee648_0    conda-forge
icu                       75.1                 he02047a_0    conda-forge
lark-parser               0.12.0             pyhd8ed1ab_0    conda-forge
ld_impl_linux-64          2.38                 h1181459_1
libblas                   3.9.0           23_linux64_openblas    conda-forge
libcblas                  3.9.0           23_linux64_openblas    conda-forge
libffi                    3.4.4                h6a678d5_1
libgcc-ng                 14.1.0               h77fa898_0    conda-forge
libgfortran-ng            14.1.0               h69a702a_0    conda-forge
libgfortran5              14.1.0               hc5f4f2c_0    conda-forge
libhwloc                  2.11.1          default_hecaa2ac_1000    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
liblapack                 3.9.0           23_linux64_openblas    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.27          pthreads_hac2b453_1    conda-forge
libsqlite                 3.46.0               hde9e2c9_0    conda-forge
libstdcxx-ng              14.1.0               hc0a3c3a_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxml2                   2.12.7               he7c6b58_4    conda-forge
libzlib                   1.3.1                h4ab18f5_1    conda-forge
llvm-openmp               18.1.8               hf5423f3_0    conda-forge
llvmlite                  0.40_pyomp      py310h84b9e52_0    Python-for-HPC
ncurses                   6.5                  h59595ed_0    conda-forge
numba                     0.57_pyomp      np1.22py3.9h84b9e52_0.57_pyomp    Python-for-HPC
numpy                     1.24.4          py310ha4c1d20_0    conda-forge
openssl                   3.3.1                h4bc722e_2    conda-forge
pip                       24.0            py310h06a4308_0
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
python                    3.10.14         hd12c33a_0_cpython    conda-forge
python_abi                3.10                    4_cp310    conda-forge
readline                  8.2                  h5eee18b_0
setuptools                69.5.1          py310h06a4308_0
tbb                       2021.12.0            h434a139_3    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tzdata                    2024a                h04d1e81_0
wheel                     0.43.0          py310h06a4308_0
xz                        5.4.6                h5eee18b_1
zstd                      1.5.6                ha6fb4c9_0    conda-forge

About the last question, i’m not sure about which informations I should look for, but:

(py-omp) pedro@DESKTOP-4081VND:~$ conda info

     active environment : py-omp
    active env location : /home/pedro/miniconda3/envs/py-omp
            shell level : 4
       user config file : /home/pedro/.condarc
 populated config files : /home/pedro/.condarc
          conda version : 24.7.1
    conda-build version : not installed
         python version : 3.10.0.final.0
                 solver : classic
       virtual packages : __archspec=1=skylake
                          __conda=24.7.1=0
                          __cuda=12.4=0
                          __glibc=2.35=0
                          __linux=5.15.153.1=0
                          __unix=0=0
       base environment : /home/pedro/miniconda3  (writable)
      conda av data dir : /home/pedro/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/pedro/miniconda3/pkgs
                          /home/pedro/.conda/pkgs
       envs directories : /home/pedro/miniconda3/envs
                          /home/pedro/.conda/envs
               platform : linux-64
             user-agent : conda/24.7.1 requests/2.32.3 CPython/3.10.0 Linux/5.15.153.1-microsoft-standard-WSL2 ubuntu/22.04.3 glibc/2.35 aau/0.4.4 c/. s/. e/.
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

And:

(py-omp) pedro@DESKTOP-4081VND:~$ screenfetch
                          ./+o+-       pedro@DESKTOP-4081VND
                  yyyyy- -yyyyyy+      OS: Ubuntu 22.04 LTS (Jammy Jellyfish)(on the Windows Subsystem for Linux)
               ://+//////-yyyyyyo      Kernel: x86_64 Linux 5.15.153.1-microsoft-standard-WSL2
           .++ .:/++++++/-.+sss/`      Uptime: 7m
         .:++o:  /++++++++/:--:/-      Packages: 558
        o:+o+:++.`..```.-/oo+++++/     Shell: bash 5.1.16
       .:+o:+o/.          `+sssoo+/    Resolution: No X Server
  .++/+:+oo+o:`             /sssooo.   WM: Not Found
 /+++//+:`oo+o               /::--:.   GTK Theme:  [GTK3]
 \+/+o+++`o++o               ++////.   Disk: 693G / 2.4T (30%)
  .++.o+++oo+:`             /dddhhh.   CPU: Intel Core i7-10700KF @ 16x 3.792GHz
       .+.o+oo:.          `oddhhhh+    GPU: NVIDIA GeForce RTX 3060 Ti
        \+.++o+o``-````.:ohdhhhhh+     RAM: 786MiB / 7890MiB
         `:o+++ `ohhhhhhhhyo++os:
           .o:`.syhhhhhhh/.oo++o`
               /osyyyyyyo++ooo+++/
                   ````` +oo+++o\:
                          `oo++.

@qedrohenrique In your py-omp environment, please run “which python” and make sure it is coming from the py-omp conda environment directory. Then, run python and import numba. Assuming that works do print(numba.__file__) and let me know the path you are getting.