Compiling function in fortran then calling in numba

Numba
1

Hi, this might have been reported somewhere, but I don’t happen to find an answer.
I have the following code in test_fortran.f90

subroutine addstuff_wrap(a,b,c) bind(C,name=‘addstuff_wrap’)
use iso_c_binding
implicit none

real(c_double), intent(in) :: a, b 
real(c_double), intent(out) :: c 

c = a + b

end subroutine addstuff_wrap

and then i do to make an executable
gfortran test_fortran.f90 -shared -fPIC -o test_fortran90.so

then i write the following test_f90.py

import ctypes as ct
from numba import njit, jit
import numpy as np

mylib = ct.CDLL(‘./test_fortran90.so’)

mylib.addstuff_wrap.argtypes =[ct.c_void_p, ct.c_void_p, ct.c_void_p]

mylib.addstuff_wrap.restype = ct.c_double

@njit(‘float64(float64,float64)’)
def test(a,b):
aa = np.array(a)
bb = np.array(b)
c = np.array(0.0)

mylib.addstuff_wrap(aa.ctypes.c_double, bb.ctypes.c_double, c.ctypes.c_double)  
return c.item()

fty = ct.CFUNCTYPE(ct.c_double,ct.c_double,ct.c_double)(test)

@njit
def fty_wrapping(a,b):
return fty
a = ct.c_double(1.0)
b = ct.c_double(2.0)
print(‘W/ numba’,fty_wrapping(a,b))

neither functions fty_wrapping nor test are working
i keep getting the following error

numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Untyped global name ‘mylib’: Cannot determine Numba type of <class ‘ctypes.CDLL’>

File “test_f90.py”, line 21:
def test(a,b):

mylib.addstuff_wrap(aa.ctypes.c_double, bb.ctypes.c_double, c.ctypes.c_double)
^

This error may have been caused by the following argument(s):

  • argument 0: Cannot determine Numba type of <class ‘ctypes.c_double’>
  • argument 1: Cannot determine Numba type of <class ‘ctypes.c_double’>

any idea on what’s going on?

2

Hi @ahmad681

The error message says that Numba does not know how to handle the CDLL object inside a jitted function. There will also be other problems related to how Ctypes is used. Let’s see how you can make it work.
I don’t know Fortran so let’s do it in C. This is what a (hopefully) equivalent example would look like in C:

// c_lib.c
#ifdef _MSC_VER
#define DLL_EXPORT __declspec(dllexport)
#else
#define DLL_EXPORT
#endif

DLL_EXPORT void 
add_two_doubles_inplace(double a, double b, double *c)
{
    *c = a + b;
}

Once the shared library is built, you can use it as follows:

# main.py
import ctypes

import numba as nb
import numpy as np

# library name may differ
dll = ctypes.CDLL("./c_lib.cp311-win_amd64.pyd")

add_two_doubles_inplace = dll.add_two_doubles_inplace
add_two_doubles_inplace.argtypes = [
    ctypes.c_double,
    ctypes.c_double,
    ctypes.c_void_p,
]


@nb.njit
def test(a, b, c):
    add_two_doubles_inplace(a, b, c)


c1 = np.array(0, dtype=np.float64)
# c.ctypes.data works also inside 'test'
test(2.0, 3.0, c1.ctypes.data)
print(c1)


# allocation and 'addressof' must be outside 'test'
c2 = ctypes.c_double(0)
test(2.0, 3.0, ctypes.addressof(c2))
print(c2.value)

You can see that only a few ctypes constructs can be used inside the jitted function. For example, you cannot use ct.c_double(1.0) and aa.ctypes.c_double. The latter does not work in Python either, because such an attribute does not exist on arrays.

If you want to run the example yourself you can build the library with the setup file below using the command python setup.py build_ext --inplace (note that you may have to change the name of the extension in main.py):

# setup.py
from distutils.command.build_ext import build_ext as build_ext_distutils

from setuptools import Extension, setup


class build_ext(build_ext_distutils):
    def get_export_symbols(self, ext):
        return ext.export_symbols


setup(
    ext_modules=[
        Extension(
            "c_lib",
            sources=["c_lib.c"],
        )
    ],
    cmdclass={
        "build_ext": build_ext,
    },
)
3

Hi @sschaer thanks a lot for the wonderful reply.
Yup it worked perfectly fine, this is fantastic. However I had to modify the C code, because it was giving me the error of not finding
AttributeError: ./c_lib.so: undefined symbol: add_two_doubles_inplace

so I modified few things and I didn’t use that python script for building the lib extension:
and did gcc -fPIC -shared c_lib.C -o c_lib.so Preformatted text

#include <stdio.h>
extern “C” {
void add_two_doubles_inplace(double a, double b, double& c)
{
c = a + b;
}
}

notice I tried passing by a reference instead of a pointer, both worked just fine. Not sure what deprecation will be in the future, but that is a nice thing to take a note of at the moment.

The python script is the same except I am loading a shared library.

dll = ctypes.CDLL(“./c_lib.so”)

thank you so much for your help

4

Hi @ahmad681

Glad you found it helpful. The reason you encountered an attribute error is because you were compiling C++ code. Without declaring the function as “extern C,” the compiler alters the function name (known as name mangling).

Were you able to get your Fortran code to work as well?