Hello,
Is it possible to export a python function with cc.export to create a module library (.so) and then call the python function from c or c++ by linking against the module library ?
I succeeded in doing it by using pybind, but I would like to know if there is a way to directly call the exported function as a c function.
My goal is to embed python functions in c/c++ library or executable.
It’s not possible as far as I know… You still need a Python interpreter.
You mentioned you did it already with pybind. I didn’t quite understand what you did, but it is possible to get the c function pointers from numba functions that are compiled in a pybind interpreter and then call them directly from the c++ process outside the interpreter.
I use this expensively to get effectively native c++ speed from numba compiled python. (We internally call it native performance embedded python)
The idea you provided is really inspiring! However, I still have some questions. Using pybind interpreter means that the numba functions are compiled during running. I wonder if there any methods that I can compile them AOT(ahead of time), and directly get the c function ptr during runtime?
Bsides，could you explain detaily about how to get the c function pointers?
Hi Kane,
I finally didn’t use numba.
I used cython to transform python to c++ and build c++.
I created a pyx file to wrap a python function and be able to call it from c++.
The reason I am trying to use numba is to getting arround GIL when calling the python function in a multithreading way. However I failed. Even though I get the C funtion pointer of numba cfunc, it still hold GIL during calling.
If you have any idea, I would really appreciate it
Numba-compiled functions don’t release the GIL unless you pass the nogil=True kwarg to the decorator. I’m not sure if this kwarg works with @cc.export though - can you try it and see if you observe a difference?
“nogil” is not avaible in the @cc.export. However, with @jit(nogil=True) decorating the function, it is can be called parallel in python.
My goal is to run the numba compiled function parallel in Cpp threads with GIL released. Would you give me some advice or hints~?
I can’t think of a straightforward solution, but perhaps you could use CPython’s C API to release the GIL from your C++ code prior to calling your Numba-compiled code in parallel? Numba does this with calls to PyEval_SaveThread() and PyEval_RestoreThread() when nogil=True.
For getting the cfunc address I started here
Really appreciate it! I thought my problems are solved.