Hi,
I have converted a few functions successfully to @njit, it took some finessing, but everything works.
Problem is that when building the py app and bundling into a single executable file, the llvmlite.dll needed to run @njit functions is a massive dependency of 76.x MB - the app w/o numba was only 20 MB.
Is there any way to reduce the size of llvmlite.dll?
I then looked at AOT compilation, but it’s unclear what the future of that is in the numba ecosystem… (?)
Also, how much slower will that be on the user’s machine given that the machine code is not optimized like @njit w/ llvmlite…?
Last, and this was a red flag:
An AOT compiled function was called (by mistake, in testing) w/ an integer of 2 instead of a float 2.0 (the fn sig specifies float) - the AOT fn did not throw an error, but it now returned an incorrect result - this is scary.
How can this be prevented realistically in larger numba code bases where AOT fns call other AOT functions w/ dynamic variables?
Here is the code to reproduce this:
numba 0.60.0
Win 10 x64
py 3.10
from numba.pycc import CC
import math
cc = CC('test')
@cc.export('ded', 'float64(float64[:], float64[:])')
def ded(p1, p2):
return math.sqrt(((p1[0] - p2[0]) ** 2) + ((p1[1] - p2[1]) ** 2) + ((p1[2] - p2[2]) ** 2))
cc.compile()
Unfortunately, I have no solution regarding the dependency issue of llvmlite.
The current process of AOT compilation in Numba has some limitations.
AOT in Numba lacks type checking, so passing incorrect types (e.g., int instead of float) can lead to incorrect results without errors. https://numba.readthedocs.io/en/stable/user/pycc.html#limitations
You could preprocess your inputs to ensure correct types or specify all necessary signatures.
You could preprocess your inputs to ensure correct types or specify all necessary signatures
could you elaborate on this?
the AOT functions require a signature, which I did specify in my example, did you mean to add another signature for int input?
but this would then require a different fn name, and that would mean to change the fn calling logic based on that…
re the preprocessing of inputs, how would this realistically work? can you share a short example?
I have numba jitted code that amounts to ca. 1,200 lines of various jitted functions and processes calling other jitted functions. Almost all numerical value are ndarray floats and defined as such, but numerical accuracy is of the utmost importance in these calculations, so cannot take a chance or risk of any process or fn compromising numerical results.
The incorrect results above do NOT occur w/ jitted fns.
@nb113 jitted functions can automatically expand their scope at runtime. The njit decorator is capable to handle a wide variety of input types and automatically generates corresponding signatures.
You can inspect the generated signatures for your examples:
If you want more control over the signatures in AOT or eager compilation (e.g., to support various combinations of float64, int64, and List), you can manually specify them using a list of signature strings (and function names in AOT).
But defining all possible combinations can quickly become very impractical:
In AOT compilation, you have to specify input and output types in advance, which can be error-prone if you miss and allow unexpected possible type combinations. A more practical approach is to limit the signature combinations to the most common ones.
Additionally, with the looming deprecation of this AOT process, it may be more beneficial to rely on jitted functions, which are more user friendly (if that’s possible).
Fyi there is an open issue on Github regarding a similar issue:
@Oyibo
thanks so much for the detailed information, it makes sense.
In AOT compilation, you have to specify input and output types in advance, which can be error-prone if you miss and allow unexpected possible type combinations. A more practical approach is to limit the signature combinations to the most common ones.
It’s probably easier to work with function wrappers that make sure, your function input types are correct.
Can you use asarray to make sure the types are in line with your operation?
ok, the ded() fn from last response when AOT compiled did not change anything, but regarding function wrappers I assume I’d use a function in Py to convert data as needed and then call AOT binary fn w/ that input data?
this would work if the AOT binary fns are called individually from Py, but my code that I want to AOT compile runs a large fn that then repeatedly calls other AOT compiled fns - so the wrapper fn approach makes little sense there but I guess I could force convert all input data always to required data type before calling other AOT fns… if this decreases execution speed then AOT compilation is not worthwhile in my case.
Just to confirm, have you compiled every function signature with a designated function name (one name per signature)?
no, I’ve not b/c that is not usable in my code, as I’d have to dynamically use a different fn name to call for the same basic functionality.
if int happens to sneak for whatever reason, I just need the fn to perform same as it does under @njit, most importantly return the correct result - but it seems this is not possible with numba AOT approach.
I’ve now tested this particular bit w/ Cython - more hassle to set up - but it handles both int and float nicely and crashes w/ error if strict types are employed - numba AOT crashes silently - not good.
Problem w/ Cython is that it is not as fast as numba (in my limited testing) and the whole reason we’re going the machine code route is to get fastest possible speed…