Cannot determine Numba type of <class 'type'>

[yiyezhilan]

Anyone can help? please. Cannot determine Numba type of <class ‘type’>. This is my code and the input xq is a (N,3) ndarray.
@njit(parallel=True)
def fraction_field(xq):
N = 15
none = np.nan
fq = np.zeros((xq.shape[0],))
delta = 5
tt = np.arange(0, 360, delta, dtype=float) / 180 * pi
pp = np.arange(0, 180, delta, dtype=float) / 180 * pi
theta, phi = np.meshgrid(tt, pp)
theta = theta.reshape((-1,))
phi = phi.reshape((-1,))
offset = np.array(
[
d / 2 * np.sin(phi) * np.cos(theta),
d / 2 * np.sin(phi) * np.sin(theta),
d / 2 * np.cos(phi),
]
).T
for j in prange(xq.shape[0]):
if j % 1000==0:
print(j)
R_xq = np.sqrt((xq[j, 0] - x) ** 2 + (xq[j, 2] - z) ** 2)
if R_xq <= r:
fq[j] = none
else:
dist = np.sqrt(np.sum((pos - xq[j, :]) ** 2, axis=1))
R_list = np.array([np.arange(dist.shape[0], dtype=float), dist]).T
R_list = R_list[R_list[:, 1].argsort(), :]
idx_N = R_list[:N, 0].astype(int)
pos_N = pos[idx_N, :]
point = np.zeros((N, theta.shape[0], 3))
for i in range(N):
point[i, :, :] = pos_N[i, :] + offset
point = point.reshape((-1, 3))
vol = ConvexHull(point).volume
fq[j] = N * 4 / 3 * pi * (d / 2) ** 3 / vol
return fq

And a error occured:
Traceback (most recent call last):
File “C:\Users\chinylan\Desktop\ConvexHull\convex_fraction.py”, line 127, in
frac = fraction_field(Xq)

File “C:\Users\chinylan\Desktop\ConvexHull\convex_fraction.py”, line 106, in fraction_field
fq=loop()

File “C:\Users\chinylan\anaconda3\lib\site-packages\numba\core\dispatcher.py”, line 482, in _compile_for_args
error_rewrite(e, ‘typing’)

File “C:\Users\chinylan\anaconda3\lib\site-packages\numba\core\dispatcher.py”, line 423, in error_rewrite
raise e.with_traceback(None)

TypingError: Cannot determine Numba type of <class ‘type’>

[Hannes]

Hi @yiyezhilan

Could you please edit your post and surround your code with triple backticks? I.e.

```python
[your code]
```

That will make it much easier to read and help.

Cheers!

[yiyezhilan]

Sorry. I missed this. And this is revised. Cannot determine Numba type of <class ‘type’>. This is my code and the input xq is a (N,3) ndarray.

‘’‘
@njit(parallel=True)
def fraction_field(xq):
N = 15
none = np.nan
fq = np.zeros((xq.shape[0],))
delta = 5
tt = np.arange(0, 360, delta, dtype=float) / 180 * pi
pp = np.arange(0, 180, delta, dtype=float) / 180 * pi
theta, phi = np.meshgrid(tt, pp)
theta = theta.reshape((-1,))
phi = phi.reshape((-1,))
offset = np.array(
[
d / 2 * np.sin(phi) * np.cos(theta),
d / 2 * np.sin(phi) * np.sin(theta),
d / 2 * np.cos(phi),
]
).T
for j in prange(xq.shape[0]):
if j % 1000==0:
print(j)
R_xq = np.sqrt((xq[j, 0] - x) ** 2 + (xq[j, 2] - z) ** 2)
if R_xq <= r:
fq[j] = none
else:
dist = np.sqrt(np.sum((pos - xq[j, :]) ** 2, axis=1))
R_list = np.array([np.arange(dist.shape[0], dtype=float), dist]).T
R_list = R_list[R_list[:, 1].argsort(), :]
idx_N = R_list[:N, 0].astype(int)
pos_N = pos[idx_N, :]
point = np.zeros((N, theta.shape[0], 3))
for i in range(N):
point[i, :, :] = pos_N[i, :] + offset
point = point.reshape((-1, 3))
vol = ConvexHull(point).volume
fq[j] = N * 4 / 3 * pi * (d / 2) ** 3 / vol
return fq
’‘’

And a error occured:
Traceback (most recent call last):
File “C:\Users\chinylan\Desktop\ConvexHull\convex_fraction.py”, line 127, in
frac = fraction_field(Xq)

File “C:\Users\chinylan\anaconda3\lib\site-packages\numba\core\dispatcher.py”, line 482, in _compile_for_args
error_rewrite(e, ‘typing’)

File “C:\Users\chinylan\anaconda3\lib\site-packages\numba\core\dispatcher.py”, line 423, in error_rewrite
raise e.with_traceback(None)

TypingError: Cannot determine Numba type of <class ‘type’>

[Hannes]

Hi again,

no worries, markdown takes some getting used to.

You did not use backticks ( ` ) but apostrophes ( ’ ) . If your keyboard does not feature backticks, just copy and paste them from my message. [EDIT: You can also press the </> button or Ctrl+E here on discourse to get the triple tick environment]

I tried to have a look at your example anyway, but unfortunately it is not a good reproducer. Please make sure your example can simply be copied and executed. Right now your code contains several undefined variables and the lack of indentation makes it impossible to reconstruct the control flow.

Also check if your code actually runs without the njit decorator (if it is too slow, reduce the input data). A conventional python error may be hiding somewhere but njit will generally make it harder to catch and decipher possibly raised error messages.

[Rutger]

Hey,

Apart from the formatting Hannes mentioned, which is important because it’s not obvious when the indentation should shift back, at the end of for-loops or the else clause for example.

Even without that, I already notice some parameters and functions/classes that are not defined within the function itself. That can perhaps be the cause of the issue, but it’s impossible to know based on the code you share. If the types of those are something that Numba njit can handle, like simple scalars, it could be fine. But it’s probably still good practice to be a bit more explicit and perhaps pass those parameters as arguments to the function.

For example the d parameters used when creating the offset array. It’s not possible for anyone to guess what that is.

Perhaps something to start with it the ConvexHull class (or function). It reads to me as if it comes from scipy.spatial, since you call the volume attribute. If that’s the case, it won’t work for sure within an njit compiled function. A simple test would be commenting those last two lines before the return and see if the function will run (with an incorrect result of course). If that’s the case you know you identified the main issue.

A next step could be to opt-in to “object mode” specifically for that line and see if that works, with the correct result. An example of entering object mode from an njit function can be seen at:
https://numba.pydata.org/numba-doc/latest/user/withobjmode.html#the-objmode-context-manager

Here’s a simple example demonstrating how that would work, you should be able to copy/paste those two lines in the function straight away.

from scipy.spatial import ConvexHull
from numba import njit, objmode

@njit
def get_volume(point):
    
    with objmode(vol='float32'):
        vol = ConvexHull(point).volume
        
    return vol

point = np.random.randn(10,3)
vol = get_volume(point)`

There will be a performance penalty for using this objmode trick. So it really depends on the volume of data you’re working with etc whether it’s acceptable, and what the performance gain is over the non-Numba implementation.

I often check to see if there are any algorithms available that are easy to implement in Numba, in this case the ConvexHull().volume. It looks like you’re working with 3D data? I do know that for a 2D case it should probably be quite easy to implement a convex hull yourself using the gift-wrapping algorithm which isn’t the most efficient, but for small amounts of points it could be good enough.

Calculating the volume of a 3D convex hull seems more difficult, I have no experience with it. It might be done with some sort of triangulation, but that’s doesn’t seem easy to write up in Numba from scratch. It’s a very common problem, so there’s probably a ton of information about that online.

Regards,
Rutger

[yiyezhilan]

I’m sorry about that. And I think I found the problem is caused by the function ConvexHull in Scipy. Also, thanks for your patience and tolerance for my problem.

[yiyezhilan]

Thank you very much! I used MATLAB before and try to use python for scientific computing recently. And I’m a beginner for python.
I think the function ConvexHull is the key for this problem. It seems that a method needs to be implemented by myself instead of by other package.
Thank you for your generosity again!

[Hannes]

According to the scipy docs it’s ConvexHull implementation calls the QHull library.

It may be possible to replicate any necessary preprocessing code that scipy implements, and then make a call into the QHull library from within numba code. But that does not feel like beginner territory. (Or my lack of C / Fortran experience is scaring me more than it should)

[yiyezhilan]

I used numba to do the works except the computing of ConvexHul and stored needed data. Then compute ConvexHull with multiprocess of python. Though this is slower than those implemented all by numba, it faster than using python only.

[Hannes]

From a quick glance at your code that actually seems close to optimal (unless you are calling this function extremely often for small data). QHull is likely already very much optimised and it is a compiled library. So probably there is not much to be gained for that step anyway.
I would guesstimate that creating the hull and computing the volume are fairly work intensive, so any runtime should not be dominated by crossing the boundary between interpreted Python and compiled code.

EDIT:

If this function should be part of a longer call chain that you want to numba-fy entirely, then do take a look at @Rutger 's suggestion. There will be a small local penalty for switching into object mode but you can then make it part of the compiled function without numba getting upset.