Numba OptionalType(array()) - Problems

Numba Support: What is this error message?
1

Hi there,
I am developing a script using numba’s njit. I found myself stuck in a tricky situation. In overall terms, the funcs. giving the error look like the following:

@jit(nopython = True)
def dot_pred(a, b, c):
    return a @ np.linalg.inv(b) @ c.T

@jit(nopython = True)
def foo(maindim):
    aa = np.random.randn(maindim, 5)
    bb = np.random.randn(5,5)
    cc = np.random.randn(maindim, 5)
    # Call the problem func (njitted)
    out = dot_pred(aa, bb, cc)
    return out

Of course, these two funcs. work with Numba (as expected), but in my code (much more large) it does not… The error is the following:

TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(\<function inv at 0x7ff7a8469f30>) found for signature:

> > > inv(OptionalType(array(float64, 2d, C)))

There are 2 candidate implementations:

- Of which 2 did not match due to:
Overload in function 'inv_impl': File: numba/np/linalg.py: Line 832.
With argument(s): '(OptionalType(array(float64, 2d, C)))':
Rejected as the implementation raised a specific error:
AttributeError: 'Optional' object has no attribute 'dtype'
raised from /home/xxxxxxxxxxxxx/python3.10/site-packages/numba/np/linalg.py:838

During: resolving callee type: Function(\<function inv at 0x7ff7a8469f30>)
During: typing of call at /tmp/ipykernel\_900007/3472101925.py (159)

To sum up,
- The demo case with two funcs. (see above) works compiled with numba
- My code works (results are ok and code is stable) without numba. Yet, when I njit I got the error above

The problem seems to be with the Optional() type and the ´dtype´ method that ´np.linalg.inv´ uses over the Optional type.

Any help on how to solve this, please?

Thanks a lot!

2

Hi @joueswant,

I get that you may shy away from sharing long code - but if you don’t show it (or better: narrow it down to the smallest failing unit and share that), it will be hard to help. If you are not allowed to share the code, then I understand that your hands may be tied. :stuck_out_tongue:

Anyway, in the docs we find Glossary — Numba 0+untagged.4124.gd4460fe.dirty documentation :

OptionalType
An OptionalType is effectively a type union of a type and None. They typically occur in practice due to a variable being set to None and then in a branch the variable being set to some other value. It’s often not possible at compile time to determine if the branch will execute so to permit type inference to complete, the type of the variable becomes the union of a type (from the value) and None, i.e. OptionalType(type).

Together with the error context you received

> > > inv(OptionalType(array(float64, 2d, C)))

we can assume that the problem is not with what inv uses or returns, but what you pass it: Under certain circumstances numba infers that the variable (presumably whatever gets passed to b of dot_pred) is None.

Possibly this won’t actually ever happen in your code due to checks you perform around the function call - but somewhere in the same scope a variable with the same name could become None. This can be enough to confuse the compiler. I think in simple cases this can be caught and inferred correctly by numba, but it depends on the exact context.

Maybe you can alleviate the problem by passing a variable to dot_pred that can never be None in any branch of your function.

1 Like
4

I see, it is a mix between the None type defined in subop2 and the try / except bug from here:
Confirm support for try/except use in objmode block · Issue #5377 · numba/numba (github.com)

5

Solved, thanks a lot @Hannes, the None type comes from a try except block, by passing a local variable instead of defining a temporal None type I solved the bug of the try/except and code runs smoothly and Justt in Time!

Thanks for your help!

1 Like