@overload function fails when calling without optional argument

Numba Development
1

I am currently working on Issue - #6611 which is to provided ddod argument support for nanstd and nanvar.

I updated numba.np.arraymath.np_nanvar as below

@overload(np.nanvar)
def np_nanvar(a, ddof=None):
    if isinstance(ddof, types.Optional):
        ddof = ddof.type

    if not isinstance(ddof, (types.Integer, types.Float, types.NoneType)):
        return

    if not isinstance(a, types.Array):
        return

    if not isinstance(a.dtype, (types.Integer, types.Float)):
        return

    isnan = get_isnan(a.dtype)

    def nanvar_impl(a, ddof=None):
        # Compute the mean
        m = np.nanmean(a)

        # Compute the sum of square diffs
        ssd = 0.0
        count = 0
        for view in np.nditer(a):
            v = view.item()
            if not isnan(v):
                val = (v.item() - m)
                ssd += np.real(val * np.conj(val))
                count += 1
        if ddof:
            count = count - ddof
        if count <= 0:
            return np.nan
        # np.divide() doesn't raise ZeroDivisionError
        return np.divide(ssd, count)

    return nanvar_impl

The above overloaded works fine when using with ddof argument. Unfortunately, it’s not working when i don’t send ddof argument in the function call.

from numba import njit
import numpy as np

a = np.array([1,2,3,4])
ddof = 2

@njit
def func1(a,ddof=None):
    return np.nanvar(a,ddof)
assert(func1(a,ddof) == np.nanvar(a,ddof=ddof))  # This works fine
assert(func1(a) == np.nanvar(a)) # This fails

Throws the following error

Traceback (most recent call last):
  File "h:/OpenSource/numba/latest/numba/temp.py", line 24, in <module>
    assert(func1(a) == np.nanvar(a))
  File "h:\OpenSource\numba\latest\numba\numba\core\dispatcher.py", line 421, in _compile_for_args
    error_rewrite(e, 'typing')
  File "h:\OpenSource\numba\latest\numba\numba\core\dispatcher.py", line 362, in error_rewrite
    raise e.with_traceback(None)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<function nanvar at 0x0000021E3F561AE8>) found for signature:

 >>> nanvar(array(int32, 1d, C), none)

There are 2 candidate implementations:
  - Of which 2 did not match due to:
  Overload in function 'np_nanvar': File: numba\np\arraymath.py: Line 921.
    With argument(s): '(array(int32, 1d, C), none)':
   Rejected as the implementation raised a specific error:
     TypingError: Failed in nopython mode pipeline (step: nopython frontend)
   No implementation of function Function(<built-in function sub>) found for signature:

    >>> sub(int64, none)

   There are 12 candidate implementations:
         - Of which 10 did not match due to:
         Overload of function 'sub': File: <numerous>: Line N/A.
           With argument(s): '(int64, none)':
          No match.
         - Of which 2 did not match due to:
         Operator Overload in function 'sub': File: unknown: Line unknown.
           With argument(s): '(int64, none)':
          No match for registered cases:
           * (int64, int64) -> int64
           * (int64, uint64) -> int64
           * (uint64, int64) -> int64
           * (uint64, uint64) -> uint64
           * (float32, float32) -> float32
           * (float64, float64) -> float64
           * (complex64, complex64) -> complex64
           * (complex128, complex128) -> complex128

   During: typing of intrinsic-call at h:\OpenSource\numba\latest\numba\numba\np\arraymath.py (951)

   File "numba\np\arraymath.py", line 951:
       def nanvar_impl(a, ddof=None):
           <source elided>
           if ddof:
               count = count - ddof
               ^

  raised from h:\OpenSource\numba\latest\numba\numba\core\typeinfer.py:1071

During: resolving callee type: Function(<function nanvar at 0x0000021E3F561AE8>)
During: typing of call at h:/OpenSource/numba/latest/numba/temp.py (9)


File "temp.py", line 9:
def func1(a,ddof=None):
    return np.nanvar(a,ddof)

How do i add support for the function call with signature nanvar(array(int32, 1d, C), none)? Please direct me.

2

I have updated nanvar which is working. Please give me feedback on if i implemented it correctly


@register_jitable
def compute_sum_of_square_diffs(a,isnan):
    m = np.nanmean(a)
    ssd = 0.0
    count = 0
    for view in np.nditer(a):
        v = view.item()
        if not isnan(v):
            val = (v.item() - m)
            ssd += np.real(val * np.conj(val))
            count += 1
    return ssd, count


@overload(np.nanvar)
def np_nanvar(a, ddof):

    if not isinstance(ddof, (types.Integer, types.Float, types.NoneType)):
        return

    if not isinstance(a, types.Array):
        return

    if not isinstance(a.dtype, (types.Integer, types.Float)):
        return

    isnan = get_isnan(a.dtype)

    if isinstance(ddof, types.NoneType):
        def nanvar_impl(a,ddof):
            # Compute the sum of square diffs
            ssd, count = compute_sum_of_square_diffs(a,isnan)

            if count <= 0:
                return np.nan
            # np.divide() doesn't raise ZeroDivisionError
            return np.divide(ssd, count)

        return nanvar_impl
    else:
        def nanvar_impl(a, ddof):
            # Compute the sum of square diffs
            ssd, count = compute_sum_of_square_diffs(a,isnan)

            count = count - ddof
            if count <= 0:
                return np.nan
            # np.divide() doesn't raise ZeroDivisionError
            return np.divide(ssd, count)

        return nanvar_impl
3

Hi @rawwar

Thanks for working on this. I suggest opening a (draft) PR to get feedback on code as it’s much easier to comment inline etc.

1 Like
4

Sure, I will do that

5

@stuartarchibald , i rasied a draft PR - nanvar with ddof-v1 by rawwar · Pull Request #6852 · numba/numba · GitHub . Thank you so much!

6

Great, thanks for doing that.

1 Like