Help with error message

Just discovered this discourse group.

Figured I would cross-post my SO question which contains the code and stack trace. https://stackoverflow.com/questions/64469950/numba-numpy-understanding-error-message

A little more context:
My dataset is a pandas dataframe with a single column. That column is an array of values, having some irregular length. I need to run union commands for all elements, as shown in the example code. I’m going to pass in the .values attribute of that column into my process_values function.

a) is this a good use case for numba? I’m trying to improve on pandas .apply method.
b) can you help be decrypt the error message?

Thanks so much,

Jeff

hi @jrjames83, I have replied on SO. Let me know if it’s clear how to solve it using lists.

Luk

Nice! Taking a look @luk-f-a. Appreciate the quick reply.

from numba import jit
from numba.typed import List

indices = np.arange(8806806, dtype=np.int64)
sizes = np.ones(8806806, dtype=np.int64)
connected_components = 8806806

@jit(npython=True)
def root(p: int) -> int:
    while p != indices[p]:
        indices[p] = indices[indices[p]]
        p = indices[p]
    return p

@jit(npython=True)
def connected( p: int, q: int) -> bool: 
    return root(p) == root(q)

@jit(npython=True)
def union( p: int, q: int) -> None:
    root1 = root(p)
    root2 = root(q)
    if root1 == root2:
        return

    if (sizes[root1] < sizes[root2]):
        indices[root1] = root2
        sizes[root2] += sizes[root1]
    else:
        indices[root2] = root1
        sizes[root1] += sizes[root2]

    connected_components -= 1
    
@jit(nopython=True)
def process_values(arr):
    for row in arr:
        for first, second in zip(arr, arr[1:]):
            union(first, second)
            
          

mylist = List()
for x in [[8018361, 4645960], [8018361, 4645960]]:
    new_list = List()
    new_list.extend(x)
    mylist.append(new_list)


process_values(mylist)

mylist evidently gets created, but now:

---------------------------------------------------------------------------
TypingError                               Traceback (most recent call last)
<ipython-input-64-6e08ff92ded5> in <module>
     48 
     49 
---> 50 process_values(mylist)
     51 
     52 

/opt/conda/lib/python3.7/site-packages/numba/core/dispatcher.py in _compile_for_args(self, *args, **kws)
    399                 e.patch_message(msg)
    400 
--> 401             error_rewrite(e, 'typing')
    402         except errors.UnsupportedError as e:
    403             # Something unsupported is present in the user code, add help info

/opt/conda/lib/python3.7/site-packages/numba/core/dispatcher.py in error_rewrite(e, issue_type)
    342                 raise e
    343             else:
--> 344                 reraise(type(e), e, None)
    345 
    346         argtypes = []

/opt/conda/lib/python3.7/site-packages/numba/core/utils.py in reraise(tp, value, tb)
     78         value = tp()
     79     if value.__traceback__ is not tb:
---> 80         raise value.with_traceback(tb)
     81     raise value
     82 

TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Internal error at <numba.core.typeinfer.CallConstraint object at 0x7f2127283ed0>.
"<class 'numba.core.cpu.CPUTargetOptions'> does not support option: 'npython'"
[1] During: resolving callee type: type(CPUDispatcher(<function union at 0x7f21272b2320>))
[2] During: typing of call at <ipython-input-64-6e08ff92ded5> (39)

Enable logging at debug level for details.

File "<ipython-input-64-6e08ff92ded5>", line 39:
def process_values(arr):
    <source elided>
        for first, second in zip(arr, arr[1:]):
            union(first, second)
            ^

What are the patterns then for getting my original array of arrays into this typed list format?

you just had a typo, @jit(npython=True) should be @jit(nopython=True)

Thanks @luk-f-a - good catch. I’ll report back with any updates and what the code ends up looking like.