How to create typed lists of structured numpy arrays?

Numba Support: How do I do ...?
1

Hi,

I am trying to create typed lists for structured NumPy arrays. Creating these outside of @njit works just fine, but creating these inside of an @njit function throws the following error:

TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Invalid use of <class ‘numba.core.types.npytypes.Array’> with parameters (typeref[Record(node_y_values[type=float64;offset=0],left_child_pointers[type=int32;offset=8],right_child_pointers[type=int32;offset=12];16;True)], Literalint, Literalstr)
No type info available for <class ‘numba.core.types.npytypes.Array’> as a callable.
During: resolving callee type: typeref[<class ‘numba.core.types.npytypes.Array’>]

Example code:

import numpy as np
from numba.typed import List
from numba.types import Array
from numba import from_dtype, njit
dtype = from_dtype(np.dtype([('node_y_values', np.float64),
                             ('left_child_pointers', np.int32),
                             ('right_child_pointers', np.int32)], align=True))
typed_list = List.empty_list(Array(dtype, 1, "C"))
@njit
def struc_arr_typed_list():
    typed_list = List.empty_list(Array(dtype, 1, "C"))
struc_arr_typed_list()

I also tried the following: typed_list = List.empty_list(dtype[:]), but that gives another typing error. Creating the typed list outside of @njit and then passing it into a jitted function works, but this results in unexpectedly slow performance in appending. Am I approaching this the wrong way? I could of course create separate arrays per datatype and use these, but putting structured arrays inside of a typed list is a much cleaner solution. Would greatly appreciate any tips or help on this! I am using Numba 0.59.0 and NumPy 1.26.4 in Python 3.12.2 if that helps.

Thanks in advance!

2

If anyone stumbles across this looking for the same functionality, I figured it out. The trick is to first create the dtype, and then create an array dtype, both outside of @njit. See code below.

import numpy as np
from numba.typed import List
from numba.types import Array
from numba import from_dtype, njit
dtype = from_dtype(np.dtype([('node_y_values', np.float64),
                             ('left_child_pointers', np.int32),
                             ('right_child_pointers', np.int32)], align=True))
numba_dtype_arr = Array(dtype, 1, 'C')
@njit
def test():
    typed_list = List.empty_list(numba_dtype_arr)
    for i in range(5):
        typed_list.append(np.empty(0, dtype=dtype))
test()