# Adding multiple types to structref.define\_boxing

**URL:** <https://numba.discourse.group/t/adding-multiple-types-to-structref-define-boxing/2907>\
**Category:** Support: How do I do ...?\
**Created:** [February 9, 2025, 5:34pm UTC](https://numba.discourse.group/t/adding-multiple-types-to-structref-define-boxing/2907 "2025-02-09T17:34:59Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![dovoxi](https://avatars.discourse-cdn.com/v4/letter/d/57b2e6/32.png) [@dovoxi](https://numba.discourse.group/u/dovoxi)\
**Post date:** [February 9, 2025, 5:34pm UTC](https://numba.discourse.group/t/adding-multiple-types-to-structref-define-boxing/2907/1 "2025-02-09T17:34:59Z")

</div>

Hi, following Danny Weitekamp’s structref example ([here](https://github.com/numba/numba/issues/6522#issuecomment-876090450)), I was trying to create a numba StructRefProxy myself.

Basically, I am predefining the types like in the example above, e.g.:

```auto
@structref.register
class TestTypeTemplate(types.StructRef):
    def preprocess_fields(self, fields):
        return tuple((name, types.unliteral(type)) for name, type in fields)
    
kb_fields = [
    ("value", types.int64),
    ("array", types.UnionType(types=(types.float64[::1], types.int64[::1]))),
]

structref.define_boxing(TestTypeTemplate, Test)

```

The idea is that the array of values can either be an array of float64 or int64 types.

The example above can’t be used to initialize structref classes unfortunately. For example, when I pass an array of int64 integers to my `Test` class, I get:

```auto
numba.core.errors.NumbaNotImplementedError: Failed in nopython mode pipeline (step: native lowering)
Cannot cast array(int64, 1d, C) to Union[array(float64, 1d, C),array(int64, 1d, C)]: %"inserted.strides" = insertvalue {i8*, i8*, i64, i64, i64*, [1 x i64], [1 x i64]} %"inserted.shape", [1 x i64] %".13", 6

```

I was wondering if it is actually possible at all in this scenario, to allow users to either provide an array of int64 or float64 values. I have found a workaround, but I don’t like it. Simply put, I can check whether the input is an int64/float64 array and then initialize the class using 2 different constructors (1 for floats, 1 for integers), but that is not something I can use unfortunately.

---

<div class="post-metadata">

**Author:** ![milton](https://avatars.discourse-cdn.com/v4/letter/m/87869e/32.png) [@milton](https://numba.discourse.group/u/milton)\
**Post date:** [February 9, 2025, 9:13pm UTC](https://numba.discourse.group/t/adding-multiple-types-to-structref-define-boxing/2907/2 "2025-02-09T21:13:11Z")

</div>

Yeah it’s possible, for instance like so:

```auto
import numpy
from numba import njit, typeof
from numba.core import types
from numba.experimental import structref

class Atest(structref.StructRefProxy):
    def __new__ (cls, value, array):
        return atest_constructor(value, array)

    @property
    @njit
    def value(self):
        return self.value

    @property
    @njit
    def array(self):
        return self.array

@structref.register
class AtestTypeTemplate(types.StructRef):
    def preprocess_fields(self, fields):
        return tuple((name, types.unliteral(type)) for name, type in fields)

structref.define_proxy(Atest, AtestTypeTemplate, ["value", "array"])

def make_Atest_constructor_sig(array_type):
    return AtestTypeTemplate([("value", types.int64), ("array", array_type)])(types.int64, array_type)

@njit([
    make_Atest_constructor_sig(types.float64[::1]),
    make_Atest_constructor_sig(types.int64[::1])
])
def atest_constructor(value, array):
    return Atest(value, array)

if __name__ == ' __main__':
    t1 = Atest(137, numpy.array([314], dtype=numpy.int64))
    t2 = Atest(-141, numpy.array([2.17], dtype=numpy.float64))

    print(t1.value, t1.array) # 137 [314]
    print(t2.value, t2.array) # -141 [2.17]

    print(typeof(t1)._fields) # (('value', int64), ('array', Array(int64, 1, 'C', False, aligned=True)))
    print(typeof(t2)._fields) # (('value', int64), ('array', Array(float64, 1, 'C', False, aligned=True)))

```

---

<div class="post-metadata">

**Author:** ![milton](https://avatars.discourse-cdn.com/v4/letter/m/87869e/32.png) [@milton](https://numba.discourse.group/u/milton)\
**Post date:** [February 9, 2025, 9:17pm UTC](https://numba.discourse.group/t/adding-multiple-types-to-structref-define-boxing/2907/3 "2025-02-09T21:17:55Z")

</div>

P.S. Here typing of `atest_constructor` is optional, you can just leave it at `njit` and let numba figure it out for you. Also, behind the scenes `structref.define_contructor` (invoked by `define_poxy` here) is doing overloads for different types of the structure’s members.

---

<div class="post-metadata">

**Author:** ![DannyWeitekamp](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/dannyweitekamp/32/144_2.png) [@DannyWeitekamp](https://numba.discourse.group/u/DannyWeitekamp)\
**Post date:** [February 9, 2025, 11:11pm UTC](https://numba.discourse.group/t/adding-multiple-types-to-structref-define-boxing/2907/4 "2025-02-09T23:11:53Z")

</div>

@dovoxi One consideration with @milton 's approach is that the constructor now outputs objects of two different structref types. While they share the same TypeTemplate, the f8 and i8 variants of the structref are not unifiable within numba’s type inference system. So you could certainly use this to instantiate these two kinds of objects on the Python side, and even send them individually into jitted functions as single arguments (jit compilation will build two different specialized endpoints for each type individually). However, you could not, for instance, add objects of these two types to the same list and pass that list into one jitted function. So this solution isn’t quite the same as having a proper union type.

I haven’t used numba much in the last 6 months or so (moved to C++ w/ nanobind in part because of these kinds of issues), but I’m fairly certain that this cannot be achieved in numba without doing some annoying, complex, and not-quite-worth-it tricks that involve treating the arrays as opaque pointers, keeping around some kind of type indicator within each structref object, and casting the opaque pointers back to the appropriate array types as indicated by the type indicator. Or alternatively, having some kind of parent class that defines the type indicator and subclasses with extra typed fields that can be casted to from the parent class. The machinery for that approach is also not directly supported by numba, and requires a lot of specialized code to get working. I talked a bit about that here: [Any numba equivalent for casting a raw pointer to a StructRef, Dict, List etc? - #12 by DannyWeitekamp](https://numba.discourse.group/t/any-numba-equivalent-for-casting-a-raw-pointer-to-a-structref-dict-list-etc/351/12).

If you’re brave enough to poke around for a solution along these lines, I did do something like this in [this project](https://github.com/DannyWeitekamp/Cognitive-Rule-Engine/blob/main/cre/utils.py). But doing so will have you doing so much non-standard stuff that it will become very difficult to maintain… so hopefully @milton 's is enough for your purposes. Good luck!

---

<div class="post-metadata">

**Author:** ![milton](https://avatars.discourse-cdn.com/v4/letter/m/87869e/32.png) [@milton](https://numba.discourse.group/u/milton)\
**Post date:** [February 10, 2025, 12:02am UTC](https://numba.discourse.group/t/adding-multiple-types-to-structref-define-boxing/2907/5 "2025-02-10T00:02:25Z")

</div>

Thanks for the comments. BTW just to make things even more pedagogical for the OP, the behind-the-scenes machinery that is responsible for creating these two different struct types happens [here](https://github.com/numba/numba/blob/release0.60/numba/experimental/structref.py#L245). One can swap the caller (`define_proxy`) with the following code, which makes things more explicit (i.e., keep the boxing for the Python proxy but roll out your own low-level constructor overloads):

```auto
from numba.extending import overload 

structref.define_boxing(AtestTypeTemplate, Atest)

@overload(Atest, strict=False)
def ol_Atest(value_type, array_type):
    Atest_type = AtestTypeTemplate([("value", value_type), ("array", array_type)])

    def _(value, array):
        Atest_obj = structref.new(Atest_type)
        Atest_obj.value = value
        Atest_obj.array = array
        return Atest_obj
    return _

```

---

<div class="post-metadata">

**Author:** ![dovoxi](https://avatars.discourse-cdn.com/v4/letter/d/57b2e6/32.png) [@dovoxi](https://numba.discourse.group/u/dovoxi)\
**Post date:** [February 10, 2025, 11:30pm UTC](https://numba.discourse.group/t/adding-multiple-types-to-structref-define-boxing/2907/6 "2025-02-10T23:30:39Z")

</div>

Thanks for the detailed answer. So I guess I won’t be able to define a numba type, such as:

```auto
AtestType = AtestTypeTemplate(kb_fields)

```

---

<div class="post-metadata">

**Author:** ![dovoxi](https://avatars.discourse-cdn.com/v4/letter/d/57b2e6/32.png) [@dovoxi](https://numba.discourse.group/u/dovoxi)\
**Post date:** [February 10, 2025, 11:32pm UTC](https://numba.discourse.group/t/adding-multiple-types-to-structref-define-boxing/2907/7 "2025-02-10T23:32:01Z")

</div>

Thanks a lot for the answer. It does indeed work.

---

<div class="post-metadata">

**Author:** ![milton](https://avatars.discourse-cdn.com/v4/letter/m/87869e/32.png) [@milton](https://numba.discourse.group/u/milton)\
**Post date:** [February 11, 2025, 2:13pm UTC](https://numba.discourse.group/t/adding-multiple-types-to-structref-define-boxing/2907/8 "2025-02-11T14:13:25Z")

</div>

You’re welcome. You can also consider using `AnyType` introduced [here](https://github.com/Goykhman/numbox/blob/0.1.1/numbox/core/any_type.py#L71), to wrap your arrays into. It leverages a variant of the type erasure technique. In application to your case, it goes like so:

```auto
import numpy
from numba import njit, typeof
from numba.core import types
from numba.experimental import structref
from numbox.core.any_type import make_any, AnyType

class Atest(structref.StructRefProxy):
    def __new__ (cls, value, array):
        return atest_constructor(value, array)

    @property
    @njit
    def value(self):
        return self.value

    @property
    @njit
    def array(self):
        return self.array

    @array.setter
    @njit
    def array(self, val):
        self.array = val

@structref.register
class AtestTypeTemplate(types.StructRef):
    def preprocess_fields(self, fields):
        return tuple((name, types.unliteral(type)) for name, type in fields)

structref.define_proxy(Atest, AtestTypeTemplate, ["value", "array"])

AtestType = AtestTypeTemplate([
    ("value", types.int64),
    ("array", AnyType) # `AnyType` from numbox package, its objects are created using `make_any`
])

@njit
def atest_constructor(value, array):
    return Atest(value, make_any(array))

if __name__ == ' __main__':
    a1 = numpy.array([314], dtype=numpy.int64)
    a2 = numpy.array([2.17], dtype=numpy.float64)
    a1_ty = typeof(a1)
    a2_ty = typeof(a2)
    t1 = Atest(137, a1)
    t2 = Atest(-141, a2)

    sigs = atest_constructor.nopython_signatures
    assert len(sigs) == 2, "for `a1_ty` and `a2_ty` arguments"
    ret_type = set([sig.return_type for sig in sigs])
    assert len(ret_type) == 1, "`ret_type` is `numba.AtestTypeTemplate(('value', int64), ('array', numba.AnyTypeClass(('p', numba.ErasedTypeClass()),)))`"

    print(t1.value, t1.array.get_as(a1_ty)) # 137 [314]
    print(t2.value, t2.array.get_as(a2_ty)) # -141 [2.17]

    assert typeof(t1) == typeof(t2)

    a3 = numpy.array([1.41], dtype=numpy.float64)
    a3_ty = typeof(a3)
    t1.array.reset(a3)
    print(t1.value, t1.array.get_as(a3_ty)) # 137 [1.41]

```