# Complex Structured Inputs **URL:** \ **Category:** Community Support\ **Created:** [July 5, 2022, 3:00pm UTC](https://numba.discourse.group/t/complex-structured-inputs/1432 "2022-07-05T15:00:13Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![jdts](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/jdts/32/635_2.png) [@jdts](https://numba.discourse.group/u/jdts)\ **Post date:** [July 5, 2022, 3:00pm UTC](https://numba.discourse.group/t/complex-structured-inputs/1432/1 "2022-07-05T15:00:13Z")
(re-posted under comm. support) I’m embarking on an optimization of a long-running `scipy.optimize` model function, and hoping to use numba to help speed things up. The optimizers allow passing additional arguments to the model function. Since the model depends on quite a few externally configured scalars and arrays, I am hoping to package these up into a structured object, like a dict or namedtuple (I see `dataclass` is not yet supported). As an analogy, an `input` to a numba-compiled function might look like: ```python param_map = {"arr1": some_array1, "arr2": {"upper": some_array2, "lower": some_array3}, "arr3": {"val": 1.0, "other": some_array4}} ``` i.e. a nested list of scalars and arrays, where the `some_array`’s can be simple 1D numpy arrays or structured arrays. Note that `param_map` is _not_ constant, i.e. it is initialized outside of numba before being passed into the njit’d model function via scipy, and some of its individual values can be updated during the run. Is there a recommended best practice for this situation, where you have dozens of individual scalars and numpy (structured) arrays you need to pass in as an argument to a numba-njit’d function, for reading/writing? ---
**Author:** ![nelson2005](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/nelson2005/32/47_2.png) [@nelson2005](https://numba.discourse.group/u/nelson2005)\ **Post date:** [July 6, 2022, 7:15pm UTC](https://numba.discourse.group/t/complex-structured-inputs/1432/2 "2022-07-06T19:15:58Z")
I’d suggest [structref](https://numba.readthedocs.io/en/stable/extending/high-level.html#defining-a-structref). If you need lots of them, it may be worth writing a code generator to create them. There’s a lot of good info about them in [this](https://numba.discourse.group/t/any-numba-equivalent-for-casting-a-raw-pointer-to-a-structref-dict-list-etc/351/12) thread, and @DannyWeitekamp’s CRE library has lots of outstanding examples. ---
**Author:** ![jdts](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/jdts/32/635_2.png) [@jdts](https://numba.discourse.group/u/jdts)\ **Post date:** [July 6, 2022, 8:35pm UTC](https://numba.discourse.group/t/complex-structured-inputs/1432/3 "2022-07-06T20:35:29Z")
Thanks for the suggestion. I neglected to mention that I’m aiming to have numba be run-time optional, so I was hoping to stick to plain python/numpy data types that numba could natively work with. The actual structure is static, i.e. all arrays and values are pre-populated in size, length, and type, before entering numbs-treated code. I suppose I could just go with a long list of arguments (array1, array2, val, val, val, array3, …) and unpack that, but that will get ugly quick! One other thought occurred: there are precious few bits of the structure that need updating. I could separate those into their own arguments. Is there a good static struct type for passing as _read-only_ into an njitted function? Edit: a [jitclass](https://numba.pydata.org/numba-doc/latest/user/jitclass.html#basic-usage) also looks quite reasonable, as it could fall back to a normal Python object if numba is not installed. ---
**Author:** ![songololo](https://avatars.discourse-cdn.com/v4/letter/s/45deac/32.png) [@songololo](https://numba.discourse.group/u/songololo)\ **Post date:** [July 12, 2022, 11:50am UTC](https://numba.discourse.group/t/complex-structured-inputs/1432/4 "2022-07-12T11:50:17Z")
I went down the `jitclass` route for some of my data structures, though regretted it a bit when I realised that this creates havoc for cacheing. I am very interested in this question and what others suggest. Some sort of named / typed tuple would seem like a good solution if there is a way to make this work. So far I’ve found `structref` a bit too complex to get my head around / implement in code, at which point it basically becomes easier just to go back to multiple individual function parameters… ---
**Author:** ![jdts](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/jdts/32/635_2.png) [@jdts](https://numba.discourse.group/u/jdts)\ **Post date:** [July 12, 2022, 1:24pm UTC](https://numba.discourse.group/t/complex-structured-inputs/1432/5 "2022-07-12T13:24:20Z")
Thanks songolo. Can you mention more about the caching issues you encountered? An advantage of jitclass is you can overload it if numba isn’t installed and it will work the same. ---
**Author:** ![songololo](https://avatars.discourse-cdn.com/v4/letter/s/45deac/32.png) [@songololo](https://numba.discourse.group/u/songololo)\ **Post date:** [July 12, 2022, 1:35pm UTC](https://numba.discourse.group/t/complex-structured-inputs/1432/6 "2022-07-12T13:35:49Z")
Basically just that the compiled JIT function can’t be cached between invocations, so it becomes painful at times waiting for the function to compile each time you run it. ---
**Author:** ![jdts](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/jdts/32/635_2.png) [@jdts](https://numba.discourse.group/u/jdts)\ **Post date:** [July 13, 2022, 12:46pm UTC](https://numba.discourse.group/t/complex-structured-inputs/1432/7 "2022-07-13T12:46:58Z")
Thanks. I gathered from [this issue](https://github.com/numba/numba/issues/4830) that a workaround for the non-caching behavior is to interact with jitclass objects only in njitted/cached _wrapper functions_. I’m not totally clear if that’s a requirement for the full lifecycle of a jitclass object (create, set attributes/properties, access/update those properties), or just part of the lifecycle. ---
**Author:** ![ananis25](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/ananis25/32/588_2.png) [@ananis25](https://numba.discourse.group/u/ananis25)\ **Post date:** [July 13, 2022, 4:09pm UTC](https://numba.discourse.group/t/complex-structured-inputs/1432/8 "2022-07-13T16:09:04Z")
For the part of the data that is nested collections of arrays, another good alternative is to use [awkward arrays](https://awkward-array.org/quickstart.html). The library comes with all the plumbing needed for it to be used in numba jit routines. ---
**Author:** ![songololo](https://avatars.discourse-cdn.com/v4/letter/s/45deac/32.png) [@songololo](https://numba.discourse.group/u/songololo)\ **Post date:** [July 13, 2022, 4:53pm UTC](https://numba.discourse.group/t/complex-structured-inputs/1432/9 "2022-07-13T16:53:21Z")
@ananis25 are there any examples of how awkward works with Numba? ---
**Author:** ![ananis25](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/ananis25/32/588_2.png) [@ananis25](https://numba.discourse.group/u/ananis25)\ **Post date:** [July 13, 2022, 6:21pm UTC](https://numba.discourse.group/t/complex-structured-inputs/1432/10 "2022-07-13T18:21:26Z")
While I don’t remember any right now, this [blog post](https://frank.sauerburger.io/2020/03/11/awkward-and-numba.html) is an example. ---
**Author:** ![jdts](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/jdts/32/635_2.png) [@jdts](https://numba.discourse.group/u/jdts)\ **Post date:** [July 13, 2022, 7:53pm UTC](https://numba.discourse.group/t/complex-structured-inputs/1432/11 "2022-07-13T19:53:27Z")
Thanks; awkward arrays looks indeed very interesting, especially when you don’t/can’t know in advance the degree of nesting, length of sub-lists, etc. For my case, it’s much simpler. I’m really just trying to neatly _package_ otherwise simple inputs that numba already knows how to deal with. I.e., rather than: ```python my_numba_func(x, y, array_1, array_2, string1, string2, array_3, string3, string4, array_4, array_5, array_5b, array_5c, string_array1, ...) ``` I want: ```python my_numba_func(x, y, params) ``` where `params` is a suitable readable/writable object that numba can compile operations on down into object code.