Thanks for confirming, I’ve updated my gist to reflect your modified print statements and also added print statements for the versions of Python, numba, numpy. These are my timings:
Python: 3.9.6 (default, Aug 18 2021, 12:38:10)
[Clang 10.0.0 ]
Numba: 0.54.0
Numpy: 1.20.3
plain 0.6843872109999998
rec array 1.9209382970000002
array 0.3137645659999997
named tuple 1.7651555309999996
rec array alt 1.9435516949999991
struct ref 0.8691741549999996
Again, this runs on a MacBook Pro with macOS High Sierra (10.13.6) in a conda virtual environment.
All my timings are slower than yours (which is fine, different machines), but for struct ref my timing is actually faster. Don’t want to claim that the difference is significant, but it is surprising.
Your potential explanation re: difference in timings between a record array and a regular array makes sense to me.
What currently is unclear to me is how I would create a record numpy array (or structured numpy array) that holds numpy arrays of different sizes and that also works in Numba.
For example: create a record/structured numpy array named x with one field a that is a (2 x 2) numpy array and another field b that is a (3 x 3) numpy array and subsequently access its fields inside a Numba jitted function like x['a'] or x.b.
Based on your first post I thought this would be possible as long as I pre-specify the sizes of the arrays, but I run into errors trying to create this myself. I think your post in this thread is also related to this: Numba failure with np.zeros in a static method of a class - #6 by luk-f-a
So, is this currently not possible in Numba?
Edit: After researching this some more I stumbled upon this thread: Structured arrays with nd-fields which seems to exactly describe my problem. In that case I cannot currently use a record/structured numpy array as a data container, unless I am missing something. I have to decide between my namedtuple approach (with the 0d numpy array hack to make the “scalars” mutable) or the structref approach, which is still experimental.