Understanding the Global Array Lowering Size Limit

Numba Development
1

Hello Numba Developers.

Numba doesn’t freeze global arrays larger than 1MB. How is this threshold chosen? What are the impacts of larger thresholds? I assume large binaries that could take too much memory and be slow? numba/base.py at 4469f04e0f4916d2541e15b38d71d10268fca73f · numba/numba · GitHub

Any context you can provide is greatly appreciated. Thanks.

1 Like
2

This was added in Improve handling of big array globals by sklam · Pull Request #2189 · numba/numba · GitHub - although the PR doesn’t state, I’d imagine it was determined empirically at the time that this was a reasonable limit.