Numba 0.54.0 RC1 and llvmlite v0.37.0 RC1

Dear all,

on behalf of the Numba community I would like to announce the immediate availability of the release candidates: Numba 0.54.0 RC1 and llvmlite 0.37.0 RC1! These release candidates result from a concentrated, three-month effort by 32 unique contributors and consist of patches from over 160 Pull-Requests!

For details, please refer to the change-logs at: Numba 0.54.0 RC1 release notes. and llvmlite v0.37.0 RC1 release notes.

Conda packages can be installed from the numba channel on anaconda.org with conda :

conda install -c numba numba=0.54.0rc1

And

conda install -c numba llvmlite=v0.37.0rc1

If you have time and inclination, please do test these release candidates with your own libraries and tools and report back any issues you may encounter!

Binary wheels are unfortunately not yet available but we hope to build those in time for the final release.

Thank you again to all contributors for helping to improve Numba!

3 Likes

All tests pass on internal libraries. Thanks for your work!

Congrats for this upcoming massive release! Will test poliastro with this later today.

I’m curious about the improved support for ndarray subclasses. Would this make jitted functions work with, say, astropy.units? Or should those types declare __numba_array_subtype_dispatch__?

Is astropy.units a subtype of ndarray? If so then the changes in this release might be sufficient to allow this to be made to work.

I’m testing the changes on Bodo. Thank you for your work!

@esc Bodo doesn’t update the scope at every location, so we had some failures in ssa.py with the line assert newtarget.name in scope.localvars.

Do you know why this is an assert statement and not:

if newtarget.name not in scope.localvars:
    newtarget = scope.define(assign.target.name, loc=assign.loc)

Are there any risks with just adding the name to the scope?

I am happy to announce some wheels for the Numba and llvmlite releases have become available. For now, they are on anaconda.org and can be installed using:

pip install -i https://pypi.anaconda.org/numba/label/manylinux2014_testing_wheels/simple llvmlite
pip install -i https://pypi.anaconda.org/numba/label/manylinux2014_testing_wheels/simple numba

The wheel build process is being overhauled at the moment, so these wheels are to be considered somewhat experimental. If you have the time and inclination to test them however that would be greately appreciated.

We hope to release wheels to PyPi for RC2.

I am not sure. Actually, the best person to ask about this is @sklam

The changes

if newtarget.name not in scope.localvars:
    newtarget = scope.define(assign.target.name, loc=assign.loc)

is just hiding the error. Very often, the actual problem is not that the scope didn’t define the variable, but that scope is incorrectly borrowed from another function. Therefore, defining the variables here may be injecting a variable into another function. The most common reason for this incorrect borrowing of scope is from inlining. As a partial fix, I have to introduce transfer_scope() to ensure the foreign ir.Block instances are transferred properly.

Conceptually this makes sense, but I’m wondering if this is practically necessary for our case. In a lot of places Bodo generates a new function with unique variable names and discards the old function (so we don’t have a scope conflict issue).

This situation passes our tests by updating the scope in ssa.py as shown above. We could implement transfer_scope() instead, but it would require us to make a lot of code changes, which we would rather avoid. In our situation I believe the two should be equivalent (unless I misunderstand this code change) since both just define a variable in a scope.

I’m not asking for this code to be changed, I just want to confirm that my understanding of the update is correct.

@sklam Bodo has a test that is failing because we assign into the same variable twice in a basic block. In Numba 0.53 DEL nodes were generated for both assignments, but in Numba 0.54 only one of the assignments seems to have DEL nodes generated.

Any idea what changes could be impacting this? How does Numba support assigning into a variable twice in the same basic block?

Assigning to scope inside SSA could be hiding bugs. But in practice, it might be ok if you are fortunate enough. In the long run, it’s best to fix it all up.

Can you open an issue and provide more details? Seeing a reproducer or the Numba IRs will help us better understand the problem.

Any thoughts about the timeline of pypi/rc2 wheels?

@nelson2005 yes, I uploaded them this morning and just announced them now.

Outstanding, thanks!