Numba 0.53.0 and llvmlite 0.36.0 Release Candidates

Hi all,

We have tagged Numba 0.53.0rc1 and llvmlite 0.36.0rc1. Their change logs are at:

Conda packages are available in the numba channel; e.g. conda install -c numba numba=0.53.0rc1.

Unfortunately, there are some hiccups with the wheels at the moment. We will update this thread as soon as the wheels are ready.

Our current problems with wheels are affecting only Numba. I have uploaded llvmlite wheels for early testing.

pip install llvmlite==0.36.0rc1

hi, I’ve tested at work and everything worked fine. Tests even ran a bit faster, although I cannot tell if it’s runtime or compilation time.

Numba wheels for 0.53.0rc1 are up. Due to post tag bug fixes for wheel building, we have to make a new tag 0.53.0rc1.post1.

Install with:

pip install numba==0.53.0rc1.post1

Now that all conda-packages and wheels are inplace, please help us test the release candidates.

Just as for luk everything seems to work great for me so far - don’t really have a well maintained test suite to prove that though :see_no_evil:

Breaks my internal statistics library, unfortunately.

Builds fine on numba 0.52.0 and llvmlite 0.35.0:

(numbastats) [root@d3c7a0bd37ae numba-stats]# ls
Dockerfile  README.ipynb  README.md  docs  examples  numbastats  references  setup.py  tests

(numbastats) [root@d3c7a0bd37ae numba-stats]# pip install .
Processing /workspaces/numba-stats
Requirement already satisfied: numpy in /opt/conda/envs/numbastats/lib/python3.8/site-packages (from numbastats==0.9.4) (1.19.2)
Requirement already satisfied: scipy in /opt/conda/envs/numbastats/lib/python3.8/site-packages (from numbastats==0.9.4) (1.6.0)
Requirement already satisfied: numba in /opt/conda/envs/numbastats/lib/python3.8/site-packages (from numbastats==0.9.4) (0.52.0)
Requirement already satisfied: llvmlite<0.36,>=0.35.0 in /opt/conda/envs/numbastats/lib/python3.8/site-packages (from numba->numbastats==0.9.4) (0.35.0)
Requirement already satisfied: setuptools in /opt/conda/envs/numbastats/lib/python3.8/site-packages (from numba->numbastats==0.9.4) (49.6.0.post20210108)
Building wheels for collected packages: numbastats
  Building wheel for numbastats (setup.py) ... done
  Created wheel for numbastats: filename=numbastats-0.9.4-cp38-cp38-linux_x86_64.whl size=319774 sha256=1a2506768caf747eb3efb5f9bfa56b7260bc8e436ab6f19b10a6ed22bb0533d3
  Stored in directory: /root/.cache/pip/wheels/bd/e9/25/9727c9df10e9a6ae3f989b4b093dce16d9855187fde250fe00
Successfully built numbastats
Installing collected packages: numbastats
Successfully installed numbastats-0.9.4

(numbastats) [root@d3c7a0bd37ae numba-stats]# conda list | grep numba
# packages in environment at /opt/conda/envs/numbastats:
icc_rt                    2020.2                intel_254    numba
llvmlite                  0.35.0           py38hf484d3e_0    numba
numba                     0.52.0          np1.11py3.8h04863e7_g18825058a_0    numba
numbastats                0.9.4                    pypi_0    pypi

(numbastats) [root@d3c7a0bd37ae numba-stats]# python -m unittest discover -s tests
.Test funcs.beta pass: True
.....Test cdf pass: True
.Test pdf pass: True
...Test gamma.gratio pass: True
.Test gamma.psi pass: True
..Time in seconds import numbastats.beta: 6.9141387939453125e-06
..F CDF test with known values pass: True
.
----------------------------------------------------------------------
Ran 16 tests in 0.732s

OK

Build fails on numba 0.53.0rc1 and llvmlite 0.36.0rc1:

(numbastats) [root@d3c7a0bd37ae numba-stats]# ls
Dockerfile  README.ipynb  README.md  docs  examples  numbastats  references  setup.py  tests

(numbastats) [root@d3c7a0bd37ae numba-stats]# pip install .
Processing /workspaces/numba-stats
    ERROR: Command errored out with exit status 1:
     command: /opt/conda/envs/numbastats/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-isc2rctd/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-isc2rctd/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-6_xfuk7u
         cwd: /tmp/pip-req-build-isc2rctd/
    Complete output (58 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-req-build-isc2rctd/setup.py", line 3, in <module>
        from numbastats.f import cc_f
      File "/tmp/pip-req-build-isc2rctd/numbastats/__init__.py", line 20, in <module>
        from .f import pdf, cdf, acdf, ppf
      File "/tmp/pip-req-build-isc2rctd/numbastats/f.py", line 3, in <module>
        from .beta import bratio, betaincinv
      File "/tmp/pip-req-build-isc2rctd/numbastats/beta.py", line 377, in <module>
        def _bpser(a, b, x, eps):
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/decorators.py", line 221, in wrapper
        disp.compile(sig)
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/dispatcher.py", line 909, in compile
        cres = self._compiler.compile(args, return_type)
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/dispatcher.py", line 83, in compile
        raise retval
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/dispatcher.py", line 93, in _compile_cached
        retval = self._compile_core(args, return_type)
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/dispatcher.py", line 106, in _compile_core
        cres = compiler.compile_extra(self.targetdescr.typing_context,
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/compiler.py", line 606, in compile_extra
        return pipeline.compile_extra(func)
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/compiler.py", line 353, in compile_extra
        return self._compile_bytecode()
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/compiler.py", line 415, in _compile_bytecode
        return self._compile_core()
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/compiler.py", line 395, in _compile_core
        raise e
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/compiler.py", line 386, in _compile_core
        pm.run(self.state)
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/compiler_machinery.py", line 339, in run
        raise patched_exception
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/compiler_machinery.py", line 330, in run
        self._runPass(idx, pass_inst, state)
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/compiler_lock.py", line 35, in _acquire_compile_lock
        return func(*args, **kwargs)
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/compiler_machinery.py", line 289, in _runPass
        mutated |= check(pss.run_pass, internal_state)
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/compiler_machinery.py", line 262, in check
        mangled = func(compiler_state)
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/typed_passes.py", line 104, in run_pass
        typemap, return_type, calltypes, errs = type_inference_stage(
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/typed_passes.py", line 83, in type_inference_stage
        typemap, restype, calltypes = infer.unify(raise_errors=raise_errors)
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/typeinfer.py", line 1204, in unify
        check_var(var)
      File "/opt/conda/envs/numbastats/lib/python3.8/site-packages/numba/core/typeinfer.py", line 1174, in check_var
        raise TypingError(msg % (var, val, loc), loc)
    numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
    Type of variable 'bins_sum.1' cannot be determined, operation: $176inplace_add.44, location: /tmp/pip-req-build-isc2rctd/numbastats/beta.py (424)
    
    File "numbastats/beta.py", line 424:
    def _bpser(a, b, x, eps):
        <source elided>
            increment = xj * (numer / denom)
            bins_sum += increment
            ^
    
    ----------------------------------------
WARNING: Discarding file:///workspaces/numba-stats. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

(numbastats) [root@d3c7a0bd37ae numba-stats]# conda list | grep numba
# packages in environment at /opt/conda/envs/numbastats:
icc_rt                    2020.2                intel_254    numba
llvmlite                  0.36.0rc1        py38hf484d3e_0    numba
numba                     0.53.0rc1       np1.11py3.8h04863e7_ga3d29a7f4_0    numba

The relevant portion of the code that fails follows, sorry for the redactions but org rules are rules:

@njit('f8(f8,f8,f8,f8)', fastmath=FMTRUE, nogil=NGTRUE)
@cc_beta.export('_bpser', 'f8(f8,f8,f8,f8)')
def _bpser(a, b, x, eps):
    """ Evaluate Ix(a, b) when b <= 1 or b*x <= 0.7 via a power series expansion. """
    
    # (code redacted)

    # Calculate the binomial series sum in Equation 7
    bins_sum = 0.0
    j = 1
    increment = 1.0 + eps

    # This series will naturally terminate when j-1 = b
    while np.abs(increment) >= eps:
        # (code redacted), suffice to say xj, numer, and denom are floats
        increment = xj * (numer / denom)
        bins_sum += increment
        j += 1

    # (code redacted)

    # Final answer
    ln_fin = ln_big_g + ln_xa - ln_a + np.log(1 + a * bins_sum)
    return( np.exp(ln_fin) )

I’ve started this thread: Helping test Numba 0.53.0 RC to capture the results of testing.

Thanks @ryanchien I can’t seem to create a set of conditions to create this failure, is there any chance you could possibly please reduce it and put in dummy things/change variable names/whatever it is you have to do, so that a reproducer is formed. Many thanks for your help with this.

@luk-f-a @Hannes Thanks for testing it and reporting back, much appreciated.

Can do, let me get back to you.

Numba 0.53.0rc2 and llvmlite 0.36.0rc2 are released. Conda and Pip packages are in the usual places. All reported regressions are fixed, including the regression due to temp variable assignment removal. Please see the changelog diff for a list of the patches.