# NumbaPendingDeprecationWarning reflected list

**URL:** <https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880>\
**Category:** Support: How do I do ...?\
**Created:** [August 31, 2021, 5:46am UTC](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880 "2021-08-31T05:46:28Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![steff](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/steff/32/207_2.png) [@steff](https://numba.discourse.group/u/steff)\
**Post date:** [August 31, 2021, 5:46am UTC](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880/1 "2021-08-31T05:46:28Z")

</div>

Am trying to `njit` the following function but getting a `NumbaPendingDeprecationWarning` which I am not sure how to best resolve. Your help would be most appreciated.

```auto
import numpy as np
from typing import List, Union

@njit()
def minmax(data: np.array, thresholds: List[Union[float, float]]) -> np.array:
    x = 0
    while x < len(data):
        if data[x] > 0:
            if data[x] > thresholds[0]:
                data[x] = thresholds[0]
            elif data[x] < 1:
                data[x] = 1
        elif data[x] < 0:
            if data[x] < thresholds[1]:
                data[x] = thresholds[1]
            elif data[x] > -1:
                data[x] = -1
        x += 1
    return data

data = np.random.uniform(low=-3, high=3, size=(1000,))
minmax(data, [2., -2.])

```

gives me the following traceback:

```auto
/Users/steffen/mambaforge/lib/python3.9/site-packages/numba/core/ir_utils.py:2119: NumbaPendingDeprecationWarning: 
Encountered the use of a type that is scheduled for deprecation: type 'reflected list' found for argument 'thresholds' of function 'minmax'.

For more information visit https://numba.pydata.org/numba-doc/latest/reference/deprecation.html#deprecation-of-reflection-for-list-and-set-types

File "../../../var/folders/xt/b71lsdgd3g1dhycp2fw80fy80000gn/T/ipykernel_92025/2811979115.py", line 3:
<source missing, REPL/exec in use?>

  warnings.warn(NumbaPendingDeprecationWarning(msg, loc=loc))

```

also tried with `from numba.typed import List` with same result.  
Python 3.9.6  
Numba 0.53.1  
Numpy 1.21.2

---

<div class="post-metadata">

**Author:** ![Rutger](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/rutger/32/446_2.png) [@Rutger](https://numba.discourse.group/u/Rutger)\
**Post date:** [August 31, 2021, 6:42am UTC](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880/2 "2021-08-31T06:42:34Z")

</div>

It has to do with converting a Python list to a Numba List, so when converting your input arguments to Numba. You current call uses a “plain” Python list as the argument: `minmax(data, [2., -2.])`.

It should be resolved by calling the function with something like:

```auto
input_list = numba.typed.List([2., 2.])
minmax(data, input_list)

```

This automatically casts to `float64` from the Python floats, but if the values in the List are already Numba datatypes, Numba will use that. So in order to use `float32` for example, you could use:

```auto
input_list = numba.typed.List([numba.float32(2.), numba.float32(-2.)])

```

Off-topic:  
Btw, your type annotation wont do anything regarding this issue, but it’s fine to have it of course. And I don’t think `Union[float, float]`, does anything, since both are the same type. A Python List has a dynamic size (by definition), so doing `List[float]` is enough to specify it’s type. For something immutable like a tuple you can specify the size with `Tuple[float, float]` for size two, versus `Tuple[float, ...]` for “any” size, etc.

---

<div class="post-metadata">

**Author:** ![steff](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/steff/32/207_2.png) [@steff](https://numba.discourse.group/u/steff)\
**Post date:** [August 31, 2021, 7:05am UTC](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880/3 "2021-08-31T07:05:21Z")

</div>

Thanks @Rutger. works and makes sense regarding the typing.  
It is just not quite as fast as I hoped. any suggestions for further improvements would be appreciated.

---

<div class="post-metadata">

**Author:** ![Hannes](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/hannes/32/188_2.png) [@Hannes](https://numba.discourse.group/u/Hannes)\
**Post date:** [August 31, 2021, 9:42am UTC](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880/4 "2021-08-31T09:42:37Z")

</div>

Hi @steff

first things first: @Rutger already said this, but I think it does not hurt to make this clear once more:

- Numba does not care at all about your type annotations. (but they are of course considered good style by many)
- `numba.typed.List` is an actual container, whereas `typing.List` is a type. They are very different things, but easy to mix up

Beyond that (not meant to be harsh):

- I don’t think `minmax` is an appropriate name for that function (given what it does).
- The function seems to have a few logical “gaps” (e.g. data[x] == 0) Unless the lack of processing the intentional
- `len(data)` might not do what you expect once data has more than 1 dimension
- It looks like for cleaner code, that while loop could be a for loop?
- When judging the speed of your function, keep in mind that the first call will trigger compilation and requires extra time compared to all subsequent calls
- Your function is both mutating data in place and returning it, I personally find that a bit odd, but okay

With that said, here are a few ideas of how one could deal with this in alternative ways (timings listed below)

```python
import numpy as np
from numba import njit # Missing import in original MWE
from numba import vectorize, prange
# Union[float, float] = float -> Union does nothing here
# There is no need for thresholds to be a List, let's be more general
from typing import Sequence

@njit
def minmax_orig(data: np.array, thresholds: Sequence[float]) -> np.array:
    x = 0
    while x < len(data):
        if data[x] > 0: # What happens if data[x] == 0 ??
            if data[x] > thresholds[0]: 
                data[x] = thresholds[0]
            elif data[x] < 1: # What happens if a threshold is in (-1,1)?
                data[x] = 1
        elif data[x] < 0:
            if data[x] < thresholds[1]:
                data[x] = thresholds[1]
            elif data[x] > -1:
                data[x] = -1
        x += 1
    return data

@njit
def minmax_scalar_args(data: np.array, lower_thr: float, upper_thr: float) -> np.array:
    x = 0
    while x < len(data):
        if data[x] > 0:
            if data[x] > lower_thr:
                data[x] = lower_thr
            elif data[x] < 1:
                data[x] = 1
        elif data[x] < 0:
            if data[x] < upper_thr:
                data[x] = upper_thr
            elif data[x] > -1:
                data[x] = -1
        x += 1
    return data

@njit(parallel=True)
def minmax_parfor(data: np.array, lower_thr: float, upper_thr: float) -> np.array:
    for x in prange(len(data)):
        if data[x] > 0:
            if data[x] > lower_thr:
                data[x] = lower_thr
            elif data[x] < 1:
                data[x] = 1
        elif data[x] < 0:
            if data[x] < upper_thr:
                data[x] = upper_thr
            elif data[x] > -1:
                data[x] = -1
    return data

@vectorize
def minmax_vec(data: np.array, lower_thr: float, upper_thr: float) -> np.array:
    if data > 0:
        if data > lower_thr:
            data = lower_thr
        elif data < 1:
            data = 1
    elif data < 0:
        if data < upper_thr:
            data = upper_thr
        elif data > -1:
            data = -1
    return data

data = np.random.uniform(low=-3, high=3, size=(1_000_000,))
small_data = data[:10]

# Trigger compilation for all testcases
minmax_orig(small_data, [2., -2.])
minmax_orig(small_data, (2., -2.))
minmax_scalar_args(small_data, 2., -2.)
minmax_parfor(small_data, 2., -2.)
minmax_vec(small_data, 2., -2.)

from timeit import timeit

for size in [10, 1_000, 1_000_000]:
    print(30*"+")
    print(f"{size} elements:\n")

    for name, expr in [
        ("Original implementation with list arg", lambda: minmax_orig(data[:size], [-2., 2.])),
        ("Original implementation with tuple arg", lambda: minmax_orig(data[:size], (-2., 2.))),
        ("Implementation with scalar threshold arg", lambda: minmax_scalar_args(data[:size], -2., 2.)),
        ("Parfor loop with scalar threshold arg", lambda: minmax_parfor(data[:size], -2., 2.)),
        ("Numba vectorised scalar threshold arg", lambda: minmax_vec(data[:size], -2., 2.))
    ]:
        t = timeit(expr, number=100)
        print(name.ljust(50), f"{t:.4e} s")

```

OUTPUT (timings are of course specific to my machine):

```nohighlight
/home/hapahl/anaconda3/lib/python3.8/site-packages/numba/core/ir_utils.py:2031: NumbaPendingDeprecationWarning:
Encountered the use of a type that is scheduled for deprecation: type 'reflected list' found for argument 'thresholds' of function 'minmax_orig'.

For more information visit https://numba.pydata.org/numba-doc/latest/reference/deprecation.html#deprecation-of-reflection-for-list-and-set-types

File "numba_discourse_880.py", line 10:
@njit
def minmax_orig(data: np.array, thresholds: Sequence[float]) -> np.array:
^

  warnings.warn(NumbaPendingDeprecationWarning(msg, loc=loc))
++++++++++++++++++++++++++++++
10 elements:

Original implementation with list arg 5.9060e-04 s
Original implementation with tuple arg 6.4200e-05 s
Implementation with scalar threshold arg 6.2100e-05 s
Parfor loop with scalar threshold arg 2.2654e-03 s
Numba vectorised scalar threshold arg 3.5370e-04 s
++++++++++++++++++++++++++++++
1000 elements:

Original implementation with list arg 1.6321e-03 s
Original implementation with tuple arg 2.3860e-04 s
Implementation with scalar threshold arg 2.4910e-04 s
Parfor loop with scalar threshold arg 2.1563e-03 s
Numba vectorised scalar threshold arg 6.4270e-04 s
++++++++++++++++++++++++++++++
1000000 elements:

Original implementation with list arg 4.5491e-01 s
Original implementation with tuple arg 4.3003e-01 s
Implementation with scalar threshold arg 4.2444e-01 s
Parfor loop with scalar threshold arg 9.4272e-02 s
Numba vectorised scalar threshold arg 4.6147e-01 s

```

Hope that helps a little 🙂

---

<div class="post-metadata">

**Author:** ![steff](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/steff/32/207_2.png) [@steff](https://numba.discourse.group/u/steff)\
**Post date:** [August 31, 2021, 10:05am UTC](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880/5 "2021-08-31T10:05:45Z")

</div>

Thank you for your elaborate answer @Hannes. Much appreciated. All good points indeed. The mutating nature I was also contemplating. Suppose it isn’t best practise, hence will change it. data[x] == 0 is intentionally not considered as it doesn’t need changing. `prange` with for loop was an alternative I also considered but the data is always one-dimensional.

---

<div class="post-metadata">

**Author:** ![Hannes](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/hannes/32/188_2.png) [@Hannes](https://numba.discourse.group/u/Hannes)\
**Post date:** [August 31, 2021, 10:07am UTC](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880/6 "2021-08-31T10:07:38Z")

</div>

Glad it helps 🙂 - one note of caution though: I am scratching my head right now, whether the mutability may affect the timings, since the data is only random on the first pass, after that all the functions are working on a dataset that’s already been processed…

---

<div class="post-metadata">

**Author:** ![steff](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/steff/32/207_2.png) [@steff](https://numba.discourse.group/u/steff)\
**Post date:** [September 1, 2021, 2:36am UTC](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880/7 "2021-09-01T02:36:27Z")

</div>

sure. conscious of this. its weird that using `prange` with `parallel=True` doesn’t work for me.

```auto
UnsupportedRewriteError: Failed in nopython mode pipeline (step: convert to parfors)
Overwrite of parallel loop index

File "../../../var/folders/xt/b71lsdgd3g1dhycp2fw80fy80000gn/T/ipykernel_92025/67808132.py", line 23:
<source missing, REPL/exec in use?>

```

Using a Mac M1. No biggie though. Speed is sufficient for research and in production the data size is limited anyway.

---

<div class="post-metadata">

**Author:** ![stuartarchibald](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/stuartarchibald/32/10_2.png) [@stuartarchibald](https://numba.discourse.group/u/stuartarchibald)\
**Post date:** [September 1, 2021, 9:36am UTC](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880/8 "2021-09-01T09:36:25Z")

</div>

@steff which Numba version and Python version are you using to see that `UnsupportedRewriteError`? Thanks.

---

<div class="post-metadata">

**Author:** ![steff](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/steff/32/207_2.png) [@steff](https://numba.discourse.group/u/steff)\
**Post date:** [September 1, 2021, 9:48am UTC](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880/9 "2021-09-01T09:48:23Z")

</div>

> [@steff](#):
>
> Python 3.9.6  
> Numba 0.53.1  
> Numpy 1.21.2

Hi @stuartarchibald please see above mentioned

---

<div class="post-metadata">

**Author:** ![stuartarchibald](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/stuartarchibald/32/10_2.png) [@stuartarchibald](https://numba.discourse.group/u/stuartarchibald)\
**Post date:** [September 1, 2021, 11:10am UTC](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880/10 "2021-09-01T11:10:10Z")

</div>

Thanks @steff

Is it the code in [NumbaPendingDeprecationWarning reflected list - #4 by Hannes](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880/4) that’s triggering it? I can’t seem to reproduce locally.

---

<div class="post-metadata">

**Author:** ![steff](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/steff/32/207_2.png) [@steff](https://numba.discourse.group/u/steff)\
**Post date:** [September 2, 2021, 12:10am UTC](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880/12 "2021-09-02T00:10:31Z")

</div>

@stuartarchibald was my bad. works now. sorry to have wasted your time.

---

<div class="post-metadata">

**Author:** ![stuartarchibald](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/stuartarchibald/32/10_2.png) [@stuartarchibald](https://numba.discourse.group/u/stuartarchibald)\
**Post date:** [September 2, 2021, 10:56am UTC](https://numba.discourse.group/t/numbapendingdeprecationwarning-reflected-list/880/13 "2021-09-02T10:56:47Z")

</div>

@steff no worries, many thanks for re-testing it and reporting back, much appreciated.