Public Numba Dev Meeting, Tuesday October 13 2020

Announcements
1

Hi all,

The upcoming Public Numba Dev meeting is on Oct 13th at 10:30 US central time. You can find instructions on how to join in the event details on this google calender . (Tips: adding it to your own calendar will convert the time to your timezone.)

Here’s are some topics that we can have:

  1. Type annotations discussions
  2. Numba devs demoing new diagnostic tool; i.e. enhanced inspect_cfg (https://github.com/numba/numba/pull/6295)
  3. < your suggestions >

Feel free to discuss which topic(s) to talk about, or indeed suggest new ones in the thread, topic selection will be based on popularity.

Hope to see you there!

2

Adding context to the “Type Annotation discussion”, the Numba devs continue to have some concerns about the idea of adding type annotations everywhere, and we are probably not as familiar with type annotations as some of our community members. So, I am inviting community members to help answer our questions and concerns about type annotations; especially:

3

Regarding readability, I at least personally find that black behaves much better than flake8 for formatting with type annotations. I think the black source code is a good example. For an example from numba, these two function definitions pass flake8

class DictType(IterableType, InitialValue):
    def __init__(self, keyty: NumbaTypeInst, valty: NumbaTypeInst,
                 initial_value: pt.Optional[pt.Dict[pt.Any, pt.Any]] = None):

and

class DictType(IterableType, InitialValue):
    def __init__(
        self, keyty: NumbaTypeInst, valty: NumbaTypeInst,
            initial_value: pt.Optional[pt.Dict[pt.Any, pt.Any]] = None):

where as black would format it as

class DictType(IterableType, InitialValue):
    def __init__(
        self,
        keyty: NumbaTypeInst,
        valty: NumbaTypeInst,
        initial_value: pt.Optional[pt.Dict[pt.Any, pt.Any]] = None,
    ):

Readability is in the eye of the beholder, but from my perspective code like this is fairly unreadable. Besides the easy formatting things (e.g. multiple arguments per row in a multi-row definition), I have no idea what the arguments should be. If that definition looked more like the black + type annotations example above, it would be a lot clearer to me what’s going on.

4

I am not a huge fan of Black for every line, but I also use the one-arg-per-line style when I have complex annotations that don’t fit in one line. I would definitely choose

class DictType(IterableType, InitialValue):
    def __init__(
        self,
        keyty: NumbaTypeInst,
        valty: NumbaTypeInst,
        initial_value: pt.Optional[pt.Dict[pt.Any, pt.Any]] = None,
    ):

over the other alternatives.

Another way to improve readability is to use more type aliases, like MaybeDict = pt.Optional[pt.Dict[pt.Any, pt.Any]], when the same type is being used in many places. IDEs will allow you to jump to the definition in one click, so it’s not a big deal.

Regarding numpy arrays, it’s think it’s an open topic for mypy and numpy themselves, so I see no option but to wait for them.

Regarding directing the effort towards llvmlite, I wonder if the effort can be re-directed. I don’t think I would be able to type llvmlite, so my efforts are limited to Numba. But it’s true that core devs effort in reviewing PRs could be re-directed, and therefore subject to choosing what has higher priority.

I would like to point out an important aspect of annotations, that goes beyond the static checking, namely the formalization of internal APIs. Look at this code

There’s no indication of what typ, val, c are. Furthermore unbox_integer(typ, obj, c) might be readable for someone familiar with numba internals, but it’s absolutely unreadable for anyone else. Even if you are familiar, you get no help from autocompletion unless you annotate. There are many such internal APIs where certain signatures are expected, but no information of what will be passed to each argument.

Just my two cents.

5

On the topic of suggestions, I am wondering if the core devs are considering new extension APIs to allow new behaviour without modifying Numba itself.

6

Yes, the “numba-extras” for additive extensions. However, that will not allow changes to existing types or behavior.