Contributing to Numba with no compiler or LLVM experience

Numba Development
1

Do you want to contribute to Numba, but haven’t worked on compilers and/or LLVM before?

This post provides a couple of pointers to resources that may be helpful - working through them should provide some background that will help to make sense of the way Numba generates code, particularly around the lowering of its IR to LLVM and code generation. Numba’s frontend is quite specific to Python and Numba, but some of the concepts you learn from the resources below might help to understand it in general.

  • My First Language Frontend with LLVM Tutorial - this tutorial walks step-by-step through implementing a compiler for a toy language that uses LLVM for code generation and JIT compilation. This tutorial uses C++.
  • Eli Bendersky’s Python / llvmlite version of the LLVM tutorial - because the above tutorial uses C++, Eli Bendersky implemented the tutorial code using llvmlite in Python - llvmlite is the library used by Numba for access to LLVM. Following the code of this tutorial in conjunction with the C++ LLVM tutorial may be helpful for understanding it better, and being able to understand Numba’s use of llvmlite. Note that this implementation is a few years old, so there might be some issues with using it with a current version of llvmlite, but it should still provide a good idea of how to use llvmlite.
  • Github repo for the Python / llvmlite version of the tutorial - Code for the Python version of the tutorial is here.
  • Mapping High Level Constructs to LLVM IR - this is a good “how-to” reference for when you know what you want to do in LLVM IR, but don’t know how to accomplish it. For example, if you’re wondering “How do I access a member of a struct?”, the section Accessing a Structure Member is very helpful and explains how to use the getelementptr instruction (or gep, as it’s written in llvmlite) to get a pointer to the member for loading / storing.

If you work through any of these resources, please do post back your experience working through it, and mention any issues working through them / translating the concepts to understanding Numba - I’d like to help out anyone who is working towards contributing to Numba - posting your experience will also help find ways to make it easier for other new contributors.

Other areas to look at to get started contributing:

  • Contributing to Numba in the manual explains how to get set up with a development environment, run tests, process / modes of communication (Gitter, Github, etc.), documentation, code reviews, etc.
  • Good First Issues - issues tagged “Good First Issue” are those that are considered good starting points for new contributors to work on. You can have a look at the list and see if you can find an issue that you think you can attempt.
  • Good Second Issues - those that are a little trickier than the “Good First Issues” - worth a look once you’ve tackled one or two issues already.

(Note: my intention is to update the list of resources as and when appropriate, to keep it relevant / improve it as time goes by)

8 Likes
2

A new addition to the list of useful resources:

3

This is the answer to my very first question for contributing to Numba. Thanks!

4

@gmarkall Can we have a similar contribution guide for numba-examples.

5

The numba-examples repo is a bit of a varied collection of items, so I’m not really sure I can write a concise and specific contribution guide for it. A couple of starting points might be to look at current and previous PRs (e.g. Add example calling cuRAND by gmarkall · Pull Request #40 · numba/numba-examples · GitHub) to see what a contribution looks like, and also to check out the Making a Benchmark section of the README.

I think the scope for numba-examples is fairly broad, so in my opinion any self-contained example of code that solves a problem in an interesting way or is a common requirement would make a good candidate for adding to the numba-examples repo.

1 Like
6

Thanks for the info @gmarkall