Contributing to Numba with no compiler or LLVM experience

Do you want to contribute to Numba, but haven’t worked on compilers and/or LLVM before?

This post provides a couple of pointers to resources that may be helpful - working through them should provide some background that will help to make sense of the way Numba generates code, particularly around the lowering of its IR to LLVM and code generation. Numba’s frontend is quite specific to Python and Numba, but some of the concepts you learn from the resources below might help to understand it in general.

  • My First Language Frontend with LLVM Tutorial - this tutorial walks step-by-step through implementing a compiler for a toy language that uses LLVM for code generation and JIT compilation. This tutorial uses C++.
  • Eli Bendersky’s Python / llvmlite version of the LLVM tutorial - because the above tutorial uses C++, Eli Bendersky implemented the tutorial code using llvmlite in Python - llvmlite is the library used by Numba for access to LLVM. Following the code of this tutorial in conjunction with the C++ LLVM tutorial may be helpful for understanding it better, and being able to understand Numba’s use of llvmlite. Note that this implementation is a few years old, so there might be some issues with using it with a current version of llvmlite, but it should still provide a good idea of how to use llvmlite.
  • Github repo for the Python / llvmlite version of the tutorial - Code for the Python version of the tutorial is here.
  • Mapping High Level Constructs to LLVM IR - this is a good “how-to” reference for when you know what you want to do in LLVM IR, but don’t know how to accomplish it. For example, if you’re wondering “How do I access a member of a struct?”, the section Accessing a Structure Member is very helpful and explains how to use the getelementptr instruction (or gep, as it’s written in llvmlite) to get a pointer to the member for loading / storing.

If you work through any of these resources, please do post back your experience working through it, and mention any issues working through them / translating the concepts to understanding Numba - I’d like to help out anyone who is working towards contributing to Numba - posting your experience will also help find ways to make it easier for other new contributors.

Other areas to look at to get started contributing:

  • Contributing to Numba in the manual explains how to get set up with a development environment, run tests, process / modes of communication (Gitter, Github, etc.), documentation, code reviews, etc.
  • Good First Issues - issues tagged “Good First Issue” are those that are considered good starting points for new contributors to work on. You can have a look at the list and see if you can find an issue that you think you can attempt.
  • Good Second Issues - those that are a little trickier than the “Good First Issues” - worth a look once you’ve tackled one or two issues already.

(Note: my intention is to update the list of resources as and when appropriate, to keep it relevant / improve it as time goes by)

6 Likes

A new addition to the list of useful resources: