# Cache behaviour

**URL:** <https://numba.discourse.group/t/cache-behaviour/1520>\
**Category:** Development\
**Created:** [August 19, 2022, 4:02pm UTC](https://numba.discourse.group/t/cache-behaviour/1520 "2022-08-19T16:02:30Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![luk-f-a](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/luk-f-a/32/56_2.png) [@luk-f-a](https://numba.discourse.group/u/luk-f-a)\
**Post date:** [August 19, 2022, 4:02pm UTC](https://numba.discourse.group/t/cache-behaviour/1520/1 "2022-08-19T16:02:30Z")

</div>

hi,

I’m working to fix the current lack of cache invalidation when a secondary file—a module from which other jit functions are imported and then called from a main jit function—is modified.  
I already have a branch where the overloads are picked from the cache according to the code signature of the calling function and all dependencies. This means that a change in the code of a dependency does not lead anymore to incorrect calculations.  
So far so good. In the process of testing, I noticed that the cache of the main function keeps growing, ie accumulating overloads.  
I found out that this is because the code of the main function has not changed, which means that the cached overloads are not considered stale, but simply unused. This could be a problem if the cache keeps growing and growing, so I started looking into that. In the process of looking into it, I noticed that currently the cache is very pessimistic about changes in the main file. Even changes outside the function invalidate the cache.  
So currently, the cache is ignoring changes in secondary files, but invalidating the cache for every change in the file, even outside the main function or its closure variables.

I would like to propose to move away from invalidating the cache index based on the timestamp of the file, and use only the code+closure signature of the function itself. Would anyone see a problem with that? When I say code+closure signature I mean the exact same information that is used to select the overload from within the cache.  
The cache index is currently a combination of (function signature, target context, code hash, closures hash). My proposal would be to use code hash + closure hash to decide whether the index should be reset (implicitly throwing away every existing cached compilation). When saving an overload in the cache, the cache machinery would compare the hashes of the latest overload against existing ones, and reset the index if they don’t match.  
All feedback and suggestions welcome.

Luk

---

<div class="post-metadata">

**Author:** ![nelson2005](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/nelson2005/32/47_2.png) [@nelson2005](https://numba.discourse.group/u/nelson2005)\
**Post date:** [August 19, 2022, 6:15pm UTC](https://numba.discourse.group/t/cache-behaviour/1520/2 "2022-08-19T18:15:38Z")

</div>

I don’t know what it would break, but I’m in favor of removing the timestamp from the cache.

This would facilitate zipping up source trees with cache and having them ‘still work’ after unzipping due to the lack of timestamp support in pkzip format.

There’s a discussion about that [here](https://numba.discourse.group/t/numba-portable-caching-logic/980/3)

---

<div class="post-metadata">

**Author:** ![luk-f-a](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/luk-f-a/32/56_2.png) [@luk-f-a](https://numba.discourse.group/u/luk-f-a)\
**Post date:** [August 19, 2022, 8:00pm UTC](https://numba.discourse.group/t/cache-behaviour/1520/3 "2022-08-19T20:00:00Z")

</div>

thanks @nelson2005 , from the related post it seems that you have been digging quite a lot into the cache too.

The cache takes a “snapshot” of the code (from `py_func. __code__.co_code`) and of the closures (from `py_func. __closure__ `). If we knew this is enough (ie, as safe as monitoring the file timestamp), I think I can find a way to replace the timestamp rule with a hashing rule.

Luk

---

<div class="post-metadata">

**Author:** ![alex](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/alex/32/61_2.png) [@alex](https://numba.discourse.group/u/alex)\
**Post date:** [August 22, 2022, 3:12pm UTC](https://numba.discourse.group/t/cache-behaviour/1520/4 "2022-08-22T15:12:59Z")

</div>

Hi @luk-f-a,

> The cache takes a “snapshot” of the code (from `py_func. __code__.co_code`) and of the closures (from `py_func. __closure__ `). If we knew this is enough (ie, as safe as monitoring the file timestamp), I think I can find a way to replace the timestamp rule with a hashing rule.

If the closure is capturing another function, that function may need to be considered in the hash, too, if I remember correctly. I implemented a simple recursive function for this in

> <https://github.com/numba/numba/pull/6284>
>
> First steps towards fixing #6264.
> 
> This PR changes the cache key to explicitly… include the bytecode of python functions captured in closures, and also their transitive dependencies. Before, the whole \`Dispatcher\` object was pickled and included in the key, which changes for each "run" (i.e. each time a new python interpreter is started) because of the UUID.
> 
> I'm not sure if this is enough, or if other attributes of the \`Dispatcher\` objects also need to be included in the key.
> 
> As @sklam noted in the issue, the referenced functions may also need to be cached (except if they are inlined?), but I'm not sure what the best approach would be, or if the user should be required to explicitly mark them to be cached.
> 
> Please let me know if this goes into the right direction, thanks!
> 
> \<!--
> 
> Thanks for wanting to contribute to Numba :)
> 
> First, if you need some help or want to chat to the core developers, please
> visit https://gitter.im/numba/numba for real time chat or post to the Numba
> mailing list https://groups.google.com/a/continuum.io/forum/#!forum/numba-users.
> 
> Here's some guidelines to help the review process go smoothly.
> 
> 0. Please write a description in this text box of the changes that are being
> made.
> 
> 1. Please ensure that you have written units tests for the changes made/features
> added.
> 
> 2. If you are closing an issue please use one of the automatic closing words as
> noted here: https://help.github.com/articles/closing-issues-using-keywords/
> 
> 3. If your pull request is not ready for review but you want to make use of the
> continuous integration testing facilities please label it with \`\[WIP\]\` and
> then remove the label when you'd like it to be reviewed.
> 
> 4. Once review has taken place please do not add features or make changes out of
> the scope of those requested by the reviewer (doing this just add delays as
> already reviewed code ends up having to be re-reviewed/it is hard to tell
> what is new etc!). Further, please do not rebase your branch on master/force
> push/rewrite history, doing any of these causes the context of any comments
> made by reviewers to be lost. If conflicts occur against master they should
> be resolved by merging master into the branch used for making the pull
> request.
> 
> Many thanks in advance for your cooperation!
> 
> \--\>

Although I’m not sure if this is enough. Hopefully some fix for [Caching: functions captured in closure · Issue #6264 · numba/numba · GitHub](https://github.com/numba/numba/issues/6264) can be included while you are touching this part of the code 🙂

Thanks,

Alex

---

<div class="post-metadata">

**Author:** ![gmarkall](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/gmarkall/32/42_2.png) [@gmarkall](https://numba.discourse.group/u/gmarkall)\
**Post date:** [August 23, 2022, 9:15am UTC](https://numba.discourse.group/t/cache-behaviour/1520/5 "2022-08-23T09:15:24Z")

</div>

I’ve added this to the agenda for the dev meeting today, as I think it will be helpful for thinking about kickstarting progress on those PRs: [Numba Meeting: 2022-08-23 - HackMD](https://hackmd.io/IFxARexrSVuL8D3ujl9gMA)