# Containerized application without recompilation at startup **URL:** \ **Category:** Support: How do I do ...?\ **Created:** [November 11, 2022, 8:27am UTC](https://numba.discourse.group/t/containerized-application-without-recompilation-at-startup/1637 "2022-11-11T08:27:11Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![JPy](https://avatars.discourse-cdn.com/v4/letter/j/8797f3/32.png) [@JPy](https://numba.discourse.group/u/JPy)\ **Post date:** [November 11, 2022, 8:27am UTC](https://numba.discourse.group/t/containerized-application-without-recompilation-at-startup/1637/1 "2022-11-11T08:27:11Z")
Hello, I am currently trying to deploy a Numba application in a docker container. As there are restrictions on the startup latency I really need to avoid the recompilation time, which is already in the order of minutes and growing. So far I have tried those methods in the container build step without success: - Ahead of time compliation My problem here is that does not support all numba features. In particular I need `objmode` at one critical point in the application (to get the elapsed time, see e.g. [I am not allowed to link]) which does not seem to work with AOT. - Trigger a cached JIT and reuse the cache after startup While this works when running the container on the same machine where it was built, running on a different host triggers a recompilation, probably due to a different “magic tuple” (again, link missing). I tried `NUMBA_CPU_NAME=generic` but the code still gets recompiled. Any help is greatly appreciated. I’m wondering if I am missing something as this use case does not seem to be that uncommon. ---
**Author:** ![nelson2005](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/nelson2005/32/47_2.png) [@nelson2005](https://numba.discourse.group/u/nelson2005)\ **Post date:** [November 12, 2022, 1:01am UTC](https://numba.discourse.group/t/containerized-application-without-recompilation-at-startup/1637/2 "2022-11-12T01:01:43Z")
What operating system are you using? How are you installing into the container? Pip? I understand you can’t post links… Have you reviewed [this](https://numba.discourse.group/t/numba-portable-caching-logic/980) and links on that topic? ---
**Author:** ![JPy](https://avatars.discourse-cdn.com/v4/letter/j/8797f3/32.png) [@JPy](https://numba.discourse.group/u/JPy)\ **Post date:** [November 14, 2022, 11:07am UTC](https://numba.discourse.group/t/containerized-application-without-recompilation-at-startup/1637/3 "2022-11-14T11:07:52Z")
Thanks @nelson2005. For development I am building the container on Mac OS, the failed caching happens when running it on a serverless cloud service. With the help of your print function I was able to extract the nbi information on both machines. For all files it looks like this: Local ```auto numba_version=0.56.0 stamp=(1668417210.6453834, 1311) overload_fname=[...]-13.py38.1.nbc overload_key_signature=(array(int64, 1d, C), int64) overload_key_magic_tuple=('x86_64-unknown-linux-gnu', 'generic', '') overload_key_code_hash=('2c74cec6df4a9612533131c0106277fd1c8c0f4d75b92ed37caef8dcf3380c7d', 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855') ``` Remote ```auto numba_version=0.56.0 stamp=(1668417210.0, 1311) overload_fname=[...]-13.py38.1.nbc overload_key_signature=(array(int64, 1d, C), int64) overload_key_magic_tuple=('x86_64-unknown-linux-gnu', 'generic', '') overload_key_code_hash=('2c74cec6df4a9612533131c0106277fd1c8c0f4d75b92ed37caef8dcf3380c7d', 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855') ``` The magic tuple and the hashes are actually the same, it’s only the timestamp that differs. Seems to be very similar to your Issue on Spark (“This process also appears to not preserve the requisite python source code file timestamp precision to match the sub-second value contained in the numba cache index .nbi file.”). However, in my case there is no unpacking/copying involved and the timestamps originate directly from the unpickled nbi. That currently does not make sense to me. Can that difference be the reason for the recompliation? ---
**Author:** ![JPy](https://avatars.discourse-cdn.com/v4/letter/j/8797f3/32.png) [@JPy](https://numba.discourse.group/u/JPy)\ **Post date:** [November 14, 2022, 3:17pm UTC](https://numba.discourse.group/t/containerized-application-without-recompilation-at-startup/1637/4 "2022-11-14T15:17:26Z")
Alright, got it. The time precision of the filesystem seems to be a feature of the docker host and the remote one did not support sub-second precision, leading to a recompilation even when the magic tuple was the same. I monkey-patched the method `_load_index` in the Class `IndexDataCacheFile` to not do the timestamp check and it finally worked. Now looking for a cleaner solution as this like messing with the timestamp values does not seem ideal. It would be nice to have an option that disables automatic cache invalidation. ---
**Author:** ![nelson2005](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/nelson2005/32/47_2.png) [@nelson2005](https://numba.discourse.group/u/nelson2005)\ **Post date:** [November 14, 2022, 3:34pm UTC](https://numba.discourse.group/t/containerized-application-without-recompilation-at-startup/1637/5 "2022-11-14T15:34:37Z")
Glad you solved it! Another option might be possibly patching the timestamp on the save-side? Kind-of-related to your objmode timing, [this](https://numba.discourse.group/t/profiling-with-a-decorator-and-njit/55/21) discussion has @stuartarchibald’s nifty implementation of doing timing with the stdlib. ---
**Author:** ![JPy](https://avatars.discourse-cdn.com/v4/letter/j/8797f3/32.png) [@JPy](https://numba.discourse.group/u/JPy)\ **Post date:** [November 15, 2022, 8:16am UTC](https://numba.discourse.group/t/containerized-application-without-recompilation-at-startup/1637/6 "2022-11-15T08:16:01Z")
But in any case I think you would need to modify both the pickled timestamps as well as the file timestamps. Otherwise you could end up with the inverse situation when the host actually supports the higher precision. However, I noticed there was already a discussion in the dev forum to remove the timestamp-based invalidation, so there is probably no need for a naive PR and I’ll diff-patch the code during the build step for now. Thanks for the other link, completely missed that. I am still not sure if that would also work with AOT, but caching is the more convenient method anyway.