How to free the GPU memory once computations are done using the device_arrays

import numpy as np
from numba import cuda
n=999999
arr = np.arange(n)
device_arr = cuda.device_array(arr)
#do some computations
#now how can we free the memory used by device_arr?

I have tried deleting the device_arr variable after the computation but it is of no use.

You could call

cuda.current_context().memory_manager.deallocations.clear()

after removing all references to the memory.

Why do you want to free the memory eagerly? Numba will eventually clear up allocations, but it defers it in order to batch cleanup for better performance: Memory management — Numba 0.58.0dev0+116.gbd5b553b0.dirty documentation

1 Like

Thank you @gmarkall for responding.

Assuming that I only have 16GB memory on my GPU device and there are two different datasets of 8GB each. Whenever I try to process them one by one on device I run into memory allocation error. I observed that numba keeps the data on device forever till that program is running. I want to replace the dataset on device with my second dataset and perform computations. How can I ideally achieve this?

If you have no reference to the first dataset remaining then when you allocate the second one it should force Numba to deallocate the first one. Can you provide an executable sample of code that reproduces the issue? If there’s another reason the deallocation is prevented, I might be able to help identify it.