I had no luck with the latest Cuda 12 (or 12.1). I used it with the latest kernel for Ubuntu (5.19.0-35-generic).
I cleaned anything NVIDIA from my system and switched to Nouveau driver.
Once I downgraded the kernel (Linux 5.15.0-46-generic x86_64) and used downgraded driver (Driver Version: 515.86.01 CUDA Version: 11.7 ), reinstalled Anaconda things started to work. There is something wrong with the latest release from Nvidia. I do not feel I know enough to try to track it down.
I am still running Ubuntu 22.04.02. So the system has just the downgraded kernel and driver which I pinned so it is not changed.
One more thing I should add: I did not bother to download anything from Nvidia. I relay on Anaconda configuration only.