Supporting sparse matrix in numba is an important feature, especially for easily speeding up big-data-set processes that require a lot of memory. This issue was found in many places a long time ago and seems not so complex because built-in sorted() has been implemented. Is there any progress or agenda about this?
Just list some numba-based python projects relies on sparse matrix to tackle big-data.
github issues.
https://stackoverflow.com/questions/62207548/how-to-use-numba-to-speed-up-sparse-linear-system-solvers-in-python-that-are-pro
1 Like
Hi @PercyLau,
Numba focuses on providing support for the Python language and NumPy. Sparse matrix representations are part of e.g. SciPy, can be expressed in the Python language and NumPy, and should probably go into some sort of extension package e.g. https://github.com/numba/numba-scipy.
Hope this helps.
It seems numba-scipy encounters some difficulty in implementing sparse matrix https://github.com/numba/numba-scipy/issues/29. Part of the problem lays on that sparse matrix require lexsort in numpy to covert coo_sparse_matrix to csr_sparse_matrix/csc_sparse_matrix. Any progress/schedule to support the lexsort in numpy ?
I don’t think it’s hugely difficult, it just requires time, and that project needs some more infrastructure setting up.
In Numba, https://github.com/numba/numba/issues/3689 tracks lexsort, I suggest subscribing to that ticket to get updates.
1 Like