# \`guvectorized\`: No performance difference between targets \`cpu\` and \`parallel\`? \`cuda\` even slower. \`vectorize\` faster?

**URL:** <https://numba.discourse.group/t/guvectorized-no-performance-difference-between-targets-cpu-and-parallel-cuda-even-slower-vectorize-faster/1482>\
**Category:** Support: How do I do ...?\
**Created:** [August 3, 2022, 1:34pm UTC](https://numba.discourse.group/t/guvectorized-no-performance-difference-between-targets-cpu-and-parallel-cuda-even-slower-vectorize-faster/1482 "2022-08-03T13:34:36Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![s-m-e](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/s-m-e/32/536_2.png) [@s-m-e](https://numba.discourse.group/u/s-m-e)\
**Post date:** [August 3, 2022, 1:34pm UTC](https://numba.discourse.group/t/guvectorized-no-performance-difference-between-targets-cpu-and-parallel-cuda-even-slower-vectorize-faster/1482/1 "2022-08-03T13:34:36Z")

</div>

Long story short: **Am I using `guvectorized` wrong?**

* * *

I have just benchmarked a relatively simple workload for both `vectorize` and `guvectorize` across all targets (`cpu`, `parallel` and `cuda`) across different sizes of input arrays.

On the `vectorize` size, stuff is as expected: For sufficiently large input arrays, I can saturate all my cores with target `parallel`. Performance scales accordingly. `cuda` is yet a little faster than all the CPU cores, again as expected.

`guvectorized` for targets `cpu` and `parallel` is basically as fast as `vectorize` for target `cpu`. It does not benefit from multiple cores, with and without the use of `prange`? Even more interestingly, when I switch `guvectorized` to target `cuda`, it gets _more_ than a solid order of magnitude _slower_ than a _single_ CPU core?

* * *

Test workload looks somewhat like this:

```python
COMPLEXITY = 2 ** 11

@jit(*args, **kwargs)
def helper(scalar: float) -> float:
    res: float = 0.0
    for idx in range(COMPLEXITY):
        if idx % 2 == round(scalar) % 2:
            res += sin(idx)
        else:
            res -= cos(idx)
    return res

@vectorize(*args, **kwargs)
def v_main(d: float) -> float:
    return helper(d)

@guvectorize(*args, **kwargs)
def gu_main(d, r):
    for idx in [nb.p]range(d.shape[0]):
        r[idx] = helper(d[idx])

```

[Full self-contained notebook with all implementations, benchmark and plots of the results](https://github.com/pleiszenburg/bulk_propagate/blob/master/test_04.ipynb).

 ![plot_all](https://global.discourse-cdn.com/free1/uploads/numba/original/1X/696ac93af25e16f92557a45d2c822d70f4d31f10.jpeg)

 ![plot_relative_speedup](https://global.discourse-cdn.com/free1/uploads/numba/original/1X/d978682d296869058df3f2ee1cf089e3577cf7a6.jpeg)

---

<div class="post-metadata">

**Author:** ![s-m-e](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/s-m-e/32/536_2.png) [@s-m-e](https://numba.discourse.group/u/s-m-e)\
**Post date:** [August 3, 2022, 3:00pm UTC](https://numba.discourse.group/t/guvectorized-no-performance-difference-between-targets-cpu-and-parallel-cuda-even-slower-vectorize-faster/1482/2 "2022-08-03T15:00:58Z")

</div>

Summary: **I did not understand how to use it correctly …**

* * *

Assuming that I have something along the following lines:

```python
@numba.guvectorize('void(f8[:],f8[:])', '(n)->(n)', target = 'parallel')
def gu_main(d: float, r: float):
    for idx in range(d.shape[0]):
        r[idx] = helper(d[idx])

```

My mistake was to give it an array of `ndim == 1`, i.e.

```python
numpy.arange(0, size, dtype = 'f8')

```

If I, on the other hand, give it the following array with `ndim > 1` …

```python
numpy.arange(0, size, dtype = 'f8').reshape(size, 1)

```

… the parallelization kicks in just as expected for both targets `parallel` and `cuda`:

 ![plot_all](https://global.discourse-cdn.com/free1/uploads/numba/original/1X/650f9c24cca024911a641720d04910cbe376b573.jpeg)

 ![plot_relative_speedup](https://global.discourse-cdn.com/free1/uploads/numba/original/1X/f794a79ef31000b8e86d1c6477f45cf55833b51b.jpeg)

Now I am beginning to understand.

* * *

[Notebook with complete test](https://github.com/pleiszenburg/bulk_propagate/blob/master/test_05.ipynb)

---

<div class="post-metadata">

**Author:** ![s-m-e](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/s-m-e/32/536_2.png) [@s-m-e](https://numba.discourse.group/u/s-m-e)\
**Post date:** [August 3, 2022, 3:20pm UTC](https://numba.discourse.group/t/guvectorized-no-performance-difference-between-targets-cpu-and-parallel-cuda-even-slower-vectorize-faster/1482/3 "2022-08-03T15:20:28Z")

</div>

Absolutely not the most intuitive thing to do, but …

```python
@numba.guvectorize('void(f8[:],f8[:])', '()->()', target = 'parallel')
def gu_main(d: float, r: float):
    r[0] = helper(d[0])

```

… scales just fine for input arrays of `ndim == 1`. For reference, see [numba#2935](https://github.com/numba/numba/issues/2935).

---

<div class="post-metadata">

**Author:** ![s-m-e](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/s-m-e/32/536_2.png) [@s-m-e](https://numba.discourse.group/u/s-m-e)\
**Post date:** [August 3, 2022, 4:47pm UTC](https://numba.discourse.group/t/guvectorized-no-performance-difference-between-targets-cpu-and-parallel-cuda-even-slower-vectorize-faster/1482/4 "2022-08-03T16:47:23Z")

</div>

Whatever loops one places inside a function decorated by `guvectorize` do not get parallelized. That’s key. So my initial workaround simply restricts the loop in my initial function to one iteration, allowing the parallelization on other dimensions to kick in. Before, there was nothing to parallelize: Everything was handled by a sequential for loop inside the function - which perfectly explains the bad benchmark results I initially observed.

Perhaps this could be made _a bit more clear_ in the documentation … ?

---

<div class="post-metadata">

**Author:** ![gmarkall](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/gmarkall/32/42_2.png) [@gmarkall](https://numba.discourse.group/u/gmarkall)\
**Post date:** [August 8, 2022, 9:55am UTC](https://numba.discourse.group/t/guvectorized-no-performance-difference-between-targets-cpu-and-parallel-cuda-even-slower-vectorize-faster/1482/5 "2022-08-08T09:55:58Z")

</div>

The warning that Numba emitted was also pointing to this:

```auto
/home/ernst/Desktop/PROJEKTE/prj.TST2/github.poliastro/env310/lib/python3.10/site-packages/numba/cuda/dispatcher.py:488: NumbaPerformanceWarning: Grid size 1 will likely result in GPU under-utilization due to low occupancy.
  warn(NumbaPerformanceWarning(msg))

```

Perhaps for gufuncs this warning could also point a bit more specifically to the relationship between grid size and the input shape.

> [@s-m-e](#):
>
> Perhaps this could be made _a bit more clear_ in the documentation … ?

I think so - would you be happy to create a PR to add a short section to the gufunc documentation that would help to make this clear please? (perhaps as a subsection of [Creating NumPy universal functions — Numba 0+untagged.4124.gd4460fe.dirty documentation](https://numba.readthedocs.io/en/stable/user/vectorize.html#the-guvectorize-decorator))

---

<div class="post-metadata">

**Author:** ![s-m-e](https://yyz2.discourse-cdn.com/free1/user_avatar/numba.discourse.group/s-m-e/32/536_2.png) [@s-m-e](https://numba.discourse.group/u/s-m-e)\
**Post date:** [August 9, 2022, 3:56pm UTC](https://numba.discourse.group/t/guvectorized-no-performance-difference-between-targets-cpu-and-parallel-cuda-even-slower-vectorize-faster/1482/6 "2022-08-09T15:56:57Z")

</div>

If I can find the right words, I will. Just made a numba dev environment work and sent a little PR (on another subject) your way.