For loop error Cannot unify list(int64)<iv=None>

Numba Support: What is this error message?
1

I’m getting an error that I dont quite understand. I’ve opened a few files, and I’m doing some basic calculations.

Here’s my code

@njit
def process(beta, INPUT, mass, energy, J_use=1, J_error=1, intype=1):
    cross_cons = 8 * np.pi * mass * mass / energy  # a constant for the cross section
    # INPUT coresponds to the dm target model ie. dwarfset.dat
    with objmode(dat_file = "float64[:,:]"):
        dat_file = np.loadtxt(INPUT)
    # dat_file = dwarfset.dat ex, set1.dat
    d_col = dat_file.shape

    if d_col[1] == 2:
        J_use = 1
        J_error = 0
    if d_col[1] >= 3:
        J_use = 1
        J_error = 1

    dwarf_list = [int(i) for i in dat_file[:,0]]
    dwarf_count = len(dat_file)
    dwarf_data = dat_file

    NOBS = FILE_PATH + '/PMFdata/Fermi_Pass8R3_239557417_585481831/NOBS.dat'

    with objmode(obs_data = "float64[:,:]"):
        obs_data = np.loadtxt(NOBS, usecols=(0, 1, 2))

    with objmode(PMF = "float64[:,:]"):    
        PMF = np.loadtxt(FILE_PATH + '/PMFdata/Fermi_Pass8R3_239557417_585481831/pmf.dat')
    # print(PMF[:,0])
    N_obs = 0
    for i in range(len(obs_data)):
        for j in range(len(dwarf_list)):
            if obs_data[i][0] == dwarf_list[j]:
                # Summing over all OBserved events ## code transelation Nobs = Nobs + obs_data[dwarf_list[i] - 1][1];
                N_obs = N_obs + obs_data[i][1]
    N_obs = int(N_obs)
    print(N_obs)
    zeros_matx = np.zeros(((abs((N_obs + 2) - len(PMF))), (gal + 1)))
    PMF = np.concatenate((PMF,zeros_matx))

    ####################################
    # Calculating P1
    ####################################

    N_bdg = 0

    P1 = []
    if dwarf_count == 1:
        for i in range(N_obs + 1):
            P1.append(PMF[i][dwarf_list[0]]) ## Nobs + 1 length

    if dwarf_count != 1:
        P1 = [0] * (N_obs + 2)
        I = [0] * (N_obs + 2) #nobs + 2 length
        X = [0] * (N_obs + 2) #nobs + 2 length
        temp = 0
        for i in range(N_obs + 1):
            I = PMF[i][dwarf_list[0]] ## The background galaxy
  

        for k in range(1,dwarf_count - 1):
            for j in range(N_obs + 2):
                for i in range(j + 1): ### Error starts here
                    temp = (PMF[i][dwarf_list[k]] * I[j - i]) ## Equation 6 paper 1802.03826, the product 
                    X[j] = X[j] + temp ## Stacking the drawrf and computing the sum from equation 6
            for j in range(N_obs + 1):
                I[j] = X[j]
            for j in range(N_obs + 1):
                X[j] = 0
        for n in range(N_obs + 1):
            for k in range(n + 1):
                X[k] = PMF[n - k][dwarf_list[dwarf_count-1]] * I[k] ## Equation 8 paper 1802.03826, the product -LR
            for k in range(n + 1):
                P1[n] = P1[n] + X[k] ## Equation 8 paper 1802.03826, computing the sum -LR
                ## P1 the total expected photon distribution -LR
            N_bdg += 1



Cannot unify list(int64)<iv=None> and float64 for 'I.2', defined at MADHATpy.py (148)


def process(beta, INPUT, mass, energy, J_use=1, J_error=1, intype=1):
    <source elided>
            for j in range(N_obs + 2):
                for i in range(j + 1):
                ^

I don’t quite understand this error. Is there something wrong with my list?
Also, I noticed that if I were to rewrite

        for i in range(N_obs + 1):
            I = PMF[i][dwarf_list[0]] ## The background galaxy

to

        for i in range(N_obs + 1):
            I[i] = PMF[i][dwarf_list[0]] ## The background galaxy

it doesn’t seem to rewrite the list for those elements in I

2

hi @luisrufino

Intuitively, it looks like

for i in range(N_obs + 1):
            I[i] = PMF[i][dwarf_list[0]] ## The background galaxy

is the right option. Because you first create the empty list I = [0] * (N_obs + 2) and then populate it.
The alternative

for i in range(N_obs + 1):
            I = PMF[i][dwarf_list[0]] ## The background galaxy

is the same as

I = PMF[N_obs][dwarf_list[0]] ## The background galaxy

is this what you want?

I cannot guess what the intention behind the code is, so only you can answer that. Is this correct I = PMF[N_obs][dwarf_list[0]] or this I[N_obs] = PMF[N_obs][dwarf_list[0]] ?

Luk

3

Not quite. Dwarf_list[0] corresponds to the column of the PMF file. I guess I could simply do I = PMF[:,dwarf_list[0]] instead of a for loop.

If I replace the for loop

for i in range(N_obs + 1):
            I = PMF[i][dwarf_list[0]] ## The background galaxy

with I = PMF[:,dwarf_list[0]] and remove the empty list I = [0] * (N_obs + 2) an error message is displayed.

    assert fromty.dtype == toty.dtype
AssertionError: Failed in nopython mode pipeline (step: Handle with contexts)
Failed in nopython mode pipeline (step: native lowering)

I don’t quite understand this error message. What is failing in nopython mode? The way I’ve set up the lists?

I also apologize I realize my original post is hard to read and figure out what I’m asking.

4

hi @luisrufino it seems that you’re having with dtypes in the arrays. The first thing I would try if I were you, is to remove the njit decorator. Sometimes the pure python error messages can be clearer.

If that doesn’t help, would you be able to make a minimal reproducer? There’s a lot going on in your original script. Could you reduce it to the absolute minimum that produces the same error?
Since your script relies on reading external files, could you replace those lines with fake ones or zeros arrays?
The process of creating a minimal reproducer many times leads to yourself finding what the problem is. But even if you don’t find the problem, the minimal reproducer makes it possible for someone else to run your code and test a solution.
All I can tell at the moment is that there’s a type incompatibility. That type seems to shift from one error message to another depending on the code changes that you make.

Luk