KeyError/PicklingError: Can't pickle class

Numba
1

Hello guys,

I am new to nunba and I am facing some issues with @jitclass decorator.
Here is the example code:

import numba as nb
from numba import jit, types, typed
from numba.experimental import jitclass
from numba import int64, float64

@jitclass([('x', float64), ('y', float64), ('z', float64)])
class Vector(object):
    def __init__(self, x, y, z):
        self.x = x
        self.y = y
        self.z = z

VectorType = Vector.class_type.instance_type

# VertexType = nb.deferred_type()
@jitclass([('pos', Vector.class_type.instance_type)])     
class Vertex(object):
    def __init__(self, pos):
        self.pos = pos

    def clone(self):
        return Vertex(self.pos)       

# VertexType.define(Vertex.class_type.instance_type)
VertexType = Vertex.class_type.instance_type   

# @jit(VertexType(VertexType), nopython=False)
# def CloneVertex(v):
#     return Vertex(Vector(v.pos.x, v.pos.y, v.pos.z))

@jitclass([('vertices', types.ListType(Vertex.class_type.instance_type)), ('shared', int64)])  
class Polygon(object):
    def __init__(self, vertices, shared):
        self.vertices = vertices
        self.shared = shared

PolygonType = Polygon.class_type.instance_type

def load_mesh(mesh_data):
    polygons = typed.List.empty_list(PolygonType)

    for polygon in mesh_data:
        vertices = typed.List.empty_list(VertexType)
        for v in polygon['vertices']:
             vertices.append( Vertex(Vector(v[0], v[1], v[2])) )
        polygons.append( Polygon(vertices, polygon['shared']) )

    return polygons

def put_back(p):
    return p[0].vertices[0].x

mesh = [{'vertices': [(0.0, 0.0, 0.0), (1.0, 0.0, 1.0), (-1.0, -1.0, 0.0)], 'shared': 0}, {'vertices': [(1.0, 0.0, 0.0), (0.0, 0.0, 1.0), (1.0, -1.0, 0.0)], 'shared': 1}, {'vertices': [(1.0, 1.0, 0.0), (0.0, 1.0, 1.0), (1.0, -1.0, 1.0)], 'shared': 2}]

Then calling the following in jupyter notebook I get a pickling error

p = load_mesh(mesh)
p[0].vertices[0]

There are more details in this bug report: https://github.com/numba/numba/issues/6032
Shortly put, the error I get goes away when I remove the clone() method which calls the constructor directly from the method. I have no idea why… Anyway, I need to be able to instantiate new objects from methods.
Do you guys have any idea why this is happening? Thanks!

I am using python 3.7.7 with numba built from master

2

I also have this problem and have switched to using named tuples when I need to be able to pickle. You can overload functions on the named tuple to provide custom behaviour.

3

@luk-f-a how? is it mutable? can I return new instances of the same ‘class’ member methods? thanks

4

BTW, I think we pin point the problem in my example. Appears to be related to the fact that I use typed lists in combination with custom classes in another class definition. The error is silent until you call a constructor from an instance methd :slight_smile: and only then you get a KeyError followed by a pickele error even thus I am not using pickeling directly