These are chat archives for ipython/ipython

27th
Sep 2018
jagan96
@jagan96
Sep 27 2018 13:26

class inputobject(object):
filepath = None

inputlist = []

for image in glob.glob(directory_location):
obj = inputobject()
obj.filepath = image
inputlist.append(obj)

view.map_async(func, inputlist[x:x+100],ordered=False)

hi guys,
can someone help me out in this
can we send a list of custom objects through view.map_async?
Min RK
@minrk
Sep 27 2018 13:31
Check out the markdown help in the chat box, it tells you how to highlight code blocks (three back-ticks: `)
You can send a list of custom input objects, but they must be pickleable. Defining classes at runtime in __main__ typically won't work because the class doesn't resolve on the engines.
If you add rc[:].use_cloudpickle(), it changes how serialization works a bit, and may get what you need.
jagan96
@jagan96
Sep 27 2018 14:23
if i use that , im getting this error : 'AsyncResult' object has no attribute 'sync_imports' @minrk
Min RK
@minrk
Sep 27 2018 14:28
Hm. Do you have the cloudpickle package?
jagan96
@jagan96
Sep 27 2018 14:28
yeah..installed it
Min RK
@minrk
Sep 27 2018 14:29
What OS, Python, ipyparallel version?
jagan96
@jagan96
Sep 27 2018 14:31
centos7 python3.6 ipyparallel6.2.2
Min RK
@minrk
Sep 27 2018 14:38
And are the engines local (same machine, same Python environment)?
jagan96
@jagan96
Sep 27 2018 14:42
right now its local with same python environment(however we will eventually be running on remote machines through mpi profile)
Min RK
@minrk
Sep 27 2018 14:44
Did you restart the engines after installing cloudpickle?
I was able to run your example after calling .use_cloudpickle() on my machine (mac, python 3.6, ipyparallel 6.2.2), but you can get weird issues if you have installed a package after starting the interpreter(s).
jagan96
@jagan96
Sep 27 2018 19:41
can you show me the code what you have tried ? @minrk
i think i might be missing something.