These are chat archives for ipython/ipython

28th
Jun 2015
Sylvain Corlay
@SylvainCorlay
Jun 28 2015 02:21
This message was deleted
Matthias Bussonnier
@Carreau
Jun 28 2015 18:03
@jasongrout , I seem to have misplaced my serialize.ts, do you have a copy if ever... (trying to get phosphor notebook working.)
Min RK
@minrk
Jun 28 2015 18:19
@ssanderson you can modify the completer, or even override the completion with your own completer object in the kernel.
Scott Sanderson
@ssanderson
Jun 28 2015 23:07

@minrk meaning I should like write an extension that looks something like:

def load_ipython_extension(ipython):
    ipython.Completer = CustomCompleter()

?

Min RK
@minrk
Jun 28 2015 23:08
I think so.
Scott Sanderson
@ssanderson
Jun 28 2015 23:09
hrm. I see references to custom_completers on the IPCompleter class but I don't see where I would pass those in
epifanio
@epifanio
Jun 28 2015 23:16
Hi, do you have any clue about this memory error : http://paste.debian.net/264232
my VM is actually running out of memory
but i can’t see which process is using it.
i’m using a bash kernel (but same happen if i use “!” prefix in a python kernel)
resterting the kernel doesn’t free up memory
to free up it .. i’ve to quit the notebook and restart it
Scott Sanderson
@ssanderson
Jun 28 2015 23:19
@epifanio that looks like you're trying to receive a gigantic message over a ZMQ socket?
@minrk would be the person to know whether that's plausible though
Min RK
@minrk
Jun 28 2015 23:20
custom completers let you add completions, but I don't think you can change existing results with them.
epifanio
@epifanio
Jun 28 2015 23:21
i see in htop the memory usage is aasigned to the ipython notebook (running from jupyter)
Min RK
@minrk
Jun 28 2015 23:22
@epifanio what code are you trying to run when this happens?
epifanio
@epifanio
Jun 28 2015 23:24
@minrk i’m working with gdal command line on a tif image (using bash kernel) i have a notebook that reproduce the behavior
Min RK
@minrk
Jun 28 2015 23:24
Does the command produce lots of output?
epifanio
@epifanio
Jun 28 2015 23:25
no i’m tring to display the image generated
and seems that the problem is in the code that convert the image (i was using convert from imagegik)
the original size is 7mb
but for weird reason (just found out that are not related to ipython) the converted jpg and png are huge
but in any case, when i clear the output
and restart the kernel
the memory should come back free
but this doesn’t happen
Min RK
@minrk
Jun 28 2015 23:31
Is the process holding memory the kernel or the notebook server?
Restarting the kernel kind of has to free its memory, since the process doesn't exist anymore.
What versions of zeromq and pyzmq?
i’ll check the 0mq version
Min RK
@minrk
Jun 28 2015 23:34
And which call is causing the problem?
epifanio
@epifanio
Jun 28 2015 23:35
zmq version : 14.6.0
i’ll restart the server and monitor the memory usage at each step of that notebook
Min RK
@minrk
Jun 28 2015 23:35
And keep track of the notebook server process and the kernel process separately.
epifanio
@epifanio
Jun 28 2015 23:36
how ?
Min RK
@minrk
Jun 28 2015 23:37
They have different PIDs
epifanio
@epifanio
Jun 28 2015 23:37
ok in htop the process that was using memory was : ipython notebook (2 processes at 65% memory each)
i’ll check again now
epifanio
@epifanio
Jun 28 2015 23:48
i re-run the entire notebook and now the memory used is 2gb
from 350mb when i start the server
the pid that are using the memory are related to the 3 ipython notebook processes
restarting the kernel the memory used is still 2gb
Scott Sanderson
@ssanderson
Jun 28 2015 23:51
presumably there are buffers on the notebook server side of those ZMQ sockets. how do those get cleaned up when a kernel is restarted?
epifanio
@epifanio
Jun 28 2015 23:53
re running the notebook the memory grows again but up to 2.250 gb (just 250 mb more than with running the notebook the first time)
i’ll do one more restart and run again
i just hit restart and the memory frws up to 2.8 gb
in the notrbook interface i received the following log :
Screen Shot 2015-06-28 at 7.54.35 PM.png
which refer to this log in the terminal : https://gist.github.com/anonymous/b6c396cfd140f6c3ee95