Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    David Leen
    @dleen
    Fantastic! Thanks!
    Peyton Murray
    @peytondmurray
    Hey all, I'm trying to deploy the base-notebook docker image using zero-to-jupyterhub with kubernetes v1.2.0. I'm trying to use a custom content manager, which I've configured in /etc/jupyter/jupyter_notebook_config.py by setting c.ServerApp.contents_manager_class = CustomContentManager. When I run the base-notebook docker image locally, the custom content manager works just fine, but when I deploy it through zero-to-jupyterhub with kubernetes, the custom content manager isn't used (although logging statements confirm that the config for the ServerApp is correctly set). Anyone know what to do here to get my custom content manager to be used?
    Ranji Raj
    @ranjiGT
    Hi all, I am working with JupyterLab (notebook version = 6.4) for culling idle kernels. From my research, I got to know https://jupyterhub.readthedocs.io/en/stable/reference/config-user-env.html#example-enable-a-jupyter-notebook-configuration-setting-for-all-users from this example that it has to be updated jupyter_server_config.py and I set the necessary parameters for culling but still It does not cull idle servers. Is there any fix to this? Attaching my logs
    [D 2022-04-14 15:36:02.908 SingleUserLabApp kernelmanager:521] activity on ff80b978-a15c-4c80-87cd-64acd851020e: status (busy)
    [D 2022-04-14 15:36:02.914 SingleUserLabApp kernelmanager:521] activity on ff80b978-a15c-4c80-87cd-64acd851020e: status (idle)
    [D 2022-04-14 15:36:03.075 SingleUserLabApp kernelmanager:521] activity on ff80b978-a15c-4c80-87cd-64acd851020e: status (busy)
    [D 2022-04-14 15:36:03.079 SingleUserLabApp kernelmanager:521] activity on ff80b978-a15c-4c80-87cd-64acd851020e: status (idle)
    [D 2022-04-14 15:36:03.081 SingleUserLabApp kernelmanager:521] activity on ff80b978-a15c-4c80-87cd-64acd851020e: status (busy)
    [D 2022-04-14 15:36:03.082 SingleUserLabApp kernelmanager:521] activity on ff80b978-a15c-4c80-87cd-64acd851020e: status (idle)
    [D 2022-04-14 15:36:03.113 SingleUserLabApp kernelmanager:521] activity on ff80b978-a15c-4c80-87cd-64acd851020e: status (busy)
    [D 2022-04-14 15:36:03.116 SingleUserLabApp kernelmanager:521] activity on ff80b978-a15c-4c80-87cd-64acd851020e: status (idle)
    [D 2022-04-14 15:36:03.207 SingleUserLabApp kernelmanager:521] activity on ff80b978-a15c-4c80-87cd-64acd851020e: status (busy)
    [D 2022-04-14 15:36:03.210 SingleUserLabApp kernelmanager:521] activity on ff80b978-a15c-4c80-87cd-64acd851020e: status (idle)
    [D 2022-04-14 15:36:03.244 SingleUserLabApp kernelmanager:521] activity on ff80b978-a15c-4c80-87cd-64acd851020e: status (busy)
    [D 2022-04-14 15:36:03.247 SingleUserLabApp kernelmanager:521] activity on ff80b978-a15c-4c80-87cd-64acd851020e: status (idle)
    Kevin Bates
    @kevin-bates

    Hi @ranjiGT,

    The culling of idle servers (i.e., shutdown_no_activity_timeout) requires that the server have no outstanding instances of managed kernels or terminal sessions. Based on your log output, your server's kernel activity appears to be fairly active - with sub-second kernel interactions. I'm assuming the configuration you're using has much smaller intervals for defining when culling should occur, but I suspect your current issue is that you have an active kernel.

    Since server culling requires no kernel activity and, thus, kernel culling, you might be interested in another one of the kernel-culling configuration properties: c.MappingKernelManager.cull_connected. Its default value is False meaning that even though the cull_idle_timeout may be exceeded, the kernel will still not be culled if there are active connections to it (i.e., a browser is viewing the kernel's execution results). Since you might have an open browser, this could be the reason why your kernel is not getting culled, and therefore, why your server is not getting culled.

    3 replies
    Ranji Raj
    @ranjiGT
    Hi all, I am working with JupyterLab for culling idle kernels. I obtained the following logs from my docker container after I closed the browser session (the container is still active). But not sure if it really worked. It would be helpful if anyone can confirm the same.
    
    [I 2022-04-20 17:33:47.185 SingleUserLabApp log:189] 200 GET /user/ranji%20raj/api/sessions?1650476027117 (ranji raj@::ffff:192.168.160.1) 2.36ms
    [D 2022-04-20 17:36:04.469 SingleUserLabApp mixins:558] Notifying Hub of activity 2022-04-20T17:33:47.185010Z
    WARNING: using start-singleuser.sh instead of start-notebook.sh to start a server associated with JupyterHub.
    Entered start.sh with args: jupyterhub-singleuser --ip=0.0.0.0
    Executing the command: jupyterhub-singleuser --ip=0.0.0.0
    [I 2022-04-20 20:30:14.238 SingleUserLabApp mixins:614] Starting jupyterhub single-user server version 2.2.0
    [I 2022-04-20 20:30:14.238 SingleUserLabApp mixins:628] Extending jupyterlab.labhubapp.SingleUserLabApp from jupyterlab 3.2.5
    [I 2022-04-20 20:30:14.238 SingleUserLabApp mixins:628] Extending jupyter_server.serverapp.ServerApp from jupyter_server 1.13.1
    [D 2022-04-20 20:30:14.259 SingleUserLabApp migrate:83] No files in /home/jovyan/.ipython/nbextensions
    [D 2022-04-20 20:30:14.260 SingleUserLabApp application:174] Searching ['/home/jovyan', '/home/jovyan/.jupyter', '/home/jovyan/.local/etc/jupyter', '/opt/conda/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
    [D 2022-04-20 20:30:14.261 SingleUserLabApp application:731] Looking for jupyter_config in /etc/jupyter
    [D 2022-04-20 20:30:14.261 SingleUserLabApp application:731] Looking for jupyter_config in /usr/local/etc/jupyter
    [D 2022-04-20 20:30:14.261 SingleUserLabApp application:731] Looking for jupyter_config in /opt/conda/etc/jupyter
    [D 2022-04-20 20:30:14.261 SingleUserLabApp application:731] Looking for jupyter_config in /home/jovyan/.local/etc/jupyter
    [D 2022-04-20 20:30:14.261 SingleUserLabApp application:731] Looking for jupyter_config in /home/jovyan/.jupyter
    [D 2022-04-20 20:30:14.262 SingleUserLabApp application:731] Looking for jupyter_config in /home/jovyan
    [D 2022-04-20 20:30:14.263 SingleUserLabApp application:731] Looking for jupyter_server_config in /etc/jupyter
    [D 2022-04-20 20:30:14.263 SingleUserLabApp application:753] Loaded config file: /etc/jupyter/jupyter_server_config.py
    [D 2022-04-20 20:30:14.264 SingleUserLabApp application:731] Looking for jupyter_server_config in /usr/local/etc/jupyter
    [D 2022-04-20 20:30:14.264 SingleUserLabApp application:731] Looking for jupyter_server_config in /opt/conda/etc/jupyter
    [D 2022-04-20 20:30:14.264 SingleUserLabApp application:731] Looking for jupyter_server_config in /home/jovyan/.local/etc/jupyter
    [D 2022-04-20 20:30:14.264 SingleUserLabApp application:731] Looking for jupyter_server_config in /home/jovyan/.jupyter
    [D 2022-04-20 20:30:14.264 SingleUserLabApp application:731] Looking for jupyter_server_config in /home/jovyan
    [W 2022-04-20 20:30:14.266 SingleUserLabApp configurable:193] Config option `open_browser` not recognized by `SingleUserLabApp`.  Did you mean `browser`?
    [D 2022-04-20 20:30:14.268 SingleUserLabApp config_manager:97] Paths used for configuration of jupyter_server_config: 
            /etc/jupyter/jupyter_server_config.json
    [D 2022-04-20 20:30:14.268 SingleUserLabApp config_manager:97] Paths used for configuration of jupyter_server_config: 
            /usr/local/etc/jupyter/jupyter_server_config.json
    [D 2022-04-20 20:30:14.269 SingleUserLabApp config_manager:97] Paths used for configuration of jupyter_server_config: 
            /opt/conda/etc/jupyter/jupyter_server_config.d/jupyterlab.json
            /opt/conda/etc/jupyter/jupyter_server_config.d/nbclassic.json
            /opt/conda/etc/jupyter/jupyter_server_config.json
    [D 2022-04-20 20:30:14.270 SingleUserLabApp config_manager:97] Paths used for configuration of jupyter_server_config: 
            /home/jovyan/.local/etc/jupyter/jupyter_server_config.json
    [D 2022-04-20 20:30:14.270 SingleUserLabApp config_manager:97] Paths used for configuration of jupyter_server_config: 
            /home/jovyan/.jupyter/jupyter_server_config.json
    [W 2022-04-20 20:30:14.292 SingleUserLabApp configurable:193] Config option `open_browser` not recognized by `SingleUserLabApp`.  Did you mean `browser`?
    [I 2022-04-20 20:30:14.292 SingleUserLabApp manager:345] jupyterlab | extension was successfully linked.
    [W 2022-04-20 20:30:14.297 NotebookApp] 'ip' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
    [W 2022-04-20 20:30:14.298 NotebookApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
    [W 2022-04-20 20:30:14.298 NotebookApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
    [D 2022-04-20 20:30:14.306 NotebookApp] Config changed: {'NotebookApp': {'open_browser': False}, 'ServerApp': {'ip': '0.0.0.0', 'port': 8888, 'open_browser': False, 'jpserver_extensions': <LazyConfigValue value={'jupyterlab': True, 'nbclassic': True}>}, 'FileContentsManager': {'delete_to_trash': False}, 'Application': {'log_level': 'DEBUG'}, 'KernelManager': {'shutdown_wait_time': 5.0}, 'MappingKernelManager': {'cull_connected': True, 'cull_idle_timeout': 300, 'cull_interval': 240, 'kernel_info_timeout': 60}, 'TerminalManager': {'cull_inactive_timeout': 360, 'cull_interval': 300}, 'SingleUserLabApp': {'ip': '0.0.0.0'}}
    [W 2022-04-20 20:30:14.308 SingleUserLabApp configurable:193] Config option `open_browser` not recognized by `SingleUserLabApp`.  Did you mean `browser`?
    [D 2022-04-20 20:30:14.562 SingleUserLabApp config_manager:97] Paths used for configuration of jupyter_notebook_config: 
            /home/jovyan/.jupyter/jupyter_notebook_config.json
    [D 2022-04-20 20:30:14.563 SingleUserLabApp config_manager:97] Paths used for configuration of jupyter_notebook_config: 
            /etc/jupyter/jupyter_notebook_config.json
    [D 2022-04-20 20:30:14.564 SingleUserLabApp config_manager:97] Paths used for configuration of jupyter_notebook_config: 
            /usr/local/etc/jupyter/jupyter_notebook_config.json
    [D 2022-04-20 20:30:14.564 SingleUserLabApp config_manager:97] Paths used for configuration of jupyter_notebook_config: 
            /opt/conda/etc/jupyter/jupyter_notebook_config.d/jupyterlab.json
            /opt/conda/etc/jupyter/jupyter_notebook_config.json
    [D 2022-04-20 20:30:14.565 SingleUserLabApp config_manager:97] Paths used for configuration of jupyter_notebook_config: 
            /home/jovyan/.local/etc/jupyter/jupyter_notebook_config.json
    [D 2022-04-20 20:30:14.566 SingleUserLabApp config_manager:97] Paths used for configuration of jupyter_notebook_config: 
            /home/jovyan/.jupyter/jupyter_notebook_config.json
    [I 2022-04-20 20:30:14.566 SingleUserLabApp manager:345] nbclassic | extension was successfully linked.
    [I 2022-04-20 20:30:14.608 SingleUserLabApp manager:367] nbclassic | extension was successfully loaded.
    [I 2022-04-20 20:30:14.609 LabApp] JupyterLab extension loaded from /opt/conda/lib/python3.9/site-packages/jupyterlab
    [I 2022-04-20 20:30:14.609 LabApp] JupyterLab application directory is /opt/conda/share/jupyter/lab
    [I 2022-04-20 20:30:14.615 SingleUserLabApp manager:367] jupyterlab | extension was successfully loaded.
    [I 2022-04-20 20:30:14.615 SingleUserLabApp mixins:640] Starting jupyterhub-singleuser server version 2.2.0
    [W 2022-04-20 20:30:14.631 SingleUserLabApp _version:68] jupyterhub version 2.3.0.dev != jupyterhub-singleuser version 2.2.0. This could cause failure to authenticate and result in redirect loops!
    [I 2022-04-20 20:30:14.632 SingleUserLabApp serverapp:2582] Serving notebooks from local directory: /home/jovyan
    [I 2022-04-20 20:30:14.632 SingleUserLabApp serverapp:2582] Jupyter Server 1.13.1 is running at:
    [I 2022-04-20 20:30:14.632 SingleUserLabApp serverapp:2582] http://7cc4352a46e6:8888/user/ranji%20raj/lab
    [I 2022-04-20 20:30:14.632 SingleUserLabApp serverapp:2582]  or http://127.0.0.1:8888/user/ranji%20raj/lab
    [I 2022-04-20 20:30:14.632 SingleUserLabApp serverapp:2583] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
    [I 2022-04-20 20:30:14.638 SingleUserLabApp mixins:596] Updating Hub with activity every 300 seconds
    [D 20
    Kevin Bates
    @kevin-bates

    Hi @ranjiGT, these statements are too early in the logs to determine if culling is enabled since debug statements won’t be produced until a kernel has been started. Here are some debug statements I just created using the following command line to start Lab:

    jupyter lab --MappingKernelManager.cull_interval=10 --MappingKernelManager.cull_idle_timeout=30 —debug

    In this case, I’m checking every 10 seconds for a kernel that’s been idle (and not connected, the default) for 30 seconds. Once the kernel has started, you should see periodic statements like the following:

    [D 2022-04-21 07:02:04.672 ServerApp] Polling every 10 seconds for kernels idle > 30 seconds...
    [D 2022-04-21 07:02:04.672 ServerApp] kernel_id=d04f4258-db1a-4a6c-94b0-d30d0941e585, kernel_name=python3, last_activity=2022-04-21 14:01:56.630070+00:00

    followed by the actual culling when all criteria has been met. In this case an extra interval took place because my kernel was still connected after the initial 30 seconds. Once I closed the tab in Lab, the following was produced:

    [W 2022-04-21 07:02:44.673 ServerApp] Culling 'idle' kernel 'python3' (d04f4258-db1a-4a6c-94b0-d30d0941e585) with 0 connections due to 48 seconds of inactivity.
    [D 2022-04-21 07:02:44.674 ServerApp] Clearing buffer for d04f4258-db1a-4a6c-94b0-d30d0941e585
    [I 2022-04-21 07:02:44.674 ServerApp] Kernel shutdown: d04f4258-db1a-4a6c-94b0-d30d0941e585
    3 replies
    Ranji Raj
    @ranjiGT
    Screenshot 2022-04-20 at 22.52.15.png
    Ranji Raj
    @ranjiGT
    Hi, I have investigated some logs from my jupyter server and got the following as above. It shows as ServerAppbut cannot be completely traced back to the file where the server loaded the config file. Any assistance would be helpful.
    Kevin Bates
    @kevin-bates
    Hi @ranjiGT, The fact that its showing Jupyter Server 1.13.1 means that, at the base level, Jupyter Server is running. However, Jupyter Hub uses a subclass of ServerApp, presumably named SingleUserLabApp. I would imagine having c.MappingKernelManager….configured in any of the files associated with the class hierarchy would be sufficient.
    While checking on SingleUserLabApp, I ran across this post (solution) regarding how to configure which Application class to use - and I suspect this is your issue. If you still have issues, I would suggest adding a post to the discourse forum (previously linked).
    1 reply
    David Leen
    @dleen

    I want to use something like aiohttp in a server extension. The aiohttp docs warn to not use a “session” per request. I can create the session object in the init of my extension handler, but I also need to call await session.close() on server shutdown to wait for outstanding requests.

    Is there a shutdown hook that I can do this in? I vaguely remember seeing a similar question a while ago but I can’t find it now...

    Kevin Bates
    @kevin-bates
    Hi @dleen - I think implementing stop_extension()jupyter-server/jupyter_server#526 may be what you want.
    David Leen
    @dleen
    Nice! That looks very promising!
    Matt Henderson
    @mlhenderson
    I'm looking to track down where and when the "{resource_dir}" kernel.json template variable gets filled out. When I call the server api via REST endpoint to request the kernelspecs, that template variable is not filled out. Looking for some pointers on tracking that down.
    Angus Hollands
    @agoose77:matrix.org
    [m]

    @mlhenderson: could you elaborate on what you're trying to do?

    I looked into where this comes from - Jupyter Server's REST API ultimately builds the JSON response for /api/kernelspecs, but loads their information from the Jupyter Client library. Jupyter Client does have a resource_dir attribute in the kernelspec, so I assume Jupyter Server is dropping it.

    2 replies
    Matt Henderson
    @mlhenderson
    I am launching another process that needs to have the same environment as the kernel, and the kernel does not have the same environment as the notebook server
    jupyter_client does not seem to have resource_dir, at least calling it from the kernel didn't get me anywhere
    I looked at the jupyter_server and jupyter_client code where resource_dir is mentioned, but have not found the answer I am looking for, which is why I am checking here as I dig into more jupyter repos to see what I may be missing
    Kevin Bates
    @kevin-bates

    Hi Matt, Angus is correct in that the REST API uses resource_dir to acquire references to the kernel "resources" (essentially icon file names that get fetched in a different request) and doesn't include the directory in its response

    launching another process that needs to have the same environment as the kernel

    By "same environment", do you mean the same set of environment variables specified in the kernel.json file's env stanza or other portions of the kernel process env that are inherited from the launching server besides the overrides in the env stanza?

    What will be launching the other process and does it know the name of the "sibling" kernel?

    If you wanted a very tight coupling between the kernel and your launched process relative to the kernel's lifecycle, one approach you could take is to subclsss the LocalProvisioner and essentially maintain a mirror of the custom process in that class, while the super class manages the local kernel. This way, you could ensure the two have precisely the same envs with no dependency on where the kernel.json file resides (although you'd have access to that information as well).

    Matt Henderson
    @mlhenderson

    The "other" process or processes (because it could be many) would be launched ad hoc and not at kernel instantiation time, and are not necessarily tied to the kernel's lifecycle (long-running remote tasks could continue to run even if the kernel is not running). It would also be cumbersome to convert a bunch of existing kernelspecs for users to use an alternative provisioner for any kernel where they may want to launch something remotely from Jupyter.

    A service (may not always be the same service) would be called from the kernel to launch a remote process based on a user's need for a particular resource or task. One of the parameters for the service call would be the how of launching it (container, custom conda env, script, etc). The underlying runtime environment (I'm being more general than just the shell when I use the term) for that process should match (not for every case but many) that of the kernel so that libraries and dependencies are compatible.

    Users can have some fairly complicated environment setups that can't easily be packed directly into the kernel.json, so it's possible to use a secondary resource to launch the kernel, like a script, which is convenient to keep in the same directory with the kernel. If I can determine the kernel spec and the path to the spec, then I can find the launch script and reuse it to launch something else, assuming that the script is written to accept a set of arguments such as the argv list in the kernel.json.

    I think I was not expecting that the server would not have a way to provide the templated resource_dir, which seems to be the piece that I am missing. The kernel.json spec is still useful, but not quite enough information for the case where there is a launch script in use that is co-located with the kernel.json and '{resource_dir}' is used instead of an absolute path.

    Kevin Bates
    @kevin-bates
    Got it.
    To associate a resource dir to a kernel, you’ll need the kernel name (or it’s display name) and can then enumerate the available kernel specs (and their resource dirs) using the KernelSpecManager pyhon API and locate the desired spec matching the kernel name (preferred) or looking in each spec to match its display name. An example of this can be found in the MainKernelSpecHandler in the server.
    Berin Aniesh
    @berinaniesh
    Hello guys, I did some calculations (t tests) on data from an excel file in python and jupyter notebook.
    I have the answers I need (t value, p value, mean etc) in python as variables.
    I am trying to create a table using markdown in the same jupyter notebook itself.
    How do I print those variables inside markdown?
    My question makes sense? Should I reword it?
    Angus Hollands
    @agoose77:matrix.org
    [m]
    @berinaniesh: you could manually format a Markdown string and display it with display_markdown from IPython.display, but I think it might be easier to put it into a pandas table, which has native rich rendering
    Berin Aniesh
    @berinaniesh
    alright, thanks for the answer @agoose77:matrix.org
    Aaron Richter
    @rikturr
    Hi all! Anyone know of a jupyterlab extension that has a window or side pane for a list of database tables? Like what you’d see in a typical db GUI. This extension: https://github.com/pbugnion/jupyterlab-sql had a widget for it but looks like it hasn't been updated since lab 1.x. If not, I'll be looking to build one so any tips are appreciated!
    1 reply
    Ranji Raj
    @ranjiGT

    Hello everyone,

    From my docker logs, I find that there are two locations from where jupyter_server_config.py is being loaded. I manually checked the location \etc\jupyter and there appears to be a file. But it does not load the file from \home\jovyan\.jupyter even though I have a file present there namedjupyter_server_config.py

    [D 2022-04-29 11:13:38.118 SingleUserLabApp application:731] Looking for jupyter_config in /etc/jupyter
    [D 2022-04-29 11:13:38.118 SingleUserLabApp application:731] Looking for jupyter_config in /usr/local/etc/jupyter
    [D 2022-04-29 11:13:38.118 SingleUserLabApp application:731] Looking for jupyter_config in /opt/conda/etc/jupyter
    [D 2022-04-29 11:13:38.119 SingleUserLabApp application:731] Looking for jupyter_config in /home/jovyan/.local/etc/jupyter
    [D 2022-04-29 11:13:38.119 SingleUserLabApp application:731] Looking for jupyter_config in /home/jovyan/.jupyter
    [D 2022-04-29 11:13:38.119 SingleUserLabApp application:731] Looking for jupyter_config in /home/jovyan
    [D 2022-04-29 11:13:38.120 SingleUserLabApp application:731] Looking for jupyter_server_config in /etc/jupyter
    [D 2022-04-29 11:13:38.121 SingleUserLabApp application:753] Loaded config file: /etc/jupyter/jupyter_server_config.py
    [D 2022-04-29 11:13:38.122 SingleUserLabApp application:731] Looking for jupyter_server_config in /usr/local/etc/jupyter
    [D 2022-04-29 11:13:38.122 SingleUserLabApp application:731] Looking for jupyter_server_config in /opt/conda/etc/jupyter
    [D 2022-04-29 11:13:38.122 SingleUserLabApp application:731] Looking for jupyter_server_config in /home/jovyan/.local/etc/jupyter
    [D 2022-04-29 11:13:38.122 SingleUserLabApp application:731] Looking for jupyter_server_config in /home/jovyan/.jupyter
    [D 2022-04-29 11:13:38.123 SingleUserLabApp application:731] Looking for jupyter_server_config in /home/jovyan
    Kevin Bates
    @kevin-bates
    Since this is a container, might /home/jovyan be a mount point that may be overriding the file that’s persisted in the image? Or vice versa - the configuration file is meant to be “brought” via a mount and the mount isn’t configured correctly? If not done already, and assuming you have access, try attaching to the container and confirm the file's existence (and content). Check permissions and things like that (especially if mounted) since the traitlets package (that’s loading these files) appears to convert OSError to an applicaion-specific file not found error that is then silently ignored.
    You might also try adding the contents of /home/jovyan/.jupyter/jupyter_server_config.py to the file in /etc/jupyter just as a data point (and to regain some sanity as this can be pretty frustrating).
    1 reply
    Dennis Reinsch
    @dreinsch:uni-bielefeld.de
    [m]
    Hi everyone,
    We used os.write(1, "...".encode()) to write some logging messages to the terminal of jupyter to enable the user to check the terminal window for this extra output if needed. However on macOS the output of os.write started to appear in the output cell of notebook as well as in the terminal.
    This now clutters the output cell.
    I hope you can provide me some pointers how I can fix this issue.
    Zelphir Kaltstahl
    @ZelphirKaltstahl

    Hi! I have a question regarding Jupyter's messaging and kernel messages. I have a Python program, which receives messages from the IOPub socket. I am using the ZeroMQ poller to poll for events of the socket:

    poller = zmq.Poller()
    socket = channel.socket
    poller.register(socket, zmq.POLLIN)
    events = dict(poller.poll(remaining_time_ms))

    Then, when there are events, I use an asynchronous kernel client to get messages:

    # with: iopub_channel = async_kernel_client.iopub_channel
    msg = await iopub_channel.get_msg(timeout=0)

    That runs in a loop to get all available messages and store them in a list, until the channel has no more messages. Then I proceed processing the messages. Then I run an outer loop again to continue polling, in case the execute_result or error message was not yet received.

    (This is not the exact code, as I cannot share the whole code, but just to make clear, what I am talking about.)

    Now I have an issue, where I am not getting all expected messages from the IOPub channel. For example when I execute code programmatically using kernel_client.execute I would expect some typical cycle of message types:

    busy, execute_input, execute_result, idle

    However, sometimes (not always!) I am only receiving busy and idle, without the messages in between, which is a problem, because I need to see execute_result or, if any, an error message. I deal with error, so error is not a problem on the meta layer, but no message is a problem.

    In which scenarios is it expected, that neither execute_result nor error is published on IOPub? Is this even known at all to be an issue?

    I am slowly beginning to doubt the kernel or ZeroMQ, because I have looked at this problem for many hours and cannot find anything I am doing wrong, when retrieving the messages. Any ideas what could be wrong? Could there be any possibility of "missing messages", because I do things in between polling the socket? Shouldn't the channel still get all the messages?

    Zelphir Kaltstahl
    @ZelphirKaltstahl
    Any ideas anyone?
    10 replies
    pepijndevos
    @pepijndevos:matrix.org
    [m]
    Can you install a server extension without restarting?
    1 reply
    pepijndevos
    @pepijndevos:matrix.org
    [m]
    right... I have access to a JupyterHub but it seems every time I reload it it discards all the packages I installed so I can't load any extensions
    Min RK
    @minrk
    That's up to how the Hub persists directories. You can contact your hub admin if it's unclear what persists. You may want to do --user installs of the extensions, if your home directory is persisted.
    minrk
    @minrk:matrix.org
    [m]
    Ah, is it possible the subsequent requests are getting aborted due to the error? There’s a stop_on_error behavior that cancels pending requests if an execution hits an error
    E.g. run all in the notebook stops execution on an error. This is handled by the kernel
    Nicholas Bollweg
    @bollwyvl
    I think some of the recently flit-migrated packages will be see breakage with pip 22.1: pypa/pip#11110
    Rogier
    @sluijs
    Hi all. I'm trying to speed up my ipywidget. Currently, it has to transfer NumPy arrays (500x500x500) from the kernel to the front-end. I'm using ipydatawidgets to convert the array to JSON and deserialization happens in the front-end. Is there a way to send the array to the front-end as bytes, for example using WebSockets?
    Angus Hollands
    @agoose77:matrix.org
    [m]
    @sluijs: I'm just going off of the source code, but it looks like ipydatawidgets should already do this - sending byte-for-byte (apart from compression). From what I can see, the array is never string-encoded as JSON. It's not really an accurate naming from what I can see - to_dict might be a better one.
    5 replies
    Ihshan Gumilar
    @ihgumilar
    Hi @everyone, I use jupyterlab (running in remote server) in VS Code (in my local computer) for running some codes, but when I pressed Shift + Tab, it does not show up a documentation. Can anyone help me please ? Thanks
    Angus Hollands
    @agoose77:matrix.org
    [m]
    @ihgumilar: the better room for this is #jupyterlab_jupyterlab:gitter.im 🙂
    1 reply
    Naman Vats
    @namanvats
    Hi, @everyone I am new to this community, can anyone suggest where, to begin with for jupyterlab contribution
    1 reply
    Zelphir Kaltstahl
    @ZelphirKaltstahl

    Continuation of: https://gitter.im/jupyter/jupyter?at=626fff91fa846847c9474429

    I am still trying to figure out this issue and I can share more code now to get input about what might be wrong, or whether there is really any bug. I can share the code, that interacts with the kernel. Since it is a bit long, I made it a paste: https://paste.debian.net/1241962/

    Do you see any mistakes in interacting with the kernel, which could lead to missing execution result and execution input, only giving me busy and next an idle message?

    (Pinging @minrk because of past context.)

    Angus Hollands
    @agoose77:matrix.org
    [m]
    @ZelphirKaltstahl: have you tried using something like https://github.com/jupyterlab-contrib/jupyterlab-kernelspy to confirm what the expected behaviour is?
    2 replies
    Frédéric Collonval
    @fcollonval
    @choldgraf and @Carreau I'm writing the CZI EOSS5 final proposal for JupyterLab project. Could you please add the Papyri and the JHub community strategic support grants to the spreadsheet: https://docs.google.com/spreadsheets/d/1kP1ggTmBji7eGlXYakBfRXKS_I7hjFxKR-gfkh-vO50 ? So I can present accurate current support received by Jupyter project (I'm mainly missing the amount of money you received for those CZI grants).
    Matthias Bussonnier
    @Carreau
    Papyri added, thanks.
    Frédéric Collonval
    @fcollonval
    Thanks Matthias
    Matthias Bussonnier
    @Carreau
    No, thanks to you.