by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • May 25 01:23
    dgergel commented #778
  • May 25 01:22
    dgergel commented #778
  • May 25 01:20
    dgergel commented #778
  • May 23 19:52
    stale[bot] unlabeled #764
  • May 23 19:52
    aaronspring commented #764
  • May 22 20:17
    rabernat commented #778
  • May 22 19:31
    drorspei commented #272
  • May 22 19:24
    TomAugspurger commented #272
  • May 22 19:19
    drorspei commented #272
  • May 22 18:54
    mrocklin commented #778
  • May 22 18:26
    stale[bot] labeled #764
  • May 22 18:26
    stale[bot] commented #764
  • May 22 18:26
    stale[bot] labeled #769
  • May 22 18:26
    stale[bot] commented #769
  • May 22 17:17
    jbusecke commented #778
  • May 20 17:01
    dcherian commented #778
  • May 20 15:59
    auraoupa commented #778
  • May 20 07:30
    apatlpo commented #778
  • May 20 04:43
    stale[bot] closed #765
  • May 20 04:43
    stale[bot] commented #765
Ryan Abernathey
@rabernat
it would be good to have some monitoring
Scott
@scollis
Hey Folks.. Thanks to some help from @jhamman I am up and running. I was running into an issue that I did not get the nice views on the panels when I was running Dask.. (profiler et al).. So I ran the example Pangeo notebooks using the binder link. http://binder.pangeo.io/v2/gh/pangeo-data/pangeo-example-notebooks/master and I get the same issue that I never get nice output..even when I click the dask icon in the right bar I can not see what workers are doing.. Could there be a network blocking issue (some stuff is blocked here at Argonne)?
is blank
Wonder if it is a Bokeh issue...
Fabien Maussion
@fmaussion
Hi Pangeo folks: how often do you actualize the base image on hub.pangeo.io (or do you even do that?)
What are the consequences for the users? I expect these to be minimal, i.e. a user's HOME is completely decoupled from the base image
but still, what real bad could happen?
Scott
@scollis
So I tried on my home computer but I still get blank panels.. Just curious to see if this on only me (browser: Chrome, OS: MacOS)
Tom Augspurger
@TomAugspurger

@scollis just to verify, are you creating a Client object?

And are the panels blank, or do they have the Dask logo?

Eduardo Gonzalez
@eddienko
@scollis same here -- panels are blank, the Dask status pages contain only the top navbar
Scott
@scollis
Thanks @eddienko
@TomAugspurger Blank with a tab with the dask logo and the name, eg “Dask Task Stream”
Opening new panels does not help
This happens on your pangeo-notebooks example and my own “roll your own"
And @TomAugspurger I am running the examples in the pangeo-example-notebook repo
Tested now on Chrome and Safari
Jim Crist-Harif
@jcrist
@scollis , I've seen this behavior with tornado 6 and jupyter-server-proxy (the proxy used to proxy the dashboard through the user's notebook session). In the user's notebook environment, you need tornado 5 installed.
Scott
@scollis
Thanks.. will play with the environment.yml file
Joe Hamman
@jhamman
@rabernat, @rsignell-usgs, @tjcrone - can I merge this: pangeo-data/pangeo-cloud-federation#362
Scott Henderson
@scottyhq
@jhamman - just left a comment
Scott Henderson
@scottyhq
@fmaussion - we aren't updating images on any fixed timeline. But separating ‘staging’ and ‘production’ hubs has been useful to test things out and minimize disruption (see https://github.com/pangeo-data/pangeo-cloud-federation). As you already noted, the user home directory is unaffected. We’ve hit a lot of snags over the last months w/ version compatibilty when trying to have the latest packages (e.g. jupyterlab1, dask + dask-labextension). see https://github.com/pangeo-data/pangeo-stacks.
Joe Hamman
@jhamman
@mrocklin - if you’re around, let’s get you sorted on Cheyenne.
Scott
@scollis
Hey @scottyhq Thanks for the idea of using the base docker container.. I will give that a go…. Still lots to learn for me!
Scott
@scollis
All this stuff is very dfferent to your quickstarts.. if you want, once the dust settles from my learning I can put some PRs together with some new quickstarts including updating the cookie cutter
Scott Henderson
@scottyhq
no problem @scollis! thanks for trying things out. PRs are more than welcome. especially w/ documentation. it’s a bit of a moving target right now as we’re constantly experimenting with things so it's hard to keep current.
Scott
@scollis
so the base docker.. does that have the dask and jupyterlab set ups? so no need to include that anymore in the binder dir environment.yml
ie the environment.yml should now only have domain specific packages?
Scott Henderson
@scottyhq
that’s the idea yes. because we’re using repo2docker you end up layering packages that are defined in the following places in order 1) https://github.com/jupyter/repo2docker/blob/master/repo2docker/buildpacks/conda/environment.yml 2) https://github.com/pangeo-data/pangeo-stacks/blob/master/base-notebook/binder/environment.yml and 3) your own environment.yml.
if you want different versions of these base packages, including those versions in your environment.yml should overwrite them
Scott
@scollis
Great
Scott Henderson
@scottyhq
https://github.com/scottyhq/pangeo-binder-test/tree/esip-tech-dive is a working current example (although I don’t automatically run the notebook to check for errors)
in the binder folder this line in Dockerfile is key FROM pangeo/base-notebook-onbuild:2019.08.08 . It fetches the dated image form https://github.com/pangeo-data/pangeo-stacks and then runs your environment.yml, postBuild, etc on top
Joe Hamman
@jhamman
@scottyhq - I think we’re good with pangeo-data/pangeo-cloud-federation#362. Please take a look around staging.hub.pangeo.io to see if you spot anything out of place.
Scott
@scollis
Thanks @scottyhq .. trying now. This is very nice
Scott
@scollis
@scottyhq assume it is the same for postBuild and other stuff in /binder/*
Scott Henderson
@scottyhq
that’s right. except unfortunately I don’t think we enabled the ‘start’ script.
Scott
@scollis
@scottyhq up and running nicely.. some issues with the dask integration into the jupyterlab panel space but stuff I can debug and iron our..
Radar Pangeo here we come!
Scott Henderson
@scottyhq
awesome! great news
Joe Hamman
@jhamman
@scottyhq - ideas on this error/warning from conda:
The following packages are causing the inconsistency:

  - conda-forge/linux-64::notebook==6.0.0=py37_0
  - conda-forge/linux-64::ipyleaflet==0.11.1=py37_1
  - conda-forge/noarch::geoviews==1.6.2=py_0
  - conda-forge/linux-64::widgetsnbextension==3.5.1=py37_0
  - conda-forge/noarch::ipywidgets==7.5.1=py_0
  - conda-forge/noarch::jupyter==1.0.0=py_2
  - conda-forge/noarch::nbconvert==5.5.0=py_0
  - conda-forge/noarch::hvplot==0.4.0=py_1
  - conda-forge/noarch::geoviews-core==1.6.2=py_0
  - conda-forge/noarch::nbgitpuller==0.7.0=py_0
  - conda-forge/linux-64::jupyterlab==1.0.4=py37_0
  - conda-forge/noarch::jupyter-server-proxy==1.1.0=py_0
  - conda-forge/noarch::jupyterlab_server==1.0.0=py_1
Matthew Rocklin
@mrocklin
@jhamman ah, sorry, I was on a flight
I'm also currently away from a full-sized USB port, maybe tomorrow though?
Joe Hamman
@jhamman
Yeah, sounds good. You’ll find me here tomorrow as well. :airplane:
Pier
@PhenoloBoy
@jhamman is it you that I've to contact to get the git-crypt symmetric key as described here? https://github.com/pangeo-data/pangeo-cloud-federation
Scott
@scollis
@jhamman is there a publication I can cite for Pangeo?
Joe Hamman
@jhamman
@scollis - nothing comprehensive just yet. However, we did publish the NSF proposal (https://figshare.com/articles/Pangeo_NSF_Earthcube_Proposal/5361094) and @guillaumeeb had this conference paper published:
Eynard-Bontemps, G., R Abernathey, J. Hamman, A. Ponte, W. Rath, 2019: The Pangeo Big Data Ecosystem and its use at CNES. In P. Soille, S. Loekken, and S. Albani, Proc. of the 2019 conference on Big Data from Space (BiDS’2019), 49-52. EUR 29660 EN, Publications Office of the European Union, Luxembourg. ISBN: 978-92-76-00034-1, doi:10.2760/848593.