Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • May 15 14:45
    echarles commented #3398
  • May 14 15:47
    welcome[bot] commented #3469
  • May 14 15:47
    satra opened #3469
  • May 14 15:47
    satra labeled #3469
  • May 14 14:49
    vpavlin commented #2779
  • May 14 14:22
    welcome[bot] commented #3468
  • May 14 14:22
    vickumar1981 labeled #3468
  • May 14 14:22
    vickumar1981 opened #3468
  • May 14 13:43
    Siecje commented #781
  • May 14 09:59
    minrk commented #3460
  • May 14 00:18
    jsuag commented #781
  • May 13 13:30
    minrk commented #3466
  • May 12 22:17
    support[bot] closed #3465
  • May 12 22:17
    support[bot] commented #3465
  • May 12 22:17
    manics labeled #3465
  • May 12 22:17
    manics unlabeled #3465
  • May 12 22:17
    manics commented #3465
  • May 12 15:17

    minrk on 1.4.1

    (compare)

  • May 12 15:17
    minrk commented #3467
  • May 12 15:17

    minrk on 1.4.x

    Backport PR #3436: ci: github w… Backport PR #3452: Fix document… Backport PR #3437: patch base h… and 7 more (compare)

rajeshK88
@rajeshK88
Currently I am using Generic authenticator .. but I want Jypyter hub to check the token once oauth access token expires..
minrk
@minrk:matrix.org
[m]
Then you probably want to subclass and define refresh_user as appropriate.
I don’t believe any Oauthenticator class defines refresh_user yet, though most probably should.
rajeshK88
@rajeshK88
gotcha and still have to mention about oauth_token_expires_in in config file right ?
minrk
@minrk:matrix.org
[m]
I wouldn’t think so. What do you want to use it for?
(Issue for oauthenticators implementing refresh_user: jupyterhub/oauthenticator#398 )
rajeshK88
@rajeshK88
my requirement is to check the user periodically after access_token expires..
minrk
@minrk:matrix.org
[m]
oauth_token_expires_in governs how often the single-user server must re-auth with the Hub, so you could set that that could be your access token expiry
rajeshK88
@rajeshK88
got it.. Let me try the setup..
Min RK
@minrk
Yeah, it's confusing, but there are two wholly separate oauth setups here. One for the notebook server authenticating with the Hub, and a second one for the Hub authenticating with KeyCloak.
Expiry, etc. are configured separately
I started working on a doc covering the various steps in jupyterhub/jupyterhub#3444
rajeshK88
@rajeshK88
cool.. this helps.. let me go through that.. Thanks for sharing..
Min RK
@minrk
rajeshK88
@rajeshK88
Perfect thanks .. neat !
rajeshK88
@rajeshK88
will try both both configuration like oauth_token_expires_in and auth_refresh_age .. will post here my observation.. We can document this later..
rajeshK88
@rajeshK88
tried the approach.. looks like I am getting None for auth_state = await user.get_auth_state()
any idea why ?
rajeshK88
@rajeshK88
never mind figure out.. need to enable auth state in config..
c.Authenticator.enable_auth_state = True
c.CryptKeeper.keys = [b'some-secret-key'] # needed for encryption
Matt Riedemann
@mriedem
We have over 5300 users in a jupyterhub DB right now which is causing cull-idle to timeout on the GET /users request but jupyterhub-idle-culler uses tornado which defaults request_timeout to 20 seconds and the API is taking about 50 seconds in some cases. Is there an easier way to set the request timeout via environment variable or are we stuck with setting max-time = 60 in $HOME/.curlrc?
7 replies
Matthew Brett
@matthew-brett

My z2jh GKE cluster stalled badly under fairly mild load today, giving "serviice refused" errors. The autohttps pod restarted many times. Last -p log on the secret-sync container has:

2021-05-10 09:29:19,247 INFO /usr/local/bin/acme-secret-sync.py watch-save --label=app=jupyterhub --label=release=jhub --label=chart=jupyterhub-0.11.1 --label=heritage=secret-sync proxy-public-tls-acme acme.json /etc/acme/acme.json
2021-05-10 09:30:24,876 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7ff5883de2b0>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/v1/namespaces/jhub/secrets/proxy-public-tls-acme

Any ideas where I could look for the problem?

Erik Sundell
@consideRatio
Hmmm, the k8s apiserver was not responding?
What version of traefik is used?
Why did the pod restart?
(kubectl logs --previous)
Matthew Brett
@matthew-brett
Sorry - that output was from a call to kubectl logs --previous autohttps-9fdcfc86c-9jdwx secret-sync - the other containers in that pod give e.g. previous terminated container "traefik" not found. Traefik image reported as traefik:v2.3.7. Is there any other way I can diagnose the pod restart?
Erik Sundell
@consideRatio
Does pods restart, or containers? Check the containerststuses - perhaps only secretsync container restarted, and perhaps that made the pod not be considered ready for a while perhaps (if it would be totally fine if it was unavailable for a long time btw - it shouldnt habe a readinessprobe)
Matthew Brett
@matthew-brett
Aha - yes - how helpful: kubectl describe pods autohttps-9fdcfc86c-9jdwx shows that, of the two containers: traefik and secret-sync, secret-sync has restarted 42 times, traefik 0 times. I didn't look specifically, but when the service went down, it seemed to me that there were many new restarts in the autohttps pod - and therefore, I now see, the secret-sync container. Is there anything I can or should do to mitigate?
Erik Sundell
@consideRatio
Can you report this in z2jh repo? Note that it may not be a problem that the container restarts in practice.
It is a problem if the pod isnt ready during that process though
Then no network traffic is accepted
But the container is only relevant on startup
Matthew Brett
@matthew-brett
Thanks @consideRatio for the suggestions.
Dan Barr
@danbarr
Question about Extension Manager - I'm gathering that since JupyterLab runs as a non-privileged user in a JupyterHub environment, it should be expected that Extension Manager doesn't work? To the user, it says "Error communicating with server extension" with "TypeError: Failed to fetch". And even if it did work, extensions installed this way wouldn't persist between notebook pod starts?
4 replies
alibama
@alibama
how do i install libraries for every user on jupyterhub if i'm using a VariableLocalProcessSpawner
i want to have a baseline pandas and stuff like that for everyone - my google fu is weak this morning ... i have tested installing via conda in the base profile, as well as in the python profile... i thought and restarted jupyterhub services... i can install it per user, just trying to onboard folks with a baseline
Angus Hollands
@agoose77:matrix.org
[m]
@alibama: you are using the UNIX users to authenticate right?
2 replies
alibama
@alibama
@agoose77:matrix.org well - we're using github but they have local accounts?
3 replies
Nick Levandoski
@metric-chicken
I've run into a potential problem and maybe someone can help. How do I handle user permissions with shared storage resources (i.e. shared drives). I have ztjh up and running. I'm mounting individual directories using NFS. I'm also mounting a shared volume using NFS. How would/could I set the owner of files to be $JUPYTERHUB_USER?
Erik Sundell
@consideRatio

@metric-chicken it is a bit tiresome, but you probably need to use a initContainer that chown the folder of relevance before the normal user-server container mounts it.

https://github.com/jupyterhub/zero-to-jupyterhub-k8s/issues/421#issuecomment-725429444

Nick Levandoski
@metric-chicken
@consideRatio I can probably make that work. Is there away to change the base user to something else? For instance, to something like user 'foo'? I'm migrating from a docker installation and I'm trying to keep the current user permissions working in the shared directory.
Erik Sundell
@consideRatio
The base user, as is the user that the container starts as? Yes singleuser.uid. The base user that the filesystem mounts as, when using NFS, currently: no. The base user that ends up running the container, yes - by starting as root, using a docker-stacks based image, and setting NB_USERID environment variable or something like that and it will during startup scripts, (which you may need to manually specify), could get it done.
Nick Levandoski
@metric-chicken
Im not familiar with docker-stacks but I think i'm following. Is 'singleiser.uid' a value passed to the helm chart? I'm using a custom spawner, I might be able to send the user name/uid from their.
Nick Levandoski
@metric-chicken
wait, the singleuser question was a dumb question. its part of the values you pass for the helm chart.
Nick Levandoski
@metric-chicken
What can potentially go wrong if the hub-bd-dir is deleted?
Erik Sundell
@consideRatio
Users manually added in some way are reset. I think it will be fine in many setups.
Nick Levandoski
@metric-chicken
Two things, first, it looks like this page moved https://github.com/berkeley-dsep-infra/datahub/blob/21e4a45c9f694578ec297c2947a0537f3bdcaa5b/deployments/data100/config/common.yaml#L65 does anyone have the new address? @yuvipanda I think you sent that to me a while back. Second, all the packages the users install are wiped out when the user's pod is restarted. How do I make them persist?
Nick Levandoski
@metric-chicken
If I create a user env from a notebook it shows up in the list of kernels. However, if I stop and then start the server, the kernel is not there. I am using 'python -m ipykernel install --user --name myenv --display-name "Python (myenv)"' to create the env. If I run the same command from a terminal, the kernel persists after I restart the server but all of the packages I installed are missing. I have added the "envs_dirs:" to the .condarc and point it to a directory in the users dir on an nfs share.
Any ideas?