ideonate/ContainDS

ContainDS Dashboards / Desktop - general support and ideas discussion

Sid Feygin

@sfwatergit

Removing the storage clause fixed this (although it seems dynamic storage would be useful in my case). However, something's still not right:

Sid Feygin

@sfwatergit

upon sshing into the node instance and inspecting the container w/ the dashboard, I'm finding that the volume with the dashboard notebook was not attached to the container

Sid Feygin

@sfwatergit

OK. So, after upgrading to cdashboards 0.9.3 (hub and singleuser) and adding dynamic storage back in, I can view my own dashboard, but as another user, I cannot view dashboards shared with me

Sid Feygin

@sfwatergit

removing https seems to fix this, but that's not really an option for us

Dan Lester

@danlester

@sfwatergit Thank for talking through all of this. It's great to hear you've (mostly) got everything working.

This last hurdle ('Auth form must be sent from auth page') is something that came up recently for someone else. Not on k8s, but behind a reverse proxy that was terminating SSL. The issue is here: https://github.com/ideonate/cdsdashboards/issues/22#issuecomment-669090980

In that case we are talking about nginx as the reverse proxy - for you it is just AWS load balancer, but it's likely the problem is the same.

It is a bug/feature in either JupyterHub's OAuth code or in the Configurable HTTP Proxy component, depending on how you look at it.

Unfortunately, the workaround suggested in the GitHub issue above won't work for you because in Zero2JH there is no easy way to specify a bespoke ConfigurableHTTPProxy.command setting to the ['configurable-http-proxy', '--no-x-forward'] which we need, as noted recently here: jupyterhub/zero-to-jupyterhub-k8s#1302

Dan Lester

@danlester

Fixing the z2jh issue 1302 would be ideal in the short term. Using an alternative proxy (Traefik instead of the standard Configurable Http Proxy is supposed to be supported soon) might make this go away: jupyterhub/zero-to-jupyterhub-k8s#1162. Or trying a different https method could work - e.g. getting z2jh to do the https termination.

But all of these have too many moving parts - I will look at building a workaround into the jupyterhub code used by ContainDS Dashboards. I am away this week but should be able to take a look at the start of next. In the meantime, I'm hoping you can continue evaluation with https off or moved to z2jh (if that solves the problem).

Dan Lester

@danlester

I have registered this as a new issue on GitHub so we can track it there: ideonate/cdsdashboards#26

Sid Feygin

@sfwatergit

Thanks Dan! I saw that GitHub post and was also puzzled as to how to achieve the same effect on AWS. Thanks for looking into this.

I will proceed with HTTPS in the meantime. Eventually, the goal is to embed ContainDS into our web portal; permitting our data scientists to develop dashboards for internal and external consumption.

Trying things out again, I seem to still have this issue with unbound PersistentVolumeClaims after replacing the suggested Dynamic Storage configuration settings (necessary for this to work, apparently):

    type: dynamic
    capacity: 10Gi
    dynamic:
      pvcNameTemplate: claim-{username}
      volumeNameTemplate: volume-{username}
      storageAccessModes: [ReadWriteMany]

which eventually times out

Sid Feygin

@sfwatergit

Also, getting an error with the default rshinydemo

although I've followed the directions as indicated

Sid Feygin

@sfwatergit

Note I resolved the issue w/ pvc above by manually deleting the unbound pvc via kubectl. I will play around w/ EFS to see if it works better.

Sid Feygin

@sfwatergit

EFS fixes issue w/ rshinydemo... perhaps instructions regarding storage classes were a bit unclear to me.

Dan Lester

@danlester

@sfwatergit Sorry not to see this earlier - will take a look

Which docker image are you using, and did you checkout the rshinydemo files from the git repo, or some other method? If you can show a screenshot of the tree that would be great.

Sid Feygin

@sfwatergit

I would need to change the pv type back to dynamic to recreate, which I'm a bit loathe to do at the moment

Sid Feygin

@sfwatergit

@danlester have been seeing some issues when trying to enable jupyterlab extensions. Please see following screenshot

Dan Lester

@danlester

No problem re PV type.

1 reply

For JupyterLab extension, this has been renamed on final publish to NPM. Please try:

jupyter labextension install @ideonate/jupyter-containds

Or you can search within the extension manager in JupyterLab. I'll update the docs.

Axel Larsson

@AxelTLarsson

Hey! I am testing out containDS on our company hub, it's very promising! I've had some minor issues, but solved them all so far. One thing that I'd like to do is to restrict non-technical users so that they can only launch already created voila dashboards, or at the very least remove their ability to launch a "normal server". I was playing around with DockerSpawner.allowed_images/image_whitelist, however, since it's the same image for both the normal server and the dashboard servers, just different commands, it won't do. Do you have any suggestions for how to proceed?

Dan Lester

@danlester

@AxelTLarsson thanks for your feedback! Yes, I think this is a good suggestion. The main question is how to convenient assign 'non-technical' versus 'technical' status.

Easiest (to start at least) would be an allow/block list to name dashboard creators

Better would be to add to JupyterHub Groups - but there is no UI for that at the moment so I would want to add one myself really

And then how to reshape the JupyterHub UI for these non-technical users so that, for example, ideally they don't see a 'My Server' or 'New Dashboard' button at all, and only get to see the list of dashboards.

Some of this functionality would sit in JupyterHub itself ideally, but doesn't really make sense to the core JupyterHub project...

Anyway, this is something I want to work on. All ideas and feedback welcome!

I have created an issue to track ideonate/cdsdashboards#37

Axel Larsson

@AxelTLarsson

I haven't found that much info on JupyterHub Groups, as far as I understand it though, there are two groups "admin", and non-admin. Or, rather, there is perhaps only one group "admin". Perhaps one could add a new group - "dashboard-users", whose users would only be allowed to list/launch already existing dashboards, and not a "normal" server. The UI would then reflect your group belonging, as it does with showing the "Admin" tab already.
However, for starters, I would be perfectly fine with some slightly hacky way to throw an error at the non-techies, if they disregard my instructions and actually click these "My Server".
What I'm thinking right now is to experiment with passing in the user name to the notebook server (Docker container), and then have a hook that checks a hard-coded allowed list of users, e.g. in /usr/local/bin/before-notebook.d, and if not allowed, refuses to start, somehow... It won't be pretty, but might do for now.

I am following the issue with great interest, and I definitely think it could be a very useful feature for many.

Dan Lester

@danlester

Actually 'admin' is just a flag that is assigned to each user (True/False). Groups are a fairly generic JupyterHub concept, but aren't really used anywhere... (other than in ContainDS to control access to Dashboards)

Without changing JupyterHub core, the UI change might be easy enough, but then ideally would also be reinforced with something like you've suggested if they do attempt to use the API to start a server anyway.

Sid Feygin

@sfwatergit

Groups would be essential for our use case as well, which I think is somewhat different from the above. Basically, we will have internal data scientists who are dashboard creators and external users (clients) who will be consuming the dashboards and not permitted to run code. Ideally, authorization to view dashboards could be assigned on a per-group basis and be well-integrated with our JupyterHub-compatible OAuth provider of choice (AWS Cognito). I can definitely make more progress towards this end once ideonate/cdsdashboards#37 is resolved and jupyterhub/jupyterhub#3133) is merged. Any other significant challenges to implementing this solution that would require changes to either containds or JupyterHub? Happy to help out if so.

Victor Tolpegin

@DarkmatterVale

I'd love to see groups integrated with dashboards as well. I've loved using the dashboards feature thus far, and I know that integrating groups would provide us additional value (both in dashboards, and in jupyter overall)

Dan Lester

@danlester

Thanks all. I think @sfwatergit and @AxelTLarsson would benefit from the same kind of approach, maybe needing some customization as well. @DarkmatterVale thank you for your encouragement too!

I will work on something based on JupyterHub groups that at least gives a reasonable UI and passes something to spawners or servers so they know if that user should be allowed to spawn or not.

Victor Tolpegin

@DarkmatterVale

Thank you! And thanks for all of your work building out the dashboards feature. It's awesome!

Dan Lester

@danlester

@DarkmatterVale Great to hear you like it!

Axel Larsson

@AxelTLarsson

Thank you so much for this project, it's going to improve my quality of life at work, immensely, I'm sure!
I was able to implement a hack to disallow "non-technical" users from launching anything but dashboard servers, by adding a script to /usr/local/bin/before-notebook.d/disallow-non-techies.sh like this:

#!/bin/bash

ALLOWED_NB_USERS=(AxelTLarsson)

function assert_allowed_user() {
    if [[ ! " ${ALLOWED_NB_USERS[@]} " =~ " ${GITHUB_USER} " ]]; then
        # whatever you want to do when array doesn't contain value
        echo "${GITHUB_USER} is not allowed to start a normal notebook server"
        exit 1
    fi
}

if [ -z ${GITHUB_USER+x} ]; then
    # If $GITHUB_USER is not set, we are attempting to start a voila (dashboard) server
    # => go ahead
    echo "Allowing dashboard start"
else
    # If GITHUB_USER is not set, we are starting a normal notebook server, check if that is ok
    assert_allowed_user
fi

It's not very pretty, and relies on the presence of $GITHUB_USER to determine if it's launching a normal server or a dashboard. So far, it seems to work alright, but I'm eagerly awaiting any development that will obviate this hack.

Dan Lester

@danlester

@AxelTLarsson thank you for your kind words, and for sharing this solution. I have made some interesting experimental commits to master if anyone wants to take a look.

Where communities thrive

People

Repo info

Activity