## Where communities thrive

• Join over 1.5M+ people
• Join over 100K+ communities
• Free without limits
##### Activity
Simon Sander
@sim-san
This happens on line 51 in the dockerfile
Lukas Masuch
@LukasMasuch
@sim-san Building full workspace image can take quite a while and might cause some build problems. If you just want to change a few things, the easiest way is to just inherit from the ml-workspace image as explained in the FAQs (https://github.com/ml-tooling/ml-workspace#faq) and do your changes. We will make it easier to configure/remove tools in the next versions. In the meantime, to remove tools from being started, you can overwrite the supervisor config (https://github.com/ml-tooling/ml-workspace/blob/develop/resources/config/supervisord.conf) and to do add/remove tools in the open-tool menu you can overwrite this file: https://github.com/ml-tooling/ml-workspace/blob/develop/resources/home/.workspace/tools/1-workspace-tools.json.
Simon Sander
@sim-san
Carlos Vivar Rios
@caviri
Thanks for the recommendation I think I'll go for rsync. I tried to run mlhub with --privileged flag but it seems this is not transfer to the servers created :s
Simon Sander
@sim-san
Hello,
Is it possible to change the parameters for the deflaut server ? For example: change "is_mount_volume" parameter.
Benjamin Räthlein
@raethlein
Hey Simon, do you mean the options in the MLHub's option page?
So just for understanding, you would like to be able to change the default settings so that, for example, is_mount_volume is unchecked by default?
Right now, you can only set the default images that are shown in the options page but no other options.
Simon Sander
@sim-san
ok, thanks for the fast answer
Simon Sander
@sim-san
At the moment I face a other Problem with ml-worksqace/ml-hub.
A user without admin rights on the ml-hub is not able to open the Visual Studio Code Web. I got the following errors in the log:
12:56:22.188 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/base/images/favicon.ico?v=845272bb369e7a52ef65139fafb905e3 socket hang up
12:56:22.195 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/services/contents.js?v=20191216125140 socket hang up
[I 2019-12-16 12:56:22.205 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fbase%2Fimages%2Ffavicon.ico%3Fv%3D845272bb369e7a52ef65139fafb905e3 (@172.17.0.2) 15.47ms
[I 2019-12-16 12:56:22.208 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fservices%2Fcontents.js%3Fv%3D20191216125140 (@172.17.0.2) 2.33ms
12:56:22.368 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/components/marked/lib/marked.js?v=20191216125140 socket hang up
12:56:22.369 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/components/codemirror/addon/runmode/runmode.js?v=20191216125140 socket hang up
2019/12/16 12:56:22 [crit] 738#0: *1804 SSL_write() failed (SSL:) while sending to client, client: 127.0.0.1, server: , request: "GET /user/test/test_vscode/static/base/js/security.js?v=20191216125140 HTTP/1.1", upstream: "http://127.0.0.1:8000/user/test/test_vscode/static/base/js/security.js?v=20191216125140", host: "192.168.1.150", referrer: "https://192.168.1.150/user/test/test_vscode/tree?"
12:56:22.372 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/components/codemirror/mode/gfm/gfm.js?v=20191216125140 socket hang up
2019/12/16 12:56:22 [crit] 738#0: *1847 SSL_write() failed (SSL:) while sending to client, client: 127.0.0.1, server: , request: "GET /user/test/test_vscode/static/notebook/js/mathjaxutils.js?v=20191216125140 HTTP/1.1", upstream: "http://127.0.0.1:8000/user/test/test_vscode/static/notebook/js/mathjaxutils.js?v=20191216125140", host: "192.168.1.150", referrer: "https://192.168.1.150/user/test/test_vscode/tree?"
12:56:22.374 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/notebook/js/codemirror-ipython.js?v=20191216125140 socket hang up
[I 2019-12-16 12:56:22.375 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fcomponents%2Fmarked%2Flib%2Fmarked.js%3Fv%3D20191216125140 (@172.17.0.2) 3.90ms
[I 2019-12-16 12:56:22.383 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fcomponents%2Fcodemirror%2Faddon%2Frunmode%2Frunmode.js%3Fv%3D20191216125140 (@172.17.0.2) 7.40ms
[I 2019-12-16 12:56:22.385 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fcomponents%2Fcodemirror%2Fmode%2Fgfm%2Fgfm.js%3Fv%3D20191216125140 (@172.17.0.2) 8.16ms
[I 2019-12-16 12:56:22.387 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fnotebook%2Fjs%2Fcodemirror-ipython.js%3Fv%3D20191216125140 (@172.17.0.2) 8.41ms
12:56:36.332 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css?v=3c2a865c832a1322285c55c6ed99abb2 socket hang up
[I 2019-12-16 12:56:36.342 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fcomponents%2Fjquery-ui%2Fthemes%2Fsmoothness%2Fjquery-ui.min.css%3Fv%3D3c2a865c832a1322285c55c6ed99abb2 (@172.17.0.2) 7.52ms
[I 2019-12-16 12:56:36.345 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fcomponents%2Fjquery-typeahead%2Fdist%2Fjquery.typeahead.min.css%3Fv%3D7afb461de36accb1aa133a1710f5bc56 (@172.17.0.2) 8.54ms
For an admin user everything works fine. Can you help me ?
Benjamin Räthlein
@raethlein
Hey Simon, could you please try it with a different username? One without an underscore "_", I think there is a problem with routing
Simon Sander
@sim-san
Hey Benjamin, your hint solve the problem. Without undersocres in username and workspace name everything works fine. Thank you.
Benjamin Räthlein
@raethlein
Great to hear! I also just added "_" to the characters that will be normalized in user names
Jędrzej Biedrzycki
@TAndronicus
Hi, I have a problem running ml-workspace container. It's pulling the images for some time and results in docker: unauthorized: authentication required.
Islam Mansour
@IslamAlam
Do you consider upgrading to cuda-10.12 after the latest commit for to cuda-10.1?
Simon Sander
@sim-san
What are the big changes in the version 0.9.1 ?
Lukas Masuch
@LukasMasuch
Hey, big changes are: Python 3.6 -> 3.7, Ubuntu 16.04 -> 18.04, CUDA 10 -> 10.1, NodeJS 8 -> 12, OpenJDK 8 -> 11. And, of course, lots of other less important version updates and fixes. Generally, we are currently working towards a stable 1.0 version. Once we have reached the 1.0 version (within the next few weeks), we will have a Changelog for all subsequent updates.
heayoungsuh
@heayoungsuh
Hi, I tried building the dockerfile without utilizing the build.py as docker build -t test .. but there seems to be problem with the ngix. Is it normal? ( yes I am a newbie )
Lukas Masuch
@LukasMasuch
@heayoungsuh what is the error you get with nginx?
Simon Sander
@sim-san
I want to mount a SMB or NFS shared folder inside the workspace. What is your recommendation for doing this ?
Simon Sander
@sim-san
And I use these ml-workspaces with ml-hub
Benjamin Räthlein
@raethlein

Hey @sim-san , if you use the Docker-local version you can either create an nfs-volume before (see https://unix.stackexchange.com/a/320717/239011) and add the volume mount to the JupyterHub config (e.g. c.DockerSpawner.volumes={'my-nfs-share': '/workspace/nfs'} or it should also be possible to add something like the following configuration to the JupyterHub config (I did not test it, so it's only meant to be a hint):

c.DockerSpawner.extra_host_kwargs={
mounts=[{
"target": "<container_path>",
"source": "nfsvolume",
"type": "volume",
"driver_config": {"name": "local", options: {"type": "nfs", "device": ":<nfs export path>", "o": "addr=<nfs-server-addr>"}}
}]
}

In the first solution, to avoid having the same nfs folder mounted in all directories, you could create a separate nfs volume (my-nfs-share-{username}) for each workspace and add the directive c.DockerSpawner.volumes={'my-nfs-share-{username}': '/workspace/nfs'}. However, this requires manual effort (as also the host directory has to be created). An automated way of handling nfs shares does not seem to be supported right now by DockerSpawner and also not by our MLHubDockerSpawner.

In a Kubernetes-setup this might be a little bit easier as you can work with dynamic PVC provisioning etc.

neer201
@neer201
Hey guys! I tried to launch ml-workspace image on vast.ai. My instance has launched and ssh runs normally. But I dont have launched jupyter app with other software. Whet I launch jupyter manually, I dont have access to other applications like VScode with the message in console: 404 GET /tools/vscode (127.0.0.1) 15.69ms referer=http://localhost:8080/tree?
Lukas Masuch
@LukasMasuch
@neer201 do you have more information on how you launched the workspace, e.g. the docker run command? When you run the workspace, jupyter should start automatically and will be accesible on the main port.
Simon Sander
@sim-san
I have copied folders und files in the mounted workspace folder. I can see the file in the notebook overview, but can't access them from the jupyterlab. The folders are not shown. Do you know why this happens ?
Lukas Masuch
@LukasMasuch
@sim-san we haven't seen this issue yet. A few questions: 1) Have you mounted a volume via NFS or is it a basic host path mount? 2) Can you open the file via Jupyter or via terminal (cat)? 3) You could check the permissions of the files and folders. Just type in ls -la within the Jupyter terminal in the /workspace folder. If you like you can also share the screenshot here.
Simon Sander
@sim-san
@LukasMasuch
1) I copy a folder with .ipynb inside the host path mount.
2) I can open the files via Jupyter and with terminal in jupyter. But inside the Jupyter Lab and Jupyter Lab terminal I do not see the files.
3) After I copied the files I chaned the ownership to root:root
The screenshot from the terminal in the normal Jupyter:
The screenshot from the terminal in the Jupyter Lab:
Simon Sander
@sim-san
But I noticed something else strange. When I opened the named server 'notebooks', I got an error message: 'Server not running. Your server is not running. Would you like to start it? '
By clicking on "Launch Server" the general server "My Server" was started. I noticed that the link to the Jupyter Lab in the named server 'notebooks' points to the general server. In the general server I can create a folder in Jupyter and see it in Jupyter Lab from the named server 'notebooks'
Simon Sander
@sim-san
And also a screenshot of the hub logs:
neer201
@neer201
@LukasMasuch Hey, I didn't run any commands. Sorry, I am a newbie in this area. My config looks like thi
s. There is an option to run some command when the instance starts but I don't know which.
Lukas Masuch
@LukasMasuch
@neer201 I just looked into how to start the workspace with vast.ai, and its possible if you select the mltooling/ml-workspace:latest image, select the SSH option, follow the vast.ai guide to setup the SSH connection, connect to the server with the ssh command (the ssh command should have the port tunneling via -L on port 8080). Once you are connected via SSH, you have to start all the workspace tools via the following command WORKSPACE_BASE_URL="/" WORKSPACE_PORT=8080 VNC_PW="vncpassword" VNC_RESOLUTION="1600x900" VNC_COL_DEPTH="24" python /resources/docker-entrypoint.py
Within a few seconds/minutes, you should be able to access the workspace from you browser via localhost:8080
@sim-san Strange issue, seems like that the proxy is redirecting the request to the user workspace instead of the named server when accessing JupyterLab. Can you confirm that what you see within Jupyterlab is the files of your user workspace? Unfortunatly, I wasn't able to reproduce this issue with the newest version of ml-hub on docker. Are you using Kubernetes or Docker?
Simon Sander
@sim-san
I use Docker
Can you confirm that what you see within Jupyterlab is the files of your user workspace? => Yes
Lukas Masuch
@LukasMasuch
@sim-san Just got an idea. Is your named server called notebooks?
Lukas Masuch
@LukasMasuch
The proxy uses the path /user/<user-name>/notebooks for actually accessing notebook objects within the user workspace. Naming the named server notebooks is therefore problematic since it is conflicting with this path. In the next version we will add blacklist some names for named servers such as notebooks. As a workaround for now, just use any other name than tree, notebooks, tools and it should (hopefully) work fine.
heayoungsuh
@heayoungsuh
@LukasMasuch Sorry I'm late.. The problem went away after ran the dockerfile with build.py. I lost my log files as I was testing out some other things.
Simon Sander
@sim-san
@LukasMasuch thanks for your help. Renaming the workspace solved the problem easily.
Besides with I have found another bug. I have opened an issue about it. ml-tooling/ml-workspace#32
Simon Sander
@sim-san
And a further Issue I found: ml-tooling/ml-workspace#33
Lukas Masuch
@LukasMasuch
@sim-san Thanks for submitting the issues :) I will try to fix them next week.
Simon Sander
@sim-san
Where can I see logs from the juypter lab if it has not started ?
I get this error in the web console: Laden fehlgeschlagen für das <script> mit der Quelle "http://192.168.1.150:8899/static/lab/vendors~main.4d6f61962d80393a514d.js".