Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    faizalam1
    @faizalam1
    @macshaggy Can you help me with starting ml-workspace?
    Edi Santoso
    @repodevs
    what is your problem?
    Jesper Vang
    @flight505
    soo after I run docker run -p 8080:8080 mltooling/ml-workspace:latest I am not able to access Jupyter .. this is a Macbook the only thing on it is OS Mojave and I am trying to run Ml workspace.. I might mention that I am also new to docker, and I don't know if I should pass a work folder or anything to the command?
    Lukas Masuch
    @LukasMasuch
    @flight505 just running docker run -p 8080:8080 mltooling/ml-workspace:latest should be enough to test the workspace (just access http://localhost:8080). The only requirement is a working docker installation. If it is not working for you, you might try a different port e.g. docker run -p 8081:8080 mltooling/ml-workspace:latest or a restart of the docker daemon/macbook. Have you tried other docker containers? What are the logs you get?
    @macshaggy We will most likely update to Python 3.7 until end of this year. There are still a few dependencies which need Python 3.6 for whatever reason, but most of the important once are working fine on 3.7.
    Derek Chia
    @DerekChia_twitter
    after creating a new conda environment, how do i start a new notebook using it?
    current there's only Python2 and Python3
    Derek Chia
    @DerekChia_twitter
    i.e. How do i add kernels to jupyter
    Lukas Masuch
    @LukasMasuch
    Hey Derek, you need to create a kernel via the ipykernel library. Here is an example:
    1. Create environment: conda create -n myenv python=3.6
    2. Activate environment: source activate myenvy
    3. Install ipykernel: conda install ipykernel
    4. Install kernel from environment: python -m ipykernel install --user --name myenv --display-name "Python (myenv)"
    After this, you just reload the jupyter website and the new kernel should be available
    Derek Chia
    @DerekChia_twitter
    Thanks Lukas, actually i'm aware of these steps, just thinking if it is already in-built in the toolbox for easy access.
    Lukas Masuch
    @LukasMasuch
    There is no easier way to create new conda environments within the workspace. However, the default Python 3 kernel is based on conda and already contains a variety of common machine learning libraries. So, in many cases you probably do not have to create a new environment.
    Derek Chia
    @DerekChia_twitter
    Yea, default Python 3 has a good range of libraries, but sometimes (most of the time) tf 1.14 is more common than tf 2.0.. Sadly..
    Naman Shukla
    @namanUIUC
    Great work guys! I see the jupyterlab got extensions already installed. Couple of questions:
    1. How to choose one of the themes (oriolmirosa/jupyterlab_materialdarker) as I can't find it under Settings -> JupyterLab Themes?
    2. How to install additional extensions ?
    Lukas Masuch
    @LukasMasuch
    @namanUIUC Thanks for the feedback! 1) materialdarker is not preinstalled, however, once you have installed it you will be able to choose it from jupyterlab menu. 2) Within the workspace you can use the terminals to install anything you like, as explained here: https://github.com/ml-tooling/ml-workspace#extensibility For example, running jupyter labextension install @oriolmirosa/jupyterlab_materialdarker in the terminal will install the material darker extension. In Jupyterlab there is also an UI to search and install Jupyterlab extensions (Accessible from the left menu - the puzzel piece)
    Naman Shukla
    @namanUIUC
    I see, thanks for the help @LukasMasuch. One thing though, when I try running jupyter labextension install @oriolmirosa/jupyterlab_materialdarker it installs perfectly but fails to build. Any comments on that? BTW here are the logs for that:
    Node v11.15.0
    
    Building jupyterlab assets (build:prod:minimize)
    > node /opt/conda/lib/python3.6/site-packages/jupyterlab/staging/yarn.js install --non-interactive
    yarn install v1.15.2
    [1/5] Validating package.json...
    [2/5] Resolving packages...
    [3/5] Fetching packages...
    warning Pattern ["plotly-icons@latest"] is trying to unpack in the same destination "/root/.cache/yarn/v4/npm-plotly-icons-1.
    3.13-31cbc2d8832df8a7ed81399ec32f03ef9043c783/node_modules/plotly-icons" as pattern ["plotly-icons@^1.1.5"]. This could resul
    t in non-deterministic behavior, skipping.
    info fsevents@1.2.9: The platform "linux" is incompatible with this module.
    info "fsevents@1.2.9" is an optional dependency and failed compatibility check. Excluding it from installation.
    [4/5] Linking dependencies...
    warning "nbdime-jupyterlab > nbdime@5.0.1" has unmet peer dependency "codemirror@^5.0.0".
    warning "jupyterlab-chart-editor > react-chart-editor > draft-js-export-html@1.2.0" has unmet peer dependency "immutable@3.x.
    x".
    warning "jupyterlab-chart-editor > react-chart-editor > draft-js-import-html@1.4.1" has unmet peer dependency "immutable@3.x.
    x".
    warning "jupyterlab-chart-editor > react-chart-editor > draft-js-utils@1.4.0" has unmet peer dependency "immutable@3.x.x".
    warning "jupyterlab-chart-editor > react-chart-editor > react-rangeslider@2.2.0" has incorrect peer dependency "react@^0.14.0
     || ^15.0.0".
    warning "jupyterlab-chart-editor > react-chart-editor > draft-js-import-html > draft-js-import-element@1.4.0" has unmet peer
    dependency "immutable@3.x.x".
    [5/5] Building fresh packages...
    success Saved lockfile.
    Done in 8.03s.
    
    > node /opt/conda/lib/python3.6/site-packages/jupyterlab/staging/yarn.js yarn-deduplicate -s fewer
    yarn run v1.15.2
    $ /opt/conda/share/jupyter/lab/staging/node_modules/.bin/yarn-deduplicate -s fewer
    Done in 0.52s.
    
    > node /opt/conda/lib/python3.6/site-packages/jupyterlab/staging/yarn.js run build:prod:minimize
    yarn run v1.15.2
    $ cross-env NODE_OPTIONS=--max_old_space_size=4096 webpack --config webpack.prod.minimize.config.js
    error Command failed with exit code 1.
    info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
    
    JupyterLab failed to build
    Traceback (most recent call last):
    
      File "/opt/conda/lib/python3.6/site-packages/jupyterlab/debuglog.py", line 47, in debug_logging
        yield
    
      File "/opt/conda/lib/python3.6/site-packages/jupyterlab/labextensions.py", line 105, in start
        core_config=self.core_config)
    
      File "/opt/conda/lib/python3.6/site-packages/jupyterlab/commands.py", line 378, in build
        command=command, clean_staging=clean_staging)
    
      File "/opt/conda/lib/python3.6/site-packages/jupyterlab/commands.py", line 583, in build
        raise RuntimeError(msg)
    
    RuntimeError: JupyterLab failed to build
    
    Exiting application: jupyter
    Carlos Vivar Rios
    @caviri
    Hello! I'm trying to mount a remote NAS inside a Machine Learning Workspace instance. But I'm receiving an error of permission denied as usual with docker. It is possible to allow another way to mount this? Thansk!
    Benjamin Räthlein
    @raethlein
    Hey Carlos, you have to perform all mounts during container startup or give the container privileged access. As you have pointed out, unprivileged containers cannot mount volumes while they are already running. So, either give the workspace privileged access (which, I think, Docker does not recommend) or mount the NAS to your host and then start the container with that location mounted.
    Besides mounting, maybe it is an option to use a tool such as rsync or similar to keep files on different systems in sync.
    Simon Sander
    @sim-san
    Hello,
    Is it possible to disable certain tools like VNC, SSH in a ml-workspace ?
    Benjamin Räthlein
    @raethlein
    Hey Simon, currently it is not configurable via environment variables or so. Currently, I guess you have to create a custom image built on top of it and remove the unwanted tools
    Simon Sander
    @sim-san

    Okay, I started it. But when I run docker build, I get the following error:

    : not foundean-layer.sh: 1: /usr/bin/clean-layer.sh:
    /usr/bin/clean-layer.sh: 11: set: Illegal option -

    This happens on line 51 in the dockerfile
    Lukas Masuch
    @LukasMasuch
    @sim-san Building full workspace image can take quite a while and might cause some build problems. If you just want to change a few things, the easiest way is to just inherit from the ml-workspace image as explained in the FAQs (https://github.com/ml-tooling/ml-workspace#faq) and do your changes. We will make it easier to configure/remove tools in the next versions. In the meantime, to remove tools from being started, you can overwrite the supervisor config (https://github.com/ml-tooling/ml-workspace/blob/develop/resources/config/supervisord.conf) and to do add/remove tools in the open-tool menu you can overwrite this file: https://github.com/ml-tooling/ml-workspace/blob/develop/resources/home/.workspace/tools/1-workspace-tools.json.
    Simon Sander
    @sim-san
    thx for your information
    Carlos Vivar Rios
    @caviri
    Thanks for the recommendation I think I'll go for rsync. I tried to run mlhub with --privileged flag but it seems this is not transfer to the servers created :s
    Simon Sander
    @sim-san
    Hello,
    Is it possible to change the parameters for the deflaut server ? For example: change "is_mount_volume" parameter.
    Benjamin Räthlein
    @raethlein
    Hey Simon, do you mean the options in the MLHub's option page?
    So just for understanding, you would like to be able to change the default settings so that, for example, is_mount_volume is unchecked by default?
    Right now, you can only set the default images that are shown in the options page but no other options.
    Simon Sander
    @sim-san
    ok, thanks for the fast answer
    Simon Sander
    @sim-san
    At the moment I face a other Problem with ml-worksqace/ml-hub.
    A user without admin rights on the ml-hub is not able to open the Visual Studio Code Web. I got the following errors in the log:
    12:56:22.188 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/base/images/favicon.ico?v=845272bb369e7a52ef65139fafb905e3 socket hang up
    12:56:22.195 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/services/contents.js?v=20191216125140 socket hang up
    [I 2019-12-16 12:56:22.205 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fbase%2Fimages%2Ffavicon.ico%3Fv%3D845272bb369e7a52ef65139fafb905e3 (@172.17.0.2) 15.47ms
    [I 2019-12-16 12:56:22.208 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fservices%2Fcontents.js%3Fv%3D20191216125140 (@172.17.0.2) 2.33ms
    12:56:22.368 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/components/marked/lib/marked.js?v=20191216125140 socket hang up
    12:56:22.369 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/components/codemirror/addon/runmode/runmode.js?v=20191216125140 socket hang up
    2019/12/16 12:56:22 [crit] 738#0: *1804 SSL_write() failed (SSL:) while sending to client, client: 127.0.0.1, server: , request: "GET /user/test/test_vscode/static/base/js/security.js?v=20191216125140 HTTP/1.1", upstream: "http://127.0.0.1:8000/user/test/test_vscode/static/base/js/security.js?v=20191216125140", host: "192.168.1.150", referrer: "https://192.168.1.150/user/test/test_vscode/tree?"
    12:56:22.372 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/components/codemirror/mode/gfm/gfm.js?v=20191216125140 socket hang up
    2019/12/16 12:56:22 [crit] 738#0: *1847 SSL_write() failed (SSL:) while sending to client, client: 127.0.0.1, server: , request: "GET /user/test/test_vscode/static/notebook/js/mathjaxutils.js?v=20191216125140 HTTP/1.1", upstream: "http://127.0.0.1:8000/user/test/test_vscode/static/notebook/js/mathjaxutils.js?v=20191216125140", host: "192.168.1.150", referrer: "https://192.168.1.150/user/test/test_vscode/tree?"
    12:56:22.374 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/notebook/js/codemirror-ipython.js?v=20191216125140 socket hang up
    [I 2019-12-16 12:56:22.375 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fcomponents%2Fmarked%2Flib%2Fmarked.js%3Fv%3D20191216125140 (@172.17.0.2) 3.90ms
    [I 2019-12-16 12:56:22.383 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fcomponents%2Fcodemirror%2Faddon%2Frunmode%2Frunmode.js%3Fv%3D20191216125140 (@172.17.0.2) 7.40ms
    [I 2019-12-16 12:56:22.385 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fcomponents%2Fcodemirror%2Fmode%2Fgfm%2Fgfm.js%3Fv%3D20191216125140 (@172.17.0.2) 8.16ms
    [I 2019-12-16 12:56:22.387 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fnotebook%2Fjs%2Fcodemirror-ipython.js%3Fv%3D20191216125140 (@172.17.0.2) 8.41ms
    12:56:36.332 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/components/jquery-ui/themes/smoothness/jquery-ui.min.css?v=3c2a865c832a1322285c55c6ed99abb2 socket hang up
    12:56:36.334 [ConfigProxy] error: 503 GET /user/test/test_vscode/static/components/jquery-typeahead/dist/jquery.typeahead.min.css?v=7afb461de36accb1aa133a1710f5bc56 socket hang up
    [I 2019-12-16 12:56:36.342 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fcomponents%2Fjquery-ui%2Fthemes%2Fsmoothness%2Fjquery-ui.min.css%3Fv%3D3c2a865c832a1322285c55c6ed99abb2 (@172.17.0.2) 7.52ms
    [I 2019-12-16 12:56:36.345 JupyterHub log:174] 200 GET /hub/error/503?url=%2Fuser%2Ftest%2Ftest_vscode%2Fstatic%2Fcomponents%2Fjquery-typeahead%2Fdist%2Fjquery.typeahead.min.css%3Fv%3D7afb461de36accb1aa133a1710f5bc56 (@172.17.0.2) 8.54ms
    For an admin user everything works fine. Can you help me ?
    Benjamin Räthlein
    @raethlein
    Hey Simon, could you please try it with a different username? One without an underscore "_", I think there is a problem with routing
    Simon Sander
    @sim-san
    Hey Benjamin, your hint solve the problem. Without undersocres in username and workspace name everything works fine. Thank you.
    Benjamin Räthlein
    @raethlein
    Great to hear! I also just added "_" to the characters that will be normalized in user names
    Jędrzej Biedrzycki
    @TAndronicus
    Hi, I have a problem running ml-workspace container. It's pulling the images for some time and results in docker: unauthorized: authentication required.
    Islam Mansour
    @IslamAlam
    Do you consider upgrading to cuda-10.12 after the latest commit for to cuda-10.1?
    Simon Sander
    @sim-san
    What are the big changes in the version 0.9.1 ?
    Lukas Masuch
    @LukasMasuch
    Hey, big changes are: Python 3.6 -> 3.7, Ubuntu 16.04 -> 18.04, CUDA 10 -> 10.1, NodeJS 8 -> 12, OpenJDK 8 -> 11. And, of course, lots of other less important version updates and fixes. Generally, we are currently working towards a stable 1.0 version. Once we have reached the 1.0 version (within the next few weeks), we will have a Changelog for all subsequent updates.
    heayoungsuh
    @heayoungsuh
    Hi, I tried building the dockerfile without utilizing the build.py as docker build -t test .. but there seems to be problem with the ngix. Is it normal? ( yes I am a newbie )
    Lukas Masuch
    @LukasMasuch
    @heayoungsuh what is the error you get with nginx?
    Simon Sander
    @sim-san
    I want to mount a SMB or NFS shared folder inside the workspace. What is your recommendation for doing this ?
    Simon Sander
    @sim-san
    And I use these ml-workspaces with ml-hub
    Benjamin Räthlein
    @raethlein

    Hey @sim-san , if you use the Docker-local version you can either create an nfs-volume before (see https://unix.stackexchange.com/a/320717/239011) and add the volume mount to the JupyterHub config (e.g. c.DockerSpawner.volumes={'my-nfs-share': '/workspace/nfs'} or it should also be possible to add something like the following configuration to the JupyterHub config (I did not test it, so it's only meant to be a hint):

    c.DockerSpawner.extra_host_kwargs={
      mounts=[{
        "target": "<container_path>",
        "source": "nfsvolume",
        "type": "volume",
        "driver_config": {"name": "local", options: {"type": "nfs", "device": ":<nfs export path>", "o": "addr=<nfs-server-addr>"}}
      }]
    }

    In the first solution, to avoid having the same nfs folder mounted in all directories, you could create a separate nfs volume (my-nfs-share-{username}) for each workspace and add the directive c.DockerSpawner.volumes={'my-nfs-share-{username}': '/workspace/nfs'}. However, this requires manual effort (as also the host directory has to be created). An automated way of handling nfs shares does not seem to be supported right now by DockerSpawner and also not by our MLHubDockerSpawner.

    In a Kubernetes-setup this might be a little bit easier as you can work with dynamic PVC provisioning etc.

    neer201
    @neer201
    Hey guys! I tried to launch ml-workspace image on vast.ai. My instance has launched and ssh runs normally. But I dont have launched jupyter app with other software. Whet I launch jupyter manually, I dont have access to other applications like VScode with the message in console: 404 GET /tools/vscode (127.0.0.1) 15.69ms referer=http://localhost:8080/tree?
    Lukas Masuch
    @LukasMasuch
    @neer201 do you have more information on how you launched the workspace, e.g. the docker run command? When you run the workspace, jupyter should start automatically and will be accesible on the main port.
    Simon Sander
    @sim-san
    I have copied folders und files in the mounted workspace folder. I can see the file in the notebook overview, but can't access them from the jupyterlab. The folders are not shown. Do you know why this happens ?
    Lukas Masuch
    @LukasMasuch
    @sim-san we haven't seen this issue yet. A few questions: 1) Have you mounted a volume via NFS or is it a basic host path mount? 2) Can you open the file via Jupyter or via terminal (cat)? 3) You could check the permissions of the files and folders. Just type in ls -la within the Jupyter terminal in the /workspace folder. If you like you can also share the screenshot here.
    Simon Sander
    @sim-san
    @LukasMasuch
    1) I copy a folder with .ipynb inside the host path mount.
    2) I can open the files via Jupyter and with terminal in jupyter. But inside the Jupyter Lab and Jupyter Lab terminal I do not see the files.
    3) After I copied the files I chaned the ownership to root:root
    The screenshot from the terminal in the normal Jupyter:
    grafik.png