Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Simon Sander
    @sim-san
    But I noticed something else strange. When I opened the named server 'notebooks', I got an error message: 'Server not running. Your server is not running. Would you like to start it? '
    grafik.png
    By clicking on "Launch Server" the general server "My Server" was started. I noticed that the link to the Jupyter Lab in the named server 'notebooks' points to the general server. In the general server I can create a folder in Jupyter and see it in Jupyter Lab from the named server 'notebooks'
    Simon Sander
    @sim-san
    And also a screenshot of the hub logs:
    grafik.png
    neer201
    @neer201
    @LukasMasuch Hey, I didn't run any commands. Sorry, I am a newbie in this area. My config looks like thi
    s. There is an option to run some command when the instance starts but I don't know which.
    image.png
    Lukas Masuch
    @LukasMasuch
    @neer201 I just looked into how to start the workspace with vast.ai, and its possible if you select the mltooling/ml-workspace:latest image, select the SSH option, follow the vast.ai guide to setup the SSH connection, connect to the server with the ssh command (the ssh command should have the port tunneling via -L on port 8080). Once you are connected via SSH, you have to start all the workspace tools via the following command WORKSPACE_BASE_URL="/" WORKSPACE_PORT=8080 VNC_PW="vncpassword" VNC_RESOLUTION="1600x900" VNC_COL_DEPTH="24" python /resources/docker-entrypoint.py
    Within a few seconds/minutes, you should be able to access the workspace from you browser via localhost:8080
    @sim-san Strange issue, seems like that the proxy is redirecting the request to the user workspace instead of the named server when accessing JupyterLab. Can you confirm that what you see within Jupyterlab is the files of your user workspace? Unfortunatly, I wasn't able to reproduce this issue with the newest version of ml-hub on docker. Are you using Kubernetes or Docker?
    Simon Sander
    @sim-san
    I use Docker
    Can you confirm that what you see within Jupyterlab is the files of your user workspace? => Yes
    Lukas Masuch
    @LukasMasuch
    @sim-san Just got an idea. Is your named server called notebooks?
    Lukas Masuch
    @LukasMasuch
    The proxy uses the path /user/<user-name>/notebooks for actually accessing notebook objects within the user workspace. Naming the named server notebooks is therefore problematic since it is conflicting with this path. In the next version we will add blacklist some names for named servers such as notebooks. As a workaround for now, just use any other name than tree, notebooks, tools and it should (hopefully) work fine.
    heayoungsuh
    @heayoungsuh
    @LukasMasuch Sorry I'm late.. The problem went away after ran the dockerfile with build.py. I lost my log files as I was testing out some other things.
    Simon Sander
    @sim-san
    @LukasMasuch thanks for your help. Renaming the workspace solved the problem easily.
    Besides with I have found another bug. I have opened an issue about it. ml-tooling/ml-workspace#32
    Simon Sander
    @sim-san
    And a further Issue I found: ml-tooling/ml-workspace#33
    Lukas Masuch
    @LukasMasuch
    @sim-san Thanks for submitting the issues :) I will try to fix them next week.
    Simon Sander
    @sim-san
    Where can I see logs from the juypter lab if it has not started ?
    I get this error in the web console: Laden fehlgeschlagen für das <script> mit der Quelle "http://192.168.1.150:8899/static/lab/vendors~main.4d6f61962d80393a514d.js".
    Simon Sander
    @sim-san
    Also in an other Browser with an other falvour I get the same error vendors~main.4d6f61962d80393a514d.js:1 Failed to load resource: net::ERR_INVALID_HTTP_RESPONSE
    Lukas Masuch
    @LukasMasuch
    @sim-san you can see all tool logs within this folder: /var/log/supervisor/ inside the workspace. JupyterLab logs are within the jupyter.log logfile. There is also a hidden supervisor UI on port 8059 (open via url path /tools/8059/), but the UI is not fully functional. If you click on the tool name, you will see the last logs of the tool. Calling this url path should give you all jupyter logs: WORKSPACE_BASE_URL/tools/8059/tail.html?processname=jupyter&limit=100000
    neer201
    @neer201
    @LukasMasuch thank you so much! =) It works.
    MangoCubes
    @MangoCubes
    Hello. I am currently trying out the workspace on my machine, and everything seems to be working great, except when I try to access it through reverse proxy. I opened the server on port 8080, and I have Let's Encrypt SSL on it to secure it. However, when I try to use tools like Terminal, the console says that "Firefox can’t establish a connection to the server at wss://domain.com/terminals/websocket/main". Some other tools display similar error message, with the aforementioned error message and error "WebSocket close with status code 1006" being included in all of them. I have a feeling that this isn't quite related to the software itself, and pardon me if it isn't, but I was unsuccessful in finding enough relevant information to fix this. What can I do to fix this, if this is related to the workspace itself?
    Victor Rodriguez-Fernandez
    @vrodriguezf
    Hi! Where is the Jupyter token logged when authenticating with the <generated> option?
    MangoCubes
    @MangoCubes
    Hello and thank you for the reply. I greatly appreciate that you are here to help. Unfortunately, I am not that familiar with this program and my knowledge about it is rather superficial. Can you explain to me what you mean by that? All I know is token authentication method, and logging in with it generates no log on the console.
    This is what it looks like when I log in: https://pastebin.com/rLnRp6Ai
    Victor Rodriguez-Fernandez
    @vrodriguezf
    I refer to this sentence in the README: "You can also use <generated> to let Jupyter generate a random token that is printed out on the container logs."
    MangoCubes
    @MangoCubes
    I just tried --env AUTHENTICATE_VIA_JUPYTER="<generated>", but I don't see any logs regarding the random generated tokens. What have I done wrong?
    MangoCubes
    @MangoCubes
    As in, there is no log about it as if I set the token manually. docker logs <containerid> doesn't reveal anything new either.
    Victor Rodriguez-Fernandez
    @vrodriguezf
    exactly. I understand that it is not logged if you set it manually, but it should be somewhere for the <generated> case
    MangoCubes
    @MangoCubes
    Where can I find them? I understand that the container log should be showing it, but it currently doesn't (https://pastebin.com/rDQtX5qW), and that you can do jupyter notebook list but I would need token to access the console of that container.
    Victor Rodriguez-Fernandez
    @vrodriguezf
    I have no idea of where they are logged. I will open an issue on Github
    MangoCubes
    @MangoCubes
    Hello. I found out that with Portainer, I can access the console without token, and was able to retrieve the token via jupyter notebook list. However, I am not sure as to how I can use this to solve the original issue I had where I could not use some of the tools at all, such as VNC, and instead gives the error message "WebSocket close with status code 1006" in the browser console.
    I have also tried clearing everything and installing NGINX instead, but this returns same result as what I had when I had Apache installed.
    Victor Rodriguez-Fernandez
    @vrodriguezf
    You are right. Just by executing docker exec with the command jupyter notebook list the problem is solved. Thanks!
    MangoCubes
    @MangoCubes
    Oh.
    N. Dorukhan Sergin
    @dorukhansergin
    Hi,
    I've been following the readme about remote development with vs-code. There's an extension recommended there and here's the link for that Remote - SSH. This extension doesn't show up in my VS Code tool?? What can I do about this?
    image.png
    krannnn
    @krannnn
    Hello, I'm having this issue when migrating to docker 19.03
    this is the commande I'm running
    docker run -d -p 5678:8080 --gpus all --name "ml-workspace" -v "${PWD}:/workspace" mltooling/ml-workspace-gpu:latest
    I keep getting this error that is related to IT renaming nvidia-container-runtime-hook to nvidia-container-toolkit .
    docker: Error response from daemon: exec: "nvidia-container-runtime-hook": executable file not found in $PATH.
    Any thoughts ?
    Christophe Lecointe
    @clec_gitlab
    Hello, I'm trying to define a working directory using config c.KubeSpawner.working_dir, but it seems to be hardcoded to /workspace somewhere. Any idea how I can change it ?
    Boris Kosmynin
    @kosmynin_gitlab
    Hi, is there a way to get a vpn client running inside the spawned container?
    Simon Sander
    @sim-san
    Hi,
    I am building my own ml-workspace based on this great ml-workspace. But right now I get this error when I open a Jupyter notebook. Can anyone help me?
    [W 13:42:20.263 NotebookApp] Config option `template_path` not recognized by `ExporterCollapsibleHeadings`.  Did you mean one of: `template_file, template_name, template_paths`?
    [W 13:42:20.271 NotebookApp] Config option `template_path` not recognized by `ExporterCollapsibleHeadings`.  Did you mean one of: `template_file, template_name, template_paths`?
    [W 13:42:20.318 NotebookApp] Config option `template_path` not recognized by `TocExporter`.  Did you mean one of: `template_file, template_name, template_paths`?
    [W 13:42:20.324 NotebookApp] Config option `template_path` not recognized by `TocExporter`.  Did you mean one of: `template_file, template_name, template_paths`?
    [W 13:42:20.353 NotebookApp] Config option `template_path` not recognized by `LenvsHTMLExporter`.  Did you mean one of: `template_file, template_name, template_paths`?
    [W 13:42:20.363 NotebookApp] Config option `template_path` not recognized by `LenvsHTMLExporter`.  Did you mean one of: `template_file, template_name, template_paths`?
    [W 13:42:20.392 NotebookApp] Config option `template_path` not recognized by `LenvsTocHTMLExporter`.  Did you mean one of: `template_file, template_name, template_paths`?
    [W 13:42:20.402 NotebookApp] Config option `template_path` not recognized by `LenvsTocHTMLExporter`.  Did you mean one of: `template_file, template_name, template_paths`?
    [E 13:42:20.731 NotebookApp] Uncaught exception GET /notebooks/welcome.ipynb (127.0.0.1)
        HTTPServerRequest(protocol='http', host='localhost', method='GET', uri='/notebooks/welcome.ipynb', version='HTTP/1.1', remote_ip='127.0.0.1')
        Traceback (most recent call last):
          File "/opt/conda/lib/python3.7/site-packages/tornado/web.py", line 1701, in _execute
            result = method(*self.path_args, **self.path_kwargs)
          File "/opt/conda/lib/python3.7/site-packages/tornado/web.py", line 3178, in wrapper
            return method(self, *args, **kwargs)
          File "/opt/conda/lib/python3.7/site-packages/notebook/notebook/handlers.py", line 96, in get
            get_frontend_exporters=get_frontend_exporters
          File "/opt/conda/lib/python3.7/site-packages/notebook/base/handlers.py", line 507, in render_template
            return template.render(**ns)
          File "/opt/conda/lib/python3.7/site-packages/jinja2/environment.py", line 1090, in render
            self.environment.handle_exception()
          File "/opt/conda/lib/python3.7/site-packages/jinja2/environment.py", line 832, in handle_exception
            reraise(*rewrite_traceback_stack(source=source))
          File "/opt/conda/lib/python3.7/site-packages/jinja2/_compat.py", line 28, in reraise
            raise value.with_traceback(tb)
          File "/opt/conda/lib/python3.7/site-packages/notebook/templates/notebook.html", line 1, in top-level template code
            {% extends "page.html" %}
          File "/opt/conda/lib/python3.7/site-packages/notebook/templates/page.html", line 154, in top-level template code
            {% block header %}
          File "/opt/conda/lib/python3.7/site-packages/notebook/templates/notebook.html", line 114, in block "header"
            {% for exporter in get_frontend_exporters() %}
          File "/opt/conda/lib/python3.7/site-packages/notebook/notebook/handlers.py", line 36, in get_frontend_exporters
            for name in get_export_names():
          File "/opt/conda/lib/python3.7/site-packages/nbconvert/exporters/base.py", line 141, in get_export_names
            e = get_exporter(exporter_name)(config=config)
          File "/opt/conda/lib/python3.7/site-packages/nbconvert/exporters/base.py", line 102, in get_exporter
            if getattr(exporter(config=config), 'enabled', True):
          File "/opt/conda/lib/python3.7/site-packages/nbinteract/exporters.py", line 125, in __init__
            self.template_path.insert(
        AttributeError: 'InteractExporter' object has no attribute 'template_path'
    Benjamin Räthlein
    @raethlein
    Hey @sim-san , the error can be fixed by uninstalling the package pip uninstall nbinteract (and probably restarting the container). We will look into fixing the Docker Image
    Benjamin Räthlein
    @raethlein
    I have removed the installation of nbinteract for now from the requirements.txt
    Simon Sander
    @sim-san

    Hey @sim-san , the error can be fixed by uninstalling the package pip uninstall nbinteract (and probably restarting the container). We will look into fixing the Docker Image

    thx. I cloud fix this error.

    Simon Sander
    @sim-san
    @raethlein When are you planning the next " full workspace image update " (https://github.com/ml-tooling/ml-workspace/blob/main/docs/update-workspace-image.md) ? I am interested in cuda 11.0 and tensorflow 2.4.
    Lukas Masuch
    @LukasMasuch
    @sim-san We plan a bigger update within the next 1-2 months, which will comprise things like Ubuntu 20.04, Cuda 11, Jupyterlab 3.x, Tensorflow 2.4 ... Back in December, CUDA 11 wasn't an option, unfortunately, since TensorFlow 2.3 did not support it. But with TensorFlow 2.4, it seems that we can now use CUDA 11.
    AKJHK2021
    @AKJHK2021
    is this compatible with Apple's M1 macbook pro?
    Benjamin Räthlein
    @raethlein
    @AKJHK2021 We have not tested it on the M1 MacBook Pro or any other ARM machine. Since we did not specifically cross-compile the image, I'd say no with the disclaimer that I have not tested it so far. I just came across this Medium article (https://medium.com/nttlabs/buildx-multiarch-2c6c2df00ca2) that explains how to cross-compile images. I might give it a shot in the coming days but can't promise anything right now. However, in the meantime if you have an M1 MacBook and want to check out the repo and try to build it on your machine, we would be very curious to learn about your experience!
    Since M1 MacBooks will grow in numbers in the future and also ARM in general, we will definitely spend some time to offer future images for both architectures (if they don't work right now).
    ahmedbendebba1
    @ahmedbendebba1
    Hello I have aleady run an image of ml-worskpace to create a container. I am developping a webapp in python in this container so I have installed streamlit and bigquerry-api libraires.
    Now i try to start this container but it gaves me this error
    2021-04-27 14:53:34,008 INFO spawned: 'jupyter' with pid 827
    
    [W 14:53:34.646 NotebookApp] All authentication is disabled. Anyone who can connect to this server will be able to run code.
    
    Traceback (most recent call last):
    
    File "/opt/conda/bin/jupyter-notebook", line 11, in <module>
    
    sys.exit(main())
    
    File "/opt/conda/lib/python3.8/site-packages/jupyter_core/application.py", line 254, in launch_instance
    
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
    
    File "/opt/conda/lib/python3.8/site-packages/traitlets/config/application.py", line 844, in launch_instance
    
    app.initialize(argv)
    
    File "/opt/conda/lib/python3.8/site-packages/traitlets/config/application.py", line 87, in inner
    
    return method(app, *args, **kwargs)
    
    File "/opt/conda/lib/python3.8/site-packages/notebook/notebookapp.py", line 2085, in initialize
    
    self.init_webapp()
    
    File "/opt/conda/lib/python3.8/site-packages/notebook/notebookapp.py", line 1757, in init_webapp
    
    success = self._bind_http_server()
    
    File "/opt/conda/lib/python3.8/site-packages/notebook/notebookapp.py", line 1764, in _bind_http_server
    
    return self._bind_http_server_unix() if self.sock else self._bind_http_server_tcp()
    
    File "/opt/conda/lib/python3.8/site-packages/notebook/notebookapp.py", line 1790, in _bind_http_server_tcp
    
    self.http_server.listen(port, self.ip)
    
    File "/opt/conda/lib/python3.8/site-packages/tornado/tcpserver.py", line 143, in listen
    
    sockets = bind_sockets(port, address=address)
    
    File "/opt/conda/lib/python3.8/site-packages/tornado/netutil.py", line 168, in bind_sockets
    
    sock.bind(sockaddr)
    
    OSError: [Errno 99] Cannot assign requested address
    
    2021-04-27 14:53:34,733 INFO exited: jupyter (exit status 1; not expected)
    
    2021-04-27 14:53:35,735 INFO gave up: jupyter entered FATAL state, too many start retries too quickly
    
    2021/04/27 14:55:01 [error] 113#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8090/", host: "localhost:8080"
    
    2021/04/27 14:55:08 [error] 113#0: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8090/", host: "localhost:8080",
    Lukas Masuch
    @LukasMasuch
    @ahmedbendebba1 thanks for reporting the issue. I wrote a response in the github issue.