Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Nikolay Vazov
    @vazovn
    If the temp files do not grow so much, we can easily set it to /tmp.
    Let me try what you said
    Helena Rasche
    @hexylena:matrix.org
    [m]
    /tmp should be in the container
    Nikolay Vazov
    @vazovn
    Yes, of course, yet this would mean that the os.link function on the galaxy host shall write there.
    No. Fails. You see, I think that since put evokes the os.link function on the galaxy host, the galaxy user on this host tries to put things to the host's defined tmp location. Which is /storage/galaxy/tmpas defined in the new_file_path. Look at the error after I made the last modificaiton:
    Helena Rasche
    @hexylena:matrix.org
    [m]
    you're running a put command inside the gxits, right?
    they have no need to access host storage at all
    there shouldn't be anything happening on the galaxy host to link things in, that's very weird
    Nikolay Vazov
    @vazovn
    ioblend.ConnectionError: Unexpected HTTP status code: 400: {"err_msg": "Error executing tool with id 'upload1': [Errno 13] Permission denied: '/storage/galaxy/tmp/tmpmqip32yx' -> '/storage/galaxy/tmp/upload_file_data_zimvku33'", "err_code": 0}
    Helena Rasche
    @hexylena:matrix.org
    [m]
    but i see now that it's a uwsgi error, I'm sorry, I missed something then
    Nikolay Vazov
    @vazovn
    I run the command from within the container and get this, yes
    Helena Rasche
    @hexylena:matrix.org
    [m]
    oh, you're not doing gxits at all! I got completely mislead on the problem from the first sentence 😅
    Sorry @vazovn
    ok, but, that part works, it's just that galaxy fails to write a newfile
    Nikolay Vazov
    @vazovn
    Exactly
    Helena Rasche
    @hexylena:matrix.org
    [m]
    ok, then you're right, you need to set new_file_path
    that's the solution
    Nikolay Vazov
    @vazovn
    Ouf, sorry for the hassle !
    Helena Rasche
    @hexylena:matrix.org
    [m]
    no worries, I should have read closer
    Maiken Pedersen
    @maikenp
    so new_file_path needs to be a folder that all systems have basically so /tmp in this example ?
    Helena Rasche
    @hexylena:matrix.org
    [m]
    it needs to be a folder the head node can write to
    Nikolay Vazov
    @vazovn
    Just one theoretical question. Why can galaxy from the host create the hard link (from the terminal) yet the put command from the container, which calls the same function, fails?
    1 reply
    Maiken Pedersen
    @maikenp
    right, but there is no way to set one tmpdir on the galaxy server and another on the compute-nodes where the docker container runs?
    1 reply
    Helena Rasche
    @hexylena:matrix.org
    [m]
    my understanding is galaxy needs new_file_path to write uploaded files, and that needs to be accessible from where the upload jobs run
    Jennifer Hillman-Jackson
    @jennaj
    Hi -- this person is having trouble connecting their Galaxy to the Main ToolShed. Help would be appreciated: https://help.galaxyproject.org/t/ssl-verfication-failure-contacting-toolshed-site/6701/2
    pdagosto
    @pdagosto

    Hi there. I'm the guy having SSL certificate verification issues with the tool shed. I'm not actually a Galaxy user or a sysadmin but I'm trying to get Galaxy in a functional state for some folks on another team at my company who are not familiar with the Linux environment.

    Let me ask another question. I noticed this in the docs:

    Uses a built-in HTTP server, written in Python. Much of the work performed by this server can be moved to nginx or Apache, which will increase performance.

    We were asked by the IT folks who are trying to help us what web server platform (e.g., Apache) Galaxy is running on. The reason is that they think that we need to install a certificate (certifying us to the tool shed site) and can't help us with that unless it's Apache or something similar.

    Is there a way to make the built-in HTTP server aware of a certificate?

    1 reply
    martenson
    @martenson:matrix.org
    [m]

    The reason is that they think that we need to install a certificate (certifying us to the tool shed site)

    You, as a client, do not need certificate. Toolshed has a certificate (a valid one) and you need a tool (in python we use certifi package) that will allow you to validate toolshed's certificate against trusted list.

    So the issue you are facing is likely in configuration of your server's environment, rather than a missing certificate on your side.
    Helena Rasche
    @hexylena:matrix.org
    [m]
    i've checked their version of certifi, it returns OK 
    (.venv) 10:02:10|(release_21.01) [hxr@cosima:~/arbeit/galaxy/galaxy]$ pip freeze | ag certifi
certifi==2020.12.5
(.venv) 10:02:16|(release_21.01) [hxr@cosima:~/arbeit/galaxy/galaxy]$ python
Python 3.8.10 (default, Jun  2 2021, 10:49:15)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get('https://toolshed.g2.bx.psu.edu')
<Response [200]>
    any chance it's using the OS' cert store? I'd bet suse is out of date there.
    Nate Coraor
    @natefoo:matrix.org
    [m]
    certifi should override that though, so it doesn't make sense.
    pdagosto
    @pdagosto
    image.png

    We do have the certifi package installed in the virtual environment that Galaxy uses. The version is the same as in your example. I don't know if it's using the OS's cert store. We did try to import the tool shed's certificate into the OS's cert store (didn't help).

    Our python version is 3.6. Could that be a problem?

    Sorry, I botched the job of trying to paste the console output into my post.
    Helena Rasche
    @hexylena:matrix.org
    [m]
    no worries. it's clear.
    yeah, odd. it'd be nice to know where the failure is, but not sure how to check that in python
    (is it the leaf cert, the root, an intermediate)
    Nate Coraor
    @natefoo:matrix.org
    [m]
    @pdagosto: If you run the same as Helena Rasche did above (start your Galaxy venv python, import requests, and get the toolshed URL) what do you get?
    Helena Rasche
    @hexylena:matrix.org
    [m]
    looks like @pdagosto is in a venv
    and ran the same as me
    Nate Coraor
    @natefoo:matrix.org
    [m]
    Oh yep sorry I see the screenshot now
    Also for the record Python 3.6 is fine, it's what I'm using.
    pdagosto
    @pdagosto
    I ran Helena's example - I got the same SSL Certificate verification error that I see in the Galaxy UI (see screen shot in my previous message). It was run in the Galaxy virtual environment.
    Nate Coraor
    @natefoo:matrix.org
    [m]
    I found some discussion showing they did some hacking to some of the Python stdlib for reading the cert store a good while back, I am not sure if this still applies but I'd be curious if you installed Python from e.g. conda and then created your venv from that if it still had the same issues: https://bugzilla.suse.com/show_bug.cgi?id=915185
    1 reply
    Helena Rasche
    @hexylena:matrix.org
    [m]
    yeah
    might try echo "" | openssl s_client -connect toolshed.g2.bx.psu.edu:443
    see if the OS is happy to resolve the chain, might give you some more info.
    I think that fails if invalid?
    or will complain at minimum