Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Marius
    @mvdbeek:matrix.org
    [m]
    I think the oldest release that required futures 3.1.1 was 20.09, which is not supported anymore
    https://github.com/galaxyproject/galaxy/blob/release_20.09/lib/galaxy/dependencies/pipfiles/default/pinned-requirements.txt#L58
    @goshng: If you switch out galaxy_commit_id to something newer, like release_21.09 this should work
    Nicola Soranzo
    @nsoranzo:matrix.org
    [m]
    Ah, git log -p --follow -- lib/galaxy/dependencies/pinned-requirements.txt lied to me!
    Marius
    @mvdbeek:matrix.org
    [m]
    yeah, the history was interrupted when we moved to poetry
    Nicola Soranzo
    @nsoranzo:matrix.org
    [m]
    It decided to follow the wrong file
    Helena Rasche
    @hexylena:matrix.org
    [m]
    huh, I didn't know about --follow, I wish I'd known that one earlier.
    Nicola Soranzo
    @nsoranzo:matrix.org
    [m]
    I use it all the time, it should be the default
    Helena Rasche
    @hexylena:matrix.org
    [m]
    testing out mamba instead of conda for one galaxy deployment I'm responsible for and wow it's faster
    fingers crossed nothing goes wrong
    Marius
    @mvdbeek:matrix.org
    [m]
    Is it much faster than adding --strict-channel-priority to the conda create flags ?
    We have this in the backlog ... galaxyproject/galaxy#12788
    Helena Rasche
    @hexylena:matrix.org
    [m]
    would be nice to benchmark
    currently I don't have an answer for that
    Marius
    @mvdbeek:matrix.org
    [m]
    I guess it really depends ... but since bioconda is recommending that we should just get around and do it
    Helena Rasche
    @hexylena:matrix.org
    [m]
    i'll run a n=1 test for ya
    mamba: 10s
    conda: 2min
    conda-strict: 42s
    # Mamba
(base) galaxy@bioinf-galactus:/srv/galaxy/var/dependencies/_conda$ time /srv/galaxy/var/dependencies/_conda/bin/mamba create -y --quiet --override-channels --channel conda-forge --channel bioconda --channel defaults hisat2=2.2.1 samtools=1.12 seqtk=1.3 --name test-mamba
...
real    0m9.918s
user    0m8.605s
sys     0m0.846s

# Conda (default)
(base) galaxy@bioinf-galactus:/srv/galaxy/var/dependencies/_conda$ time /srv/galaxy/var/dependencies/_conda/bin/conda create -y --quiet --override-channels --channel conda-forge --channel bioconda --channel defaults hisat2=2.2.1 samtools=1.12 seqtk=1.3 --name test-conda
# The first bit is VERY slow
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working...
...
real    1m59.373s
user    1m45.825s
sys     0m4.085s


# Conda (--strict-channel-priority)

(base) galaxy@bioinf-galactus:/srv/galaxy/var/dependencies/_conda$ time /srv/galaxy/var/dependencies/_conda/bin/conda create -y --quiet --override-channels --channel conda-forge --channel bioconda --channel defaults hisat2=2.2.1 samtools=1.12 seqtk=1.3 --name test-conda-strict --strict-channel-priority
# The first bit fails much faster.
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working...
...
real    0m42.747s
user    0m31.254s
sys     0m2.513s
    some notes inline in the snippet. anyway I'm out for the weekend.
    Marius
    @mvdbeek:matrix.org
    [m]
    Enjoy!
    Nicola Soranzo
    @nsoranzo:matrix.org
    [m]
    Also from my experience mamba is much faster than conda (and I've had the channel priority strict turned on since a while ago), and it resolves complex sets of requirements that conda fails to.
    Sang Chul Choi
    @goshng
    @mvdbeek:matrix.org Thank you! The change to galaxy_commit_id passed the step.
    Marius
    @mvdbeek:matrix.org
    [m]
    That's already on dev, do we need to backport this ?
    It's just a warning, so I thought that won't be necessary
    Nate Coraor
    @natefoo:matrix.org
    [m]
    No, definitely not necessary to backport, I didn't know it was on dev.
    Nikolay Vazov
    @vazovn
    Hi, which is the last version of the ansible role usegalaxy_eu.galaxy_systemd I am running 0.1.4.
    Nikolay Vazov
    @vazovn
    Ok found it - this is actually the latest
    Helena Rasche
    @hexylena:matrix.org
    [m]
    for what it's worth: it existed as a separate role largely because of the zergling setup we did, there's a strong chance we'll merge the features from that into the main galaxy role at some point in the near future.
    1 reply
    Nikolay Vazov
    @vazovn
    Hi, I forgot I was running an old role for galaxyproject - 0.9.7. Now I updated it and I am configuring it step by step. There are changes which are exactly the changes I needed - enabling me to unstall systemd galaxy as galaxy user. We are having a slightly different setup now - instead of mounting partitions from the cluster, we have setup a directory in our IBM Elastic Storage System (ESS) which will be used to host the data from Galaxy, like job dirs, file dirs ect. We are mounting it a root_squash and I have some issues adapting the roles to this setup. But I am working on it.
    Helena Rasche
    @hexylena:matrix.org
    [m]
    sounds fun!
    Nikolay Vazov
    @vazovn
    Will let you know :)
    Nikolay Vazov
    @vazovn
    Hi, I am running the latest ansible-playbook and installing version 21.09. The systemd would fail throwing three errors about the location of three files: 
    File "/galaxy/srv/galaxy/server/lib/galaxy/config/schema.py", line 48, in _read_schema
    with open(path) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/galaxy/srv/galaxy/server/lib/galaxy/config/../config_schema.yml'

 File "/galaxy/srv/galaxy/server/lib/galaxy/util/yaml_util.py", line 25, in include
    with open(filename) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/galaxy/srv/galaxy/server/lib/galaxy/config/../uwsgi_schema.yml'

  File "/galaxy/srv/galaxy/server/lib/galaxy/util/yaml_util.py", line 25, in include
    with open(filename) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/galaxy/srv/galaxy/server/lib/galaxy/config/../job_config_schema.yml'
    I copied them manually :
    cp /galaxy/srv/galaxy/server/lib/galaxy/webapps/galaxy/config_schema.yml /galaxy/srv/galaxy/server/lib/galaxy/ cp /galaxy/srv/galaxy/server/lib/galaxy/webapps/galaxy/uwsgi_schema.yml /galaxy/srv/galaxy/server/lib/galaxy/ cp /galaxy/srv/galaxy/server/lib/galaxy/webapps/galaxy/job_config_schema.yml /galaxy/srv/galaxy/server/lib/galaxy/
    Helena Rasche
    @hexylena:matrix.org
    [m]
    those are part of the galaxy repository, odd that they'd be missing
    Nikolay Vazov
    @vazovn
    And now it seems to be working. Is it something I am doing wrong?
    The were misplaced
    Look where I copied them from ...
    /galaxy/srv/galaxy/server/lib/galaxy/webapps/galaxy/
    Helena Rasche
    @hexylena:matrix.org
    [m]
    hmm
    galaxy_commit_id?
    Nikolay Vazov
    @vazovn
    galaxy_commit_id: release_21.09
    But I guess you need the exact commit?
    Helena Rasche
    @hexylena:matrix.org
    [m]
    no, no that's fine
    just wondered if you were on a weird branch or so
    Nikolay Vazov
    @vazovn
    He, you are asking too much from me :)
    If this can help : 
    changed: [galaxy01.educloud.no] => {
    "msg": "Galaxy version changed from '' to '60ab1d5b7ae4385ca31307b0c23c52c979e337eb'"
}
    Morten Johansen
    @morj-uio
    :point_up: March 27, 2020 2:58 PM
    failed to download http://cvmfs0-psu0.galaxyproject.org/cvmfs/singularity.galaxyproject.org/data/6b/81f0681da21040ce0b4a31c3602255a19c5032P (9 - host returned HTTP error) unexpected HTTP error code 200 - please check the stratum 0 health
    Still getting this same error when trying to replicate stratum 0 singularity repo. Was it not rebuilt? Or has the stratum 0 URL changed? The data repo is replicating successfully. Thanks for any pointers.
    Nikolay Vazov
    @vazovn
    Hi, in the previous versions of the ansible playbook the systemd configuration contained the following variables : galaxy_systemd_zergling_envand galaxy_zergpool_listen_addr. In the latest version where mule is the only systemd mode, there is only galaxy_systemd_envvariable. I am using it but the DRMAA_LIBRARY_PATH=.... variable which I set there is now missing in the galaxy.service file. Is this correct?