Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    akankshabafna
    @akankshabafna
    @bgruening .. I am trying to join using Operate on genomic intervals in chrome.
    Björn Grüning
    @bgruening
    can you share those history?
    Patrick Bovio
    @pbovio
    "This job was resubmitted to the queue because it encountered a tool detected error condition on its compute resource.
    Couldn't zUncompress 13631 bytes: zlib data error"
    bigWigToBedGraph
    anny idea whats wrong? the other bigwigs were fine
    Jonathan Manning
    @pinin4fjords

    Hi all- I'm using https://github.com/ebi-gene-expression-group/galaxy-workflow-executor to try and run a workflow via the API from the command line. This works fine on our local Galaxy, but on usegalaxy.eu it stops after uploading the first input dataset with

    Failed due to ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

    I can consult @pcm32 on the code, but is there something I need to do (other than setting the API key which I've done) to enable successful running via the API? I'm pointing at "https://usegalaxy.eu".

    akankshabafna
    @akankshabafna
    @bgruening ... "history_id": "ed4a9a03cab10c2f",
    "tool_id": "toolshed.g2.bx.psu.edu/repos/devteam/join/gops_join_1/1.0.0",
    "tool_version": "1.0.0",
    "inputs": {
    "input1": {
    "values": [
    {
    "id": "11ac94870d0bb33aa6263f6bde3656be",
    "hid": 701,
    "name": "MACS2 callpeak on data 437 and data 430 (narrow Peaks)",
    "tags": [],
    "src": "hda",
    "keep": false
    }
    ],
    "batch": false
    },
    "input2": {
    "values": [
    {
    "id": "11ac94870d0bb33a57542bd429bc178e",
    "hid": 706,
    "name": "MACS2 callpeak on data 438 and data 431 (narrow Peaks)",
    "tags": [],
    "src": "hda",
    "keep": false
    }
    ],
    "batch": false
    },
    "min": 1,
    "fill": "none"
    }
    }
    Arthur Eschenlauer
    @eschen42
    Here is what I think is going in, if my interpretation is correct; if I get time I may get to producing a toy example to reproduce this, but over the past two weeks I have had trouble specifically with Query Tabular on usegalaxy.eu; either re-running the tool by pushing the rerun button or creating a workflow from a history has often reverted the "Table Options" "Use first line as columns" from Yes to the default value No. This is comparatively new behavior; it didn't used to exhibit this problem, and AFAIK the version of Query Tabular has not changed since it was working without issue. I think that the version of Galaxy (or at least the UI) has changed and fear that there may be a subtle issue lurking below the surface. The good (perhaps) news is that this is the only tool parameter that I have found mutable (or revertible if you prefer); the bad news is that I may indeed be experiencing other parameters reverting that do not cause tool failure and consequently I may not know that my workflow results are different.
    Arthur Eschenlauer
    @eschen42
    Also, and perhaps related, clicking the duplicate button for the Query Tabular tool (having two datasets as input) in the workflow editor results in the duplicate of Query Tabular whose tool details start with {label, annotation, label, annotation} and then the rest, i.e., the first two fields are repeated even though they should be one-per-tool-instance.
    I fully realize that this and the previous notes are not a UseGalaxy.eu specific, but this is kind of a "distant early warning" that the UI is behaving in unexpected ways.
    Gabriele Arcari
    @arcari_gabriele_twitter
    Hi everybody, I'm having an issue with roary on usegalaxy.eu!
    I annotated several FASTA files with prokka and then selected the .gff output as inputs for roary. After a few minutes I always receive an error message which says "The job creating this dataset has been resubmitted" and "This job was resubmitted to the queue because it encountered a tool detected error condition on its compute resource".
    What can I do? Thanks a lot :)
    Arthur Eschenlauer
    @eschen42
    Hi, Gabriele. My experience on usegalaxy.eu is that I have seen this when the tool itself ends with an error. I don't know how or why this process unfolds this way with these messages, but my experience has always been that I need to correct the tool parameters or input data until I can get the tool to run successfully. So, what can you do? Please check your inputs. You might click on the bug icon to see if the tool wrote a helpful message, otherwise, check the black circle with a white letter "i" and then click on stdout and stderr to see if there are helpful messages written there.
    If that doesn't work for you then you may need to track down the author of the tool or the team that supports it.
    Bram van Dijk
    @bramvandijk88_twitter
    My spades-nodes are being queued forever... it's bin in que all night
    Is this a spades-specific thing?
    Björn Grüning
    @bgruening
    @bramvandijk88_twitter yes Spades are requesting a lot of memory.
    @arcari_gabriele_twitter @eschen42 I try to answer during the day. But I'm currently swamped ... sorry
    Bram van Dijk
    @bramvandijk88_twitter

    @bramvandijk88_twitter yes Spades are requesting a lot of memory.

    Alright. Are there subscription services available to prioritise things?

    Helena Rasche
    @hexylena
    The subscription/paid option is currently: setup your own galaxy, e.g. using cloudman. (Judging by your name, SURFSara.nl might be an option for you if you're in NL. They have a galaxy environment.)
    Bram van Dijk
    @bramvandijk88_twitter
    Well, I'm in germany right now, but no worries. So the take home message is: there are services that provide this, but not here?
    Helena Rasche
    @hexylena
    There are not services that provide a "fast lane" in any public galaxy that I'm aware of. If you want such a fast lane, you generally need to setup, host, manage it yourself on some infrastructure you have access to.
    Bram van Dijk
    @bramvandijk88_twitter
    Alright, that's always an option too. Thanks
    Helena Rasche
    @hexylena
    I believe there is work being done to allow attaching your own compute, but it will be some years? before this is generally available.
    Bram van Dijk
    @bramvandijk88_twitter
    So, does it estimate beforehand how much memory I will use, or is spades just very lowly prioritised?
    Because I'm assembling relatively simple things...
    Helena Rasche
    @hexylena

    we need to schedule the job, so spades requests a set amount of memory before it can be scheduled.

    We do not have a good, accurate model yet for how much memory spades uses based on input settings, so, we set it to a high number that can accommodate the full range of assemblies being conducted on EU.

    Unfortunately that means for even small assemblies, spades is a "big" job that is hard to find space for in our a queue.

    https://github.com/usegalaxy-eu/infrastructure-playbook/blob/264a20ab8e3136ffc0b6c259f64396cc94b917f9/files/galaxy/dynamic_rules/usegalaxy/tool_destinations.yaml#L961 here is our file which defines how much memory / how many CPU cores each tool gets, and spades requests 20 cores and 400 GB of ram. These machines are in high demand for similar jobs and so it takes some time unfortunately :(
    EU is working towards modelling tools based on input sizes to guess at how much memory will be needed, but that work is some months away I think.
    Bram van Dijk
    @bramvandijk88_twitter
    Hmmm, well that's unfortunate. Megahit works too, but it assembles these things a bit less well.
    Different question: what do the checkboxes in the workflow-nodes actually do?
    Helena Rasche
    @hexylena
    It's a free service, we offer what we can to a wide audience :)
    the checkboxes control if the outputs will be shown following a workflow run, or hidden
    they let you check e.g. your final report only. So when the user runs the workflow, they see the jobs run + disappear, and only the final datasets that are interesting to them are left.
    Bram van Dijk
    @bramvandijk88_twitter

    Ah that's actually great. I couldn't figure this out sorry.

    Yeah, this is great for a free service. I think our institute would be willing to pay for dedicated servers though. You should consider this.

    Björn Grüning
    @bgruening
    @bramvandijk88_twitter we do. Please contact me on this issue.
    For the time being, the only spades jobs that I see queued have been actually resubmitted because they crashed.
    Helena Rasche
    @hexylena
    I guess in general it has been "instutites adding capacity for everyone" and not just specific groups/institutes but maybe EU's policies can be changed.
    Björn Grüning
    @bgruening
    Crash, most likely due to out of memory.
    Bram van Dijk
    @bramvandijk88_twitter

    For the time being, the only spades jobs that I see queued have been actually resubmitted because they crashed.

    Yeah, I cancelled the ones that weren't crashing about an hour ago

    Björn Grüning
    @bgruening

    I guess in general it has been "instutites adding capacity for everyone"

    That is still true.

    10 Spades jobs are currently running, requesting each 400GB memory.
    Bram van Dijk
    @bramvandijk88_twitter
    Both could be the case. I'm sure we can make all ships rise with the same tide.
    Oh really?
    I thought I cancelled them
    Also: it should take no more than 2 GB :')
    It's really a small number of reads
    Where can I find these jobs though? Or do you mean there not my jobs?
    they're (sorry, not native :')))
    Björn Grüning
    @bgruening
    Sorry, I meant overall spades jobs
    @bramvandijk88_twitter sorry that we are slow in responding, not a good week here.
    Bram van Dijk
    @bramvandijk88_twitter
    No worries, I totally understand.