Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Marius
    @mvdbeek:matrix.org
    [m]
    Yeah, but I think we fixed that in a more recent galaxy-tool-util release ?!
    Is that still failing in CI ?
    Simon Bray
    @simonbray
    Ah, good to know, I am testing locally. I'll try updating galaxy-tool-util
    Simon Bray
    @simonbray:matrix.org
    [m]
    Hello, I'm currently encountering the following Planemo error for the last step of my workflow:
    Traceback (most recent call last):
      File "/home/simon/GitRepos/planemo/planemo/galaxy/activity.py", line 50, in execute
        return _execute(ctx, config, runnable, job_path, **kwds)
      File "/home/simon/GitRepos/planemo/planemo/galaxy/activity.py", line 219, in _execute
        run_response.collect_outputs(ctx, output_directory)
      File "/home/simon/GitRepos/planemo/planemo/galaxy/activity.py", line 366, in collect_outputs
        output_src = self.output_src(runnable_output)
      File "/home/simon/GitRepos/planemo/planemo/galaxy/activity.py", line 567, in output_src
        raise Exception("Failed to find output [%s] in invocation outputs [%s]" % (output_name, invocation["outputs"]))
    There is a corresponding message in the Galaxy logs: galaxy.workflow.run DEBUG 2021-10-27 17:46:24,479 [pN:main.web.1,p:11036,w:1,m:0,tN:WorkflowRequestMonitor.monitor_thread] Failed to find expected workflow output [outp_col] in step outputs [{'outp': <galaxy.model.HistoryDatasetAssociation(2228) at 0x7f4a0e35a668>, 'outp_frict': <galaxy.model.HistoryDatasetAssociation(2229) at 0x7f4a0e35aa20>}]
    Marius
    @mvdbeek:matrix.org
    [m]
    ah, nice, a reproducible case!
    Simon Bray
    @simonbray:matrix.org
    [m]
    Wondering if anyone has seen something similar to that before?
    Marius
    @mvdbeek:matrix.org
    [m]
    can you share this, we need to improve the error reporting
    yes
    there's a bunch of different possible causes, like filtered out outputs or bugs
    Simon Bray
    @simonbray:matrix.org
    [m]
    fwiw, maybe it's related, but whenever I open the workflow in the editor, it always tries to delete the final connection (to the problematic output) if I try and modify the tool
    anyway, I was planning to make an IWC PR, so I will push it there
    Marius
    @mvdbeek:matrix.org
    [m]
    yeah, that could well be possible
    Simon Bray
    @simonbray:matrix.org
    [m]
    Marius galaxyproject/iwc#68 if you want to have a look
    I can also try and create a more minimal case
    Marius
    @mvdbeek:matrix.org
    [m]
    that would be awesome
    Simon Bray
    @simonbray:matrix.org
    [m]
    it seems it isn't possible to add citations to a workflow, as it is for tools?
    would be a useful feature imo
    Marius
    @mvdbeek:matrix.org
    [m]
    I agree, can you open an issue for this on the galaxy repo ?
    Marius
    @mvdbeek:matrix.org
    [m]
    I'm prepping to look into the minimal workflow you posted Simon Bray , one thing I noticed is that gmx_sim isn't at the latest version. Might be totally unrelated of course
    Marius
    @mvdbeek:matrix.org
    [m]
    arg, I can't read, I just didn't install the right version it looks like
    Marius
    @mvdbeek:matrix.org
    [m]

    Simon Bray:

            <collection name="outp_col" type="list" label="Free energy data">
                <discover_datasets pattern="(?P&lt;designation&gt;^cluster\d+)_outp\.txt$" ext="tabular"/>
                <filter>
                    json
                </filter>
            </collection>
            <collection name="outp_frict_col" type="list" label="Friction data">
                <discover_datasets pattern="(?P&lt;designation&gt;^cluster\d+)_ofrict\.txt$" ext="tabular"/>
                <filter>
                    json
                </filter>
            </collection>
            <data name="outp" from_work_dir="outp.txt" format="tabular" label="Free energy data">
                <filter>
                    not json
                </filter>
            </data>
            <data name="outp_frict" from_work_dir="ofrict.txt" format="tabular" label="Friction data">
                <filter>
                    not json
                </filter>
            </data>

    that seems to be the problem, the workflow defines a workflow output outp_col, but it seems that output is being filtered out because there's no input connected to Ensemble subgroups (the json parameter). Does that make sense ? I think this might be one of the cases where the workflow editor cannot accurately predict the result of the filter

    Since otherwise you'd expect to have labelled outp, correct ?
    Simon Bray
    @simonbray:matrix.org
    [m]
    Argh, so simple??
    Thanks for having a look Marius!
    I will test it tomorrow on the real workflow, but I'm sure that's it
    Simon Bray
    @simonbray:matrix.org
    [m]
    Could I have a review of galaxyproject/iwc#68, when someone has time, from a technical, IWC point of view?
    Chris Barnett agreed to review it from the scientific side
    Marius
    @mvdbeek:matrix.org
    [m]
    I'm at the biohackathon, but I can take a look early next week
    Simon Bray
    @simonbray:matrix.org
    [m]
    that would be great, thanks Marius
    Marius
    @mvdbeek:matrix.org
    [m]
    If you want to get the tests passing you could take a look into either installing the tools on usegalaxy.org via https://github.com/galaxyproject/usegalaxy-tools or seeing if we can remove --no_shed_install from https://github.com/galaxyproject/planemo-ci-action/blob/main/planemo_ci_actions.sh#L6
    We could probably copy /cvmfs/main.galaxyproject.org/config/shed_tool_conf.xml to a writable location and then planemo would install the tools that are not available on cvmfs ... or that would be my hope
    Simon Bray
    @simonbray:matrix.org
    [m]
    hm, in the latter case you would switch to testing on a local galaxy, not on usegalaxy.org, right?
    Marius
    @mvdbeek:matrix.org
    [m]
    it is all local
    just the tools and reference data come from cvmfs
    Simon Bray
    @simonbray:matrix.org
    [m]
    ah ok, that makes sense
    so you don't have to reinstall the tools each time
    Marius
    @mvdbeek:matrix.org
    [m]
    exactly ... although it also takes a long time to start up, if we load all the tools that we don't actually use, maybe we'd be better off to just have the reference data on cvmfs
    and test against live servers as separate jobs
    Helena Rasche
    @hexylena:matrix.org
    [m]
    Do we have anything for generating RO Crates from workflows? I've had a quick google and found nothing
    but the WorkflowHub people finally seem to have an API so maybe we can do that in the GTN
    wm75 (Wolfgang Maier)
    @wm75:matrix.org
    [m]
    Helena Rasche
    @hexylena:matrix.org
    [m]
    ahh future plans
    that might be just the ticket, thanks wm75 (Wolfgang Maier)
    oh and built in upload to WFH? fab