Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    ok so the "format" is the Python code?
    John Chilton
    @jmchilton:matrix.org
    [m]
    that question is a bit too philosophical for me 😅, if I was forced to give a definition of the format I'd make it an empirical one based on what Galaxy runs.
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    yes that's kind of what I meant... so where is the schema actually used?
    John Chilton
    @jmchilton:matrix.org
    [m]
    linting, documentation, code generation for Java models
    pvanheusden
    @pvanheusden:matrix.org
    [m]
    aha, so it should co-evolve with the code for linting purposes... nod
    John Chilton
    @jmchilton:matrix.org
    [m]
    yes
    Marius
    @mvdbeek:matrix.org
    [m]
    We've got the montly WG call in 10 minutes: meet.google.com/myu-vozy-bsj
    wm75 (Wolfgang Maier)
    @wm75:matrix.org
    [m]
    Shoot, didn't realize. I joined another meeting.
    bgruening
    @bgruening:matrix.org
    [m]
    Carole just proposed we should be renaming to "Intergalactic Workflow Constellation" :)
    Simon Bray
    @simonbray:matrix.org
    [m]
    Question: does the attributes: {} in IWC tests for collection outputs have any purpose?
    Apart from causing tests to fail if it's not included, I mean 😛
    Marius
    @mvdbeek:matrix.org
    [m]
    Yeah, but I think we fixed that in a more recent galaxy-tool-util release ?!
    Is that still failing in CI ?
    Simon Bray
    @simonbray
    Ah, good to know, I am testing locally. I'll try updating galaxy-tool-util
    Simon Bray
    @simonbray:matrix.org
    [m]
    Hello, I'm currently encountering the following Planemo error for the last step of my workflow:
    Traceback (most recent call last):
      File "/home/simon/GitRepos/planemo/planemo/galaxy/activity.py", line 50, in execute
        return _execute(ctx, config, runnable, job_path, **kwds)
      File "/home/simon/GitRepos/planemo/planemo/galaxy/activity.py", line 219, in _execute
        run_response.collect_outputs(ctx, output_directory)
      File "/home/simon/GitRepos/planemo/planemo/galaxy/activity.py", line 366, in collect_outputs
        output_src = self.output_src(runnable_output)
      File "/home/simon/GitRepos/planemo/planemo/galaxy/activity.py", line 567, in output_src
        raise Exception("Failed to find output [%s] in invocation outputs [%s]" % (output_name, invocation["outputs"]))
    There is a corresponding message in the Galaxy logs: galaxy.workflow.run DEBUG 2021-10-27 17:46:24,479 [pN:main.web.1,p:11036,w:1,m:0,tN:WorkflowRequestMonitor.monitor_thread] Failed to find expected workflow output [outp_col] in step outputs [{'outp': <galaxy.model.HistoryDatasetAssociation(2228) at 0x7f4a0e35a668>, 'outp_frict': <galaxy.model.HistoryDatasetAssociation(2229) at 0x7f4a0e35aa20>}]
    Marius
    @mvdbeek:matrix.org
    [m]
    ah, nice, a reproducible case!
    Simon Bray
    @simonbray:matrix.org
    [m]
    Wondering if anyone has seen something similar to that before?
    Marius
    @mvdbeek:matrix.org
    [m]
    can you share this, we need to improve the error reporting
    yes
    there's a bunch of different possible causes, like filtered out outputs or bugs
    Simon Bray
    @simonbray:matrix.org
    [m]
    fwiw, maybe it's related, but whenever I open the workflow in the editor, it always tries to delete the final connection (to the problematic output) if I try and modify the tool
    anyway, I was planning to make an IWC PR, so I will push it there
    Marius
    @mvdbeek:matrix.org
    [m]
    yeah, that could well be possible
    Simon Bray
    @simonbray:matrix.org
    [m]
    Marius galaxyproject/iwc#68 if you want to have a look
    I can also try and create a more minimal case
    Marius
    @mvdbeek:matrix.org
    [m]
    that would be awesome
    Simon Bray
    @simonbray:matrix.org
    [m]
    it seems it isn't possible to add citations to a workflow, as it is for tools?
    would be a useful feature imo
    Marius
    @mvdbeek:matrix.org
    [m]
    I agree, can you open an issue for this on the galaxy repo ?
    Marius
    @mvdbeek:matrix.org
    [m]
    I'm prepping to look into the minimal workflow you posted Simon Bray , one thing I noticed is that gmx_sim isn't at the latest version. Might be totally unrelated of course
    Marius
    @mvdbeek:matrix.org
    [m]
    arg, I can't read, I just didn't install the right version it looks like
    Marius
    @mvdbeek:matrix.org
    [m]

    Simon Bray:

            <collection name="outp_col" type="list" label="Free energy data">
                <discover_datasets pattern="(?P&lt;designation&gt;^cluster\d+)_outp\.txt$" ext="tabular"/>
                <filter>
                    json
                </filter>
            </collection>
            <collection name="outp_frict_col" type="list" label="Friction data">
                <discover_datasets pattern="(?P&lt;designation&gt;^cluster\d+)_ofrict\.txt$" ext="tabular"/>
                <filter>
                    json
                </filter>
            </collection>
            <data name="outp" from_work_dir="outp.txt" format="tabular" label="Free energy data">
                <filter>
                    not json
                </filter>
            </data>
            <data name="outp_frict" from_work_dir="ofrict.txt" format="tabular" label="Friction data">
                <filter>
                    not json
                </filter>
            </data>

    that seems to be the problem, the workflow defines a workflow output outp_col, but it seems that output is being filtered out because there's no input connected to Ensemble subgroups (the json parameter). Does that make sense ? I think this might be one of the cases where the workflow editor cannot accurately predict the result of the filter

    Since otherwise you'd expect to have labelled outp, correct ?
    Simon Bray
    @simonbray:matrix.org
    [m]
    Argh, so simple??
    Thanks for having a look Marius!
    I will test it tomorrow on the real workflow, but I'm sure that's it
    Simon Bray
    @simonbray:matrix.org
    [m]
    Could I have a review of galaxyproject/iwc#68, when someone has time, from a technical, IWC point of view?
    Chris Barnett agreed to review it from the scientific side
    Marius
    @mvdbeek:matrix.org
    [m]
    I'm at the biohackathon, but I can take a look early next week
    Simon Bray
    @simonbray:matrix.org
    [m]
    that would be great, thanks Marius
    Marius
    @mvdbeek:matrix.org
    [m]
    If you want to get the tests passing you could take a look into either installing the tools on usegalaxy.org via https://github.com/galaxyproject/usegalaxy-tools or seeing if we can remove --no_shed_install from https://github.com/galaxyproject/planemo-ci-action/blob/main/planemo_ci_actions.sh#L6
    We could probably copy /cvmfs/main.galaxyproject.org/config/shed_tool_conf.xml to a writable location and then planemo would install the tools that are not available on cvmfs ... or that would be my hope
    Simon Bray
    @simonbray:matrix.org
    [m]
    hm, in the latter case you would switch to testing on a local galaxy, not on usegalaxy.org, right?
    Marius
    @mvdbeek:matrix.org
    [m]
    it is all local