common-workflow-language/common-workflow-language

https://cwl.discourse.group

Kaushik Ghose

@kaushik-work

@bernt-matthias For example this workflow https://github.com/genome/analysis-workflows/blob/2cdf6f9877661a7c2460f674947007ebb984799d/definitions/pipelines/alignment_exome.cwl imports common types used across the project.

Sorry, I read "types" where you said "parameters".

If I understand, this is like a set of globals that you want to use? Are these inputs?

Or did I understand correctly and "parameters" means inputs. In such a case these examples should work.

M Bernt

@bernt-matthias

Can be inputs / outputs

Kaushik Ghose

@kaushik-work

@bernt-matthias For example in https://github.com/genome/analysis-workflows/blob/2cdf6f9877661a7c2460f674947007ebb984799d/definitions/pipelines/alignment_exome.cwl there is a shared input type called ../types/labelled_file.yml which is imported and then referred to as

per_base_intervals:
        type: ../types/labelled_file.yml#labelled_file[]

M Bernt

@bernt-matthias

Not sure if this is really what I want. Lets say I have the parameters:

    bait_intervals:
        type: File
    final_name:
        type: string?
    target_intervals:
        type: File

which appear in many tools. So I want to define them in an external file which is then included in the tools.

Kaushik Ghose

@kaushik-work

Could you define them as a record type and import that?

Peter Amstutz

@tetron

@kaushik-work the binary distribution of benten for linux, shockingly (/s), has compatibility problems across linux distributions rabix/benten#110

Pjotr Prins

@pjotrp

@tetron you should consider GNU Guix. It is the only sane way of dealing with dependencies and versioning all the way down to glibc. We are using it with great effect.

with CWL

Not to start a war. It is one of those things. Around 2000 I told people not to ignore Linux. Hard to imagine today.

I am happy to help.

Michael R. Crusoe

@mr-c

I suspect that @tetron is trying to support users on many Linux distributions 😄

Peter Amstutz

@tetron

yes, that

Pjotr Prins

@pjotrp

It is hard. I do that with my tools too. People always want binaries they can download and run.

Static linking and/or Guix does the job for me.

Peter Amstutz

@tetron

specifically we're trying to distribute binaries of the benten CWL language server which is written in python, so we're using 'pyinstaller' which builds a binary distribution for your python program

Michael Franklin

@illusional

Sorry, running a little late. CWL APAC meeting starting now :)

Peter Amstutz

@tetron

@mr-c the weekly cwl meeting was deleted from the calendar probably because we had the conference scheduled, what did you want to do?

Michael R. Crusoe

@mr-c

@/all no CWL meeting this week ; take care and talk to you next Tuesday!

Michael R. Crusoe

@mr-c

More CWL workflows released from NCBI: https://github.com/ncbi/cwl-ngs-workflows-cbb/

Via https://doi.org/10.1093/gigascience/giaa141

Kaushik Ghose

@kaushik-work

:clap:

Pjotr Prins

@pjotrp

Excellent :)

pvanheus

@pvanheus

somehow that gigascience DOI url didn't work for me so: https://academic.oup.com/gigascience/article/10/1/giaa141/6067195?login=true

Michael R. Crusoe

@mr-c

Fixed DOI link, thank you @pvanheus https://doi.org/10.1093/gigascience/giaa141

Huh, they are the same, but it was slow to resolve. Maybe a DB issue at doi.org

Ward Vandewege

@cure

is there any interest in getting https://github.com/common-workflow-language/cwltool/compare/main...wtsi-hgi:mini_racer_js merged?

(much faster js engine)

Michael R. Crusoe

@mr-c:matrix.org

[m]

Is there a PR? Does it pass the conformance tests?

bogdang989

@bogdang989

Hi all, quick question regarding conformance test ids. I see that a bunch of tests have string IDs (e.g. all conditional tests, or this one https://github.com/common-workflow-language/cwl-v1.2/blob/main/conformance_tests.yaml#L331). I didn't really play with it so far does someone have exp, does that mess with usability of test subset definitions for cwltest, e.g. if wanting to specify a range of tests?

Michael R. Crusoe

@mr-c

@bogdang989 eventually the numerical IDs will go away, but you will always be able to specify tests by range, tag (and soon the label)

If you want to add testing-by-label to cwltest, that PR would be favorably received by me :-)

I think we have open issues documenting this plan

bogdang989

@bogdang989

Ah, sad to hear about numerical IDs, the "-n" option was really useful for me for quick checks and validations and also for easy communication. Would a PR for specifying tests by IDs also be acceptable? Not promising anything at all, just asking casually :) I'll most likely go with a quick script to re-enumerate all tests and dump a separate test file, as that fits into the system we had for v1.0/v1.1 testing. If I do that I can do a PR if it makes sense to have it anywhere at all within cwltest/cwl repos :)

Michael R. Crusoe

@mr-c:matrix.org

[m]

-n will always work for the (sub)set of tests selected

bogdang989

@bogdang989

oh, I presumed it would only work with integer ids (at least failed for me when wanting to run test with id cond-1, but I could be missing something cwltest --test conformance_tests.yaml -ncond-1 is what I optimistically attempted)

Michael R. Crusoe

@mr-c

@bogdang989 common-workflow-language/cwltest#110 :-)

so it will be -s for the label ; -n2 will still run the 2nd test

Kaushik Ghose

@kaushik-work

I have a Python package that needs a version of ruamel.yaml cwltool is unhappy with it. What is the typical workaround for this? On local I got away with installing via pipx. Sadly this messes up in github actions for some reason.

Peter Amstutz

@tetron

@kaushik-work is this benten or something else?

you can have multiple virtualenvs

put cwltool in one and the other package in the other, unless the other package is importing cwltool

is this requiring a version that's too new or too old?

Kaushik Ghose

@kaushik-work

Turns out it's an issue with pipx and ubuntu specifically. Filed a bug report: pipxproject/pipx#597

@tetron how would you call the cwltool in a different virtualenv? (pipx is my go to solution for this exactly because it takes care of all the plumbing, but I want to know how you would do this manually)

Peter Amstutz

@tetron

if you just need to run cwltool as an executable you just put the bin directory of the other virtualenv in PATH

Kaushik Ghose

@kaushik-work

:thumbsup:

bogdang989

@bogdang989

Awesome, thanks for the explanation @mr-c :) I mistakenly thought the -n was related to id, but this is even better 👍

Where communities thrive

People

Repo info

Activity