by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 10:19
    mr-c synchronize #1316
  • 10:17
    giannisdoukas synchronize #1316
  • Jul 09 21:25
    tetron synchronize #1320
  • Jul 09 21:25

    tetron on pack-mixed

    Pass packing tests (compare)

  • Jul 09 20:39
    pvanheus opened #1321
  • Jul 09 20:36
    tetron opened #1320
  • Jul 09 20:35

    tetron on pack-mixed

    Support packing mixed versions … Fix mypy & format (compare)

  • Jul 09 18:15

    mr-c on secondary_toil_s3

    (compare)

  • Jul 09 18:15

    mr-c on main

    fix for https://github.com/Data… (compare)

  • Jul 09 18:15
    mr-c closed #1319
  • Jul 09 17:33
    cwl-bot commented #608
  • Jul 09 17:06
    stain commented #608
  • Jul 09 17:06
    stain commented #608
  • Jul 09 17:05
    stain commented #608
  • Jul 09 11:25
    codecov[bot] commented #1319
  • Jul 09 11:25
    codecov[bot] commented #1319
  • Jul 09 11:23
    codecov[bot] commented #1319
  • Jul 09 11:22
    codecov[bot] commented #1319
  • Jul 09 11:21
    codecov[bot] commented #1319
Michael R. Crusoe
@mr-c
For restricting a particular value (like a string or a number) to a subset there is a proposal under development at common-workflow-language/common-workflow-language#764
@yangchoo Maybe you could join our next weekly video chat and introduce your project and we can discuss further?
@yangchoo Here are the notes from this week's call which has the time and date for the next meeting https://docs.google.com/document/d/1p9Ielg_Icxd2Nlyw5qReFXPn9LrrxgdhlVcs7_dwrf0/edit
2020-07-14 14:30 UTC
Yang
@yangchoo
thanks for the responses @mr-c!
Sounds good, that proposal looks promising, will leave comments there. And I'll be happy to join the call :)
Michael R. Crusoe
@mr-c
@yangchoo you are welcome! I look forward to hearing more on Tuesday
Jasper Koehorst
@jjkoehorst_gitlab
I am currently trying to get some javascript working in cwl and I am working under cwlVersion: v1.2.0-dev3 but some mapping is not accepted due to a version issue: cwls/assembly/spades_test.cwl:19:9: JSHINT: W119: 'arrow function syntax (=>)' is only available in ES6. CWL only supports ES5.1
is there a way to enable this support?
I noticed this issue: common-workflow-language/common-workflow-language#329 which is already open since 2016 but i was wondering if there is already any support?
Stian Soiland-Reyes
@stain
I think we just got EcmaScript 11 last month, - EcmaScript 6 was released in 2015 - is CWL 1.2 also bound to 5.1? Because of ISO/IEC 16262:2011?
we are trying to fix https://github.com/common-workflow-library/bio-cwl-tools/blob/release/spades/spades.cwl -- thinking EcmaScript 6 that generates yaml might be a bit better than ecmascript 5.1 that generates a shells script that generates a shell script that runs a Python script
Peter Amstutz
@tetron
we talked about it at several calls already, the spec needs to set a ceiling version, but we could add a vendor requirement that explicitly raises the ceiling version and then incorporate it in some form into the next CWL spec
somebody just needs to do the work
so a use could add requirements: {JavascriptVersionRequirement: {jsVersion: 6}}
Nathaniel Maki
@nathanielmki
A bit of a general question, but will versions of cwl scripts written in say, cwlVersion v1.0, work with 2.0, and future implementations?
Peter Amstutz
@tetron
@nathanielmki yes
the file declares its version so the runner is supposed to run it as that version
but you can also have a v1.1 workflow that includes v1.0 tools or the other way around (v1.0 workflow and v1.1 tools)
and then for the v1.x series in most cases you can trivially change the number from v1.0 to v1.1 to v1.2 etc and it'll still be valid
but v2.0 will likely have syntax-breaking changes so being able to run v1.0 from a v2.0 workflow would still work (as separate files), but being able to trivially change the version number on the file from v1.0 to v2.0 might not
Nathaniel Maki
@nathanielmki
Ok, thank you for clarifying! A goal of the organization I'm in is to have robust, reproducible, and portable bioinformatics tools and pipelines. We use Docker, and have begun to build our own tool-specific Docker images, as opposed to relying upon those made by others. CWL is the other component of that, and I just wanted to verify that scripts we write today in a certain version format will be able to be executed at some point down the line, regardless of how far CWL evolves.
Nathaniel Maki
@nathanielmki
Additionally, is there a preferred way to go about overriding the Docker mountpoints generated by CWLtool or to provide our own --volume options?
Peter Amstutz
@tetron
it depends on what you need to do
@mr-c I remember discussing that you could have full paths in InitialWorkDir to set mount points in Docker, I thought we were going to formalize that behavior but I can't find any documentation
(this might possibly be something that was added as a conformance test but not added into the text)
Peter Amstutz
@tetron
@nathanielmki it is an explicit goal of the CWL project that workflows written in CWL are can still be executed far in the future
@nathanielmki about mounting things in docker -- this would be a good topic for the forum https://cwl.discourse.group/
@mr-c @kaushik-work please add your thoughts common-workflow-language/cwl-v1.2#37
pvanheus
@pvanheus
@tetron I have noticed that my update to the bio-cwl-tools BWA-Index tool break when I switch version from 1.0 to 1.1 - I'll do a PR for the tool updates first though to get them reviewed before investigating this issue further.
Peter Amstutz
@tetron
@pvanheus validation error or runtime error?
pvanheus
@pvanheus
runtime error. validates fine.
pvanheus
@pvanheus
anyway relevant PR is here - not sure if my approach is the "correct approach": common-workflow-library/bio-cwl-tools#94
Peter Amstutz
@tetron
yea, I'm a little confused, the usual approach is for the reference FASTA to be the primary file and the index files to be the secondaryFiles
which is what the original code was doing I think
so I don't understand the problem that you're trying to solve
pvanheus
@pvanheus
well strictly the FASTA file is not part of the index
and the existing tools do not match up the secondaryFiles between the two tools - in fact the BWA-Index tool does not produce secondaryFiles.
Peter Amstutz
@tetron
it probably should?
pvanheus
@pvanheus
sorry, messed up git things so re-doing the PR
and yes it shoud.
should
pvanheus
@pvanheus
honestly I cannot actually get the existing BWA-Index.cwl to work and there is no tests/ directory with an example. Re those tests, do they need to be JSON or is YAML fine?
pvanheus
@pvanheus
anyway here is the new PR common-workflow-library/bio-cwl-tools#95 - and it all works in local testing (as individual tools or with a workflow) but if you flip the cwlVersion to v1.1 for the BWA-Index tool then the workflow throws a runtime error in the cwltool checker.
I guess this is a cwltool bug?
Peter Amstutz
@tetron
if cwltool crashes then that's a bug
pvanheus
@pvanheus
Jasper Koehorst
@jjkoehorst_gitlab

Is it possible to have 2 arrays as input and for each first element run a step and for the second element in each array run that step as well? As we have a workflow for input > quality > assembly in which quality accepts only 1 input file and the assembly accepts multiple input files. Meaning that if we have a paired set of input1 and input2 and another set of input3 and input 4 the inputs will be

forward input1, input3
reverse input2, input4

then the first group (1,2) runs through the quality and in parallel or separately (3,4) are sent through. Then the result of that are two arrays input1a,input3a and input2a,input4a which is then sent to the next application.

Thinking about this i might make things more complicated than it should be?

hmm i could generate a bash script for each input set
Jasper Koehorst
@jjkoehorst_gitlab
@stain pointed me towards scatter