Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 16 15:15
    lgtm-com[bot] commented #63
  • Jun 16 15:02
    codecov[bot] commented #81
  • Jun 16 15:01
    codecov[bot] commented #81
  • Jun 16 14:58
    codecov[bot] commented #81
  • Jun 16 14:57
    dependabot[bot] synchronize #81
  • Jun 16 14:57

    dependabot[bot] on pip

    Bump mypy from 0.901 to 0.902 … (compare)

  • Jun 16 14:57
    dependabot[bot] edited #81
  • Jun 16 14:57
    mr-c synchronize #63
  • Jun 16 14:57

    mr-c on norm_tool

    fix typo Bump mypy from 0.812 to 0.901 … requests type hints are now in … and 5 more (compare)

  • Jun 16 14:57
    dependabot[bot] edited #81
  • Jun 16 14:57

    mr-c on cwl-utils-release

    (compare)

  • Jun 16 14:57

    mr-c on main

    update release test (compare)

  • Jun 16 14:57
    mr-c closed #84
  • Jun 16 14:55
    codecov[bot] commented #84
  • Jun 16 14:54
    mr-c auto_merge_enabled #84
  • Jun 16 14:54

    mr-c on v0.10

    (compare)

  • Jun 16 14:53
    mr-c opened #84
  • Jun 16 14:53

    mr-c on cwl-utils-release

    update release test (compare)

  • Jun 16 14:52

    mr-c on v0.10

    (compare)

  • Jun 16 14:52

    mr-c on cwl-utils-release

    update release test (compare)

Peter Amstutz
@tetron
coupled with a formal model of "here is a command to run, which modifies the environment in such a way that the requirement 'import somepackage' now succeeds" (which may have its own requirements, and is composable)
8 replies
and that both of those descriptions are well specified, standardized, and have multiple interoperable implementations, and conformance test suites
Peter Amstutz
@tetron
CWL is presently focused on functional workflows with each step running in a isolated, ephemeral environment, rather than coordinating a series of stateful modifications starting from a base environment, which is what a package manager does
2 replies
"a file format that uses spack to iteratively bootstrap any software from top-level description" would indeed be pretty awesome, even better if the specification were separate from the implementation so spack is an implementation of a standard
2 replies
Danny McClanahan
@cosmicexplorer
i am super super glad that is the case, because (1) that file format is so far just a me idea and possibly not something spack is interested in alone (i haven't pitched the idea yet) and (2) i am very serious that spack (or any single tool) should not be the single method to satisfy such a bootstrapped setup, just as a personal conviction (because spack's goals may not align with everyone else's at some point)
i have a document i created which describes this vision in more detail. i was going to share it with the pex project (https://github.com/pantsbuild/pex, it's awesome), but haven't because the pex project tries pretty hard to maintain a specific focus on reproducible/hermetic python environments
Danny McClanahan
@cosmicexplorer
i have turned the document into a gist, but i'm going to flesh out a section i hadn't finished yet first. it sounds like this is compatible with the project, which is awesome
pvanheusden
@pvanheusden:matrix.org
[m]
Yep, I worked on Galaxy Dependency Resolvers some years back. Its a pluggable framework for going from tool + (optional) version specification to how to get a runnable version of the tool.
crusoe
@mr-c:matrix.org
[m]
crusoe
@mr-c:matrix.org
[m]
Michael Kotliar
@michael-kotliar
@tetron @glassofwhiskey:matrix.org Thanks a lot for your answers! Probably, will come back to this question again somewhere in July when that laboratory get their computer:)
unikzforce
@unikzforce
Hey There everybody. As many of you may know the Tekton.dev or argo-proj out there ... Is there any plan to make alternative kubernetes native backends for CWL?
pvanheusden
@pvanheusden:matrix.org
[m]
I think the Toil folks were talking about K8s at some point. Might be wrong.
Peter Amstutz
@tetron:matrix.org
[m]
CWL is a spec, so it is a question of what the individual projects that support CWL decide to do. https://github.com/Duke-GCB/calrissian is one that supports k8s
unikzforce
@unikzforce
Yes, searched it right now. It seems good. I didn't know the Toil. Don't know how serious is that kubernetes Integration. Will check it Thanks
Sure, Thanks
Peter Amstutz
@tetron:matrix.org
[m]
Arvados has it on the roadmap, you can set up Arvados through a helm chart but it isn't really "kubernetes native" yet
there are probably others
unikzforce
@unikzforce
👍
glassofwhiskey
@glassofwhiskey:matrix.org
[m]
StreamFlow (https://streamflow.di.unito.it) can offload tasks to a kubernetes cluster, but is not managed through Kubernetes manifests like argo. What it can do is deploy an helm chart, execute a CWL task on one of its pods and then undeploy the environment
crusoe
@mr-c:matrix.org
[m]
Michael Franklin
@illusional
Hi CWL! I've been a little disconnected from gitter for a while, I noticed that there's a WDL -> CWL lab for Outreachy contributors which is awesome to see! It doesn't exactly solve all the things, but I've got a WDL parser to Janis (which you could then translate to CWL), it doesn't solve the exact described problem because it uses the MiniWDL parser, but thought I'd link it here for interest:
1 reply
crusoe
@mr-c:matrix.org
[m]
An implementor had a question about the dir3 test and the canonical order of the directory listings. I feel like this has been discussed before, but I can't find any references in the spec or the issue tracker. @tetron and others, what am I forgetting? https://github.com/common-workflow-language/cwl-v1.2/blob/main/tests/dir3.cwl
Peter Amstutz
@tetron
@mr-c:matrix.org the discussion that I remember is that the files are supposed to be consistently sorted based on the current locale's collation rules
4 replies
Rupert Nash
@rupertnash
There is a job opening for people into bio workflows + cloud computing at University of Edinburgh: https://twitter.com/BioSScotland/status/1403407859442978821
4 replies
Melclic
@Melclic
Hello CWL community! I would like to test CWL workflows and tools using test data (like unit testing). What do you guys recommend? Is there a way to specify test input and output data and perhaps perform some type of checksum? Thanks!
Peter Amstutz
@tetron:matrix.org
[m]
Melclic
@Melclic
I've come across that project, but cannot find any documentation on how to use it (full disclosure, I'm new to the CWL game)
4 replies
glassofwhiskey
@glassofwhiskey:matrix.org
[m]

Hello. I have a question on the evaluation order of CWL CommandLineTool inputs. I've always supposed (maybe uncorrectly) that the order of evaluation of different input elements in the inputs field of a CommandLineTool should be undefined, i.e., input A should not rely on the value of input B to determine its value.

Now, in some CWL conformance tests it seems that an evaluation order must actually exist. For example, in test 101 one of the inputs (fasta_path) has a secondaryFiles spec which relies on the value of the other input (require_dat) through expressions. But why a CWL interpreter should evaluate require_dat before evaluating the secondaryFiles spec in fasta_path?

crusoe
@mr-c:matrix.org
[m]
Hello glassofwhiskey , https://www.commonwl.org/v1.0/CommandLineTool.html#Running_a_Command has all the canonical answers to your questions. Let us know if it doesn't cover one of the scenarios you posted.
Whoops, I misread your comment. Let me think on this further
glassofwhiskey
@glassofwhiskey:matrix.org
[m]
Yes unfortunately I didn't find anything on the official specification. I am currently terminating the StreamFlow journey to full CWL compatibility (25 tests failed out of 336), but those that remain are the trickiest ones XD
Peter Amstutz
@tetron
@glassofwhiskey:matrix.org at that point in the evaluation, the inputs variable has the value of the input object that it was invoked with. the secondaryFiles spec in this case are assertions that certain secondaryFiles are present.
glassofwhiskey
@glassofwhiskey:matrix.org
[m]
So secondaryFiles specs are evaluated after the value of ALL inputs has been evaluated? I was evaluating inputs independently in parallel at the moment
Rupert Nash
@rupertnash
I need an IntialWorkDirRequirement that creates some empty directories with names that depend on input parameters. Am I correct in understanding that I can't set a directory's basename from an Expression?
crusoe
@mr-c:matrix.org
[m]
you totally can, yes
Rupert Nash
@rupertnash
Hmm
I just get a directory named '$(inputs.species)'
crusoe
@mr-c:matrix.org
[m]
entryname can be a CWL expression
Rupert Nash
@rupertnash
I was hoping that to be "aegypti" which was the value set for the species param
I'm using CWL 1.1
4 replies
crusoe
@mr-c:matrix.org
[m]
For your subdir situation, it might be simpler to just return an expression for the root listing
1 reply
crusoe
@mr-c:matrix.org
[m]
listing: ${ return [ {"class": "Directory", "basename": "exp", "listing": [ {"class": Directory", "basename": inputs.species} ] } ] }
Rupert Nash
@rupertnash
lovely - I take it this requires JS?
crusoe
@mr-c:matrix.org
[m]
Yes
Rupert Nash
@rupertnash
I like this new CWL debugging aaS
1 reply
Peter Amstutz
@tetron
the shop opens up for 30 minutes each day, at a different time every day, and serves a different meal every day
today it will be directory soup. tomorrow it will be roasted javascript kale chips
glassofwhiskey
@glassofwhiskey:matrix.org
[m]
Can an input value have multiple bindings in a CommandLineTool?
I solved it using two entries in arguments that refer to the same input, but was wondering if the same result can be obtained via inputBinding
1 reply