and that both of those descriptions are well specified, standardized, and have multiple interoperable implementations, and conformance test suites
"a file format that uses spack to iteratively bootstrap any software from top-level description" would indeed be pretty awesome, even better if the specification were separate from the implementation so spack is an implementation of a standard
2 replies
i am super super glad that is the case, because (1) that file format is so far just a me idea and possibly not something spack is interested in alone (i haven't pitched the idea yet) and (2) i am very serious that spack (or any single tool) should not be the single method to satisfy such a bootstrapped setup, just as a personal conviction (because spack's goals may not align with everyone else's at some point)
i have a document i created which describes this vision in more detail. i was going to share it with the pex project (https://github.com/pantsbuild/pex, it's awesome), but haven't because the pex project tries pretty hard to maintain a specific focus on reproducible/hermetic python environments
Yep, I worked on Galaxy Dependency Resolvers some years back. Its a pluggable framework for going from tool + (optional) version specification to how to get a runnable version of the tool.
Any topics for today's CWL meeting? https://docs.google.com/document/d/1m81F4E1uqFT89th3ljUiJr4oJxJVp3dARyvYcZMj7_4/edit
:point_up: Edit: Any topics for today's CWL meeting? https://docs.google.com/document/d/1aWIgSaJPKaahjmLB4QalRAojSDH5OC1Ddi1i4q-JXgE/edit?usp=sharing
I think the Toil folks were talking about K8s at some point. Might be wrong.
CWL is a spec, so it is a question of what the individual projects that support CWL decide to do. https://github.com/Duke-GCB/calrissian is one that supports k8s
Sure, Thanks
Arvados has it on the roadmap, you can set up Arvados through a helm chart but it isn't really "kubernetes native" yet
there are probably others
StreamFlow (https://streamflow.di.unito.it) can offload tasks to a kubernetes cluster, but is not managed through Kubernetes manifests like argo. What it can do is deploy an helm chart, execute a CWL task on one of its pods and then undeploy the environment
Any topics for today's CWL meeting? https://docs.google.com/document/d/1WIryon-Q-3UPGbTDGCxrXJ1UP8sLAnhENTblnOjOgRU/edit
Hi CWL! I've been a little disconnected from gitter for a while, I noticed that there's a WDL -> CWL lab for Outreachy contributors which is awesome to see! It doesn't exactly solve all the things, but I've got a WDL parser to Janis (which you could then translate to CWL), it doesn't solve the exact described problem because it uses the MiniWDL parser, but thought I'd link it here for interest:
1 reply
An implementor had a question about the
dir3 test and the canonical order of the directory listings. I feel like this has been discussed before, but I can't find any references in the spec or the issue tracker. @tetron and others, what am I forgetting? https://github.com/common-workflow-language/cwl-v1.2/blob/main/tests/dir3.cwl
There is a job opening for people into bio workflows + cloud computing at University of Edinburgh: https://twitter.com/BioSScotland/status/1403407859442978821
4 replies
Hello. I have a question on the evaluation order of CWL CommandLineTool inputs. I've always supposed (maybe uncorrectly) that the order of evaluation of different input elements in the inputs field of a CommandLineTool should be undefined, i.e., input A should not rely on the value of input B to determine its value.
Now, in some CWL conformance tests it seems that an evaluation order must actually exist. For example, in test 101 one of the inputs (fasta_path) has a secondaryFiles spec which relies on the value of the other input (require_dat) through expressions. But why a CWL interpreter should evaluate require_dat before evaluating the secondaryFiles spec in fasta_path?
Hello glassofwhiskey , https://www.commonwl.org/v1.0/CommandLineTool.html#Running_a_Command has all the canonical answers to your questions. Let us know if it doesn't cover one of the scenarios you posted.
Whoops, I misread your comment. Let me think on this further
Yes unfortunately I didn't find anything on the official specification. I am currently terminating the StreamFlow journey to full CWL compatibility (25 tests failed out of 336), but those that remain are the trickiest ones XD
So secondaryFiles specs are evaluated after the value of ALL inputs has been evaluated? I was evaluating inputs independently in parallel at the moment
I just get a directory named
'$(inputs.species)'
I'm using CWL 1.1
4 replies
For your subdir situation, it might be simpler to just return an expression for the root
listing
1 reply
listing: ${ return [ {"class": "Directory", "basename": "exp", "listing": [ {"class": Directory", "basename": inputs.species} ] } ] }
today it will be directory soup. tomorrow it will be roasted javascript kale chips
Can an input value have multiple bindings in a CommandLineTool?
I solved it using two entries in
arguments that refer to the same input, but was wondering if the same result can be obtained via inputBinding
1 reply