These are chat archives for nextflow-io/nextflow

19th
Aug 2016
Johan Viklund
@viklund
Aug 19 2016 09:12
what where clause?
Paolo Di Tommaso
@pditommaso
Aug 19 2016 09:13
oops sorry I mean when
:)
updated
Johan Viklund
@viklund
Aug 19 2016 09:13
then the bug is that the var_check process works?
it's a syntax error that is not reported?
Paolo Di Tommaso
@pditommaso
Aug 19 2016 09:14
well, maybe you are right, it should be reported a syntax error in both cases
Johan Viklund
@viklund
Aug 19 2016 09:17
but yes, this worked like I expected:
process var_check { when: params.run script: """ echo """ }
hmpf
can't paste multiline code??
Thanks for the help
Paolo Di Tommaso
@pditommaso
Aug 19 2016 09:18
yes, that the reason why you need to start the script with a script:
Mike Smoot
@mes5k
Aug 19 2016 17:59
@pditommaso Hi Paolo, do you know much about the DNANexus executor for nextflow?
In particular, we're curious how data is managed or shared between nodes.
Mike Smoot
@mes5k
Aug 19 2016 18:23
To further clarify, assume that process A running on node A generates a fasta file that process B uses, which is running on node B. What's the behind-the-scenes magic that makes that fasta file appear as if it's in the work dir used by process B on node B?
Paolo Di Tommaso
@pditommaso
Aug 19 2016 18:36
@mes5k dnanexus has a storage somehow similar to AWS S3, thus file are produced locally and uploaded to a shared storage, and so on
to be honest the dnanexus executor is a three years old experimental feature, that we don't develop any more
Mike Smoot
@mes5k
Aug 19 2016 18:37
So, the work "directory" is basically an S3 bucket?
Is that transparent or do I need to make each file a s3 reference instead?
And it's good to know about the development status!
Paolo Di Tommaso
@pditommaso
Aug 19 2016 18:40
it has a similar approach but they have a proprietary protocol, it was planned also a POSIX shared file system but at that time never worked properly, frankly I have no idea of the current status
Mike Smoot
@mes5k
Aug 19 2016 18:41
Ok, good to know. We may be doing some work with them.
Thanks for the insight!
Paolo Di Tommaso
@pditommaso
Aug 19 2016 18:44
finally I've found it
this is the jsr203 java client for the dnanexus file system used by NF
Mike Smoot
@mes5k
Aug 19 2016 18:47
Cool, thanks!