These are chat archives for nextflow-io/nextflow

7th
Apr 2016
Tiffany Delhomme
@tdelhomme
Apr 07 2016 06:34
Hi,
I have this input in a process set val(region_tag), file("${region_tag}.pileup") from pileup and I would like to add a when condition to execute the process. Actually I want to execute iff the input file is not empty, I do not find the syntax...
Also file("${region_tag}.pileup").empty() returns true for a not empty file...
Paolo Di Tommaso
@pditommaso
Apr 07 2016 07:16
instead of when I would suggest
set val(region_tag), file("${region_tag}.pileup") from pileup.filter { tag, file -> !file.isEmpty() }
Tiffany Delhomme
@tdelhomme
Apr 07 2016 07:18
good idea I will try
Paolo Di Tommaso
@pditommaso
Apr 07 2016 07:18
ok
Tiffany Delhomme
@tdelhomme
Apr 07 2016 07:20
It seems to work fine :clap:
Paolo Di Tommaso
@pditommaso
Apr 07 2016 07:20
great
Tiffany Delhomme
@tdelhomme
Apr 07 2016 07:24
why do I need tag, file and not only file in the filter syntax?
Paolo Di Tommaso
@pditommaso
Apr 07 2016 07:25
because the pipeup channel is emitting a pair of values
thus you need reference both of them
Tiffany Delhomme
@tdelhomme
Apr 07 2016 07:25
yes sure!
this does not work:
input:
file '*.vcf' from vcf.toList().filter { list -> !list.isEmpty() }
Paolo Di Tommaso
@pditommaso
Apr 07 2016 07:31
this, should work
input:
file '*.vcf' from vcf.filter { list -> !list.isEmpty() }.toList()
Tiffany Delhomme
@tdelhomme
Apr 07 2016 07:36
it does not seem to work
but this creates a list with no file isn't it?
Paolo Di Tommaso
@pditommaso
Apr 07 2016 07:37
what is vcf ?
a channel producing many files, right?
Tiffany Delhomme
@tdelhomme
Apr 07 2016 07:38
yes I think I see, vcf is a channel created by the previous process, but with my filter the previous process can not be executed and in this case vcf does not exist (this is the case 'm trying)
Paolo Di Tommaso
@pditommaso
Apr 07 2016 07:39
ahh
Tiffany Delhomme
@tdelhomme
Apr 07 2016 07:39
here I have to use when? because I can have in a case the channel existing and in another case the channel not existing
Paolo Di Tommaso
@pditommaso
Apr 07 2016 07:41
wait, in any case the filter should produce an empty (channel) result
do you have a runtime error or a logical problem?
Tiffany Delhomme
@tdelhomme
Apr 07 2016 07:46
I don't think so
Paolo Di Tommaso
@pditommaso
Apr 07 2016 07:47
not clear, anyway this is correct
input:
file '*.vcf' from vcf.filter { file -> !file.isEmpty() }.toList()
the problem should be somewhere else
Tiffany Delhomme
@tdelhomme
Apr 07 2016 07:49
this should work even if I do not have emit a vcf channel?
Paolo Di Tommaso
@pditommaso
Apr 07 2016 07:49
if you mean vcf produce nothing, yes
if you mean vcf is not declared no
Tiffany Delhomme
@tdelhomme
Apr 07 2016 07:50
yes I am dealing with the case where vcf is not decared...
Paolo Di Tommaso
@pditommaso
Apr 07 2016 07:52
but why it should not be declared?
Tiffany Delhomme
@tdelhomme
Apr 07 2016 08:00
actually I have 3 processes:
The first is based on your filter in input.
The second takes in input the output of the first.
The third takes in input the output the the second (this output is vcf).
If the first outputs nothing because it does not take anything in input (filter), the second will not be executed but the third will be (I can't understand why)...
and me I would like in this case to break the chain just after the second process.
Tell if me if it is not clear
Paolo Di Tommaso
@pditommaso
Apr 07 2016 08:05
I need to leave now, if you can share the code much better
Paolo Di Tommaso
@pditommaso
Apr 07 2016 10:07
even when it does produce nothing the channel emits an empty value that triggers the execution of the flow as expected.
if you are still experiencing this problem please provide an example that reproduce the problem
Tiffany Delhomme
@tdelhomme
Apr 07 2016 10:13
yes I am creating an easier example
Paolo Di Tommaso
@pditommaso
Apr 07 2016 10:14
open an issue on github eventually
Tiffany Delhomme
@tdelhomme
Apr 07 2016 10:25
ok I will
Jason Byars
@jbyars
Apr 07 2016 17:47
is there a doc/example anywhere on mid run clean up strategies with nextflow? I.E. after all of a given job's dependencies succeed, intermediate files are deleted.
I'm think of situations like GATK where you go through a lot intermediate bam's and vcf's before reaching the end of the workflow