These are chat archives for nextflow-io/nextflow

18th
Dec 2017
Tim Diels
@timdiels
Dec 18 2017 10:06
That's a good point as well. Coworker thinks JSON is more readable than YAML, so I can use all the points I can get :P. For one, I keep forgetting to add/remove , in JSON which is not a problem in YAML.
Paolo Di Tommaso
@pditommaso
Dec 18 2017 10:41
I hate both of them in the same manner :)
Phil Ewels
@ewels
Dec 18 2017 15:30
Hi @pditommaso! I'm trying to copy a static file using publishDir and having difficulties.. Code snippet:
process trim_galore {
    tag "$name"
    publishDir "${params.outdir}/trim_galore", mode: 'copy',
        saveAs: {filename ->
            if (filename.indexOf("_fastqc") > 0) "FastQC/$filename"
            else if (filename.indexOf("trimming_report.txt") > 0) "logs/$filename"
            else if (!params.saveTrimmed && filename == "where_are_my_files.txt") filename
            else params.saveTrimmed ? filename : null
        }

    input:
    set val(name), file(reads) from read_files_trimming
    file wherearemyfiles
the input wherearemyfiles is defined further up: wherearemyfiles= file("$baseDir/assets/where_are_my_files.txt")
it gets to the work directory fine, but isn't copied out to the publishDir
Can you see what I'm doing wrong?
Paolo Di Tommaso
@pditommaso
Dec 18 2017 15:31
else params.saveTrimmed ? filename : null
params.saveTrimmed may be false ?
Phil Ewels
@ewels
Dec 18 2017 15:31
that bit works fine - it's the else if (!params.saveTrimmed && filename == "where_are_my_files.txt") filename bit that isn't working
Paolo Di Tommaso
@pditommaso
Dec 18 2017 15:32
bit hard to debug
I would suggest to refactor that snippet to a method and add some logging
eg
Phil Ewels
@ewels
Dec 18 2017 15:32
so I'm trying to save the main output (trimmed FastQ in this case) to publishDir when params.saveTrimmed is True. When it's false I want a static file called where_are_my_files.txt to be copied
as people keep asking us where their intermediate BAM files are
Paolo Di Tommaso
@pditommaso
Dec 18 2017 15:33
def saveMyStuff(filename) {
            if (filename.indexOf("_fastqc") > 0) "FastQC/$filename"
            else if (filename.indexOf("trimming_report.txt") > 0) "logs/$filename"
            else if (!params.saveTrimmed && filename == "where_are_my_files.txt") filename
            else params.saveTrimmed ? filename : null
}
Phil Ewels
@ewels
Dec 18 2017 15:34
I don't think it's that code that's causing the problem
If I remove it all (eg. just publishDir "${params.outdir}/trim_galore", mode: 'copy') then where_are_my_files.txt still isn't copied out
do I have to reference it in the script block or something?
I guess that Nextflow is doing something clever to avoid copying out intermediate files somehow
Paolo Di Tommaso
@pditommaso
Dec 18 2017 15:35
wait, that's the name of the missing file?
Phil Ewels
@ewels
Dec 18 2017 15:35
where_are_my_files.txt
ah crap
Paolo Di Tommaso
@pditommaso
Dec 18 2017 15:36
have you declared as an output ?
Phil Ewels
@ewels
Dec 18 2017 15:36
needs to be in output
Paolo Di Tommaso
@pditommaso
Dec 18 2017 15:36
:clap:
Paolo Di Tommaso
@pditommaso
Dec 18 2017 15:36
:satisfied:
Phil Ewels
@ewels
Dec 18 2017 15:37
sorry - couldn't see it for looking!
Paolo Di Tommaso
@pditommaso
Dec 18 2017 15:37
no pb
Phil Ewels
@ewels
Dec 18 2017 15:37
yup, now it's working
ok ace, thanks!
Paolo Di Tommaso
@pditommaso
Dec 18 2017 15:37
:+1:
Phil Ewels
@ewels
Dec 18 2017 16:06
Working nicely now: SciLifeLab/NGI-RNAseq#185
Paolo Di Tommaso
@pditommaso
Dec 18 2017 16:11
nice to hear that