These are chat archives for nextflow-io/nextflow

24th
Apr 2017
Mike Smoot
@mes5k
Apr 24 2017 19:50

Hi @pditommaso I'm back with another spread problem. Here is the code:

fasta_scaffolds_2
    .view{"just fasta_scaffolds_2 ${it}"}
    .into{ fasta_scaffolds_x2 }

blastable_fasta_db_2
    .view{"just blastable_fasta_db_2: ${it}"}
    .spread( fasta_scaffolds_x2 )
    .view{"spread fasta_scaffolds_2: ${it}"}

and here is the output generated:

just fasta_scaffolds_2 /local_scratch/msmoot/nextflow_eukaryotic_annotation/work/25/2e87c8bef2f1b685b325cad1698087/scaffolds.fa
just blastable_fasta_db_2: /local_scratch/msmoot/nextflow_eukaryotic_annotation/work/05/54adf0266175bd06332b3d29967fd3/blastable.fasta
spread fasta_scaffolds_2: [/local_scratch/msmoot/nextflow_eukaryotic_annotation/work/05/54adf0266175bd06332b3d29967fd3/blastable.fasta, local_scratch]
spread fasta_scaffolds_2: [/local_scratch/msmoot/nextflow_eukaryotic_annotation/work/05/54adf0266175bd06332b3d29967fd3/blastable.fasta, msmoot]
spread fasta_scaffolds_2: [/local_scratch/msmoot/nextflow_eukaryotic_annotation/work/05/54adf0266175bd06332b3d29967fd3/blastable.fasta, nextflow_eukaryotic_annotation]
spread fasta_scaffolds_2: [/local_scratch/msmoot/nextflow_eukaryotic_annotation/work/05/54adf0266175bd06332b3d29967fd3/blastable.fasta, work]
spread fasta_scaffolds_2: [/local_scratch/msmoot/nextflow_eukaryotic_annotation/work/05/54adf0266175bd06332b3d29967fd3/blastable.fasta, 25]
spread fasta_scaffolds_2: [/local_scratch/msmoot/nextflow_eukaryotic_annotation/work/05/54adf0266175bd06332b3d29967fd3/blastable.fasta, 2e87c8bef2f1b685b325cad1698087]
spread fasta_scaffolds_2: [/local_scratch/msmoot/nextflow_eukaryotic_annotation/work/05/54adf0266175bd06332b3d29967fd3/blastable.fasta, scaffolds.fa]

Basically what I'm seeing is that the file in fasta_scaffolds is being split by spread on the directory separator rather than being treated as a single file. I know this code used to work. Any ideas? I'm going to try to come up with a standalone example.

Paolo Di Tommaso
@pditommaso
Apr 24 2017 19:51
yes please, create an issue with a reproducible test case
Mike Smoot
@mes5k
Apr 24 2017 20:28

So, this code behaves exactly as expected and does not duplicate the error I see above:

Channel.from(file("/local_scratch/msmoot/nextflow_eukaryotic_annotation/work/25/2e87c8bef2f1b685b325cad1698087/scaffolds.fa")).into{ fasta_scaffolds_2 }
Channel.from(file("/local_scratch/msmoot/nextflow_eukaryotic_annotation/work/05/54adf0266175bd06332b3d29967fd3/blastable.fasta")).into{ blastable_fasta_db_2 }

fasta_scaffolds_2
    .view{"just fasta_scaffolds_2 ${it} ${it.getClass()}"}
    .into{ fasta_scaffolds_x2 }

blastable_fasta_db_2
    .view{"just blastable_fasta_db_2: ${it} ${it.getClass()}"}
    .spread( fasta_scaffolds_x2 )
    .view{"spread fasta_scaffolds_2: ${it}"}

Any debugging tips on how I can examine the channels involved? Would cached processes impact how the channels are created or the values in the channels? The pipeline where I'm seeing this error consistently does the wrong thing, so if I can instrument the code in some way I might be able to get to the bottom of what's going on.