These are chat archives for nextflow-io/nextflow

30th
Nov 2018
Robin
@rduque1
Nov 30 2018 10:04
Hi, is it possible to see or print the final configuration used by nextflow after all config files where merged?
Paolo Di Tommaso
@pditommaso
Nov 30 2018 10:06
Nextflow config command
Alexander Peltzer
@apeltzer
Nov 30 2018 10:15
Does the fromFilePairs directive produce a sorted n-tuple ? I have multiple files (R1,R2, I1) that I'd like to access but need to do different things with these - do I need to use channel-ordering-logic outside of processes for this kind of thing?
Robin
@rduque1
Nov 30 2018 10:17
@pditommaso sorry I can't find it back in the doc, could you give the link. Thanks
Paolo Di Tommaso
@pditommaso
Nov 30 2018 10:17
The pair list yes, not the pairs sequence
Type nextflow config -h
Alexander Peltzer
@apeltzer
Nov 30 2018 10:18

The pair list yes, not the pairs sequence

So that means i can assume that my n-tuple contains first [blabla_R1.fastq.gz, blabla_R2.fastq.gz, blabla_I1.fastq.gz]

?
I think i don't understand this, but maybe I'll just try it with dump and figure out how it behaves
Paolo Di Tommaso
@pditommaso
Nov 30 2018 10:20
Yep
Robin
@rduque1
Nov 30 2018 10:37
@pditommaso thanks I get it now
Robin
@rduque1
Nov 30 2018 12:40
When executing nextflow with aws cloud are there some files from the home dir of the master that are copied to the workers?
I am asking because I would like to have ~/.docker/config.jsoncopied to the workers
Félix C. Morency
@fmorency
Nov 30 2018 15:22
@pditommaso I'm not sure if you answered or not about 481
Joe Brown
@brwnj
Nov 30 2018 15:51
I'm trying to debug an AWS Batch configuration. The instance and service roles both have S3FullAccess. The process executes and writes successfully to the working directory. When nextflow attempts to copy to publishDir the S3 location appears read-only. I'm not certain how to debug further, so any advice is helpful!
KochTobi
@KochTobi
Nov 30 2018 18:46

@brwnj I'm having the same issue. I can only think of two reasons why that might be.

  • the directory (in my case /tmp) was mounted read-only for some reason
  • there is no space left on the volume so the volume becomes read-only

I will try to increase my volume size and somehow check whether everything is in order in /proc/mounts.

Joe Brown
@brwnj
Nov 30 2018 19:05
@KochTobi It's not clear where the aws s3 cp command is being run as the container running the job has succeeded. Is it being run by the local machine executing nextflow run?
Joe Brown
@brwnj
Nov 30 2018 22:19
I figured out my issue. I was joining file paths so the server (s3://) was being clipped from my publishDir path.