These are chat archives for nextflow-io/nextflow

9th
Apr 2019
Rad Suchecki
@rsuchecki
Apr 09 00:42
@veeravalli I expect it should work by definition, something along these lines
profiles {
  awsFoo {
    aws {
      accessKey = '<YOUR S3 ACCESS KEY>'
      secretKey = '<YOUR S3 SECRET KEY>'
      region = '<REGION IDENTIFIER>'
    } 
  }
  awsBar {
    aws {
      accessKey = '<YOUR S3 ACCESS KEY>'
      secretKey = '<YOUR S3 SECRET KEY>'
      region = '<REGION IDENTIFIER>'
    }
  }
}
Rad Suchecki
@rsuchecki
Apr 09 00:54
@fmorency beforeScript is bash log.info is NF - what are you trying to achieve?
Lavanya Veeravalli
@veeravalli
Apr 09 02:21
@rsuchecki yes, Thanks.
Rafal Gumienny
@guma44
Apr 09 09:19
Hi everybody, I would like to ask if there is a possibility do change the .nextflow directory which is located in the CWD? I run my pipeline with cron and I struggle with resuming it ie. resume is not working because (as I think) the cache is not found. I would like to set the .nextflow dir to be the same for each run or at least the cache dir to be the same. Thanks!
sureshhewa
@sureshhewabi
Apr 09 10:23
Hi All, If I am running 10 files in nextflow pipeline, say 3 files got failed to run. I need a report at the end of the pipeline reporting filename and the status of each 10 files. Is this possible in nextflow? any suggestions or any examples?
Paolo Di Tommaso
@pditommaso
Apr 09 10:24
@guma44 No, but you can move in the directory you whish and specificy instead the pipeline work for with the -w option
Rafal Gumienny
@guma44
Apr 09 11:48
@pditommaso Thanks! I would prefer some more systematic solution (like an option) but this could (in principle) solve this problem. I will give it a try.
Rad Suchecki
@rsuchecki
Apr 09 12:52
@sureshhewabi Assuming a process is executed for each of the input files, then a simple way of achieving that would be
  1. Use the tag directive in the relevant process e.g. tag{filename}
  2. Use -with-report and/or -with-trace and pick up the relevant info from the generated file(s)
sureshhewa
@sureshhewabi
Apr 09 12:56
Thanks @rsuchecki . Let me try
Félix C. Morency
@fmorency
Apr 09 13:13
@rsuchecki Got it figured out, thanks!
Chadi Saad
@chadisaad
Apr 09 13:47
Hello everyone. When I use $workflow.commitId, I'm getting 'Null'. Can you help me plz ?
it is a git repo
cloned locally
Maxime Garcia
@MaxUlysse
Apr 09 13:56
you can access workflow.commitId only when pulled from github
Chadi Saad
@chadisaad
Apr 09 13:56
oh ok. thanks :)
Maxime Garcia
@MaxUlysse
Apr 09 13:57
I'm using this snippet to get the same string:
def grabRevision() {
  // Return the same string executed from github or not
  return workflow.revision ?: workflow.commitId ?: workflow.scriptId.substring(0,10)
}
sureshhewa
@sureshhewabi
Apr 09 14:57
can I pass the nextflow log filename into the pipeline. I need to pass /path/of/the/dir/{accession}_nextflow.log ?
Stephen Kelly
@stevekm
Apr 09 17:46
@guma44 I have a pipeline that I run with cron here: https://github.com/NYU-Molecular-Pathology/lyz-nf can't remember if the .nextflow dir is one of the things I redirect to another location, but I know I redirect the work dir and other things. However its set up to not resume, so not sure how helpful that might be

Hi All, If I am running 10 files in nextflow pipeline, say 3 files got failed to run. I need a report at the end of the pipeline reporting filename and the status of each 10 files. Is this possible in nextflow? any suggestions or any examples?

@sureshhewabi

I ended up implementing my own logging solution for samples that failed, its in here:

https://github.com/NYU-Molecular-Pathology/NGS580-nf/blob/1cbba495437776daa3a69b1ba14cb3bfbc9e157a/main.nf#L2596

if you follow the trail of channels that starts with failed_samples you can see how I use it to collect samples that failed at different stages and then write messages about them to a common log file

Rad Suchecki
@rsuchecki
Apr 09 23:58
Are there any cluster-specific guidelines for installing NF? The standard user-install works fine fore me, but curious if it should be different to make it available as module?