These are chat archives for nextflow-io/nextflow

8th
May 2017
Anthony Underwood
@aunderwo
May 08 2017 11:08
Thanks @ewels , @pditommaso has responded and said that as of 0.24.x this should no longer be necessary. And hey presto it worked!
Thanks @pditommaso
Phil Ewels
@ewels
May 08 2017 12:11
:+1:
Anthony Underwood
@aunderwo
May 08 2017 13:28
When using nextflow and Docker how does nextflow decide how many parallel threads to run?
Paolo Di Tommaso
@pditommaso
May 08 2017 13:29
in the same manner how it does w/o docker
docker usage does not have any impact on parallelism
why do you think it should ?
Anthony Underwood
@aunderwo
May 08 2017 13:30
No - does it use a local executor therefore and use cpus as specified in the config?
Paolo Di Tommaso
@pditommaso
May 08 2017 13:31
yes of course
Anthony Underwood
@aunderwo
May 08 2017 13:32
OK - does it autodetect no. of cores by default?
Paolo Di Tommaso
@pditommaso
May 08 2017 13:32
yes
Anthony Underwood
@aunderwo
May 08 2017 13:35
Thanks - yep that makes sense my VM has 2 cores and nextflow with docker executed 2 processes in parallel :)
flowchart.png
love the flow chart feature btw :heart: :bar_chart:
Paolo Di Tommaso
@pditommaso
May 08 2017 13:35
ahah ! cool !
Anthony Underwood
@aunderwo
May 08 2017 13:42
Simple for now and tests most of the things I wanted to checkout. I have other code that splits files into separate pathways/channels based on contents of outputs. All looking v good. Next thing to figure out is how to orchestrate instantiation of docker hosts and pulling of versioned docker images :)
Paolo Di Tommaso
@pditommaso
May 08 2017 13:43
have a look to singularity
it spares you to manage the deployment of the docker daemons
Félix C. Morency
@fmorency
May 08 2017 13:44
+1
Paolo Di Tommaso
@pditommaso
May 08 2017 13:44
we have a lot of Singularity cheerleaders here ;)
Anthony Underwood
@aunderwo
May 08 2017 13:45
I just have to convince others that the relative unknown of Singularity is worth exploring given the hype around Docker :)
Paolo Di Tommaso
@pditommaso
May 08 2017 13:46
you don't have to convince, stress test both
you will see that's more stable and easier to maintain
this not mean that you have to get rid of Docker
Anthony Underwood
@aunderwo
May 08 2017 13:47
OK - good advice
Paolo Di Tommaso
@pditommaso
May 08 2017 13:47
develop with Docker and deploy with Singularity
Anthony Underwood
@aunderwo
May 08 2017 14:33
Just used docker2singularity to make singularity image and ran via singularity - seamless. And actually took slightly less time overall n=1
mitul-patel
@mitul-patel
May 08 2017 14:39
Hello, how do I stop being root while run script from nextflow.?
Maxime Garcia
@MaxUlysse
May 08 2017 14:40
@mitul-patel are you using docker ?
mitul-patel
@mitul-patel
May 08 2017 14:41
yes
This option will changes ownership of the files, once the docker process is over, but you can also pass option to docker to fix that during the process as it was discussed here last week, you should find this out if you go further up ;-)
mitul-patel
@mitul-patel
May 08 2017 14:53
Got it.... Thanks @MaxUlysse .... Dumb question.. where should I find nextflow.config file...or do I have to create one?
Maxime Garcia
@MaxUlysse
May 08 2017 15:23
you should create one in the root of your script
Definitively a good thing to look into
mitul-patel
@mitul-patel
May 08 2017 15:40
Many thanks @MaxUlysse .... I am looking into it...
Maxime Garcia
@MaxUlysse
May 08 2017 15:43
Good luck
;-)
mitul-patel
@mitul-patel
May 08 2017 15:56
Just out of interest...is it possible to call process from another process (script block) ?
Paolo Di Tommaso
@pditommaso
May 08 2017 15:56
nope
mitul-patel
@mitul-patel
May 08 2017 15:57
Thanks @pditommaso ..
Paolo Di Tommaso
@pditommaso
May 08 2017 15:57
unless you externalise your script with a template or a plain bash script file
mitul-patel
@mitul-patel
May 08 2017 15:58
you mean if i got template can I call another process from the same NF file?
Paolo Di Tommaso
@pditommaso
May 08 2017 15:59
you can share the same process command script
but you cannot have a process that invokes another process
mitul-patel
@mitul-patel
May 08 2017 16:18
Thanks @pditommaso ....
Paolo Di Tommaso
@pditommaso
May 08 2017 16:30
welcome
Shawn Rynearson
@srynobio
May 08 2017 18:35
Is there a way to specify (in .nf file or config) the number of jobs you want each node to process (slurm)? i.e some process you may give the entire node (bwa) but other (gzip) you may want to co-process 20 jobs.
Paolo Di Tommaso
@pditommaso
May 08 2017 18:54
not NF directly, but you can rely on slurm options provided it allows that
Shawn Rynearson
@srynobio
May 08 2017 19:15
Thanks for the reply!
Paolo Di Tommaso
@pditommaso
May 08 2017 19:16
:+1: