These are chat archives for nextflow-io/nextflow

27th
Oct 2017
rspreafico
@rspreafico
Oct 27 2017 01:30
@pditommaso I am trying to set the Docker registry at the individual process level, but it seems it's not supported, is that correct? I am already setting containers at the individual process level, and having the ability to set the registry as well would allow me to pull containers from different registries (e.g. a private registry and a public registry, depending on the process/tool).
Maxime Garcia
@MaxUlysse
Oct 27 2017 06:36
@rspreafico you can specify a container for each process, no problem at all
Use:```
process { $nameOfProcess.container = container }
Maxime Garcia
@MaxUlysse
Oct 27 2017 06:42
Not very good looking on the phone, but you can of course specify containers for each process, and even add conditions when specifying the container of you need
Paolo Di Tommaso
@pditommaso
Oct 27 2017 06:47
yes, the target registry makes part of the fully qualified container name eg quay.io/biocontainers/tophat
you can even use a private registry, however NF does not handle the registry authentication
@amacbride yes, you can use any groovy/java statement, maybe in your case you can create an helper method of even an helper class stored in the PROJECT/lib/ folder
Paolo Di Tommaso
@pditommaso
Oct 27 2017 06:54
however have a look at the syntax for dynamic directives https://www.nextflow.io/docs/latest/process.html#dynamic-computing-resources
Maxime Garcia
@MaxUlysse
Oct 27 2017 09:19
@pditommaso Hi, we sometimes have a problem with some process that hangs up forever on one of the cluster we're using, we've been investigting with UPPMAX people, but I think you could help us answer there latest questions SciLifeLab/CAW#420
mitul-patel
@mitul-patel
Oct 27 2017 09:53
Hello, I was wondering whether Nexflow can show the progress in percentage on command line.
Paolo Di Tommaso
@pditommaso
Oct 27 2017 09:53
quick answer, no
long answer, the streaming nature of NF makes very difficult to predict it
mitul-patel
@mitul-patel
Oct 27 2017 10:50
Thanks @pditommaso .
Paolo Di Tommaso
@pditommaso
Oct 27 2017 10:50
:+1:
Francesco Strozzi
@fstrozzi
Oct 27 2017 11:24

naive question: I am retrieving simple sample name + data folder from a TSV file in this way:

set val(sample),val(folder) from Channel.fromPath(params.samples).splitCsv(sep:'\t')

What is the best way to also define a file as a combination of the two val so that NF will also perform the staging of the files directly in the same task ? The data directory to stage- in is something like

${params.dir}/$folder/$sample
Paolo Di Tommaso
@pditommaso
Oct 27 2017 11:51
@fstrozzi the file path is a column in the tsv ?
Francesco Strozzi
@fstrozzi
Oct 27 2017 11:51
yes, but I need to combine that with the sample name. Fairly trivial, just I don’t know which can be the best way to do that
in NF
:)
Paolo Di Tommaso
@pditommaso
Oct 27 2017 11:52
you should chain a .map to the .splitCsv
Francesco Strozzi
@fstrozzi
Oct 27 2017 11:52
:+1:
Paolo Di Tommaso
@pditommaso
Oct 27 2017 11:53
to reformat the structure as you need
Francesco Strozzi
@fstrozzi
Oct 27 2017 12:03
thanks, that works nicely
Francesco Strozzi
@fstrozzi
Oct 27 2017 12:09
I keep forgetting that I can use groovy code everywhere in NF :)
Rickard Hammarén
@Hammarn
Oct 27 2017 13:18

@pditommaso is it possible to change process params like this:

--process.PullSingularityContainers.maxForks 1

I thought I remembered that it was possible but I can't seem to get the syntax right

Paolo Di Tommaso
@pditommaso
Oct 27 2017 13:19
nope sorry
it's possibile trough the config file, but not using the command line
Rickard Hammarén
@Hammarn
Oct 27 2017 13:19
That's too bad, I guess I can stop trying now then :D
Paolo Di Tommaso
@pditommaso
Oct 27 2017 13:20
in the config you can add
process.$PullSingularityContainers.maxForks =1
Maxime Garcia
@MaxUlysse
Oct 27 2017 14:13
@pditommaso Thanks for your help
Paolo Di Tommaso
@pditommaso
Oct 27 2017 14:13
welcome
Maxime Garcia
@MaxUlysse
Oct 27 2017 14:13
I think we're getting closer to an explanation
Paolo Di Tommaso
@pditommaso
Oct 27 2017 14:14
good
Luca Cozzuto
@lucacozzuto
Oct 27 2017 14:14
ehr... they just declare the independence... since I joint this chat there was a terrorist attack and a declaration of Independence :)
Paolo Di Tommaso
@pditommaso
Oct 27 2017 14:15
to me it looks always the same shit
rspreafico
@rspreafico
Oct 27 2017 16:15
@pditommaso Thanks for the suggestion! I will drop the registry parameter then and will use the container directive for both registry and container! Thanks again!