These are chat archives for nextflow-io/nextflow

15th
Feb 2018
Simone Baffelli
@baffelli
Feb 15 2018 10:51
Good morning.
Is there a way to get the current workdir in a process using shell mixed with native code? I tried inspecting the task variable but had no luck.
Does nextflow only create the workdir when the native code execution completes? task.workDir is null before the native code execution...
This message was deleted
Paolo Di Tommaso
@pditommaso
Feb 15 2018 11:14
now, that's not allowed (a batch scheduler can assign a workdir at runtime)
Simone Baffelli
@baffelli
Feb 15 2018 11:20
I see
chdem
@chdem
Feb 15 2018 15:34
good afternoon everyone !
I have a bit complex question as I'm using groovy code in a process to fill a List (java)
I want to put this list in the output Channel of my process
to get it back in the input process of the next process
Paolo Di Tommaso
@pditommaso
Feb 15 2018 15:39
Channel.value(your_list)
chdem
@chdem
Feb 15 2018 15:39
ok
ok, if my output is :
set val(customDbFilesList) into customDbFileChannel
and my List is customDbFilesList
where do I have to do the Channel.value(your_list)
Paolo Di Tommaso
@pditommaso
Feb 15 2018 15:44
oh, if so you don't need any special
just
val(customDbFilesList) into customDbFileChannel
not set
chdem
@chdem
Feb 15 2018 15:45
ah but of course what an idiot!! No set !!!
Paolo Di Tommaso
@pditommaso
Feb 15 2018 15:45
:)
chdem
@chdem
Feb 15 2018 15:47
Okay, I'm making progress.... Now I have a Missing value declared as output parameter: customDbFilesList
Why my list is considered as a parameter ?
It have been declared as a List outside of any process with List<String> customDbFilesList = new ArrayList<String[]>();
Paolo Di Tommaso
@pditommaso
Feb 15 2018 15:48
how is declared customDbFilesList
chdem
@chdem
Feb 15 2018 15:48
:smile:
Paolo Di Tommaso
@pditommaso
Feb 15 2018 15:49
do customDbFilesList = []
I hope that's an immutable list . .
chdem
@chdem
Feb 15 2018 15:51
what do you mean by immutable list ?
Paolo Di Tommaso
@pditommaso
Feb 15 2018 15:52
that you don't change the list content during the execution
chdem
@chdem
Feb 15 2018 15:52
there's only one execution of this process
so it could be condidered as an immutable list I think
Paolo Di Tommaso
@pditommaso
Feb 15 2018 15:54
:+1:
but if you declared it in the global scope, why are you using as a process output ?
chdem
@chdem
Feb 15 2018 15:55
this is a good question !! 'cause I need to wait for many process to finish with varFreqIndicatorChannel.last() as an input....
Paolo Di Tommaso
@pditommaso
Feb 15 2018 15:56
still I don't see how it can help
chdem
@chdem
Feb 15 2018 15:56
is that a good reason? You're making me doubt it. :smile:
Just to understand, if I'm putting code between 2 process, when is it executed ?
I think it's because I'm was not able to answer to this question that I'm using process :worried:
Paolo Di Tommaso
@pditommaso
Feb 15 2018 16:00
if I'm putting code between 2 process, when is it executed ?
what does it mean?
Edgar
@edgano
Feb 15 2018 16:05
If you are working with some input of other process, I will bet that this "middle" code is not executed until the process is finished (bc it will not have the output before it...)
chdem
@chdem
Feb 15 2018 16:06
ok, understood
@pditommaso my explanation was incomprehensible because I forgot the very principle of NextFlow.
:smile:
thank you @edgano for understanding the incomprehensible :fire:
Edgar
@edgano
Feb 15 2018 16:09
hahah thats the good part of be a noob to :P we have similar "problems" hahaha
chdem
@chdem
Feb 15 2018 16:14
Okay, still a behaviour that I don't understand....
this is my list creation in the script part :
customDbFilesList << [result.getLabel(),file(result.getFilePath()).baseName,result.getFilePath()].toArray();
my output, following @pditommaso advises :
output:
        val(customDbFilesList) into customDbFileChannel
and in the input in the following process :
input:
        set val(vepShortname), val(customDbBasename), file(customDbFile) from customDbFileChannel
but array is not well recognized 'cause I have input.1 in customDbFile and the complete array in the customDbBasename
chdem
@chdem
Feb 15 2018 16:19
(result.getLabel() and result.getFilePath() generate strings)
chdem
@chdem
Feb 15 2018 16:27
The console help me :
test = [[1,2,3],[4,5,6]]
Channel.from(test).subscribe { println "value1: $it" }
Channel.value(test).subscribe { println "value2: $it" }

value1: [1, 2, 3]
value1: [4, 5, 6]
value2: [[1, 2, 3], [4, 5, 6]]
I would like to have the "value1" behavior, like 'from' factory
Paolo Di Tommaso
@pditommaso
Feb 15 2018 16:31
Channel.value(test).flatMap().subscribe { println "value2: $it" }
Michael Hall
@mbhall88_twitter
Feb 15 2018 16:40
Hi everyone. New here. I have a question regarding using a Singularity container in Nextflow. Specifically I have my container setup with apps (as outlined in Singularity's Scientific Filesystem (SCIF) Apps).
To run an app in my container called foo I would run singularity exec --app foo singularity.simg echo "Hello world". My question is, how would I setup the config for Singularity in Nextflow in order to have the --app foo part added into the command. I believe Nextflow will effectively run singularity exec singularity.simg?
Paolo Di Tommaso
@pditommaso
Feb 15 2018 16:43
You are welcome Michael
What is doing in practice the --app option?
Michael Hall
@mbhall88_twitter
Feb 15 2018 16:53
it is calling an app inside my container called foo in this case. If you take a look at that link it gives a nice explanation. Effectively they're modular programs within a container. If you have two apps inside a container you can set environmental variables specifically for each. Also helps manage things like if app foo needs numpy<=1.2 but app bar needs numpy>=1.3 you can have separate installs in each app's environment and they dont clash.
I was thinking that potentially runOptions within the singularity scope might insert the --app foo piece between exec and singularity.simg?
Paolo Di Tommaso
@pditommaso
Feb 15 2018 16:55
NF has not support for it and frankly I don't think it's a good idea
At this time runOptions it's global setting, so you cannot use for that
Michael Hall
@mbhall88_twitter
Feb 15 2018 16:58
Why do you think it's not a good idea?
Paolo Di Tommaso
@pditommaso
Feb 15 2018 17:00
Because it's mixing container concept with a package manager
Michael Hall
@mbhall88_twitter
Feb 15 2018 17:01
Well that's not it's real purpose, I just picked a poor example.
Anyway, if it's not supported it's fine, I'll just have to explicitly type my singularity command in NF.
Thanks for the speedy response :smile:
Paolo Di Tommaso
@pditommaso
Feb 15 2018 17:03
:+1:
Tintest
@Tintest
Feb 15 2018 17:04

Hi, can you tell me, why I can not have my tool stdoud displayed ?

process bcl2fastq {

    publishDir "${params.resultDir}/bcl2fastq", mode: 'copy'

    output:
  stdout result
    file '*.fastq.gz' into fastq_list mode flatten
    file "SampleSheet-bcl2fastq.csv" into samplesheet_modif
    file 'Reports' into bcl2fastq_reports
    file 'Stats' into bcl2fastq_stats
    file "bcl2fastq.log" into bcl2fastq_log

    script:
    """
    bcl2fastq -R ${params.runDir} -o . --barcode-mismatches  ${params.missMatch} --sample-sheet SampleSheet-bcl2fastq.csv -r ${params.maxJob} -p ${params.maxJob} -w ${params.maxJob} &> bcl2fastq.log
    """

}


result.subscribe { println "$it" }

This code in my nextflow script is a "sub-nextflow" script called by a master one. The master one also does have a .subscribe method ... It is working with a simple process with an echo.

Thank you.

chdem
@chdem
Feb 15 2018 17:05
@pditommaso Thank you !!! :smile: :+1: