These are chat archives for nextflow-io/nextflow

7th
Jun 2018
Maxime Garcia
@MaxUlysse
Jun 07 2018 08:51
@micans we split our pipeline as well, works quite well
micans
@micans
Jun 07 2018 09:06
@MaxUlysse splitting meaning having different repos? @Paolo what are your thoughts about having an option to invoke a different file than main.nf from the command line, similar to the -r option for branches (I know about the manifest setting)?
Alexander Peltzer
@apeltzer
Jun 07 2018 09:18
@micans No, same repository but several scripts for functionality. In principle that wuold make it easier to move to different repositories later too. Check out https://github.com/SciLifeLab/Sarek for some insight
@micans You can already do that nextflow run micans/blabla/test.nf .... should work IIRC
micans
@micans
Jun 07 2018 09:19
Ooooooo. Thanks :smile:
Alexander Peltzer
@apeltzer
Jun 07 2018 09:19
Np ;-)
micans
@micans
Jun 07 2018 09:22
(to both!)
Maxime Garcia
@MaxUlysse
Jun 07 2018 09:31
@apeltzer Thanks ;-)
micans
@micans
Jun 07 2018 12:49

I have a process that generates a file, and the next process should read lines from that file, say

process makefile {
    output:
    file('thefile') into mylines
   script:
   """
   echo -e "aaa\nbbb\nccc\n" > thefile
   """
}

I have trouble creating the next process; I can make a channel receiving the file, but how do I couple it to readLines? -- I have trouble getting the input directive to work.

Paolo Di Tommaso
@pditommaso
Jun 07 2018 12:54
what do you mean? does the downstream process be executed once for each line ?
micans
@micans
Jun 07 2018 12:54
yes
I have a case like that ..
Paolo Di Tommaso
@pditommaso
Jun 07 2018 12:54
input:
file x from mylines.splitText()
micans
@micans
Jun 07 2018 12:55
aha, thanks!
is that file x or val x?
Paolo Di Tommaso
@pditommaso
Jun 07 2018 12:56
file
micans
@micans
Jun 07 2018 12:56
I was expecting x to be a string corresponding to a single line. Is that what it is?
Paolo Di Tommaso
@pditommaso
Jun 07 2018 12:57
is up to you, if you want a file use file, if you want a string use val
should make sense :)
micans
@micans
Jun 07 2018 12:57
Testing it now ...
with fabulous bhaskara
micans
@micans
Jun 07 2018 13:19
I need to experiment interactively with groovy ... does nextflow come with a groovy interpreter as well?
Paolo Di Tommaso
@pditommaso
Jun 07 2018 13:19
nextflow console
micans
@micans
Jun 07 2018 13:20
wow :smile: this will help me hopefully fix some problems on my own
micans
@micans
Jun 07 2018 15:29
In the nextflow console, can I play with channels? At the moment I have type discrepancies; I have made a file('eee') in the console and I can use .readLines() on it; but .readLines() on a channel in NF does not work. splitText() does something different on a channel than it does on a file in the NF console.
micans
@micans
Jun 07 2018 15:50
OK, not perfect yet, but with val foo from mylines.splitText().map { it.trim() } at least everything goes through - the newline needed removing.
Ryo Kita
@rkita
Jun 07 2018 18:17
I have a process runGenes that is creating N files.
process runGenes {

   output:
   file "output*" into resultFiles

   """
   mycommand.py
   """
}
This process is running several times because it depends on another input. Is there a way to concatenate all of the files together from all of the processes?
Ryo Kita
@rkita
Jun 07 2018 18:29

Actually, I just got this to work:

process concatenateResults {
    input:
    file results from resultFiles.collect()

    output:
    stdout output
    file "total_results.txt" into all_results

    """
    echo $results
    cat $results >> total_results.txt
    """
}

But the problem now is that if the number of files created by the processes runGenes is large > 20000, then the cat statement becomes too large, and it breaks.. Is there a way to handle this?

Ryo Kita
@rkita
Jun 07 2018 18:40
Sorry, I just found this as the solution! https://www.nextflow.io/docs/latest/faq.html
Dave Istanto
@DaveIstanto
Jun 07 2018 20:20
Hi everyone, is there any way to exit nextflow after using watchPath? because like the doc says, it will keep watching that path even after the file appears in the path