These are chat archives for nextflow-io/nextflow

22nd
Mar 2018
Maxime Garcia
@MaxUlysse
Mar 22 2018 13:38
Hi, I'm trying to use the scratch directive
But it doesn't seems to work when I'm specifying process.scratch=true
Paolo Di Tommaso
@pditommaso
Mar 22 2018 13:39
it should
Maxime Garcia
@MaxUlysse
Mar 22 2018 13:42
OK, let me try again then, I probably made another mistake somewhere
Paolo Di Tommaso
@pditommaso
Mar 22 2018 13:43
check the .command.run
there should a variable NXF_SCRATCH defined
Maxime Garcia
@MaxUlysse
Mar 22 2018 13:47
it is
NXF_SCRATCH="$(set +u; nxf_mktemp $TMPDIR)"
te work directory should be on scratch too?
Paolo Di Tommaso
@pditommaso
Mar 22 2018 13:50
usually TMPDIR is a path in the local disk
work dir cannot
Maxime Garcia
@MaxUlysse
Mar 22 2018 13:51
ok
in this cluster TMPDIR is set to /scratch
Paolo Di Tommaso
@pditommaso
Mar 22 2018 13:51
and?
Maxime Garcia
@MaxUlysse
Mar 22 2018 13:52
so it should work
I'm trying to change the way we handled using /scratch, currently we're launching our nextflow command line in a single job on a single core
Paolo Di Tommaso
@pditommaso
Mar 22 2018 13:53
local executor ?
Maxime Garcia
@MaxUlysse
Mar 22 2018 13:54
currently yes
but I'm trying scratch with slurm
Paolo Di Tommaso
@pditommaso
Mar 22 2018 13:54
I'm missing the big picture ..
Maxime Garcia
@MaxUlysse
Mar 22 2018 13:56
Wait, I think I'm onto something
Sorry for the messy train of thoughts
Phil Ewels
@ewels
Mar 22 2018 14:01
Big picture: Currently running on a single node on scratch using local executor. This comes with known limitations - eg no restarting if stuff fails, limited ability to parallelise etc.
We want to run with slurm using scratch per-task instead. More overhead to copy to and from scratch for each task, but the above stuff works :+1:
Nothing special here I think
Paolo Di Tommaso
@pditommaso
Mar 22 2018 14:07
make sense, the scratch stuff is how HPC people want to do that
Maxime Garcia
@MaxUlysse
Mar 22 2018 14:40
OK, so it's working well
I was just too impatient and our queue very slow
sorry for everything
Paolo Di Tommaso
@pditommaso
Mar 22 2018 14:40
:+1:
Shellfishgene
@Shellfishgene
Mar 22 2018 15:48
So, I have a channel that emits A.bam B.bam C.bam. Next I want to run find_snps A.bam B.bam and find_snps A.bam C.bam. What's the best way of doing that?
Luca Cozzuto
@lucacozzuto
Mar 22 2018 15:55
flatten
on the input of the find_snps project
input:
    file aln from aligns.flatten()
no sorry
Luca Cozzuto
@lucacozzuto
Mar 22 2018 16:00
I did something similar with a config file that specifies the comparisons (A -> B and A->C)
you need to read that file and create a channel with combinations
Shellfishgene
@Shellfishgene
Mar 22 2018 16:02
That makes sense, thanks!
Luca Cozzuto
@lucacozzuto
Mar 22 2018 16:16
I don't remember where I got this from... :)
Channel
    .from(config.readLines())
    .map { line ->
        list = line.split("\t")
        sample = list[0]
        control = list[1]    
        [ sample, control ]
    }
    .set{ comparisons }
Shellfishgene
@Shellfishgene
Mar 22 2018 16:24
cool, thanks
Luca Cozzuto
@lucacozzuto
Mar 22 2018 17:12
Hi, I'm trying to install nextflow on my mac but I have a problem with java..
curl -s https://get.nextflow.io | bash
ERROR: Cannot find Java or it's a wrong version -- please make sure that Java 8 is installed
NOTE: Nextflow is trying to use the Java VM defined by the following environment variables:
 JAVA_CMD: /Library/Java/JavaVirtualMachines/jdk-10.jdk/Contents/Home//bin/java
 JAVA_HOME: /Library/Java/JavaVirtualMachines/jdk-10.jdk/Contents/Home/
do I need Java 8 and not 10?
Luca Cozzuto
@lucacozzuto
Mar 22 2018 17:33
yes it was wrong version :)