These are chat archives for nextflow-io/nextflow

20th
Oct 2017
Paolo Di Tommaso
@pditommaso
Oct 20 2017 07:03
The error reason is reported /bin/bash: .command.sh: Permission denied
Venkat Malladi
@vsmalladi
Oct 20 2017 12:11
@pditommaso I am running on slurm cluster, but I am failing to make .commond.env, but rest of config file is being read in
any idea why?

process {
executor = 'slurm'
queue='super'

// Process specific configuration
$checkDesignFile {
module = ['python/3.6.1-2-anaconda']
executor = 'local'
}
$trim_galore {
module = ['python/3.6.1-2-anaconda', 'trimgalore/0.4.1']
cpus = 32
}
$alignReads{
module = ['python/3.6.1-2-anaconda', 'bwa/intel/0.7.12', 'samtools/intel/1.3']
cpus = 32
}
$filterReads{
module = ['python/3.6.1-2-anaconda', 'samtools/1.4.1', 'sambamba/0.6.6', 'bedtools/2.26.0']
cpus = 32
}
$experimentQC {
module = ['python/3.6.1-2-anaconda', 'deeptools/2.5.0.1']
executor = 'local'
}
}

Paolo Di Tommaso
@pditommaso
Oct 20 2017 12:24
What do you mean, the module is not applied?
Venkat Malladi
@vsmalladi
Oct 20 2017 12:55
The .command.env file is not created for the process
Paolo Di Tommaso
@pditommaso
Oct 20 2017 12:56
that file is created only when needed, what problem is reported
Venkat Malladi
@vsmalladi
Oct 20 2017 12:57
no problem is reported, from what I can tell
so it fails at this process: $trim_galore {
module = ['python/3.6.1-2-anaconda', 'trimgalore/0.4.1']
cpus = 32
}
and doesn't load the python or trimgalore module when i should be requesting a new node for the process
Paolo Di Tommaso
@pditommaso
Oct 20 2017 13:00
and it doesn't create the .command.env for that task, right?
Venkat Malladi
@vsmalladi
Oct 20 2017 13:00
yep
Paolo Di Tommaso
@pditommaso
Oct 20 2017 13:01
what if you put at the end of you script
process.$trim_galore.module = ['python/3.6.1-2-anaconda', 'trimgalore/0.4.1']
Venkat Malladi
@vsmalladi
Oct 20 2017 13:02
where would I put this in the config?
Paolo Di Tommaso
@pditommaso
Oct 20 2017 13:03
at the end of the nextflow.config outside any scope
Venkat Malladi
@vsmalladi
Oct 20 2017 13:03
okay i will try that
it does currently work for when it automatically get invoked by gitlab.ci
Paolo Di Tommaso
@pditommaso
Oct 20 2017 13:04
ah
Venkat Malladi
@vsmalladi
Oct 20 2017 13:04
so wasn't sure why it wouldn't work when I try to run it as a different person
Paolo Di Tommaso
@pditommaso
Oct 20 2017 13:04
are you sure the config file is parsed when running from the CI ?
you can see that from the .nextflow.log file
Venkat Malladi
@vsmalladi
Oct 20 2017 13:05
ya the config file is parsed
and the correct .command.env files are created for each process
and I can see the process run in my slurm queue
Paolo Di Tommaso
@pditommaso
Oct 20 2017 13:06
and the correct .command.env files are created for each process
so why it is not working ?
Venkat Malladi
@vsmalladi
Oct 20 2017 13:08
it doesn't work for when i run it as different person with the default config
Paolo Di Tommaso
@pditommaso
Oct 20 2017 13:10
is the config file read in this case ?
Venkat Malladi
@vsmalladi
Oct 20 2017 13:11
ya it reads in the config of setting up the correct local and slurm executors, and makes the trace file that is specified in the config
Paolo Di Tommaso
@pditommaso
Oct 20 2017 13:19
this looks very strange
Venkat Malladi
@vsmalladi
Oct 20 2017 13:19
ya not sure what is going on
Paolo Di Tommaso
@pditommaso
Oct 20 2017 13:20
can you share the log files for both working and not working ?
Venkat Malladi
@vsmalladi
Oct 20 2017 13:20
ya how should I share them?
Paolo Di Tommaso
@pditommaso
Oct 20 2017 13:21
pastebin.com ?
Venkat Malladi
@vsmalladi
Oct 20 2017 13:21
ya works for me
Venkat Malladi
@vsmalladi
Oct 20 2017 13:31
Will post it once I get out of this meeting
Paolo Di Tommaso
@pditommaso
Oct 20 2017 13:31
no hurry
Venkat Malladi
@vsmalladi
Oct 20 2017 14:06
@pditommaso Nextflow CI: https://pastebin.com/r3F0p0hg
Paolo Di Tommaso
@pditommaso
Oct 20 2017 14:11
em, but this are two different pipelines
ah, no
Venkat Malladi
@vsmalladi
Oct 20 2017 14:11
just different locations
Paolo Di Tommaso
@pditommaso
Oct 20 2017 14:14
I see the process failing is named trimReads
instead you were reporting
$trim_galore {
module = ['python/3.6.1-2-anaconda', 'trimgalore/0.4.1']
cpus = 32
}
?
Venkat Malladi
@vsmalladi
Oct 20 2017 14:20
oh that would make sense
thanks for the catch
stupid user error
Paolo Di Tommaso
@pditommaso
Oct 20 2017 14:21
I will try to add a warning for such problem
Venkat Malladi
@vsmalladi
Oct 20 2017 14:21
thanks
i am supprised that it works
on my ci
Eric Davis
@davisem
Oct 20 2017 15:29
Wanted to discuss this issue:https://groups.google.com/forum/#!topic/nextflow/2fXRdcnV-W4. Basically, running the nextflow command via bsub on LSF fails to find the input files.
submitting the job from an interactive session works fine
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:30
do you still have the this path /cgs01/scratch_space/edavis5/work/0d/b4895579040f42c6313a865a2c8399/ ?
Eric Davis
@davisem
Oct 20 2017 15:31
yes
input.1 and input.2 are the only things in the folder
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:31
can you copy&paste the output of ls -la that folder
no hidden files ?
Eric Davis
@davisem
Oct 20 2017 15:31
yes those are there
sec getting interrupted
afk few mins
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:32
no pb
Eric Davis
@davisem
Oct 20 2017 15:40
sry bout that
back
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:41
can you copy&paste the output of ls -la that folder
Eric Davis
@davisem
Oct 20 2017 15:41
[edavis5(BASH)@c-head001]: ls -la
total 512
drwxr-xr-x 2 edavis5 zhanggrp  512 Oct 19 14:05 .
drwxr-xr-x 3 edavis5 zhanggrp  512 Oct 19 14:05 ..
-rw-r--r-- 1 edavis5 zhanggrp    0 Oct 19 14:05 .command.begin
-rw-r--r-- 1 edavis5 zhanggrp   42 Oct 19 14:05 .command.env
-rw-r--r-- 1 edavis5 zhanggrp   73 Oct 19 14:05 .command.err
-rw-r--r-- 1 edavis5 zhanggrp 2315 Oct 19 14:05 .command.log
-rw-r--r-- 1 edavis5 zhanggrp    0 Oct 19 14:05 .command.out
-rw-r--r-- 1 edavis5 zhanggrp 2194 Oct 19 14:05 .command.run
-rw-r--r-- 1 edavis5 zhanggrp   39 Oct 19 14:05 .command.sh
-rw-r--r-- 1 edavis5 zhanggrp    1 Oct 19 14:05 .exitcode
lrwxrwxrwx 1 edavis5 zhanggrp   79 Oct 19 14:05 input.1 -> /cgs01/scratch_space/edavis5/work/tmp/0f/5fd41a50c9b1fc77644216ebdbe4d5/input.1
lrwxrwxrwx 1 edavis5 zhanggrp   79 Oct 19 14:05 input.2 -> /cgs01/scratch_space/edavis5/work/tmp/82/a38b034ddf3afda53e449a8fdcebe5/input.2
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:41
umm wrap in triple ` then new-line
Eric Davis
@davisem
Oct 20 2017 15:41
haha yeah trying
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:42
:+1:
Eric Davis
@davisem
Oct 20 2017 15:42
used to slack
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:42
now, what the error message says ?
we are rustic people on gitter ;)
Eric Davis
@davisem
Oct 20 2017 15:43
it was saying file not found, referring to the inputs. I can't find the exact log
though in .command.err it says samtools index: "input.2" is in a format that cannot be usefully indexed
so im guessing it couldn't find the file
usually, in that folder, the dynamic name of the input file is present
but not in this case
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:45
the command you are trying to execute is
samtools index input.2
what if you try to execute this task as
bsub < .command.run
Eric Davis
@davisem
Oct 20 2017 15:47
it didn't throw up immediately, so that's interesting
oh but I am on a head node
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:48
running ?
Eric Davis
@davisem
Oct 20 2017 15:48
seems to be
but I feel like I should be on a worker node, again doing work on the head node is frowned upon
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:49
in principle there should be difference
NF is doing exactly the same, for each task run that command
Eric Davis
@davisem
Oct 20 2017 15:49
I agree!
where would I find that output?
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:51
in the same dir
Eric Davis
@davisem
Oct 20 2017 15:51
I don't see it, and I don't see anything active under bjobs
and I didn't get an "it crashed" email
weird
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:52
can you copy and paste the ls -la again ?
Eric Davis
@davisem
Oct 20 2017 15:53
[edavis5(BASH)@nodec003]: ls -la
total 768
drwxr-xr-x 2 edavis5 zhanggrp 131072 Oct 20 10:49 .
drwxr-xr-x 3 edavis5 zhanggrp    512 Oct 19 14:05 ..
-rw-r--r-- 1 edavis5 zhanggrp      0 Oct 20 10:49 .command.begin
-rw-r--r-- 1 edavis5 zhanggrp     42 Oct 19 14:05 .command.env
-rw-r--r-- 1 edavis5 zhanggrp     73 Oct 20 10:49 .command.err
-rw-r--r-- 1 edavis5 zhanggrp   6946 Oct 20 10:49 .command.log
-rw-r--r-- 1 edavis5 zhanggrp      0 Oct 20 10:49 .command.out
-rw-r--r-- 1 edavis5 zhanggrp   2194 Oct 19 14:05 .command.run
-rw-r--r-- 1 edavis5 zhanggrp     39 Oct 19 14:05 .command.sh
-rw-r--r-- 1 edavis5 zhanggrp      1 Oct 20 10:49 .exitcode
lrwxrwxrwx 1 edavis5 zhanggrp     79 Oct 20 10:49 input.1 -> /cgs01/scratch_space/edavis5/work/tmp/0f/5fd41a50c9b1fc77644216ebdbe4d5/input.1
lrwxrwxrwx 1 edavis5 zhanggrp     79 Oct 20 10:49 input.2 -> /cgs01/scratch_space/edavis5/work/tmp/82/a38b034ddf3afda53e449a8fdcebe5/input.2
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:53
umm
what's the content of .command.log
Eric Davis
@davisem
Oct 20 2017 15:55
Sender: LSF System <lsfadmin@nodec038>
Subject: Job 246473: <nf-IndexNormal_(1)> in cluster <hpcf_cluster_clinical> Exited

Job <nf-IndexNormal_(1)> was submitted from host <nodec033> by user <edavis5> in cluster <hpcf_cluster_clinical>.
Job was executed on host(s) <nodec038>, in queue <normal>, as user <edavis5> in cluster <hpcf_cluster_clinical>.
</home/edavis5> was used as the home directory.
</cgs01/scratch_space/edavis5/work/0d/b4895579040f42c6313a865a2c8399> was used as the working directory.
Started at Thu Oct 19 14:05:29 2017
Results reported at Thu Oct 19 14:05:30 2017

Your job looked like:

------------------------------------------------------------
# LSBATCH: User input
#!/bin/bash
#BSUB -o /cgs01/scratch_space/edavis5/work/0d/b4895579040f42c6313a865a2c8399/.command.log
#BSUB -J nf-IndexNormal_(1)

# NEXTFLOW TASK: IndexNormal (1)
set -e
set -u
NXF_DEBUG=${NXF_DEBUG:=0}; [[ $NXF_DEBUG > 1 ]] && set -x

nxf_env() {
    echo '============= task environment ============='
    env | sort | sed "s/\(.*\)AWS\(.*\)=\(.\{6\}\).*/\1AWS\2=\3xxxxxxxxxxxxx/"
    echo '============= task output =================='
}

nxf_kill() {
    declare -a ALL_CHILD
    while read P PP;do
        ALL_CHILD[$PP]+=" $P"
    done < <(ps -e -o pid= -o ppid=)

    walk() {
        [[ $1 != $$ ]] && kill $1 2>/dev/null || true
        for i in ${ALL_CHILD[$1]:=}; do walk $i; done
    }

    walk $1
}

nxf_mktemp() {
    local base=${1:-/tmp}
    if [[ $(uname) = Darwin ]]; then mktemp -d $base/nxf.XXXXXXXXXX
    else TMPDIR="$base" mktemp -d -t nxf.XXXXXXXXXX
    fi
}

on_exit() {
  exit_status=${ret:=$?}
  printf $exit_status > /cgs01/scratch_space/edavis5/work/0d/b4895579040f42c6313a865a2c8399/.exitcode
  set +u
  [[ "$COUT" ]] && rm -f "$COUT" || true
  [[ "$CERR" ]] && rm -f "$CERR" || true
  exit $exit_status
}

on_term() {
    set +e
    [[ "$pid" ]] && nxf_kill $pid
}


(... more ...)
------------------------------------------------------------

Exited with exit code 1.

Resource usage summary:

    CPU time :               0.05 sec.
    Total Requested Memory : 3800.00 MB
    Delta Memory :           -
    (Delta: the difference between total requested memory and actual max usage.)

The output (if any) follows:

samtools index: "input.2" is in a format that cannot be usefully indexed
Sender: LSF System <lsfadmin@nodec033>
Subject: Job 247435: <nf-IndexNormal_(1)> in cluster <hpcf_cluster_clinical> Exited

Job <nf-IndexNormal_(1)> was submitted from host <c-head001> by user <edavis5> in cluster <hpcf_cluster_clinical>.
Job was executed on host(s) <nodec033>, in queue <normal>, as user <edavis5> in cluster <hpcf_cluster_clinical>.
</home/edavis5> was used as the home directory.
</cgs01/scratch_space/edavis5/work/0d/b4895579040f42c6313a865a2c8399> was used as the working directory.
Started at Fri Oct 20 10:46:32 2017
Results reported at Fri Oct 20 10:46:33 2017

Your job looked like:

------------------------------------------------------------
# LSBATCH: User input
#!/bin/bash
#BSUB -o /cgs01/scratch_space/edavis5/work/0d/b4895579040f42c6313a865a2c8399/.command.log
#BSUB -J nf-IndexNormal_(1)

# NEXTFLOW TASK: IndexNormal (1)
set -e
set -u
NXF_DEBUG=${NXF_DEBUG:=0}; [[ $NXF_DEBUG > 1 ]] && set -x

nxf_env() {
    echo '============= task environment ============='
    env | sort | sed "s/\(.*\)AWS\(.*\)=\(.\{6\}\).*/\1AWS\2=\3xxxxxxxxxxxxx/"
    echo '============= task output =================='
}

nxf_kill() {
    declare -a ALL_CHILD
    while read P PP;do
        ALL_CHILD[$PP]+=" $P"
    done < <(ps -e -o pid= -o ppid=)

    walk() {
        [[ $1 != $$ ]] && kill $1 2>/dev/null || true
        for i in ${ALL_CHILD[$1]:=}; do walk $i; done
    }

    walk $1
}

nxf_mktemp() {
    local base=${1:-/tmp}
    if [[ $(uname) = Darwin ]]; then mktemp -d $base/nxf.XXXXXXXXXX
    else TMPDIR="$base" mktemp -d -t nxf.XXXXXXXXXX
    fi
}

on_exit() {
  exit_status=${ret:=$?}
  printf $exit_status > /cgs01/scratch_space/edavis5/work/0d/b4895579040f42c6313a865a2c8399/.exitcode
  set +u
  [[ "$COUT" ]] && rm -f "$COUT" || true
  [
I think it truncated some
[[ "$CERR" ]] && rm -f "$CERR" || true
  exit $exit_status
}

on_term() {
    set +e
    [[ "$pid" ]] && nxf_kill $pid
}


(... more ...)
------------------------------------------------------------

Exited with exit code 1.

Resource usage summary:

    CPU time :               0.05 sec.
    Total Requested Memory : 3800.00 MB
    Delta Memory :           -
    (Delta: the difference between total requested memory and actual max usage.)

The output (if any) follows:

samtools index: "input.2" is in a format that cannot be usefully indexed
should be it
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:56
it's not a samtools error ?
Eric Davis
@davisem
Oct 20 2017 15:57
I think eventually it is a samtools error, only because its not finding the file
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:57
do this
edit the .command.run and add
cd /cgs01/scratch_space/edavis5/work/0d/b4895579040f42c6313a865a2c8399
on top of the script just after the last #BSUB .. line
then to execute again with bsub < .command.run
Eric Davis
@davisem
Oct 20 2017 15:59
ok so I added it here
#!/bin/bash
#BSUB -o /cgs01/scratch_space/edavis5/work/0d/b4895579040f42c6313a865a2c8399/.command.log
#BSUB -J nf-IndexNormal_(1)
cd /cgs01/scratch_space/edavis5/work/0d/b4895579040f42c6313a865a2c8399
# NEXTFLOW TASK: IndexNormal (1)
set -e
set -u
Paolo Di Tommaso
@pditommaso
Oct 20 2017 15:59
exactly
Eric Davis
@davisem
Oct 20 2017 16:00
:+1:
what am I checking for?
same error is there
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:04
same error ?
Eric Davis
@davisem
Oct 20 2017 16:04
yes
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:04
if so this has to be a samtools related error
Eric Davis
@davisem
Oct 20 2017 16:05
I just repeated this for a sanity check
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:05
make sure that the content is not corrupted and that the tools is not file name extension sensitive
I would not exclude that
Eric Davis
@davisem
Oct 20 2017 16:06
and that could explain the bsub vs no bsub?
[params.tumor_normal_pair] failed to open: 'p-sequenza/SJHGG003_A_10000.bam' : No such file or directory
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:07
what's this ?
Eric Davis
@davisem
Oct 20 2017 16:07
I guess this is the stdout error message from nextflow?
so the no such file, error is thrown first, and then samtools crashes later in the log
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:08
I have no idea where this error is coming from
Eric Davis
@davisem
Oct 20 2017 16:08

The output (if any) follows:

N E X T F L O W  ~  version 0.25.7
Launching `p-sequenza/sequenza_v2.nf` [condescending_bernard] - revision: b3244510a7

[params.tumor_normal_pair] failed to open: 'p-sequenza/SJHGG003_A_10000.bam' : No such file or directory
[warm up] executor > lsf
WARN: Input tuple does not match input set cardinality declared by process `IndexNormal` -- offending value: null
WARN: Input tuple does not match input set cardinality declared by process `IndexTumor` -- offending value: null
[32/88f80d] Submitted process > IndexTumor (1)
[94/4fdb7f] Submitted process > IndexNormal (1)
ERROR ~ Error executing process > 'IndexTumor (1)'

Caused by:
  Process `IndexTumor (1)` terminated with an error exit status (1)

Command executed:

  samtools index input.1

Command exit status:
  1

Command output:
  (empty)

Command error:
  samtools index: "input.1" is in a format that cannot be usefully indexed

Work dir:
  /cgs02/home/edavis5/work/32/88f80d55c4791bf1cf928aa734785a

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

-- Check '.nextflow.log' file for details
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:09
is this the complete output ?
Eric Davis
@davisem
Oct 20 2017 16:09
sorry
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:10
[params.tumor_normal_pair] failed to open: 'p-sequenza/SJHGG003_A_10000.bam' : No such file or directory
Eric Davis
@davisem
Oct 20 2017 16:11
So, I can do bsub samtools index p-sequenza/SJHGG003_A_10000.bam
that works fine
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:11
this message is weird, but doesn't look like a NF message
unless there's something weird in your code
WARN: Input tuple does not match input set cardinality declared by process IndexNormal -- offending value: null
also this doesn't look good
is there a public version of your code?
Eric Davis
@davisem
Oct 20 2017 16:12
so I get that error message every time I pass it the wrong glob expression
sadly no
I wish
the code works under every circumstance, except when executed with bsub
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:14
first, I would try to understand why the samtool task is not working as expected when running bsub < .command.run
that's the main problem
Eric Davis
@davisem
Oct 20 2017 16:14
ok
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:14
a sysadmin could help
second, I would try to identify what line in your code is producing that error
Eric Davis
@davisem
Oct 20 2017 16:15
thats complicated
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:15
[params.tumor_normal_pair] failed to open: 'p-sequenza/SJHGG003_A_10000.bam' : No such file or directory
Eric Davis
@davisem
Oct 20 2017 16:15
ok
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:15
third, you need to solve also
WARN: Input tuple does not match input set cardinality declared by process IndexNormal -- offending value: null
Eric Davis
@davisem
Oct 20 2017 16:15
Channel
    .fromFilePairs(params.tumor_normal_pair, flat:true)
    .ifEmpty { return tumor_normal_pair_error(params.tumor_normal_pair) }
    .into{tumor_sample; normal_sample}
?
I'm guessing it's this
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:16
oook
I guess tumor_normal_pair_error is printing that message
Eric Davis
@davisem
Oct 20 2017 16:16
indeed
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:16
but should run or stop the pipeline in that condition ?
Eric Davis
@davisem
Oct 20 2017 16:17
*should
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:17
what ?
Eric Davis
@davisem
Oct 20 2017 16:17
it does not
I should have it return 1
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:17
tumor_normal_pair_error is supposed to return the error message string ?
Eric Davis
@davisem
Oct 20 2017 16:17
yes
it just prints it
it doesn't actually return anything
def tumor_normal_pair_error(def input) {
    println ""
    println "[params.tumor_normal_pair] failed to open: '" + input + "' : No such file or directory"
}
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:18
hence, you need to use error method
like this
Eric Davis
@davisem
Oct 20 2017 16:18
bad design on my part
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:18
Channel
    .fromFilePairs(params.tumor_normal_pair, flat:true)
    .ifEmpty { error tumor_normal_pair_error(params.tumor_normal_pair) }
    .into{tumor_sample; normal_sample}
Eric Davis
@davisem
Oct 20 2017 16:18
ah
nice
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:18
this explain a lot
Eric Davis
@davisem
Oct 20 2017 16:19
in a good way?
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:20
when you where running with bsub the pipeline is running in a different directory, and fromFilePairs(params.tumor_normal_pair, flat:true) is not finding the files you are passing I guess with a relative path
Eric Davis
@davisem
Oct 20 2017 16:20
that makes sense
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:21
indeed
nextflow run /home/edavis5/p-sequenza/sequenza_v2.nf --genome_gc GRCh37-lite.fa.gc50Base.txt.gz --genome GRCh37-lite.fa --tumor_normal_pair 003_A_10000.bam 003_G_10000.bam
specifies all inputs with an abs path
it should work
Eric Davis
@davisem
Oct 20 2017 16:21
trying
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:21
in that case please comment on the google group for the records
Eric Davis
@davisem
Oct 20 2017 16:21
of course
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:22
:)
Eric Davis
@davisem
Oct 20 2017 16:23
for sure
Eric Davis
@davisem
Oct 20 2017 16:40
sadly this did not fix it.
:(
so bizarre
bsub ls /cgs02/home/$(whoami)/p-sequenza/003_[A,G]_10000.bam
works fine
but I still get file not found with the full bsub nextflow run ... with that parameter as an input.
Eric Davis
@davisem
Oct 20 2017 16:46
in the working dir of that command
input.1 -> /cgs02/home/edavis5/p-sequenza/work/tmp/a1/65c8108c806584573d57c51d169cd6/input.1
^ There is nothing here. Also, why is at /work/tmp?
that's not usually what I see in the work dir
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:52
I guess you have still crap input
head that file
Eric Davis
@davisem
Oct 20 2017 16:52
its fine
the links to the bams don't point to that temp dir
I think its some env thing with the way our lsf is set up
maybe when you mktemp
Paolo Di Tommaso
@pditommaso
Oct 20 2017 16:55
yes, you that when it's working fine you get bam files
instead in the other case you get dummy files
this means that your input files are not valid messing
check if you more code like the previous one and very that all input paths are valid
Eric Davis
@davisem
Oct 20 2017 16:57
are you saying the problem is my input files? I think the paths are valid, I just hit up arrow, and removed bsub and used the exact same paths.
the files are clearly there and valid
maybe I'm missunderstanding
Paolo Di Tommaso
@pditommaso
Oct 20 2017 17:00
the input of the IndexNormal are not the expected files
are not files at all
Eric Davis
@davisem
Oct 20 2017 17:00
yes that is true
the breakdown seems to be happening during the copying of the bams to the working dir.
Paolo Di Tommaso
@pditommaso
Oct 20 2017 17:02
there should not be any copy of bams to the working dir
Eric Davis
@davisem
Oct 20 2017 17:02
should be a link no?
Paolo Di Tommaso
@pditommaso
Oct 20 2017 17:03
yes a link
Eric Davis
@davisem
Oct 20 2017 17:03
in the .command.run
NXF_SCRATCH=''
[[ $NXF_DEBUG > 0 ]] && nxf_env
touch /cgs02/home/edavis5/work/d4/76fd0df6e151db263071ab3eeb1017/.command.begin
[ -f /cgs02/home/edavis5/work/d4/76fd0df6e151db263071ab3eeb1017/.command.env ] && source /cgs02/home/edavis5/work/d4/76fd0df6e151db263071ab3eeb1017/.command.env
[[ $NXF_SCRATCH ]] && echo "nxf-scratch-dir $HOSTNAME:$NXF_SCRATCH" && cd $NXF_SCRATCH
rm -f 003_A_10000.bam
rm -f 003_G_10000.bam
ln -s /cgs02/home/edavis5/p-sequenza/003_A_10000.bam 003_A_10000.bam
ln -s /cgs02/home/edavis5/p-sequenza/003_G_10000.bam 003_G_10000.bam
I don't see any of this block in the .command.run for the failed job
Paolo Di Tommaso
@pditommaso
Oct 20 2017 17:04
but I repeat, NF is creating a dummy file instead of a linking the bams because that process is not getting the bams at all
Eric Davis
@davisem
Oct 20 2017 17:04
oh ok I see
I under stand
so seems like the LSF configuration is to blame. I will try to get someone else on board to see if we can trace this down.
Eric Davis
@davisem
Oct 20 2017 17:11
thanks for your help. I will post on the google group whatever I figure out.
Paolo Di Tommaso
@pditommaso
Oct 20 2017 17:12
:+1:
Shawn Rynearson
@srynobio
Oct 20 2017 21:26
I think I have a simple question that I'm struggling with. If I have a comma/tab delimited string coming as input from a channel, how can I split into independent val with out splitting into independent channels?
Mike Smoot
@mes5k
Oct 20 2017 21:27
would your_channel.map{ it.tokenize(',')} do what you want?
Shawn Rynearson
@srynobio
Oct 20 2017 21:28
but would it split the values into two channels?
Mike Smoot
@mes5k
Oct 20 2017 21:30
Sorry, I misread that: your_channel.map{ it.tokenize(',')}.into{ ch1, ch2 }
that will give you two channels.
Wait, do you want one channel or two channels?
Shawn Rynearson
@srynobio
Oct 20 2017 21:30
@mes5k sorry for the confusion, i don't want to channels
Mike Smoot
@mes5k
Oct 20 2017 21:31
Then the first answer is what you want. The result of the map call will be a list within the channel.
So if your input channel has 5 elements, you'll end up with a channel with 5 lists.
Shawn Rynearson
@srynobio
Oct 20 2017 21:37
@mes5k that worked, thanks! tokenize isn't listed in the docs so I'm glad I came here.
Mike Smoot
@mes5k
Oct 20 2017 21:39
tokenize is a groovy method and not part of nextflow
glad it worked!