These are chat archives for nextflow-io/nextflow

16th
Apr 2018
Johan Viklund
@viklund
Apr 16 2018 07:09

I get the following error when running a pipeline

  /bin/bash: line 0: cd: /opt/test/K9-WGS-Pipeline.git/work/fc/0457fce94b853cb67b7672be8a7c8b: No such file or directory
  /bin/bash: .command.sh: No such file or directory

But the directory (and files) exists if I run ls directly from the terminal. I am running in a vm in vagrant and the /opt filesystem is a separate volume image that I have created. My suspicion is that nextflow is faster than the filesystem.

If I don't run on a separate volume it works (but that volume is to small for my analysis).
Caspar
@caspargross
Apr 16 2018 09:44
How can I write the output files from a process into the current working directory? Is there an environment variable I can give to publishDir?
Edgar
@edgano
Apr 16 2018 09:58
@caspargrossbaseDir
Paolo Di Tommaso
@pditommaso
Apr 16 2018 09:59
Dollar and curly brackets only when interpolating it in a string... 😉
Thomas Zichner
@zichner
Apr 16 2018 12:05
Hi Paolo, is it possible to mount directories read-only into Docker containers to prevent accidental data loss?
You implemented this some time ago (nextflow-io/nextflow#346), but likely this caused other problems so the feature was removed as far as I can see.
Recently, you added 'docker.mountFlags', which is very useful. However, setting this to 'ro' seems to not work because also the PWD will be mounted read-only.
Is there any other solution to this?
Thank you very much!
Thomas
Paolo Di Tommaso
@pditommaso
Apr 16 2018 12:07
It has not been removed, only disabled by default
Thomas Zichner
@zichner
Apr 16 2018 12:08
Ah, very good. How can one enable it?
Tim Diels
@timdiels
Apr 16 2018 12:10
Any way to see for sure whether a channel is a value channel or whatever the other type is named?
Paolo Di Tommaso
@pditommaso
Apr 16 2018 12:17
@zichner I've realized only now that's not documented. Can't check now, you need to dig in the code, sorry
Thomas Zichner
@zichner
Apr 16 2018 12:17
Ok. No problem. I will check the code. Thanks!
Thomas Zichner
@zichner
Apr 16 2018 12:52
Just for completeness: the option is "docker.readOnlyInputs = true"
Thanks again!
Pierre Lindenbaum
@lindenb
Apr 16 2018 13:03

I'm back here ;-) ok, current problem. How can I map the following output:

output:
   set sampleName,lane,bam to bam_by_lane

to

input:
  set sampleName,lane,list_of_bam_per_lane from bam_by_lane

is there an operation to do this ? I cannot find a simple way to do it ?

Paolo Di Tommaso
@pditommaso
Apr 16 2018 13:10
It should be enough to use .. from bam_by_lane.groupTuple(by:1)
Check the groupTuple documentation online
Pierre Lindenbaum
@lindenb
Apr 16 2018 13:17
@pditommaso it worked !, thanks Paolo
hahahine
@hahahine
Apr 16 2018 13:41

@pditommaso I'm just starting with workflows (so hi everyone ;) ) and I'm stuck with this problem...

I'd like to have one file in entry, which contains a list of other file names.
Example : "nextflow project.nf --file file.lst ..."
So in this example, "file.lst" contains a list of three files : 31_pep.fas, 32_pep.fas, 33_pep.fas.

What I'd like to do is iterate on each element on the list (I'm applying Muscle and Mafft on them). So here what it looks like for now :

listSeq=params.file

Channel.fromPath(listSeq)
        .splitText()
        .map { file(it) }
        .set { file_list }


process listAlign {
    input :
    file seq from file_list

    output:
    file "*.aln" into listeAln, listeAln2, listeAln3, listeAln4, listeAln5
    stdout out_list

    script:
    print "${seq}" > out_list
    if( mode == 'muscle' || mode == 'mafft' || mode == 'clustalo' || mode == 'all' ){

        template "${mode}list_command.sh"   

    }else
        error "Invalid alignment mode: ${mode}"

}

My template musclelist_command.sh contains :

#!/bin/bash
echo "process started at `date`"
muscle -in ${seq} -out tmp_out.aln
t_coffee -other_pg seq_reformat -in tmp_out.aln -output phylip -out ${seq} _AS_${mode}.aln
echo "Done !"

And it just doesn't work ... And I don't know why ! If I could get some help, that would save my life
Thanks

By the way, here's what I've got :
N E X T F L O W  ~  version 0.28.0
Launching `projetc.nf` [loving_nobel] - revision: 0e62dacb19
[warm up] executor > local
COG.lst
falsefalse[b4/8308a6] Submitted process > listAlign (1)
[24/8352ec] Submitted process > listAlign (2)
ERROR ~ Error executing process > 'listAlign (1)'

Caused by:
  Process `listAlign (1)` terminated with an error exit status (127)

Command executed [/home/Tests/templates/musclelist_command.sh]:

  #!/bin/bash
  echo "process started at `date`"
  muscle -in 31450_pep.fas
   -out tmp_out.aln
  t_coffee -other_pg seq_reformat -in tmp_out.aln -output phylip -out 31450_pep.fas
  _AS_muscle.aln
  echo "Done !"

Command exit status:
  127

Command output:
  (empty)

Command wrapper:
  .command.run: line 61: 31450_pep.fas: command not found
Paolo Di Tommaso
@pditommaso
Apr 16 2018 13:45
:ok_hand:
hahahine
@hahahine
Apr 16 2018 13:51
:ok_hand: Nothing wrong with it ?
Paolo Di Tommaso
@pditommaso
Apr 16 2018 13:57
There should have been a gitter hiccup because when I wrote that your comment was not displayed
However do the input file paths contain blanks
hahahine
@hahahine
Apr 16 2018 14:01
@pditommaso Okay !
No, they don't
And I've try to suppress them with a basic cut or tr but it doesn't work
hahahine
@hahahine
Apr 16 2018 14:11
@pditommaso Okay, they're obviously blanks insered (I don't know how) after 31450_pep.fas, but I can't find a way to suppress them
Paolo Di Tommaso
@pditommaso
Apr 16 2018 14:29
Sorry, I'm out of office this week. With mobile can't help much.
Check the error at the line reported in the error message
hahahine
@hahahine
Apr 16 2018 14:48
I think I found the problem
Channel.fromPath(listSeq)
.map { file(it) }
is what causes the blanks
Vladimir Kiselev
@wikiselev
Apr 16 2018 23:01
I have a problem with the bin folder again… I am preparing a workshop and created this pipeline with the bin folder: https://github.com/cellgeni/nf-workshop
However, when I run it (either from Github or locally) I get an error that NF does not find the sc3.R file (which is in this folder):
Launching `nf-workshop/main.nf` [determined_legentil] - revision: 363f3dd3d4
[warm up] executor > local
[93/e85d27] Submitted process > run_sc3 (1)
[fa/b797c3] Submitted process > run_sc3 (3)
[88/0720be] Submitted process > run_sc3 (2)
ERROR ~ Error executing process > 'run_sc3 (1)'

Caused by:
  Process `run_sc3 (1)` terminated with an error exit status (2)

Command executed:

  Rscript sc3.R 1

Command exit status:
  2

Command output:
  Fatal error: cannot open file 'sc3.R': No such file or directory

Command wrapper:
  Fatal error: cannot open file 'sc3.R': No such file or directory
Can anyone advise on how to use the bin folder
?