These are chat archives for nextflow-io/nextflow

1st
May 2019
Tobias "Tobi" Schraink
@tobsecret
May 01 14:35
Oooooh, the shebang needs to be in the script.py - :facepalm: ok, yeah that makes sense
thanks guys!
micans
@micans
May 01 14:58
@pditommaso -- I have work directories where something went wrong; possibly a truncated file due to disk quota. The .exitcode there is 0 (I am not 100% sure what went wrong). I'd like to do something there so that NF will resume those tasks rather than use the cached results; I tried removing the output files, but that does not seem to work. I should write a small test case, but in the meantime if you have sage advice ...
Paolo Di Tommaso
@pditommaso
May 01 15:13
remove the task work dir
micans
@micans
May 01 15:14
:beers:
Paolo Di Tommaso
@pditommaso
May 01 15:15
I think next week I deserve one :joy:
micans
@micans
May 01 15:15
I haven't been paying attention ... are you very very busy right now?
Paolo Di Tommaso
@pditommaso
May 01 15:16
as usual . .
micans
@micans
May 01 15:17
don't you deserve :beer: every day then?
Paolo Di Tommaso
@pditommaso
May 01 15:17
ahah
Ólavur Mortensen
@olavurmortensen
May 01 15:27

I have a very newbie question. I have the error below, and I think it's because I'm launching the same workflow in the same working directory, with different parameters. I'm basically running different samples through a workflow. I tried changing Nextflow's working directory using the -w parameter, but I still got the same error. I should note that I haven't containerized the workflow (yet), otherwise this wouldn't be a problem (I think).

Is there any way I can do this? Running my exolink/single_sample.nf workflow on different samples simultaneously?

N E X T F L O W  ~  version 19.04.0
Launching `exolink/single_sample.nf` [elated_lamarck] - revision: 6989dd7626
ERROR ~ Unable to acquire lock on session with ID 35b02a48-6d30-4baa-9807-bbb7df303bbf

Common reasons of this error are:
 - You are trying to resume the execution of an already running pipeline
 - A previous execution was abruptly interrupted leaving the session open

You can check what process is holding the lock file by using the following command:
 - lsof /mnt/fargen/experiments/joint_call/.nextflow/cache/35b02a48-6d30-4baa-9807-bbb7df303bbf/db/LOCK

 -- Check '.nextflow.log' file for details
micans
@micans
May 01 15:32
Nextflow also wants to make .nextflow.log and .nextflow/ in that directory. It seems neater to just run those workflows in different (sub)directories. But I don't know whether this is the actual problem.
Ólavur Mortensen
@olavurmortensen
May 01 15:32
I'll try.
Ólavur Mortensen
@olavurmortensen
May 01 15:38
This seems to work. Bit clunky, but does the job. Guess Nextflow isn't designed to be used like this, but it's only for early stage development purposes so it's fine.
Thanks :)
micans
@micans
May 01 15:39
@olavurmortensen nextflow has a -log option; perhaps there are ways around.
Ólavur Mortensen
@olavurmortensen
May 01 15:42
Can't find documentation on it. Do you have a link?
micans
@micans
May 01 15:43
I ran nextflow help!
Ólavur Mortensen
@olavurmortensen
May 01 15:44
Ah, of course :D
micans
@micans
May 01 15:45
Fair question, np
Ólavur Mortensen
@olavurmortensen
May 01 15:48
I also see this -log parameter, yet this is the message I get when trying to use it:
Unknown option: -log -- Check the available commands and options and syntax with 'help'
micans
@micans
May 01 15:52
nextflow -log lmz run cf.nf works for me, but still uses .nextflow/ I think.
Ólavur Mortensen
@olavurmortensen
May 01 15:54
Ah, the -log arg goes before run alright.
But yeah, still doesn't work.
Probably because of the .nextflow/ folder, as you say.
micans
@micans
May 01 15:57
Folders are a nice encapsulation mechanism in the NF world. I've embraced it :-)
Ólavur Mortensen
@olavurmortensen
May 01 15:58
I'll just change directories, as we discussed earlier, that seemed to work :)
micans
@micans
May 01 15:58
:+1: good luck, NF is worth it!
Ólavur Mortensen
@olavurmortensen
May 01 15:58
Thanks :D
I agree, so far it is a game changer :)
Shawn Rynearson
@srynobio
May 01 22:01

Based on this documentation example

seq_to_align = ...
mode = 'tcoffee'

process align {
    input:
    file seq_to_aln from sequences

    script:
    if( mode == 'tcoffee' )
        """
        t_coffee -in $seq_to_aln > out_file
        """

    else if( mode == 'mafft' )
        """
        mafft --anysymbol --parttree --quiet $seq_to_aln > out_file
        """

    else if( mode == 'clustalo' )
        """
        clustalo -i $seq_to_aln -o out_file
        """

    else
        error "Invalid alignment mode: ${mode}"

}

Does Nextflow have the ability to somehow assign values to 'mode' from a previous process? Or is there another way?

I want to be able to use the if/else method based on a previous process output of a file.
Shawn Rynearson
@srynobio
May 01 22:07
Possable input type: env
Shawn Rynearson
@srynobio
May 01 22:13
Try it, didn't work
process test {
    tag {"$params.project"}
    label 'localterm'

    input:
    each source_file from source_type_file_ch

    output:
    stdout into test_out_ch

    shell:
    '''
    export TYPE=$(cat !{source_file} | cut -f1)
    echo $TYPE
    '''

}

// ------------------------------------- //

process meh {
    tag {"$params.project"}
    label 'localterm'

    input:
    env HELLO from test_out_ch

    script:
    if ($HELLO == 'paired_fq')
    """
    echo $HELLO
    """
}
Rad Suchecki
@rsuchecki
May 01 23:14
@srynobio Here is a simplified, working version of your code which you may be able to adapt to your needs:
source_type_file_ch = Channel.from('fq', 'paired_fq')
process test {
    input:
     val s from source_type_file_ch

    output:
      stdout into test_out_ch

    """
    echo $s
    """
}

process meh {
  echo true
  input:
     val HELLO from test_out_ch

  when:
    HELLO.trim() == 'paired_fq'

  script:
    """
    echo $HELLO
    """
}
if when is not suitable for your needs, this will work as well
  script:
  if(HELLO.trim() == 'paired_fq')
    """
    echo $HELLO
    """