These are chat archives for nextflow-io/nextflow

23rd
Apr 2018
Paolo Di Tommaso
@pditommaso
Apr 23 2018 13:35
@/all anybody tried version 0.29.0-RC1 ?
hahahine
@hahahine
Apr 23 2018 14:14

Do you know why it doesn't generate any file ?
'''
process jackKnife{
input :
file "concatenation.lst" from concat

output:
file "*" into aln_jackKnife
stdout v

script:
if ( count != 1 ){
    """
    basename `/share/gsi/bin/jackknife_concat_family.pl --data_dir $HOME/Tests --file_list "concatenation" --erase 0 --verbose 0 --nb_otus_min 100 --exclude NequA --one_strain 1 --cluster 1 --suffix fas  | grep "concatenated sequences : " | cut -c 27-` > aln_jackKnife
    """
}

}

process readAl {
input :
file read from aln_jackKnife

output : 
file "*" into readal_phy
stdout dt

script:
if ( count != 1 ){
            """
    readal -in `cat ${read}` -out test_readal2.phy -phylip 
           """
    }

}

Paolo Di Tommaso
@pditommaso
Apr 23 2018 14:15
code formatting is:
triple ```
new-line
code
new-line
triple ```
hahahine
@hahahine
Apr 23 2018 14:16
Do you know why it doesn't generate any file ?
process jackKnife{
    input :
    file "concatenation.lst"  from concat

    output:
    file "*" into aln_jackKnife
    stdout v

    script:
    if ( count != 1 ){
        """
        basename `/share/gsi/bin/jackknife_concat_family.pl --data_dir $HOME/Tests --file_list "concatenation" --erase 0 --verbose 0 --nb_otus_min 100 --exclude NequA --one_strain 1 --cluster 1 --suffix fas  | grep "concatenated sequences : " | cut -c 27-` > aln_jackKnife
        """
    }

}

process readAl {
    input :
    file read from aln_jackKnife

    output : 
    file "*" into readal_phy
    stdout dt

    script:
    if ( count != 1 ){
                """
        readal -in `cat ${read}` -out test_readal2.phy -phylip 
               """
        }
}
Paolo Di Tommaso
@pditommaso
Apr 23 2018 14:16
no ''' but ```
:)
good ! ;)
hahahine
@hahahine
Apr 23 2018 14:17
Got it XD
sorry I'm exhausted ;)
can't think about anything properly
Paolo Di Tommaso
@pditommaso
Apr 23 2018 14:17
so, which should create what ?
hahahine
@hahahine
Apr 23 2018 14:18
so the readal is supposed to create a file in output
test_readal2.phy
I'm just beginning and I'm kind of struggling
It doesn't create anything
Paolo Di Tommaso
@pditommaso
Apr 23 2018 14:19
is the process executed ?
hahahine
@hahahine
Apr 23 2018 14:19
yes
Paolo Di Tommaso
@pditommaso
Apr 23 2018 14:20
the best way to debug this is to change the task work dir and check the command executed and the inputs are the ones expected
hahahine
@hahahine
Apr 23 2018 14:21
okay
I'm gonna try, thx
Paolo Di Tommaso
@pditommaso
Apr 23 2018 14:21
also you can executed only that task using bash .command.run
and see what happens
also check the .command.out and .command.err files
Ashley S Doane
@DoaneAS
Apr 23 2018 15:25
hi, I fixed the sge issue mentioned above (admins fixed submit host issue to ignore the virtual_mem line).
A seperate question about singularity and using sge executor. On cluster nodes, I need to call a few environment setup commands in order for signularity to work. Is there a place to put commands for setting up environment before the process execution script is called? These commands would need to be called outside of the container. thanks
Félix C. Morency
@fmorency
Apr 23 2018 15:33
@DoaneAS Take a look at the beforeScript process directive https://www.nextflow.io/docs/latest/process.html#beforescript
Ashley S Doane
@DoaneAS
Apr 23 2018 15:33
thanks @fmorency
Félix C. Morency
@fmorency
Apr 23 2018 15:34
However, I don't know if this is executed outside of the container. You'll need to test.
There is also the module directive if your cluster is using it https://www.nextflow.io/docs/latest/process.html#module
Ashley S Doane
@DoaneAS
Apr 23 2018 15:35
hmm interesting
Paolo Di Tommaso
@pditommaso
Apr 23 2018 15:37
yes, it is
Ashley S Doane
@DoaneAS
Apr 23 2018 15:43
we are using spack, which is basically a wrapper around modules that allows for different build environments depending on the host. Our compute nodes are runing linux kernel 2.6 or 3.1. So I need to call spack load singularity on cluster nodes.
Paolo Di Tommaso
@pditommaso
Apr 23 2018 15:43
you can put that in the beforeScript
Ashley S Doane
@DoaneAS
Apr 23 2018 15:49
thanks. everything gets set up correctly if the cluster node process gets a new login shell, or if it sources ~/.bashrc, so I will try source $HOME/.bashrc in beforeScript
Venkat Malladi
@vsmalladi
Apr 23 2018 20:48
does anyone have a good implmentation example of nextflow on azure?
Paolo Di Tommaso
@pditommaso
Apr 23 2018 20:51
not sure how much it could help
Venkat Malladi
@vsmalladi
Apr 23 2018 20:51
@pditommaso thanks
Trying to move our pipelines to use Azure on the fly by reading in the config file
Paolo Di Tommaso
@pditommaso
Apr 23 2018 20:54
interesting, let me know if you manage to do that
Venkat Malladi
@vsmalladi
Apr 23 2018 20:59
Thanks
Paolo Di Tommaso
@pditommaso
Apr 23 2018 21:00
is that useful ?
Diogo Silva
@ODiogoSilva
Apr 23 2018 21:29
Sorry for double posting, but this may have been lost in the chat: Are there any updates on the interactive command line tool that's mentioned in the GH projects page (https://github.com/nextflow-io/nextflow/projects/1)? I would be really nice to have some way of checking the progress of a pipeline run, specially for large data volumes where one quickly loses track :smile:
Paolo Di Tommaso
@pditommaso
Apr 23 2018 21:30
hi there, not in the short run
Diogo Silva
@ODiogoSilva
Apr 23 2018 21:36
I see. I would love to contribute to that but my java knowledge is very limited
Paolo Di Tommaso
@pditommaso
Apr 23 2018 21:37
a good opportunity to learn it :)
Paolo Di Tommaso
@pditommaso
Apr 23 2018 21:46
BTW congrats regarding assemblerflow, it looks an amazing project !
Diogo Silva
@ODiogoSilva
Apr 23 2018 21:57
wow, thanks! that means a lot. I'll stay in touch to see how I can help in that matter