These are chat archives for nextflow-io/nextflow

27th
Sep 2017
Simone Baffelli
@baffelli
Sep 27 2017 09:04 UTC
Good morning
Any elegant way to flatten only one levels of nested list in groovf/nf?
nvm, found it probabily
Paolo Di Tommaso
@pditommaso
Sep 27 2017 09:37 UTC
mmm, .flatMap { it }
Toni Hermoso Pulido
@toniher
Sep 27 2017 10:33 UTC
Hi @pditommaso, sorry if I double-posted in the group
Paolo Di Tommaso
@pditommaso
Sep 27 2017 10:33 UTC
no pb, you are welcome :)
I missed your question ?
Toni Hermoso Pulido
@toniher
Sep 27 2017 10:39 UTC
Did you get deleted, or is it a spam thing
?
I pass you the question here:
Paolo Di Tommaso
@pditommaso
Sep 27 2017 10:40 UTC
sorry what question ?
Toni Hermoso Pulido
@toniher
Sep 27 2017 10:40 UTC
I sent this, I tried:

count = 0process blast {

publishDir 'results', mode: 'copy', overwrite: false

count = count + 1

input:
file 'query.fa' from fasta
file db_path

output:
file 'blast_result_'+count into blast_results

"""
blastp -db $db_path/$db_name -query query.fa -outfmt 5 -evalue ${params.evalue} -max_target_seqs ${params.hits} > blast_result_${count}
"""

}

but didn't work
what should I look for understing what shoud l change?
Paolo Di Tommaso
@pditommaso
Sep 27 2017 10:42 UTC
weird, I can't find it the google group, anyhow
Toni Hermoso Pulido
@toniher
Sep 27 2017 10:42 UTC
basically trying to copy results from processes using publishDir
it must be marked as spam I guess
Paolo Di Tommaso
@pditommaso
Sep 27 2017 10:43 UTC
to format the code use ``` new-line, then your-code, then ```
it should work, what's the problem ?
Toni Hermoso Pulido
@toniher
Sep 27 2017 10:43 UTC
Missing output file(s) blast_result_1 expected by process blast (2)

Command executed:

blastp -db db/uniprot_sprot.fasta -query query.fa -outfmt 5 -evalue 0.01 -max_target_seqs 100 > blast_result_0

Paolo Di Tommaso
@pditommaso
Sep 27 2017 10:45 UTC
it missing a * in the output file declaration
it should be
ahh, no wait
now I see, it doesn't work in this way ..
stated that usually it's not necessary to produce output files with a different suffix, because each task is executed in its own folder
you can do implement this by: 1) removing the count because it's useless, 2) use task.index in place of count, then modify the output declaration as show below
output:
file 'blast_result_*' into blast_results
(need to go later, happy to help more later)
Toni Hermoso Pulido
@toniher
Sep 27 2017 10:50 UTC
thanks @pditommaso I will play
@pditommaso it work! Thanks a lot!
Paolo Di Tommaso
@pditommaso
Sep 27 2017 11:45 UTC
good!
Félix C. Morency
@fmorency
Sep 27 2017 13:42 UTC
This message was deleted
This message was deleted
Ghost
@ghost~598345d2d73408ce4f6ff925
Sep 27 2017 13:59 UTC
I have the following csv file
Sample_Name,Alignment
ggal_alpha,path/to/ggal_alpha.bam
Do you see anything in this that is blatantly wrong? When I try to use bam_files in following processes, it seems it can't open the file correctly
  Channel
    .fromPath(params.alignments).splitCsv(header: true)
    .map {row -> [row.Sample_Name, [row.Alignment]]}
    .ifEmpty {error "File ${params.alignments} not parsed properly"}
    .into {bam_files; bam_counts; bam_stringtie1; bam_stringtie2}

for example, this process results in the following error

process test {
  tag "$sampleid"
  publishDir "${params.outdir}/${sampleid}/test", mode: 'copy'

  input:
  set sampleid, file(bamfiles) from bam_files
  file gtf from file(params.gtf)

  output:
  file '*.gtf' into bamout

  script:
  """
  stringtie ${bamfiles[0]} -o ${bamfiles[0]}_transcripts.gtf -v -G $gtf
  """
}

Command error:
Running StringTie 1.3.3. Command line:
stringtie input.1 -o test_transcripts.gtf -v -G genome_annotation.gff
[samopen] no @SQ lines in the header.
[sam_read1] missing header? Abort!

I've confirmed the file works by running it manually and replacing ${bamfiles[0]} with the absolute path to the bam file
Paolo Di Tommaso
@pditommaso
Sep 27 2017 14:19 UTC
yes
you need to convert the path string to a file object eg
  Channel
    .fromPath(params.alignments).splitCsv(header: true)
    .map {row -> [row.Sample_Name, file(row.Alignment)]}
also it is just one file, the [ .. ] is not necessary (as above)
we are thinking to remove this need in a future release because it's a bit tricky
Ghost
@ghost~598345d2d73408ce4f6ff925
Sep 27 2017 14:36 UTC
sweet it worked
thank you, I see now
Paolo Di Tommaso
@pditommaso
Sep 27 2017 14:41 UTC
:+1:
apoholek
@apoholek
Sep 27 2017 17:30 UTC
Thanks Paolo. One additional question. We are having trouble getting bash command "wait" in nextflows execution. any ideas?
Paolo Di Tommaso
@pditommaso
Sep 27 2017 17:34 UTC
what kind of problem?
apoholek
@apoholek
Sep 27 2017 17:47 UTC
ok, we are running bash scripts in nextflow, and had some background processes running, using a & and wait bash commands - these seem to not be recognized? we keep getting empty files.
Paolo Di Tommaso
@pditommaso
Sep 27 2017 17:48 UTC
?
is there an error message ?
apoholek
@apoholek
Sep 27 2017 17:56 UTC
ok - we figured it out. we had a different problem all along and had not loaded modules.
thanks!
Paolo Di Tommaso
@pditommaso
Sep 27 2017 17:56 UTC
ok nice, may ask what's your organisation ?
apoholek
@apoholek
Sep 27 2017 18:34 UTC
We are a team participating in an NCBI Hackathon at University of Pittsburgh. See our repo ( in development) https://github.com/NCBI-Hackathons/Automatic_Packaging_Bioinformatics
Paolo Di Tommaso
@pditommaso
Sep 27 2017 18:37 UTC
Oops
CWL was widely dismissed by pretty much all members present, as being too labor intensive to use. A few people with CWL experience relayed how difficult and frustrating it was to use, and the time it took to learn considered not worth the effort.
Snakemake was dismissed as being less flexible than NextFlow. Many users thought that it is mostly Python oriented, although others confirmed that is not the case.
happy to help is needed :)
Francesco Strozzi
@fstrozzi
Sep 27 2017 18:49 UTC
Well, it's the same thing I keep hearing over and over. :+1:
Félix C. Morency
@fmorency
Sep 27 2017 18:51 UTC
Neuro university folks here in Quebec are starting to use NF more and more (because of us ^^)
Paolo Di Tommaso
@pditommaso
Sep 27 2017 18:51 UTC
nice, spread the word
@fmorency BTW have you seen this singularity pull singularityware/singularity#977
that's a very bad news, isn't it? what do you think ?
Félix C. Morency
@fmorency
Sep 27 2017 18:58 UTC
I didn't saw that issue previously. It shouldn't impact us that much but this indeed sounds like a bad news
Félix C. Morency
@fmorency
Sep 27 2017 19:41 UTC
it seems that a fix is in the work and will be in 2.4
Paolo Di Tommaso
@pditommaso
Sep 27 2017 19:42 UTC
a fix that will allow pull w/o sudo ?
Félix C. Morency
@fmorency
Sep 27 2017 19:42 UTC
yes. user mibauer confirmed it on the singularity slack 2mins ago
Paolo Di Tommaso
@pditommaso
Sep 27 2017 19:44 UTC
Thanks good
I will comment in any case
ah, they have just updated the issue