These are chat archives for nextflow-io/nextflow

29th
May 2018
Trevor Tanner
@tantrev
May 29 2018 04:37
So this is probably a stupid question, but is there any way to filter a channel where its constituents are individual arrays/sets? For example, something like ch1.filter{ it[0] == "SRR6049692"}?
Bioninbo
@Bioninbo
May 29 2018 06:35
Hello everybody,
Is it possible to use the multiple input file parameter together with single input. i.e.:
input: set id_cond1, file(*), id_cond2, file(*) from multiple_input_channel
My aim is to load replicate files for condition1 and replicate files for condition2. But there could be a varying number of replicate files, so I am not sure how to catch this in a process.
Maxime Garcia
@MaxUlysse
May 29 2018 08:12
have you tried something like?:
input:
  set id_cond1, file(*) from input1.collect()
  set id_cond2, file(*) from input2.collect()
Luca Cozzuto
@lucacozzuto
May 29 2018 09:00
dear all
Luca Cozzuto
@lucacozzuto
May 29 2018 09:16
I have a process failing with nextflow but when running with qsub .command.run it works...
Maxime Garcia
@MaxUlysse
May 29 2018 11:29
Can you give some more details on your process?
What is your error?
Ryo Kita
@rkita
May 29 2018 14:16
Another question: One of my scripts is creating a file in another directory. And I want to use that file as the output in a channel. However, when I try to do that, I end up getting a "out of scope of process working dir" error.
process create_chunk_file {
    input:
    val run_dir from params.runDir

    output:
    file "${run_dir}/run_chunks.txt" into all_chunks                                                                                                                                                

    """
    run_bayenv.py \
    chunk \
    -d $run_dir \
    """
}
I can avoid that by just creating the file run_chunks.txt file in the current working directory, but is there a way to keep this organization structure?
Paolo Di Tommaso
@pditommaso
May 29 2018 14:19
task outputs need to be produced in the task work dir
to organise the output use publishDir directive
Ryo Kita
@rkita
May 29 2018 14:20
ok thanks
Luca Cozzuto
@lucacozzuto
May 29 2018 14:48
@MaxUlysse I spoke with @pditommaso... it is likely a java error thanks the same
Maxime Garcia
@MaxUlysse
May 29 2018 14:50
Good luck @lucacozzuto
Luca Cozzuto
@lucacozzuto
May 29 2018 14:52
@MaxUlysse thanks. I hate this
LukeGoodsell
@LukeGoodsell
May 29 2018 15:18
Hi @pditommaso, I’ve raised a PR (#719) for an old issue (#296) that’s been causing my team to need a lot of tedious boiler plate. Please let me know if it’s something you’d consider adding and/or what I’d need to do to make it possible to consider.
Paolo Di Tommaso
@pditommaso
May 29 2018 15:19
thanks, I will review it soon
LukeGoodsell
@LukeGoodsell
May 29 2018 15:21
Great, thanks
Paolo Di Tommaso
@pditommaso
May 29 2018 15:30
:+1:
Anthony Underwood
@aunderwo
May 29 2018 15:45
Hi @pditommaso Hopefully getting a chance to get back to using Nextflow soon :) Quick Q re AWS. Which would you recommend as the easiest/most stable to set up and use - cluster or batch?
Vladimir Kiselev
@wikiselev
May 29 2018 15:48
what is the command to update NF?
Edgar
@edgano
May 29 2018 15:49
nextflow self-update
Félix C. Morency
@fmorency
May 29 2018 15:49
nextflow self-update
Vladimir Kiselev
@wikiselev
May 29 2018 15:49
Thanks guys!
Paolo Di Tommaso
@pditommaso
May 29 2018 16:31
@aunderwo Hi Anthony, likely the AWS Batch is the simplest way
micans
@micans
May 29 2018 17:18
Is it possible to have a queueSize that's specific to a single process block? When we obtain files using irods we need to throttle it, but subsequent steps can use more parallelisation. I'm probably missing something: I've used withLabel: irods (with corresponding label 'irods') and $irods selection, but so far I've not been able to reduce queueSize for this specific step in the pipeline.
Paolo Di Tommaso
@pditommaso
May 29 2018 17:20
you can set maxForks directive which limits the number of number or parallel tasks
micans
@micans
May 29 2018 17:23
ahhh. I'll give it a few months before I ask something substantial. I did have the nagging feeling that queueSize indicated a different thing from what I wanted, but had overlooked maxForks! As ever, very grateful!
I need to work on my NF search foo clearly