Nextflow community chat moved to Slack! https://www.nextflow.io/blog/2022/nextflow-is-moving-to-slack.html
jorgeaguileraseqera on 2678-dsl2-emit-tuples-with-optional-values
feature: new allowNull attribut… (compare)
pditommaso on master
Preview gbch support [alpha] S… (compare)
pditommaso on master
pditommaso on master
Merge gbtch alpha Signed-off-b… Bump version 22.05.0-SNAPSHOT … (compare)
jorgeaguileraseqera on 2678-dsl2-emit-tuples-with-optional-values
feature: emit nullable path in … feature: new allowNull attribut… (compare)
process generate_readset {
tag "$sample_id"
cpus 48
input:
tuple val(read_name), val(chromosome1), val(chromosome2), val(cuteSV_pos1), val(cuteSV_pos2),
val(sniffle_pos1), val(sniffle_pos2),
path(cuteSV_vcf), path(sniffles_vcf) from vcf_input
output:
path 'complete_read_set.txt' into receiver
script:
"""
${bcftools_1_11} view --threads ${task.cpus} $cuteSV_vcf -r chr$chromosome1:$cuteSV_pos1-$cuteSV_pos2 > complete.txt
"""}
Remote resource not found: https://api.github.com/repos/PATH/TO/contents/main.nf
. What am I doing wrong?
Hey All,
I have a error relate to nextflow azurebatch. The first process using a default D4_v3 vm work alright, but the second process I fail to request a larger vm (I set it in queue, but apparently, it is not working, do I make some naive mistake?)
'''
Error executing process > 'secondprocess'
Caused by:
Cannot find a VM for task 'secondprocess' matching this requirements: type=Standard_D4_v3, cpus=16, mem=14 GB, location=eastus
'''
The config file I used:
process {
executor = 'azurebatch'
}
docker {
enabled = true
}
azure {
batch {
location = 'eastus'
accountName = 'xxxbatch'
accountKey = 'xxx'
autoPoolMode = true
allowPoolCreation = true
deletePoolsOnCompletion = true
deleteJobsOnCompletion = true
pools {
small {
autoScale = true
vmType = 'Standard_D4_v3'
vmCount = 5
maxVmCount = 50
}
large {
autoScale = true
vmType = 'Standard_D16_v3'
vmCount = 5
maxVmCount = 50
}
}
}
storage {
accountName = "xxx"
accountKey = "xxx"
}
}
process {
withName: firstprocess {
queue = 'small'
}
withName: secondprocess {
queue = 'large'
}
}
nextflow run
, despite specifying an unquota'd path with -w
aka -work-dir
. Any ideas? In this thread, Paolo suggests -w
is the solution… https://groups.google.com/g/nextflow/c/401Tp_6H57k/m/va8ACNeTAQAJ
$ nextflow run nf-core/viralrecon -w /users/xxx/test --help
N E X T F L O W ~ version 21.04.0
Pulling nf-core/viralrecon ...
Disk quota exceeded
How can I use this as the 'size' part of a groupTuple? I tried:
aligned_bams
.groupTuple(by:0,size:lane_calc)
But it did not like it - complained about the value type etc. All thoughts gladly received!
.collect()
the files from each tuple at the same time. Anyone has done this before? I've already read about each
and combine
, so my input channel is of the right format. The problem I have is .collect()
- not sure how to incorporate it in the input tuple.
Hi, in DSL2, do functions allow the passing in of channels? I was hoping to clean up a workflow where a similar sequence of operations is applied to channels, but it looks like channels passed into functions are demoted to plain Java collections.
The error reads:
No signature of method: java.util.LinkedHashMap.collectFile() is applicable for argument types: (LinkedHashMap, Script_77e02758$_collect_file_tuples_closure1) values: [[storeDir:null, sort:hash], Script_77e02758$_collect_file_tuples_closure1@54e2fe]
process {
withName: structural_alignment {
if (task.exitStatus in 140..143 )
"""
errorStrategy = 'retry'
cpus = {2 * task.attempt}
maxRetries = 5
"""
else
errorStrategy = 'retry'
maxRetries = 10
}
}
process {
withName: structural_alignment {
if (task.exitStatus in 140..143 ) {
errorStrategy = 'retry'
cpus = {2 * task.attempt}
maxRetries = 5
}
else {
errorStrategy = 'retry'
maxRetries = 10
}
}
Hi guys.
I have a simple doubt (I think). I just need to save my output process in an external directory so I'm using the publishDir
function for it. However I need 2 things: i) use variables (collected from input tuple); ii) Create the directory because it doesn't exist.
I successfully run it like this:
publishDir "${params.out_dir}/${tcga_project}/${tcga_barcode}/RSEM_quantification", mode: 'move'
input:
file STAR_bam_file from STAR_alignment_bam
set val(sample_UUID), val(tcga_barcode), val(tcga_project) from samples_ch_3
But It doesn't create the directory. So I tried switching to this:
outDir = file("${params.out_dir}/${tcga_project}/${tcga_barcode}/RSEM_quantification")
outDir.mkdirs()
publishDir outDir, mode: 'move'
input:
file STAR_bam_file from STAR_alignment_bam
set val(sample_UUID), val(tcga_barcode), val(tcga_project) from samples_ch_3
However it states the error of:No such variable: tcga_project
Any help in this situtation? Thanks :)
-profile docker
at the command line level, and instead I would like to have in my nextflow.config file profile = docker
such that it successfully imports all of the config options specified by the docker profile. thanks
collectFile
in my workflow to combine the results of several upstream processes, and setting storeDir
. However, when I resume my workflow (using AWS Batch executor), even though the upstream processes are correctly cached, the downstream process that consumes the output of collectFile
always re-executes. I've tried sorting the input to collectFile
with no luck.
How come
Channel
.fromFilePairs( "${params.reads}/*{1,2}*.fastq.gz", size: 2 )
.into { paired_reads }
does not work when the file starts with a number (like 7126-MS-1_100_S1_L005_R1_001.fastq.gz
and 7126-MS-1_100_S1_L005_R2_001.fastq.gz
or 21VR067049-211104_S1_L001_R1_001.fastq.gz
and21VR067049-211104_S1_L001_R2_001.fastq.gz
.collate()
in my pipeline and then an implementation of this which works elegantly. My problem is with the -resume
option. Some processes get cached, but the rest are re-scheduled. I am assuming the order changes from run to run before the collate operator, and thus assumes input has changed. Could this be it?
Channel.from(2,1,3,4).toSortedList().fromList()
Hello Group members, I am trying a sample nextflow script using Azure Blob storage for work directory. I am getting an error and couldn't understand if my setup has anything to do, appreciate your response. Here is the error when trying to run my sample script connecting to Azure Blob storage
nextflow run tutorial.nf -w az://nextflow-work/work
How to deal with the empty path channel in process (dsl2)? I saw a solution for empty channel of value (https://github.com/nextflow-io/patterns/blob/master/docs/process-when-empty.adoc)
nextflow.enable.dsl=2
params.a = '*.txt'
params.b = ''
params.c = ''
process test {
tag "test "
echo true
input:
path file1
path file2
val value1
"""
echo $file1
echo $file2
echo $value1
"""
}
workflow {
a_ch = Channel.fromPath(params.a)
b_ch = Channel.fromPath(params.b)
test(a_ch,b_ch,params.c)
}
The Error message:
N E X T F L O W ~ version 21.04.3
Launching `code/test2.nf` [modest_colden] - revision: b4078eef6a
Missing `fromPath` parameter
a_ch = params.a ? Channel.fromPath(params.a) : []
maybe?
Hi,
I'm trying to use Docker in Nextflow script and I'm getting the following error:
"connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Fr
un%2Fdocker.sock/v1.24/containers/create?name=nxf-8IoBF1Fj7HBmfAtOGRGpW5Fp": dial unix /var/run/docker.sock: connect: permission denied.
See 'docker run --help'."
Does anyone have any advice on how to handle this?
:point_up: Edit: Hi,
I'm trying to use Docker in Nextflow script and I'm getting the following error:
"connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Fr
un%2Fdocker.sock/v1.24/containers/create?name=nxf-8IoBF1Fj7HBmfAtOGRGpW5Fp": dial unix /var/run/docker.sock: connect: permission denied.
See 'docker run --help'."
Does anyone have any advice on how to handle this?