Nextflow community chat moved to Slack! https://www.nextflow.io/blog/2022/nextflow-is-moving-to-slack.html
I could not figure out how to uninstall Nextflow. So I simply ran the following command one more time.
curl -s https://get.nextflow.io | bash Then I moved the created nextflow file to the path. Now it works properly.
Is it possible for me to access the params such that I can write them as an output? In my particular workflow, several of the params used are things like "password needed to unzip file", which I'd like to save for posterity in the output location. I'm basically trying to do:
params_json = new JsonBuilder(params).toString()
process xxx {
shell:
'''echo !{params_json} > params.json'''
}But it's not letting me. Is there some other nice way for me to do this?
9 replies
+([0-9]). How do you all deal with this?Channel.fromFilePairs(["*_S[0-9]_L00[1-9]_{R1,R2}_001.fastq.gz",
"*_S[0-9][0-9]_L00[1-9]_{R1,R2}_001.fastq.gz",
"*_S[0-9][0-9][0-9]_L00[1-9]_{R1,R2}_001.fastq.gz",
"*_S[0-9][0-9][0-9][0-9]_L00[1-9]_{R1,R2}_001.fastq.gz"]).set { illumina_q }
Hey all. I have trouble cat two files together under shell. Outside of nextflow run, everything work flawlessly.
shell:
'''
echo "read1: NA12878_S1_L001_R1_001.fastq.gz NA12878_S1_L002_R1_001.fastq.gz"
echo "read2: NA12878_S1_L001_R2_001.fastq.gz NA12878_S1_L002_R2_001.fastq.gz"
echo $(ls)
cat NA12878_S1_L001_R1_001.fastq.gz NA12878_S1_L002_R1_001.fastq.gz > read1.fastq.gz
cat NA12878_S1_L001_R2_001.fastq.gz NA12878_S1_L002_R2_001.fastq.gz > read2.fastq.gz
'''
Command exit status:
1
Command output:
read1: NA12878_S1_L001_R1_001.fastq.gz NA12878_S1_L002_R1_001.fastq.gz
read2: NA12878_S1_L001_R2_001.fastq.gz NA12878_S1_L002_R2_001.fastq.gz
NA12878_S1_L001_R1_001.fastq.gz NA12878_S1_L001_R2_001.fastq.gz NA12878_S1_L002_R1_001.fastq.gz NA12878_S1_L002_R2_001.fastq.gz
Command error:
cat: NA12878_S1_L001_R1_001.fastq.gz: No such file or directory
cat: NA12878_S1_L002_R1_001.fastq.gz: No such file or directory
13 replies
Hi all, How much does it cost to run a workflow using tower and aws-batch + fsx lustre? I'm working on a customer project right now and want to evaluate this option. Perhaps some guidance to the documentation and pricing model is needed.
Using aws-batch mode with s3 only work directory is not very efficient.
What is the commercial license price
Hi all, I'm looking for a method to convert paired end data from bcl2fastq process output into fastq file pair tuple. @happykhan's method is the only I found. Is there some more elegant way with DSL2? Currently I have a standard bcl2fastq process that emits
*.fastq.gzand I can read in the sample sheet, but I'd need to match sample sheet names and filename. Is there some way to use something like the.fromFilePairs factory on an existing channel?
nextflow.enable.dsl=2
process createFiles {
output:
path("*.txt", emit: apath)
script:
"""
#!/usr/bin/env python
filenames = ['a.txt', 'b.txt', 'c.txt']
for f in filenames:
with open(f, "w") as wf:
wf.write("hello\\n")
print(f)
"""
}
process printContent {
input:
path(x)
script:
"""
cat $x
"""
}
workflow {
createFiles()
printContent(createFiles.out.apath)
}executor > local (2)
[67/0c496d] process > createFiles [100%] 1 of 1 ✔
[1b/36a13c] process > printContent [100%] 1 of 1 ✔
workflow {
createFiles()
printContent(createFiles.out.apath.flatten())
}
Is it possible to create files with the native execution mode of a process? For example, I attempted the following:
process WRITE_FASTP_METRICS{
input:
val (rna_result)
val (adt_result)
output:
path "fastp_metrics.csv"
exec:
write_out = file("fastp_metrics.csv")
rna_result.forEach{key, value ->
write_out << key << ',' << value << '\n'
}
adt_result.forEach{key, value ->
write_out << key << ',' << value << '\n'
}
}But the fastp_metrics.csv is not created in the work directory, causing this error: Missing output file(s) ``fastp_metrics.csv`` expected by process ``WRITE_FASTP_METRICS (1)``
i'm wondering if we can pass in a container as a variable, as i want to test the same process over various versions of a software. something like this:
process A {
container= container_label
input:
tuple val(container_label), path(inputFile)
...
}this code did not work, however. can it be done in another way?
process A {
container= params.container_label
input:
path(inputFile)
...
}process A {
input:
tuple val(container_label), path(inputFile)
...
script:
task.container = container_label
...
}
Does anyone know if there will be any nextflow workshop in the near future? I see that there was one in July of 2020 and another one in May of this year hosted by ecseq and was wondering if anyone knows if a similar event will be taking place in the future.
2 replies
5 replies
process generate_readset {
tag "$sample_id"
cpus 48
input:
tuple val(read_name), val(chromosome1), val(chromosome2), val(cuteSV_pos1), val(cuteSV_pos2),
val(sniffle_pos1), val(sniffle_pos2),
path(cuteSV_vcf), path(sniffles_vcf) from vcf_input
output:
path 'complete_read_set.txt' into receiver
script:
"""
${bcftools_1_11} view --threads ${task.cpus} $cuteSV_vcf -r chr$chromosome1:$cuteSV_pos1-$cuteSV_pos2 > complete.txt
"""}
Remote resource not found: https://api.github.com/repos/PATH/TO/contents/main.nf. What am I doing wrong?
Hey All,
I have a error relate to nextflow azurebatch. The first process using a default D4_v3 vm work alright, but the second process I fail to request a larger vm (I set it in queue, but apparently, it is not working, do I make some naive mistake?)
'''
Error executing process > 'secondprocess'
Caused by:
Cannot find a VM for task 'secondprocess' matching this requirements: type=Standard_D4_v3, cpus=16, mem=14 GB, location=eastus
'''
The config file I used:
process {
executor = 'azurebatch'
}
docker {
enabled = true
}
azure {
batch {
location = 'eastus'
accountName = 'xxxbatch'
accountKey = 'xxx'
autoPoolMode = true
allowPoolCreation = true
deletePoolsOnCompletion = true
deleteJobsOnCompletion = true
pools {
small {
autoScale = true
vmType = 'Standard_D4_v3'
vmCount = 5
maxVmCount = 50
}
large {
autoScale = true
vmType = 'Standard_D16_v3'
vmCount = 5
maxVmCount = 50
}
}
}
storage {
accountName = "xxx"
accountKey = "xxx"
}
}
process {
withName: firstprocess {
queue = 'small'
}
withName: secondprocess {
queue = 'large'
}
}
3 replies