chA = Channel.from('A')
chB = Channel.from('B')
// if I declare process here is ok
process proc {
input:
val x from chA
output:
stdout chA2
"""
echo $x
"""
}
chC = Channel.empty().mix(chA2, chB)
// If I declare process here
// I get an error "no such variable chA2"// all my channels declaration logic here
// all my processes there
full example
#!/usr/bin/env nextflow
/* Def channel then proc then channel */
chA = Channel.from('A')
chB = Channel.from('B')
process procAtoA2 {
input: val x from chA
output: stdout chA2
"echo $x$x"
}
chC = Channel.empty().mix(chA2, chB)
.subscribe {println "channel C out: $it"}
///* Def channel then proc */
//chE = Channel.from('E')
//chF = Channel.from('F')
//chG = Channel.empty().mix(chE2, chF)
// .subscribe {println "channel G out: $it"}
//
//process procEtoE2 {
// input: val x from chE
// output: stdout chE2
// "echo $x$x"
//}as is, chC print is working... if you uncomment the second section of the code an error pop up "No such variable: chE2"
mix(), so operate, on a channel which is declared in procEtoE2.
subscribe() fire the error :-/
output: file "my folder" into chPath ?
Or a way to make groovy-mode play nice with Nextflow indentation?
rm -r $someinputfile as an afterScript, Nextflow will rerun the upstream task even when I use the -resume flag?
Hi everyone,
I got some play-money to run some TCGA analysis on AWS :)
So after doing quite some reading on AWS manuals and also the Nextflow documentation, I have some very basic beginner questions - I hope somebody might be kind enough to help out an absolute AWS novice.
The major difference between setting up a cluster on AWS and AWS batch is that for AWS batch you predefine your setup (number of EC2 instances + resources) beforehand and can launch pipelines locally vs with nextflow cloud you would dynamically launch instances and run pipelines on the masternode?
Next: The typical workflow is to put input files into s3 -> then you launch the pipeline which will load any required files (containers + input files from s3) into ebs of the respective ec2 instance -> do the computation -> copy results back to s3
Is that correct?
I was just wondering how to scale the EBS of the custom image, do I need to calculate all image files, input files, intermediate files and output files or will input be directly read from s3 and output be directly written to s3?
The documentation says "the pipeline execution must specifies a S3 bucket where jobs intermediate results are stored with the -bucket-dir command line options.", so it's not clear to me where intermediate files are stored and why they would be stored both on EBS and S3.
Finally: The documentation says that the AMI needs the aws cli available. However in the documentation it says to create the AMI and only after install the aws CLI. Should that not be installed when creating the AMI so it's available within each AMI?
Any input on that would be highly appreciated to understand the whole process more and also save money. Thanks in advance.