These are chat archives for nextflow-io/nextflow

19th
Jan 2017
Eivind Gard Lund
@eivindgl
Jan 19 2017 09:35
shoot, I guess I shouldn't develop a workflow on real datasets, but I did a change and I forgot to add the -resume option when I ran nextflow. I quickly aborted the from scratch run. Is there anyway I can resume the old workflow cache? I tried this: (where infallible_fermat is the run I would like to resume)
$ nextflow run preprocess.nf -resume -name infallible_fermat
Run name `infallible_fermat` has been already used -- Specify a different one
Paolo Di Tommaso
@pditommaso
Jan 19 2017 09:45
yes, you need to specify infallible_fermat as an argument of -resume (by default it's the last one)
$ nextflow run preprocess.nf -resume infallible_fermat
Eivind Gard Lund
@eivindgl
Jan 19 2017 10:00
@pditommaso That did not work for me. It did work when I specified the session-id (from the log command). Many thanks for the help!
Paolo Di Tommaso
@pditommaso
Jan 19 2017 10:00
:+1:
Fredrik Boulund
@boulund
Jan 19 2017 12:24
how do I ensure a reference database file is copied to the scratch dir when running a process?
Can i just define
input:
file somename from input_channel
file 'database.sqlite3' from params.database_file
Paolo Di Tommaso
@pditommaso
Jan 19 2017 12:25
nope
params.database_file is a string
Fredrik Boulund
@boulund
Jan 19 2017 12:25
make a single file channel out of it somehow?
Paolo Di Tommaso
@pditommaso
Jan 19 2017 12:26
thus you will need to convert it to a file object with the file function
Fredrik Boulund
@boulund
Jan 19 2017 12:26
like this (from the docs)?
myFileChannel = Channel.fromPath( '/data/some/bigfile.txt' )
Paolo Di Tommaso
@pditommaso
Jan 19 2017 12:27
yes, but I guess the that the reference db is always the same
Fredrik Boulund
@boulund
Jan 19 2017 12:27
or do I just do
databaseFile = file(params.database_file)
Paolo Di Tommaso
@pditommaso
Jan 19 2017 12:27
yep
much better
Fredrik Boulund
@boulund
Jan 19 2017 12:27
Ok, thanks! :)
then I don't need to specify it as file databaseFile in the input declarations, it should be enough with databaseFile, right?
Would it work with file dbfile from databaseFile btw?
Paolo Di Tommaso
@pditommaso
Jan 19 2017 12:31
No. Input file must be always typed, even if they are file objects ..
Fredrik Boulund
@boulund
Jan 19 2017 12:31
ok, makes sense
Paolo Di Tommaso
@pditommaso
Jan 19 2017 12:31
yes, you need to use file dbfile from databaseFile
or
file 'database.sqlite3' from databaseFile
Fredrik Boulund
@boulund
Jan 19 2017 12:33
Great! Slowly getting there :)
Paolo Di Tommaso
@pditommaso
Jan 19 2017 12:33
:)
Fredrik Boulund
@boulund
Jan 19 2017 14:23
another quick question, how do I limit the maximum number of concurrent jobs submitted via slurm?
Trevor Tanner
@tantrev
Jan 19 2017 14:49
Are Singularity images mounted as writable?
Félix C. Morency
@fmorency
Jan 19 2017 14:49
I hope not.
iirc you can't mount them as writable as a normal user
Trevor Tanner
@tantrev
Jan 19 2017 14:56
Interesting. The .img file seems to be updated when I exec one as "--writable" under non-sudo? I'm new to them and just trying to figure out how to save files from programs executed within them.
Félix C. Morency
@fmorency
Jan 19 2017 14:57
I might be wrong. However, what is your usage exactly?
Trevor Tanner
@tantrev
Jan 19 2017 14:59
Pretty basic - I just want to capture the output of a program like "echo hi > hi.txt". Eventually want to get an RNA-Seq pipeline going though.
That's how I'd do it in docker anyways. Singularity complains about being read-only.
Félix C. Morency
@fmorency
Jan 19 2017 15:00
Why not save the file outside of the container?
Trevor Tanner
@tantrev
Jan 19 2017 15:02
I'm probably doing something fundamentally wrong but I guess I don't really understand how? Do you mean like a bind point?
Félix C. Morency
@fmorency
Jan 19 2017 15:08
Singularity mounts a lot of stuff automagically. It's not like docker where you need to explicitly specify an external volume
Trevor Tanner
@tantrev
Jan 19 2017 15:09
Thanks, just figured it out. :) The directory I was trying to write to just wasn't being automatically mounted by Singularity (I thought it was).
Félix C. Morency
@fmorency
Jan 19 2017 15:09
\o/
Trevor Tanner
@tantrev
Jan 19 2017 15:13
Scratch that, it was a permissions issue. Used to not having to worry about these things lol.
Félix C. Morency
@fmorency
Jan 19 2017 15:53
@pditommaso Does NF download the necessary input file locally when using a scratch?
Paolo Di Tommaso
@pditommaso
Jan 19 2017 16:12
@boulund Use the queueSize config setting and the -qs command line option
@fmorency Um, well NF creates symlinks for the input files.
I'm missing what's the problem
Félix C. Morency
@fmorency
Jan 19 2017 16:15
No problem. I was just curious.