These are chat archives for nextflow-io/nextflow

11th
Jun 2018
spaceturtle
@spaceturtle
Jun 11 2018 02:25
Hi, I have a question. When I used nohup to run nextflow pipeline in the background, how can I kill the pipeline process? I don't think I cound find the process ID in ps. Any suggestion?
Barry Moore
@barrymoore
Jun 11 2018 02:31
You'll see it if you ps aux | grep nextflow. If you have htop installed you can always search and kill interactively with that.
spaceturtle
@spaceturtle
Jun 11 2018 02:59
@barrymoore Thanks. It works fine.
Paolo Di Tommaso
@pditommaso
Jun 11 2018 06:28
@barrymoore params is a Map object, therefore you can use either params.get(key) or params[key] that in a string become "${params.get(key)}" or "${params[key]}"
@spaceturtle if you use the -bg command line option the pid is save to .nextflow.pid file
Vladimir Kiselev
@wikiselev
Jun 11 2018 09:30
hi @pditommaso, is the nextflow.config loaded before execution of the main.nf
I have an error in 0.30.0, which does not happen in 0.29.1
Paolo Di Tommaso
@pditommaso
Jun 11 2018 09:31
Yep
Fill a bug please
Vladimir Kiselev
@wikiselev
Jun 11 2018 09:31
ok
Vladimir Kiselev
@wikiselev
Jun 11 2018 09:51
nextflow-io/nextflow#738
Maxime Garcia
@MaxUlysse
Jun 11 2018 10:55
Hi @pditommaso I'm trying to use file() in a function in a library, but it doesn't seem to work, should I import some Nextflow library?
Paolo Di Tommaso
@pditommaso
Jun 11 2018 12:25
Yes import static nextflow.Nextflow.file
Maxime Garcia
@MaxUlysse
Jun 11 2018 12:31
Thanks a lot ;-D
Rohan Shah
@rohanshah
Jun 11 2018 16:48
what sets the current directory for processes that run in AWS Batch. Whenever I try to echo $PWD the .command.sh file contains echo /usr/src/nextflow/ in it. Where is that value set?
Paolo Di Tommaso
@pditommaso
Jun 11 2018 16:49
not sure to understand
Rohan Shah
@rohanshah
Jun 11 2018 16:51
ah sorry, what I mean is, how to the working directory for the batch executor determined?
when a process runs in batch
it seems to run from the directory /usr/src/nextflow
but that doesnt seem to be set anywhere by me or the nextflow code
the only place that exists is the location where I run nextflow
Paolo Di Tommaso
@pditommaso
Jun 11 2018 16:53
it should be a path in the /tmp directory automatically crated by NF
Rohan Shah
@rohanshah
Jun 11 2018 16:54
hmm it seems to be using the same directory as the main nextflow process
ah wait I could be wrong
Paolo Di Tommaso
@pditommaso
Jun 11 2018 16:54
I guess so :smile:
Rohan Shah
@rohanshah
Jun 11 2018 16:54
if I do echo $PWD with the script section of a process
Paolo Di Tommaso
@pditommaso
Jun 11 2018 16:54
escape it
Rohan Shah
@rohanshah
Jun 11 2018 16:54
will nextflow replace $PWD with the current directory of nextflow
ah
ty
Paolo Di Tommaso
@pditommaso
Jun 11 2018 16:55
otherwise is resolved by NF
Rohan Shah
@rohanshah
Jun 11 2018 16:55
this makes sense ty
Paolo Di Tommaso
@pditommaso
Jun 11 2018 16:55
welcome
Shawn Rynearson
@srynobio
Jun 11 2018 17:05
I created an feature nextflow-io/nextflow#631 and have a related question. Is it possible to run a nf script from within a nf script without causing a collision issue?
Paolo Di Tommaso
@pditommaso
Jun 11 2018 17:07
what kind of collision ?
Rohan Shah
@rohanshah
Jun 11 2018 17:11
how do I override the tmp directory created by nextflow in an executor?
i tried using the scratch directive as suggested before
but it still sets it to /tmp/nxf.MoUPd5tBvU or such even though I have scratch "/docker_scratch" in my process and in the nextflow.config file
Paolo Di Tommaso
@pditommaso
Jun 11 2018 17:12
process.scratch = '/some/path'
Rohan Shah
@rohanshah
Jun 11 2018 17:13
yes I have
process foo {
  scratch "/docker_scratch"
}
Paolo Di Tommaso
@pditommaso
Jun 11 2018 17:13
and ?
ah
Rohan Shah
@rohanshah
Jun 11 2018 17:13
and I have
process {
  scratch = "/docker_scratch"
}
in the nextflow.config
neither work to change the tmp directory
Paolo Di Tommaso
@pditommaso
Jun 11 2018 17:13
it should work, it doesnt?
please fill an issue
Rohan Shah
@rohanshah
Jun 11 2018 17:14
no I still get the above /tmp/nxf. MoUPd5tBvU type tmp folder
will do
Paolo Di Tommaso
@pditommaso
Jun 11 2018 17:14
:+1:
Rohan Shah
@rohanshah
Jun 11 2018 17:14
thank you
Paolo Di Tommaso
@pditommaso
Jun 11 2018 17:14
you r welcome
Shawn Rynearson
@srynobio
Jun 11 2018 17:25
The collision issue I'm thinking would be how the nf cache system works. If I'm running a nf script within a nf script would they try to read/use the same baseDir info? Also, would it cause issue when using the --resume option.
Paolo Di Tommaso
@pditommaso
Jun 11 2018 17:31
well, each task would have it's own cache
Shawn Rynearson
@srynobio
Jun 11 2018 17:40
I'll do some testing. Thanks @pditommaso
Paolo Di Tommaso
@pditommaso
Jun 11 2018 17:40
:+1:
take care that each NF job will try to use all available cpus