These are chat archives for nextflow-io/nextflow

6th
Apr 2016
Matthieu Foll
@mfoll
Apr 06 2016 14:35
Hi Paolo
Paolo Di Tommaso
@pditommaso
Apr 06 2016 14:35
Hi
Matthieu Foll
@mfoll
Apr 06 2016 14:35
We sometimes (often) have pipes in a shell of a process
Paolo Di Tommaso
@pditommaso
Apr 06 2016 14:35
what do you mean ?
Matthieu Foll
@mfoll
Apr 06 2016 14:35
cat test | grep hello
for ex
Paolo Di Tommaso
@pditommaso
Apr 06 2016 14:36
ok
Matthieu Foll
@mfoll
Apr 06 2016 14:36
if the first command returns an error (exit code 1) but the second works the overall exitcode is 0
I know in linux you can change this behavior with set -o pipefail
Paolo Di Tommaso
@pditommaso
Apr 06 2016 14:37
yes you are right
Matthieu Foll
@mfoll
Apr 06 2016 14:37
Don’t you think it would be a good idea to systematically do this the same way you put set -u and set -o in all scripts nextflow generates?
Paolo Di Tommaso
@pditommaso
Apr 06 2016 14:38
in principle it could, though it could be dangerous to enforce it because it could break come existing code
Matthieu Foll
@mfoll
Apr 06 2016 14:41
ok
but we can put it ourself in the shell section
Paolo Di Tommaso
@pditommaso
Apr 06 2016 14:41
surely
Matthieu Foll
@mfoll
Apr 06 2016 14:41
ok thanks
we will try
Paolo Di Tommaso
@pditommaso
Apr 06 2016 14:42
it should be also possible to specify the your shebang header line
by default it is #!/bin/bash -ue
It should be possible to add in the config file
process.shell = ['/bin/bash', '-ue', .. ]
but I'm not sure there's a pipefile option for that
Matthieu Foll
@mfoll
Apr 06 2016 14:57
It seems that /bin/bash -ueo pipefail is working
Paolo Di Tommaso
@pditommaso
Apr 06 2016 14:58
ah!
cool
Matthieu Foll
@mfoll
Apr 06 2016 14:59

So I should put:

process.shell = ['/bin/bash', '-ueo','pipefail' ]

?

Paolo Di Tommaso
@pditommaso
Apr 06 2016 14:59
I think so
Matthieu Foll
@mfoll
Apr 06 2016 14:59
ok I’ll try
Paolo Di Tommaso
@pditommaso
Apr 06 2016 14:59
good let me know
Matthieu Foll
@mfoll
Apr 06 2016 15:13
Great it works. Thanks for your help.
Paolo Di Tommaso
@pditommaso
Apr 06 2016 15:13
:+1:
Jason Byars
@jbyars
Apr 06 2016 17:38
if s3 buckets have a policy requiring SSE, is there any nextflow setting to use SSE?
Paolo Di Tommaso
@pditommaso
Apr 06 2016 17:39
I don't think so
could you open a feature request for that?
Jason Byars
@jbyars
Apr 06 2016 17:41
sure, but first, is the implementation just wrapping awscli? If so I might be able to find a setting there
Paolo Di Tommaso
@pditommaso
Apr 06 2016 17:41
no, it's based on the aws-s3-sdk
It shouldn't too difficult
Jason Byars
@jbyars
Apr 06 2016 17:45
ok, so where do official feature requests go? github issues, google groups forum?
Paolo Di Tommaso
@pditommaso
Apr 06 2016 17:51
thanks!