These are chat archives for nextflow-io/nextflow

4th
Feb 2019
Vladimir Kiselev
@wikiselev
Feb 04 10:43
How does NF deal with the cases when the user is not root in the docker container? We get a permission error when nextflow try to touch it’s own files
Paolo Di Tommaso
@pditommaso
Feb 04 10:45
plain docker or k8s ?
Vladimir Kiselev
@wikiselev
Feb 04 10:45
k8s
Paolo Di Tommaso
@pditommaso
Feb 04 10:47
uff, the rocker image !
you can try try to use pod.runAsUser setting
Vladimir Kiselev
@wikiselev
Feb 04 10:55
Oh, that’s great, sorry, haven’t checked the k8s docs for a while. Thanks and sorry for bothering!
Paolo Di Tommaso
@pditommaso
Feb 04 10:55
no pb
let me know if it solves
Vladimir Kiselev
@wikiselev
Feb 04 10:55
:+1:
Vladimir Kiselev
@wikiselev
Feb 04 14:21

@pditommaso, all worked with the following settings:

# in the process itself
pod = [runAsUser: 0]

# add user specific environment
env.PATH = '/home/user/dropEst/build:$PATH'
env.LD_LIBRARY_PATH = '/usr/local/lib/R/lib/'
env.R_PROFILE = '/home/user/.Rprofile'

many thanks!

Paolo Di Tommaso
@pditommaso
Feb 04 14:26
uhh
the docker user model is completely broken BTW
hopefully singularity should be available also for K8s soon
Vladimir Kiselev
@wikiselev
Feb 04 14:31
well, yes, I suppose they had to do that to be able to run on an HPC cluster
Vladimir Kiselev
@wikiselev
Feb 04 14:35
oh, cool! :+1:
Paolo Di Tommaso
@pditommaso
Feb 04 14:37
even though, we'll install amazon in house, soon .. :satisfied:
micans
@micans
Feb 04 15:25
I'm just starting to use shelll: blocks -- I gather that bash parameter expansion (e.g. ${sam%.sam}) still needs escaping of dollar sign, as well as command substitution (e.g. $(basename sth)).
Paolo Di Tommaso
@pditommaso
Feb 04 15:26
?
micans
@micans
Feb 04 15:40
oh oh ... I will recheck this and make a small example in case what I said made sense
Paolo Di Tommaso
@pditommaso
Feb 04 15:41
ahh
no, you don't need to escape $(basename sth) as long as you use ''' instead of """
micans
@micans
Feb 04 15:42
oooh. I see ...... makes sense. !{} will be escaped in ''' I guess
Paolo Di Tommaso
@pditommaso
Feb 04 15:44
!{} works in both, both ${} is interpreted only by """
micans
@micans
Feb 04 15:44
super, thanks :+1:
Paolo Di Tommaso
@pditommaso
Feb 04 15:44
:+1:
micans
@micans
Feb 04 15:55

I tried this:

  shell:
  def S = samplename
  def B = bcd
  def P = psg
  '''
  dir="demux-!{S}"

it does not work. If I change !{S} to !{samplename} it works. The documentation suggests this mechanism works for process input variables .... so there is a slight difference with script sections, is that correct? I believe in a script section I could have ${S} similar to above.

Paolo Di Tommaso
@pditommaso
Feb 04 15:55
no def
micans
@micans
Feb 04 15:55
gaaah. Thanks again :-)
Paolo Di Tommaso
@pditommaso
Feb 04 15:57
welcome
Stephen Kelly
@stevekm
Feb 04 16:30
@pditommaso , @tobsecret & I were wondering how you make sure that when you hold Nextflow tutorial seminar meetings, everyone that arrives has a Nextflow compatible system. For example, we were planning to do a Nextflow workshop soon but were concerned that some people might show up with only a Windows laptop and no access to either a Mac or Linux machine. What do you do to prevent that?
Paolo Di Tommaso
@pditommaso
Feb 04 16:31
cloud preconfigured environment
Stephen Kelly
@stevekm
Feb 04 17:45
thanks, @tobsecret we might want to look into this, it would also solve the problem of making bioinformatics tools available for everyone to use e.g. Fastqc, etc.
@pditommaso do you encounter a lot of trouble with getting first-timers able to use the cloud environment and subsequently program in Nextflow? For example, Windows users would need to all get PuTTY, make ssh keys, get them propagated to the server, get something like CyberDuck to do local script file editing with, get an appropriate text editor...
is this something you factor into your workshops?
Paolo Di Tommaso
@pditommaso
Feb 04 17:49
in our workshop, instances are password less, data is already provisioned, tools preinstalled
user only need to access with ssh
Stephen Kelly
@stevekm
Feb 04 17:50
what size instances do you use?
Paolo Di Tommaso
@pditommaso
Feb 04 17:50
50 GB
Stephen Kelly
@stevekm
Feb 04 17:51
I am not sure how much RAM & CPU would be required for e.g. 20 people to work simultaneously on a demo Nextflow pipeline development
Paolo Di Tommaso
@pditommaso
Feb 04 17:52
I used t2.large instances