[Adam Novak, UCSC GI] I don't believe that Toil makes any attempt to preserve file permissions when copying them around, even the execute bit. A workaround might be to tar up the script and then untar it before executing it, or just cram a chmod +x in somewhere in the CWL workflow somehow.
It would be good if Toil could make sure to set the execute bit when downloading stuff that was executable when it was stored. You can open a bug for that and we migth be able to do it.
prom/node-exporter(the only dockerhub image we currently pull during initialization) or provide more built-in ways for folks to authenticate to dockerhub with pro/team accounts when running large workflows?
cwlrunnerto do all its Docker stuff anyway.
cwltoolrunning wthin a pod can handle a
DockerRequirementunless you write a bunch of logic to specially prepare the pod and bring up the Docker daemon inside it.
singularitywith user-mode namespaces to handle
DockerRequirementwhen the Docker daemon isn't available and/or can't start, that would solve the problem.
[2020-10-18T21:48:20+0000] [MainThread] [I] [toil.provisioners.node] Attempting to establish SSH connection... [2020-10-18T21:48:20+0000] [MainThread] [I] [toil.provisioners.node] Executing the command "ps" on the appliance returned a non-zero exit code 255 with stdout b'' and stderr b"Warning: Permanently added '188.8.131.52' (ECDSA) to the list of known hosts.\r\email@example.com: Permission denied (publickey,password,keyboard-interactive).\r\n" [2020-10-18T21:48:20+0000] [MainThread] [I] [toil.provisioners.node] Connection rejected, waiting for public SSH key to be propagated. Trying again in 10s.
After about a minute, the process fails with the following error.
RuntimeError: Key propagation failed on machine with ip 184.108.40.206
However, the instance itself seems to be configured correctly? I am able to ssh into the instance manually and the docker processes are running:
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 4b746d4ee9ba quay.io/ucsc_cgl/toil:4.2.0-3aa1da130141039cb357efe36d7df9b9f6ae9b5b-py3.6 "mesos-master --log_…" 14 minutes ago Up 14 minutes toil_leader f4b09dc519c1 prom/node-exporter:v0.15.2 "/bin/node_exporter …" 15 minutes ago Up 14 minutes 0.0.0.0:9100->9100/tcp node-exporter
ssh firstname.lastname@example.org), and then I think
toil ssh-clustershould work for everyone to log into the appliance.