Where communities thrive

Join over 1.5M+ people
Join over 100K+ communities
Free without limits
Create your own community

Explore more communities

resbaz/Arizona

All things Research Bazaar Arizona! ☀️🌵💻 Need help with computing? Your fellow researchers are here to help/commiserate.

People

Repo info

Activity

Julian Pistorius

@julianpistorius

Heh... Great.

aconkli4

@aconkli4

thanks for the advice.

Julian Pistorius

@julianpistorius

My pleasure! Happy to help.

Elaheh Alizadeh

@Elaheh-Alizadeh

Hello, I am having similar issue. When I want to go from Docker to Singularity , my script does not run!

I am not sure if I don't bind my data properly or what !

In Docker

docker run -it -v $PWD/Input:/Input -v $PWD/Output:/Output elizadeh/batch:4.0 -p /Output/Batch_data2019_05_29.h5

runs with no error.
But with Singularity
singularity run -B $PWD/Input:/Input -B $PWD/Output:/Output cellprofiler_3.0.0.sif -p /Output/Batch_data2019_05_29.h5
does not run. I only get a warning : "seccomp requested but not enabled, seccomp library is missing or too old"

Julian Pistorius

@julianpistorius

Hi @Elaheh-Alizadeh I think it might be the -p parameter. You don't need it. It's used in Docker only for binding ports. Try running singularity without -p.

Elaheh Alizadeh

@Elaheh-Alizadeh

Thanks julian I tried it. Now the error is:

s6-mkdir: warning: unable to mkdir /var/run/s6: Read-only file system.

Julian Pistorius

@julianpistorius

Hmm... are you using Singularity on HPC or on Atmosphere?

Elaheh Alizadeh

@Elaheh-Alizadeh

on HPC

Elaheh Alizadeh

@Elaheh-Alizadeh

I get the same warning on Atmosphere

Julian Pistorius

@julianpistorius

Actually, I think that -p might be necessary for the cellprofiler command.

I misread where the -p was. I thought it was before the Singularity image, but it's after. Can you please paste the whole singularity command here?

And maybe send the image to me?

You can use this to send me the file: https://send.firefox.com/

Elaheh Alizadeh

@Elaheh-Alizadeh

-p is for determining the pipeline. But it does not make any difference. I get the same error

the image is from docker://cellprofiler/cellprofiler:3.0.0

singularity run -B $PWD/Input:/Input -B $PWD/Output:/Output cellprofiler_3.0.0.sif -p $PWD/Output/Batch_data2019_05_29.h5 -f 1 -l 1

Julian Pistorius

@julianpistorius

Ah.

Well, before I say 'ah', let me check something.

OK, I think I've figured it out.

Julian Pistorius

@julianpistorius

It's the Docker entry point which is causing problems when run with Singularity. It tries to create directories in /var/run and all kinds of things. I don't think you need any of that.

Trying to get you a different command.

Try this:

singularity exec -B $PWD/Input:/Input -B $PWD/Output:/Output cellprofiler_3.0.0.sif cellprofiler -p /Output/Batch_data2019_05_29.h5 -f 1 -l 1

I changed:

run to exec
-p $PWD/Output/Batch_data2019_05_29.h5 to -p /Output/Batch_data2019_05_29.h5

Julian Pistorius

@julianpistorius

And I also added the cellprofiler command after the image file and before -p

Elaheh Alizadeh

@Elaheh-Alizadeh

It's working !!

Thanks a ton Julian

Julian Pistorius

@julianpistorius

My pleasure! Glad you got it working.

Julian Pistorius

@julianpistorius

https://twitter.com/ResBazAZ/status/1134161726935408640

@channel ^^

Julian Pistorius

@julianpistorius

I’m running a bit late for Coffee & Code. See y’all soon.

Julian Pistorius

@julianpistorius

https://github.com/smorks/keepassnatmsg

Devin Bayly

@DevinBayly

@julianpistorius hi! I never got to hear your question from yesterday morning about native messaging, what did you figure out?

Julian Pistorius

@julianpistorius

Hi @DevinBayly! No, I was going to ask whether you had found anything else out. I have a use-case for it. :D

Devin Bayly

@DevinBayly

sadly not yet, I started learning about MessageChannels which I need for messages between iframes, but no more on native.

what was you use case?

Julian Pistorius

@julianpistorius

Kinda tricky to explain. It involves Docker. :D

Devin Bayly

@DevinBayly

your*
oh interesting! I've never had success with webby docker things but maybe this could change all that. I'll get the scoop on thursday eh?

Julian Pistorius

@julianpistorius

Yep! Maybe even a proof of concept!

Devin Bayly

@DevinBayly

@julianpistorius just ordered my first zero phone part .. here we go side project hahah

Elaheh Alizadeh

@Elaheh-Alizadeh

Has anyone used job array to run matlab code on cluster that launches parpool?

I am using UAHPC . I am having issues for some of the jobs. To me it looks like only limited number of jobs can use parpool at the same time!

Here is examples of errors for some jobs:

Caused by:
Error using parallel.internal.pool.InteractiveClient>iThrowWithCause (line
678)
Failed to start pool.
Error using parallel.Cluster/createConcurrentJob (line 1145)
Unable to write to MAT-file
/home/u5/elahehalizadeh/.matlab/local_cluster_jobs/R2019a/Job265.in.mat.
The file may be corrupt.

==> ThirdStep_TISMorph.e2586980.21 <==
Parallel pool failed to start with the following error. For more detailed
information, validate the profile 'local' in the Cluster Profile Manager.

Caused by:
Error using parallel.internal.pool.InteractiveClient>iThrowWithCause (line
678)
Failed to start pool.
Error using parallel.Job/createTask (line 320)
Only one task may be created on a concurrent Job.

==> ThirdStep_TISMorph.e2586978.38 <==
Caused by:
Error using parallel.internal.pool.InteractiveClient>iThrowWithCause (line
678)
Failed to start pool.
Error using
parallel.internal.customattr.CustomGetSet/hSetAllowNonPublic (line 87)
Cannot create 'Task1.in.mat' because
'/home/u5/elahehalizadeh/.matlab/local_cluster_jobs/R2019a/Job253' does
not exist.

Julian Pistorius

@julianpistorius

Hi Elaheh. Let me check.

Julian Pistorius

@julianpistorius

Somebody at Hacky Hour says:

Tell her to contact HPC consult, may be a limit we set on the number of jobs. Tell her to make sure to include code

Elaheh Alizadeh

@Elaheh-Alizadeh

Thanks. I have. No response yet! I emailed them again.