Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    aconkli4
    @aconkli4
    python: can't open file './hdbscan_script_for_hpc.py': [Errno 2] No such file or directory
    Julian Pistorius
    @julianpistorius
    Hmm... Is this on your laptop I assume?
    Oh, and you get the error on the second sudo command?
    @aconkli4 - I think there might be an issue with the path in that container. I assume running this as a Docker container works?
    aconkli4
    @aconkli4
    yes this is on my laptop and i got the same error on hpc (without the sudo)
    it does work as a docker container
    Julian Pistorius
    @julianpistorius

    What happens if you run this?

    ```
    singularity run qtl-clustering.simg ls -al
    singularity run qtl-clustering.simg bash -c 'echo $PATH'

    aconkli4
    @aconkli4
    same error. it doesn't seem to be able to get past its inability to find the python script
    so I have ENTRYPOINT [ "python", "./hdbscan_script_for_hpc.py"] in my Dockerfile. should i change that to "hdbscan_script_for_hpc.py" only and rebuild?
    or ENTRYPOINT [ "python", "hdbscan_script_for_hpc.py"] rather?
    Julian Pistorius
    @julianpistorius
    Oh you know what.
    aconkli4
    @aconkli4
    what's that?
    Julian Pistorius
    @julianpistorius
    I bet it's because Singularity overwrites the home directory.
    singularity run qtl-clustering.simg pwd
    That's one of the gotchas when going from Docker to Singularity.
    aconkli4
    @aconkli4
    ok. do i need to rebuild my docker image?
    Julian Pistorius
    @julianpistorius

    Yeah. Probably. It's best to not use the home directory for storing stuff in containers. Use a path like /my-software/whatever.py or /opt/my-software/whatever.py.

    Background: The home directory inside the container will be replaced with the home directory of the user on the host (your laptop or HPC) that runs the container.

    aconkli4
    @aconkli4
    oh! that's what you meant. I'll give it a shot
    thanks!
    Julian Pistorius
    @julianpistorius
    Sure thing! Let us know what happens.
    aconkli4
    @aconkli4
    works on my laptop!
    Julian Pistorius
    @julianpistorius
    Heh... Great.
    aconkli4
    @aconkli4
    thanks for the advice.
    Julian Pistorius
    @julianpistorius
    My pleasure! Happy to help.
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Hello, I am having similar issue. When I want to go from Docker to Singularity , my script does not run!
    I am not sure if I don't bind my data properly or what !
    In Docker
    docker run -it -v $PWD/Input:/Input -v $PWD/Output:/Output elizadeh/batch:4.0 -p /Output/Batch_data2019_05_29.h5
    runs with no error.
    But with Singularity
    singularity run -B $PWD/Input:/Input -B $PWD/Output:/Output cellprofiler_3.0.0.sif -p /Output/Batch_data2019_05_29.h5
    does not run. I only get a warning : "seccomp requested but not enabled, seccomp library is missing or too old"
    Julian Pistorius
    @julianpistorius
    Hi @Elaheh-Alizadeh I think it might be the -p parameter. You don't need it. It's used in Docker only for binding ports. Try running singularity without -p.
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Thanks julian I tried it. Now the error is:
    s6-mkdir: warning: unable to mkdir /var/run/s6: Read-only file system.
    Julian Pistorius
    @julianpistorius
    Hmm... are you using Singularity on HPC or on Atmosphere?
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    on HPC
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    I get the same warning on Atmosphere
    Julian Pistorius
    @julianpistorius
    Actually, I think that -p might be necessary for the cellprofiler command.
    I misread where the -p was. I thought it was before the Singularity image, but it's after. Can you please paste the whole singularity command here?
    And maybe send the image to me?
    You can use this to send me the file: https://send.firefox.com/
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    -p is for determining the pipeline. But it does not make any difference. I get the same error
    the image is from docker://cellprofiler/cellprofiler:3.0.0
    singularity run -B $PWD/Input:/Input -B $PWD/Output:/Output cellprofiler_3.0.0.sif -p $PWD/Output/Batch_data2019_05_29.h5 -f 1 -l 1
    Julian Pistorius
    @julianpistorius
    Ah.
    Well, before I say 'ah', let me check something.
    OK, I think I've figured it out.
    Julian Pistorius
    @julianpistorius
    It's the Docker entry point which is causing problems when run with Singularity. It tries to create directories in /var/run and all kinds of things. I don't think you need any of that.
    Trying to get you a different command.
    Try this: singularity exec -B $PWD/Input:/Input -B $PWD/Output:/Output cellprofiler_3.0.0.sif cellprofiler -p /Output/Batch_data2019_05_29.h5 -f 1 -l 1
    I changed:
    • run to exec
    • -p $PWD/Output/Batch_data2019_05_29.h5 to -p /Output/Batch_data2019_05_29.h5
    Julian Pistorius
    @julianpistorius
    And I also added the cellprofiler command after the image file and before -p