Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Julian Pistorius
    @julianpistorius
    And I also added the cellprofiler command after the image file and before -p
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    It's working !!
    Thanks a ton Julian
    Julian Pistorius
    @julianpistorius
    My pleasure! Glad you got it working.
    Julian Pistorius
    @julianpistorius
    Julian Pistorius
    @julianpistorius
    I’m running a bit late for Coffee & Code. See y’all soon.
    Julian Pistorius
    @julianpistorius
    Devin Bayly
    @DevinBayly
    @julianpistorius hi! I never got to hear your question from yesterday morning about native messaging, what did you figure out?
    Julian Pistorius
    @julianpistorius
    Hi @DevinBayly! No, I was going to ask whether you had found anything else out. I have a use-case for it. :D
    Devin Bayly
    @DevinBayly
    sadly not yet, I started learning about MessageChannels which I need for messages between iframes, but no more on native.
    what was you use case?
    Julian Pistorius
    @julianpistorius
    Kinda tricky to explain. It involves Docker. :D
    Devin Bayly
    @DevinBayly
    your*
    oh interesting! I've never had success with webby docker things but maybe this could change all that. I'll get the scoop on thursday eh?
    Julian Pistorius
    @julianpistorius
    Yep! Maybe even a proof of concept!
    Devin Bayly
    @DevinBayly
    @julianpistorius just ordered my first zero phone part .. here we go side project hahah
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Has anyone used job array to run matlab code on cluster that launches parpool?
    I am using UAHPC . I am having issues for some of the jobs. To me it looks like only limited number of jobs can use parpool at the same time!
    Here is examples of errors for some jobs:

    Caused by:
    Error using parallel.internal.pool.InteractiveClient>iThrowWithCause (line
    678)
    Failed to start pool.
    Error using parallel.Cluster/createConcurrentJob (line 1145)
    Unable to write to MAT-file
    /home/u5/elahehalizadeh/.matlab/local_cluster_jobs/R2019a/Job265.in.mat.
    The file may be corrupt.

    ==> ThirdStep_TISMorph.e2586980.21 <==
    Parallel pool failed to start with the following error. For more detailed
    information, validate the profile 'local' in the Cluster Profile Manager.

    Caused by:
    Error using parallel.internal.pool.InteractiveClient>iThrowWithCause (line
    678)
    Failed to start pool.
    Error using parallel.Job/createTask (line 320)
    Only one task may be created on a concurrent Job.

    ==> ThirdStep_TISMorph.e2586978.38 <==
    Caused by:
    Error using parallel.internal.pool.InteractiveClient>iThrowWithCause (line
    678)
    Failed to start pool.
    Error using
    parallel.internal.customattr.CustomGetSet/hSetAllowNonPublic (line 87)
    Cannot create 'Task1.in.mat' because
    '/home/u5/elahehalizadeh/.matlab/local_cluster_jobs/R2019a/Job253' does
    not exist.

    Julian Pistorius
    @julianpistorius
    Hi Elaheh. Let me check.
    Julian Pistorius
    @julianpistorius

    Somebody at Hacky Hour says:

    Tell her to contact HPC consult, may be a limit we set on the number of jobs. Tell her to make sure to include code

    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Thanks. I have. No response yet! I emailed them again.
    Julian Pistorius
    @julianpistorius
    Hmm... @bjoyce3?
    Julian Pistorius
    @julianpistorius
    @Elaheh-Alizadeh, Blake says he'll get in touch with you. Apparently this involves some code specific stuff. πŸ€“πŸ’»
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    okay thanks!
    Blake Joyce
    @bjoyce3
    Hey Elaheh, we're looking into the issue. The first of the year has been kinda crazy for us.
    We think it may be a licensing issue....in that you're asking for a number of processes that exceeds how many Matlab licenses we have. But we're not sure
    Blake Joyce
    @bjoyce3
    @Elaheh-Alizadeh this line has me a bit confused: /home/u5/elahehalizadeh/.matlab/local_cluster_jobs/R2019a/Job265.in.mat.
    The file may be corrupt.
    Can you check the matlab code you're using to make sure that file transferred properly and isn't corrupt?
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    I am running a job array and it pulls the same code for all the jobs. The code processes bunch of images and for different jobs it just pulls different set of images. I do not think the code is corrupted since it ran successfully for some jobs. The job is supposed to launch parpool and run the images. So, if there is an issue with launching the parpool it will stop in first 2-3 minutes. I just reran that specific job with the error you mentioned and it is running with no error. I feel like I can only launch parpool once at a given time. That's because I see the jobs that start at the same time fail launching parpool except one job. Is there a way to delay array jobs so each starts 4 minutes after the job before?
    Blake Joyce
    @bjoyce3
    I would assume there's a way to do that
    I'm not familiar with a wait function off the top of head for PBS arrays, but I'll ask the HPC consultants tomorrow to get in touch with you
    It is interesting that rerunning the job meant it worked fine. Have you sent us the PBS array file too?
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    I have in my first email.
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Now I am 100% sure that it is some kind of license issue. I ran one job at a time (10 jobs that had error before) with no issue.
    Blake Joyce
    @bjoyce3
    We'll dig in. It'll take some time though
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Okay. Let me know when you know how to delay the each job for couple of minutes. This should solve my problem for now.
    Blake Joyce
    @bjoyce3
    Will do. Thanks for being patient with us. Things will get better on our response time once the first month is over!
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Thanks
    aconkli4
    @aconkli4
    Blake, Julian, do either of you know where I could get help implementing locality sensitive hashing?
    Julian Pistorius
    @julianpistorius
    Hi @aconkli4! What's the context?
    aconkli4
    @aconkli4
    Hi Julian!
    I am trying to implement an implicit correlation network analysis (finding modules in a network without calculating all the edge weights). One of the steps is to convert negative edge weights into positive edge weights. This is accomplished by squaring the edge weights. The goal (I think) is to utilize a set of hashing functions so that I can approximate the squared dot products implicitly.
    Julian Pistorius
    @julianpistorius
    Interesting! What have you looked at so far? Are there any libraries which do approximately what you need?
    aconkli4
    @aconkli4

    Maybe. My primary problem is figuring out how to sausage grind the math into code. There may be some library that can help me, but I probably wouldn't know if I saw it.
    Unfortunately, my Youtube level education on LSH is not measuring up well to the math in this dissertation I am reading.

    So far I have googled a lot of LSH tutorials and had min-hashing and Jaccard distance between text documents explained to me a half dozen times.

    aconkli4
    @aconkli4

    So maybe a better question is, do you know really a patient math/computer science tutor that would be willing to put up with my math inadequacy for a while?

    @Elaheh-Alizadeh I just realized you are in this group! Would you maybe be willing to help me some time this week?

    Julian Pistorius
    @julianpistorius
    Hmm... I wonder if Ian Bertolacci might be able to help.
    I've asked on the Python Meetup Slack.
    Alex Pakalniskis
    @alex-pakalniskis
    @aconkli4 I have weekly Python drop-in hours on Monday evenings (https://new.library.arizona.edu/events/python-programming-drop-hours) and I'm patient. Happy to help however I can. Feel free to message me.
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Sure, How about this afternoon?