Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Here is examples of errors for some jobs:

    Caused by:
    Error using parallel.internal.pool.InteractiveClient>iThrowWithCause (line
    678)
    Failed to start pool.
    Error using parallel.Cluster/createConcurrentJob (line 1145)
    Unable to write to MAT-file
    /home/u5/elahehalizadeh/.matlab/local_cluster_jobs/R2019a/Job265.in.mat.
    The file may be corrupt.

    ==> ThirdStep_TISMorph.e2586980.21 <==
    Parallel pool failed to start with the following error. For more detailed
    information, validate the profile 'local' in the Cluster Profile Manager.

    Caused by:
    Error using parallel.internal.pool.InteractiveClient>iThrowWithCause (line
    678)
    Failed to start pool.
    Error using parallel.Job/createTask (line 320)
    Only one task may be created on a concurrent Job.

    ==> ThirdStep_TISMorph.e2586978.38 <==
    Caused by:
    Error using parallel.internal.pool.InteractiveClient>iThrowWithCause (line
    678)
    Failed to start pool.
    Error using
    parallel.internal.customattr.CustomGetSet/hSetAllowNonPublic (line 87)
    Cannot create 'Task1.in.mat' because
    '/home/u5/elahehalizadeh/.matlab/local_cluster_jobs/R2019a/Job253' does
    not exist.

    Julian Pistorius
    @julianpistorius
    Hi Elaheh. Let me check.
    Julian Pistorius
    @julianpistorius

    Somebody at Hacky Hour says:

    Tell her to contact HPC consult, may be a limit we set on the number of jobs. Tell her to make sure to include code

    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Thanks. I have. No response yet! I emailed them again.
    Julian Pistorius
    @julianpistorius
    Hmm... @bjoyce3?
    Julian Pistorius
    @julianpistorius
    @Elaheh-Alizadeh, Blake says he'll get in touch with you. Apparently this involves some code specific stuff. 🤓💻
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    okay thanks!
    Blake Joyce
    @bjoyce3
    Hey Elaheh, we're looking into the issue. The first of the year has been kinda crazy for us.
    We think it may be a licensing issue....in that you're asking for a number of processes that exceeds how many Matlab licenses we have. But we're not sure
    Blake Joyce
    @bjoyce3
    @Elaheh-Alizadeh this line has me a bit confused: /home/u5/elahehalizadeh/.matlab/local_cluster_jobs/R2019a/Job265.in.mat.
    The file may be corrupt.
    Can you check the matlab code you're using to make sure that file transferred properly and isn't corrupt?
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    I am running a job array and it pulls the same code for all the jobs. The code processes bunch of images and for different jobs it just pulls different set of images. I do not think the code is corrupted since it ran successfully for some jobs. The job is supposed to launch parpool and run the images. So, if there is an issue with launching the parpool it will stop in first 2-3 minutes. I just reran that specific job with the error you mentioned and it is running with no error. I feel like I can only launch parpool once at a given time. That's because I see the jobs that start at the same time fail launching parpool except one job. Is there a way to delay array jobs so each starts 4 minutes after the job before?
    Blake Joyce
    @bjoyce3
    I would assume there's a way to do that
    I'm not familiar with a wait function off the top of head for PBS arrays, but I'll ask the HPC consultants tomorrow to get in touch with you
    It is interesting that rerunning the job meant it worked fine. Have you sent us the PBS array file too?
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    I have in my first email.
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Now I am 100% sure that it is some kind of license issue. I ran one job at a time (10 jobs that had error before) with no issue.
    Blake Joyce
    @bjoyce3
    We'll dig in. It'll take some time though
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Okay. Let me know when you know how to delay the each job for couple of minutes. This should solve my problem for now.
    Blake Joyce
    @bjoyce3
    Will do. Thanks for being patient with us. Things will get better on our response time once the first month is over!
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Thanks
    aconkli4
    @aconkli4
    Blake, Julian, do either of you know where I could get help implementing locality sensitive hashing?
    Julian Pistorius
    @julianpistorius
    Hi @aconkli4! What's the context?
    aconkli4
    @aconkli4
    Hi Julian!
    I am trying to implement an implicit correlation network analysis (finding modules in a network without calculating all the edge weights). One of the steps is to convert negative edge weights into positive edge weights. This is accomplished by squaring the edge weights. The goal (I think) is to utilize a set of hashing functions so that I can approximate the squared dot products implicitly.
    Julian Pistorius
    @julianpistorius
    Interesting! What have you looked at so far? Are there any libraries which do approximately what you need?
    aconkli4
    @aconkli4

    Maybe. My primary problem is figuring out how to sausage grind the math into code. There may be some library that can help me, but I probably wouldn't know if I saw it.
    Unfortunately, my Youtube level education on LSH is not measuring up well to the math in this dissertation I am reading.

    So far I have googled a lot of LSH tutorials and had min-hashing and Jaccard distance between text documents explained to me a half dozen times.

    aconkli4
    @aconkli4

    So maybe a better question is, do you know really a patient math/computer science tutor that would be willing to put up with my math inadequacy for a while?

    @Elaheh-Alizadeh I just realized you are in this group! Would you maybe be willing to help me some time this week?

    Julian Pistorius
    @julianpistorius
    Hmm... I wonder if Ian Bertolacci might be able to help.
    I've asked on the Python Meetup Slack.
    Alex Pakalniskis
    @alex-pakalniskis
    @aconkli4 I have weekly Python drop-in hours on Monday evenings (https://new.library.arizona.edu/events/python-programming-drop-hours) and I'm patient. Happy to help however I can. Feel free to message me.
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Sure, How about this afternoon?
    aconkli4
    @aconkli4
    Thanks @alex-pakalniskis and @Elaheh-Alizadeh ! Elaheh, that sounds good. Are you free after seminar this afternoon?
    Alex, I might take you up on that. I won't make it this coming Monday, but the following drop-in you may see me
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    I will not be at the seminar. You can find me at cancer center #4906B after the seminar.
    aconkli4
    @aconkli4
    Okay thanks