Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Blake Joyce
    @bjoyce3
    Can you check the matlab code you're using to make sure that file transferred properly and isn't corrupt?
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    I am running a job array and it pulls the same code for all the jobs. The code processes bunch of images and for different jobs it just pulls different set of images. I do not think the code is corrupted since it ran successfully for some jobs. The job is supposed to launch parpool and run the images. So, if there is an issue with launching the parpool it will stop in first 2-3 minutes. I just reran that specific job with the error you mentioned and it is running with no error. I feel like I can only launch parpool once at a given time. That's because I see the jobs that start at the same time fail launching parpool except one job. Is there a way to delay array jobs so each starts 4 minutes after the job before?
    Blake Joyce
    @bjoyce3
    I would assume there's a way to do that
    I'm not familiar with a wait function off the top of head for PBS arrays, but I'll ask the HPC consultants tomorrow to get in touch with you
    It is interesting that rerunning the job meant it worked fine. Have you sent us the PBS array file too?
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    I have in my first email.
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Now I am 100% sure that it is some kind of license issue. I ran one job at a time (10 jobs that had error before) with no issue.
    Blake Joyce
    @bjoyce3
    We'll dig in. It'll take some time though
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Okay. Let me know when you know how to delay the each job for couple of minutes. This should solve my problem for now.
    Blake Joyce
    @bjoyce3
    Will do. Thanks for being patient with us. Things will get better on our response time once the first month is over!
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Thanks
    aconkli4
    @aconkli4
    Blake, Julian, do either of you know where I could get help implementing locality sensitive hashing?
    Julian Pistorius
    @julianpistorius
    Hi @aconkli4! What's the context?
    aconkli4
    @aconkli4
    Hi Julian!
    I am trying to implement an implicit correlation network analysis (finding modules in a network without calculating all the edge weights). One of the steps is to convert negative edge weights into positive edge weights. This is accomplished by squaring the edge weights. The goal (I think) is to utilize a set of hashing functions so that I can approximate the squared dot products implicitly.
    Julian Pistorius
    @julianpistorius
    Interesting! What have you looked at so far? Are there any libraries which do approximately what you need?
    aconkli4
    @aconkli4

    Maybe. My primary problem is figuring out how to sausage grind the math into code. There may be some library that can help me, but I probably wouldn't know if I saw it.
    Unfortunately, my Youtube level education on LSH is not measuring up well to the math in this dissertation I am reading.

    So far I have googled a lot of LSH tutorials and had min-hashing and Jaccard distance between text documents explained to me a half dozen times.

    aconkli4
    @aconkli4

    So maybe a better question is, do you know really a patient math/computer science tutor that would be willing to put up with my math inadequacy for a while?

    @Elaheh-Alizadeh I just realized you are in this group! Would you maybe be willing to help me some time this week?

    Julian Pistorius
    @julianpistorius
    Hmm... I wonder if Ian Bertolacci might be able to help.
    I've asked on the Python Meetup Slack.
    Alex Pakalniskis
    @alex-pakalniskis
    @aconkli4 I have weekly Python drop-in hours on Monday evenings (https://new.library.arizona.edu/events/python-programming-drop-hours) and I'm patient. Happy to help however I can. Feel free to message me.
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Sure, How about this afternoon?
    aconkli4
    @aconkli4
    Thanks @alex-pakalniskis and @Elaheh-Alizadeh ! Elaheh, that sounds good. Are you free after seminar this afternoon?
    Alex, I might take you up on that. I won't make it this coming Monday, but the following drop-in you may see me
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    I will not be at the seminar. You can find me at cancer center #4906B after the seminar.
    aconkli4
    @aconkli4
    Okay thanks
    alekseynovikov78
    @alekseynovikov78
    @julianpistorius This is a reminder to check in on how the Russian NLP thing is going.
    alekseynovikov78
    @alekseynovikov78
    @julianpistorius , still the same error: it can't open edu/stanford/nlp/models/parser/nndep/english_UD.gz
    alekseynovikov78
    @alekseynovikov78

    @julianpistorius, so I added stanford-corenlp-3.9.2-models.jar to the lib directory, and now I get a different error: Execution failed for task ':run'.

    Process 'command '/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1

    Julian Pistorius
    @julianpistorius
    Hmm... Any more detailed errors?
    alekseynovikov78
    @alekseynovikov78
    • What went wrong:
      Execution failed for task ':run'.

      Process 'command '/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1

    • Try:
      Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

    Julian Pistorius
    @julianpistorius
    Hmm... There is probably a way to make Maven run that way, but I'd have to poke at it.
    Julian Pistorius
    @julianpistorius
    @alekseynovikov78 - Any progress?
    alekseynovikov78
    @alekseynovikov78
    @julianpistorius, have not had a chance yet.
    Would you be able to meet sometime this week to work on it? I understand if not!
    Julian Pistorius
    @julianpistorius
    @alekseynovikov78 - Yes, come to Hacky Hour on Thursday and we'll figure it out. I will try to run it on my own system before then.
    alekseynovikov78
    @alekseynovikov78
    Cool beans! Thanks, @julianpistorius
    Joseph D. Long
    @joseph-long

    Hi all, hoping someone more numerically savvy can help me understand this unexpected output from an eigenvector solver. For a real, symmetric matrix, the syevr (eigenvectors + values) and gesdd (SVD) functions should produce the same results. I'm finding the eigenvalues/singular values agree to numerical precision, but the vectors do not. The vectors in both outputs behave as expected (A v = lambda v and |v| = 1), and I don't have any repeated eigenvalues in the output so I don't suspect degenerate eigenvalues.

    Any ideas?

    Joseph D. Long
    @joseph-long
    Aha, there is a sign flip
    image (5).png
    Elaheh Alizadeh
    @Elaheh-Alizadeh
    Hey. I brought the lab computer home because remoting was sometimes problematic. Now my jupyter notebook is very slow for some reasons. I can not even read my data( it would take seconds to read before). Does anyone know how to solve this? I have not heard from our IT people yet!
    Julian Pistorius
    @julianpistorius
    Hmmm... Where is the data? Maybe it’s accessible it over the network?
    eranaj
    @eranaj
    I was just curious, do you know if the people behind the software installation requests for HPC are actually from the general tech support team at UITS or if they are actually dedicated HPC people?
    Julian Pistorius
    @julianpistorius
    I think the latter. I’ll ask Blake.
    Blake Joyce
    @bjoyce3
    Hello Erana, it's just us. There's no wizard of Oz :)
    But we need to collect a lot of information for software installs, and getting the usual "we need this" email to hpc-consult got real old every time we had to send 4 additional emails to get all the info we need to asssess software
    Blake Joyce
    @bjoyce3
    We do everything associated with the HPC. Software installs take a lot of time, so it make take a bit to get back. Containers allow you to do the installs yourself though. So we'll look at what you sent us, but feel free to update us if you figure anything out
    eranaj
    @eranaj
    Okay, thanks. It helps to know what kind of person is on the other end so that I know how to construct the request (to suit the audience level's understanding)
    I was going to do that whole container install myself but in the end it requires a superuser
    StefanMPopp
    @StefanMPopp
    Hey, I'm stuck on the same issue for the 3rd time and was hoping to get some experienced perspective on it:
    I am trying to compute the efficiency and other things of a search path of ants by getting where and how much the ant has crossed her own path. The search path is defined as the area between the left & right antenna, for which I have x,y,t coordinates calculated from the centroid x,y,t of the ant.
    I need to get a) the points where the ant bumped into her own track, b) whether she has crossed her track subsequently, and c) the area of path overlap.
    The challenge is that I'm dealing with 3k+ tracks of 100k+ points each, for which my approaches so far have been too slow.
    The data are positions sampled at a constant, high frequency, where the path width is ~4x the mean distance between subsequent points.
    I'm working in MATLAB (but am open to other software as well) and have tried the available functions of path intersection and inpolygon, which are either too slow or require too much RAM, as well as preselecting points for these functions based on rough distance, which gave only me improvements of less than an order of magnitude. Thus, now I'm looking for information on what the best top-level approach might be. Any kind of help would be appreciated and lmk if you need more information etc.