Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    pierce314159
    @pierce314159
    you need to set the CHPL_RE2 env variable and rebuild chapel
    export CHPL_RE2=bundled
    Zhihui Du
    @zhihuidu
    Thanks!
    login-1-84 chapel >: export CHPL_RE2=bundled
    login-1-85 chapel >: make
    ....
    make[3]: Nothing to be done for `re2'.
    .....
    It seems I missed something else?
    I need to
    make clean
    first?
    Must I build it from clean? If I rebuild the chapel, it will need a long time.
    pierce314159
    @pierce314159
    if arkouda is still failing to make, then try make clean and then make
    Brad Chamberlain
    @bradcray
    I would expect you not to have to re-build if re2 is already built.
    What version of Chapel are you using? (chpl --version)
    Zhihui Du
    @zhihuidu
    chpl version 1.24.0 pre-release (8ca0d00)
    I have make clean and make
    Brad Chamberlain
    @bradcray
    I think the problem is actually in the checkRE2.chpl file itself. There’s a Makefile variable you can set to skip this check, and I would recommend doing that and seeing if Arkouda proper builds for you.
    make “ARKOUDA_SKIP_CHECK_DEPS=true from the arkouda directory.
    Zhihui Du
    @zhihuidu
    Thanks and I will try it.
    Brad Chamberlain
    @bradcray
    @pierce314159 : I see you worked on checkRE2.chpl last. The issue that I think I’m seeing is that use statements are resolved early in the compiler, such that both Regex and Regexp will both attempt to be parsed by the compiler regardless of version number.
    pierce314159
    @pierce314159
    ohhhhh
    Brad Chamberlain
    @bradcray
    Arkouda proper deals with this by bringing in ArkoudaRegexCompat.chpl, which publicly uses one module or the other depending on version number. (another approach would’ve been to add a module named Regex to the lt-125 directory that publicly used Regexp).
    We’ve recently been talking about what could be done to make uses get processed more lazily (or other things to be processed more aggressively) so that the code would’ve worked as you’d hoped. But that’s a ways off yet, I’m afraid.
    pierce314159
    @pierce314159
    okay yeah that makes sense, thanks Brad!
    Zhihui Du
    @zhihuidu
    @bradcray Yes, make ARKOUDA_SKIP_CHECK_DEPS=true can pass the checking and now everything seems good.
    Brad Chamberlain
    @bradcray
    Great!
    @pierce314159 : I added a comment to #1032 here: https://github.com/Bears-R-Us/arkouda/pull/1032/files#r802103781 but probably it deserves an issue to make sure it doesn’t get lost. Can you take care of that?
    pierce314159
    @pierce314159
    yeah I just created: Bears-R-Us/arkouda#1079
    Brad Chamberlain
    @bradcray
    :+1:
    pierce314159
    @pierce314159
    @zhihuidu, the issue you were having should be resolved by Bears-R-Us/arkouda#1084. You should be able to make without skipping checks on the latest version of arkouda master
    Zhihui Du
    @zhihuidu
    @pierce314159 yes, I have tested it and the new version works! Thanks!
    Zhihui Du
    @zhihuidu
    @bradcray Hi, Brad, our tests show that on a shared memory multicore computer, it will take a much longer time if we run a program with two locales compared with one locale. I wonder what chaple will do when it cannot have multiple physical resources to execute a program?
    Brad Chamberlain
    @bradcray
    Hi @zhihuidu — Sorry not to have noticed this earlier. I believe that you're correct that, today by default, Chapel processes assume they own the full compute node, and so will (for example) create a thread per core and pin those threads to the cores. When both processes do this, it can hurt performance of course.
    There are some advanced features that can be used to try and carve up a compute node (like a shared memory system like this) between multiple Chapel processes, and we've had some success with them, but it's still not optimal.
    I expect that we'll be doing more here to support a user-facing "locales per node" setting that will cause the processes to be more aware of each other and cooperate better, but that doesn't exist today.
    As far as best practices for the advanced features I refer to above, Elliot Ronaghan (@ronawho on gitter) is the resident expert. I think he was giving tips on this to others at Arkouda recently, let me see if I can find them.
    Zhihui Du
    @zhihuidu
    @bradcray thanks and now I understand it!
    Brad Chamberlain
    @bradcray

    I’m not finding what I was remembering, but did find these two settings:

    CHPL_RT_OVERSUBSCRIBED=yes
CHPL_RT_NUM_THREADS_PER_LOCALE=8  # or whatever

    That said, I’m not confident that this is sufficient. Specifically, I don’t recall that anything will make sure that the two processes won’t use the same cores for each of their 8 threads (?). Elliot’s definitely the expert here, so I’d suggest checking with him next week (though actually he may still be online now… even though it’s late on a Friday on the east coast. Of course, this is true for you as well… :) )

    Zhihui Du
    @zhihuidu
    I appreciate your quick reply on weekend! Since now the distributed memory computer is not available, we want to do some experiments on a shared memory computer using multiple locales.
    Brad Chamberlain
    @bradcray
    Not a problem (it’s not quite the weekend here yet :) ). I’ll be curious if the trick above improves things at all. It probably won’t be worse, but it probably also won’t be optimal.
    Zhihui Du
    @zhihuidu
    Let's try your method and see if there are some performance difference.
    Michael Merrill
    @mhmerrill
    Does anyone have a topic for today's Arkouda Weekly Call?
    Michael Merrill
    @mhmerrill
    I think since I haven't heard from anyone today's Arkouda Weekly Call topic will be the new https://github.com/Bears-R-Us/arkouda-contrib repo
    Michael Merrill
    @mhmerrill
    I am going to cancel tomorrow’s Arkouda Weekly Call
    Michael Merrill
    @mhmerrill
    today on the Arkouda Weekly Call we will discuss the structure of the arkouda-contrib repo
    Engin Kayraklioglu
    @e-kayrakli
    Hello all! In case you missed it, Chapel Implementers and Users Workshop (CHIUW) call for submissions is out with an April 15 deadline. If you have Arkouda/Chapel related work, we encourage you submit your work there. Even if you don’t, we hope to see you at CHIUW 2022 on June 10th. It is free and virtual.
    Michael Merrill
    @mhmerrill
    Anyone have a topic for today's Arkouda Weekly Call ?
    Michael Merrill
    @mhmerrill
    I am going to cancel today's call if no one has a subject to talk about.
    Michael Merrill
    @mhmerrill
    Todays meeting is cancelled.
    Michael Merrill
    @mhmerrill
    I am going to cancel todays Arkouda weekly call unless someone has a topic to discuss. Please put topics in this channel
    Michael Merrill
    @mhmerrill
    todays meeting is canceled
    Zhihui Du
    @zhihuidu
    When I compile the latest arkouda version, it gives the following warnings.
    WARNING: ParquetMsg module declared in ServerModules.cfg but ARKOUDA_SERVER_PARQUET_SUPPORT is not set.
    WARNING: ParquetMsg module will NOT be built.
    Any suggestion to remove the warning safely?
    Michael Merrill
    @mhmerrill
    @zhihuidu this is just informational if you want Parquet support to be built then set the env variable otherwise leave it unset
    @zhihuidu you could also remove the ParquetMsg module from the ServerModules.cfg file, we should probably rework the build script to not include the Parquet module if the env var is not set
    Zhihui Du
    @zhihuidu
    Got it and thanks!