Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Engin Kayraklioglu
    @e-kayrakli
    And get a string that is only part of the data?
    hokiegeek2
    @hokiegeek2
    @e-kayrakli AFAIK, Arkouda strings = Chapel strings = uint(8) arrays?
    Engin Kayraklioglu
    @e-kayrakli
    I remember something about segmented arrays (?) at somepoint that made Arkouda strings = special uint(8) arrays. But I can be wrong
    hokiegeek2
    @hokiegeek2
    @e-kayrakli Yup, you're correct re: Arkouda strings = segmented arrays. I don't, however, have to get one string. This is for writing "strings" to/reading from hdf5, and I am thinking I need to make it so each string AKA array of uint(8) elements go to the same locale if I need to enable reading strings from one locale only.
    Brad Chamberlain
    @bradcray
    @hokiegeek2: Not really, unfortunately. I mean, Chapel strings are buffers of uint(8)s at their core, but there’s metadata that describes them and implements their string-ness which isn’t present in Arkouda.
    hokiegeek2
    @hokiegeek2
    @e-kayrakli you are correct! Given you answer, I am guessing Chapel strings ain't the same
    @bradcray gotcha, okay
    Brad Chamberlain
    @bradcray
    Think of a Chapel string as a record pointing to a uint(8) buffer. An array of Chapel strings is an array of records, each of which points off to somewhere else. So the string metadata is consecutive in memory and blocked such that no string is spanning locales, but the strings themselves are all disjoint on the heap.
    Arkouda’s strings are block-distributed arrays of uint(8)s which Arkouda interprets as strings by storing segments of where each string begins. So all the string data is consecutive in memory; but a string may be split between locales.
    Each approach has tradeoffs, obviously.
    hokiegeek2
    @hokiegeek2
    @bradcray ah, yes, excellent summary, thank you!
    hokiegeek2
    @hokiegeek2
    @bradcray @e-kayrakli thank you both for allowing me to borrow your respective brains -> excellent discussion that really helped
    So @e-kayrakli, when are you getting started, LOL
    Engin Kayraklioglu
    @e-kayrakli
    :) FWIW, looking at Brad’s description of Arkouda strings, you may need to adapt what I was suggesting to Arkouda strings. i.e. keep storing them using regular Block dist as it already is, but locally buffer and write that buffer when you need to write it. Reading may get a bit complicated though. i.e. I am not sure how to answer the question “I read these bytes here, but where do they go??”, but it sounds like you may have an answer by looking at Arkouda’s string implementation
    hokiegeek2
    @hokiegeek2
    @e-kayrakli Gotcha, cool, excellent info, will ponder this a bit more. Thanks again to both you and @bradcray!
    hokiegeek2
    @hokiegeek2
    @bradcray @e-kayrakli I think there's a way to do this in a straightforward and deterministic way. In the first screenshot there's the entire array of strings in the form of a byte array where the strings are separated by null bytes. Also in this first screenshot is the list of indices, in other words, where the string segments start as well as the overall length of the byte array. The list of indices I think provides a means of manually partitioning the byte data so that complete strings are written to the locales.
    image.png
    hokiegeek2
    @hokiegeek2
    The second screenshot show the array data slice, the domain corresponding to the array data slice, and the slice data type, uint(8). If the byte array was partitioned as I just described, there would be a zero at the end of each locale slice, which would ensure each string is written in it's entirety to the same locale.
    image.png
    In this case, the second "tech" string and the "foster" string are split between two locales as shown in the python return values where the variable new_strings variable points to the Arkouda/Chapel array and the nsr variable points to the Arkouda/Chapel array read out of hdf5 files, in this case, three of them since I am running Arkouda with three locales.
    image.png
    hokiegeek2
    @hokiegeek2
    I am hoping there's a hook in the codebase involving a BlockDist where I can manually partition the byte array based upon indices into separate locales. With the indices there's a deterministic means of partitioning chunks of bytes to locales. In this example, the locales would need to be {0...31}, {32...59}, and {60...82}
    hokiegeek2
    @hokiegeek2
    It not, perhaps the partitioning logic can be refactored so one could pass in a first-class function a la Python or Scala to define what chunks are assigned to which locale. @bradcray @e-kayrakli Are the hooks there to pass in or leverage custom partitioning logic? Re-reading what @bradcray wrote above, I am guessing not, but just want to double-check, thanks!
    hokiegeek2
    @hokiegeek2
    @bradcray is it the boundingBox that would have to be customized so that the values are not spread evenly across locales?
    image.png
    Thomas Rolinger
    @thomasrolinger
    @hokiegeek2 A problem with blocks in a distribution that are not all the same size is how the underlying indexing works. When all blocks are the same size, you can rely on div/mod tricks to figure out where the “logical” index is mapped to within the distribution. When there are blocks with different sizes, you are basically left with performing a binary search to find out where your element is. GlobalArrays supports variable size blocks and does just that. But doing that is very expensive compared to what happens when all blocks are the same size.
    hokiegeek2
    @hokiegeek2
    @thomasrolinger wow, gotcha, great point
    Thomas Rolinger
    @thomasrolinger
    “Problem” may not be the best way to describe it, it’s more of a consideration i suppose. I imagine it is something that you can extend/customize but it’s a lot more involved and will not give you the same performance.
    hokiegeek2
    @hokiegeek2
    @thomasrolinger so this is really a consideration for when a strings array is written out to hdf5. Is it possible to read in from disk and then "rebalance" so the locales are in the same sized blocks?
    In other words, read into a sub-optimal locale setup and then rebalance? Dunno if there's such a notion in Chapel (or if there should be!). The analogous thing is Spark or Dask is something to avoid, so it's likely the same in Chapel
    Thomas Rolinger
    @thomasrolinger
    I’m not sure if there is a built-in mechanism in Chapel for that. But it seems like something you could do as a pre-processing step prior to creating the distribution, or perhaps by introducing some sort of padding to “balance” things. I’m not all to familiar with distributions in Chapel but have spent a good amount of time doing similar things in other frameworks. Hopefully someone with a bit more knowledge will chime in
    hokiegeek2
    @hokiegeek2
    I just thought of something that I believe is important. The Arkouda approach to arrays of strings to persist the strings as a contiguous array of uint(8) elements with a corresponding indices array, the latter of which is used to access the string elements from each locale. Since the indices are there, I would there still be the binary search requirement?
    Thomas Rolinger
    @thomasrolinger
    If I understand correctly, there is an array that tells you the index of each element? If so, then you wouldn’t need to do a search to find an element, as you essentially have a pre-computed mapping. The only downside to that is memory/space.
    hokiegeek2
    @hokiegeek2
    Yep, the's an array with indices
    Cool, thanks for the confirmation!
    Michael Merrill
    @mhmerrill
    Is there a preferred way to have multi and single locale coexist? is it just as simple as setting export CHPL_COMM=none ?
    Ben Albrecht
    @ben-albrecht
    @mhmerrill - What do you mean by coexist? Having 2 built binaries of CHPL_COMM={gasnet || ugni} and CHPL_COMM=none?
    Michael Merrill
    @mhmerrill
    yes comm=none vs comm=gasnet
    i built a comm=none first but then I needed to simulate multi-locale
    oh i wasn't clear, sorry, I meant I built the chapel compiler with CHPL_COM=none then I needed to build it again with CHPL_comm=gasnet
    Ben Albrecht
    @ben-albrecht
    OK, the Chapel compiler can be built with multiple configurations simultaneously, that you can switch configuration on the fly with env vars or compiler flags. Switching the env var or using the compiler flag —comm={none, gasnet} (and using different -o names to avoid overwriting each other obviously) is a reasonable approach.
    Michael Merrill
    @mhmerrill
    ok, i was afraid I was going to have to make clean and rebuild completely
    Brad Chamberlain
    @bradcray
    @mhmerrill: To put a bit more detail into Ben’s response: The chapel compiler itself is independent of whether it’s built with CHPL_COMM=none or gasnet or whatever. The runtime is not. So if you look under your $CHPL_HOME/bin/ directory, you’ll see a small amount of uniquification due to platform, architecture, etc. but most configurations will share the same path to chpl and binary. Whereas in the $CHPL_HOME/lib directory, you’ll see a lot more diversity in paths based on each setting of CHPL_COMM, CHPL_TASKS, etc.
    @hokiegeek2: Sorry that I missed your questions last week (I took Th-Fri off). Your notion of wanting to be able to specify where the blocks should start/stop is similar to the “cut” distribution that I mentioned which we’ve had a user proof-of-concept, yet don’t support on master. The big difference between how “cut” and “block” would work, as @thomasrolinger notes, is that “cut” would need to store some sort of "directory structure" indicating how to map from indices to locales whereas for “block” it’s “just math.” This is not a huge deal to implement, simply not something we have today.
    @thomasrolinger’s suggestion about potentially padding the strings such that they block up evenly would be another way to get strings to not be split across locales, but has that tricky chicken-and-egg problem where you have to know your data pretty intimately in order to pad them correctly.
    Brad Chamberlain
    @bradcray
    The other approach (which may or may not be appropriate, depending on your context and constraints) would be Engin’s notion of just letting the strings fall where they may, and then when you do an operation that wants them all to be on a single locale, make sure to get them there by hook or by crook (e.g., “this string is all local, so I’ll just do what I want to it” vs. “this string is split, so let me make a temporary buffer locally to store the whole thing and concatenate my local portion with my neighbor’s portion into that buffer before operating on it”).
    hokiegeek2
    @hokiegeek2
    @bradcray gotcha, cool, good info, thanks for the follow-up!
    Michael Merrill
    @mhmerrill
    has anyone build chapel on the new raspberry pi 4 with 8 GB of RAM???
    Elliot Ronaghan
    @ronawho
    Not on raspberry pi, but we have some nightly testing that builds on machines with 2 and 4 GB RAM
    I don't think arkouda will build with 8GB of RAM though, I think you need 11-12 currently