Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Daniel
    @dnorman
    what I think is pushing me over the edge, is that significant memo prefetch is required in order to make unbase operate efficiently. How does one facilitate this prefetch? –– by creating a query structure that parallels the code path. Broad Query (possibly with hints about the next thing) -> Index traversal ( getting somewhat more specific about what we want to do here ) -> specific memo that I need right now, but don't have -> UNKNOWN NEXT STEPS where prefetch will be of great assistance
    Leonora Tindall
    @NoraCodes
    I agree that would be an interesting and useful feature. How hierarchical do you want it? Could we potentially write something that examines the stack at each function call and inserts information about it? Or we could have logging calls at the beginning and end of each function (facilitated with macros) and emit XML or something else hierarchical.
    Daniel
    @dnorman
    this nested query execution structure would be built in such a way that (incidentally) reflected the code path, and also be serialized as part of any memo request: Slab A would say: "Hey Slab B, can you send me memo 123? Here's the query / query part that I need it for. if you want to partially execute this query yourself, and send me additional stuff, that'd be cool, but no worries either way. Thanks!"
    essentially, a dag of query parts pointing to their parent query part, finally ending up at the rootmost user inquiry
    this could be used to make for some sweet / powerful logging, and also be serialized either fully or partially ( however much we have room for in the packet, for instance ) to try to accelerate the query process and reduce roundtripping
    what do you think @SilverWingedSeraph ? ( have to step away for a bit, but will be back on later )
    ( I think a DAG, maybe not. need to mull it over a bit more )
    Leonora Tindall
    @NoraCodes
    Ah, I see. Yeah that could be incredibly useful and I, also, need to think about it more.
    Daniel
    @dnorman
    FYI, just registered the #unbase channel on irc.mozilla.org
    do you have a nick on irc.mozilla.org @SilverWingedSeraph / @mmacfadden ?
    Leonora Tindall
    @NoraCodes
    This looks like a potential place where Unbase could shine: https://staltz.com/an-off-grid-social-network.html
    Daniel
    @dnorman
    Interestingg
    That does seem like a particularly apt situation
    That said, IMO it's applicable to all social networking, but with somewhat subtler benefits
    Xavier Lange
    @xrl
    @dnorman some cool serde news with zero-copy deserialization. it's a fun feature to brag about :smile:
    it'll add yet another lifetime annotation since the struct is now tied to the life of the buffer. but it's safe and will definitely reduce memory pressure!
    it's on a big win if you're doing a lot of string-y stuff -- it's a no brainer for standard JSON-y web-y stuff. does not make sense for number stuff that is Copy.
    Leonora Tindall
    @NoraCodes
    I'm very excited about Serde 1.0 tbh. Also remote derive is interesting, if not so useful for this project.
    Xavier Lange
    @xrl
    that feature didn't quite click with me, but zero copy is like "yeah, duh, that's why I signed up for this rust thing"
    "it's 2017, why are we still copying out of strings?"
    Leonora Tindall
    @NoraCodes
    Definitely. And for something that will do as much buffer processing as Unbase it's a huge win.
    Xavier Lange
    @xrl
    yeah, keep that svelte profile
    Daniel
    @dnorman
    @SilverWingedSeraph @xrl I'm excited about this: https://github.com/serde-rs/serde/issues/881#issuecomment-294366077
    Will allow me to delete .9 kloc ^_^
    (whenever it arrives of course)
    btw... pushed v 0.1.0 to crates.io :)
    Still a long way to go, but a lot more test cases passing
    also getting started on proper documentation: https://docs.rs/unbase/0.1.0/unbase/
    Leonora Tindall
    @NoraCodes
    That would be very nice, for sure. Also, woot for docs!
    Daniel
    @dnorman
    Andddd, moving to https://github.com/unbase/unbase/projects/1 for task management
    As always, feedback encouraged! :rainbow:
    Leonora Tindall
    @NoraCodes
    A question about client implementation: how would one go about getting a subject with only the key, no SubjectID? For instance, in the case of the ping-pong example, could the spawned thread get the subject if it only knew "animal_sound" was the key and couldn't get the ID? The use case for this is any situation in which two slabs (creator and interested party) can't communicate out of band.
    Or am I misunderstanding how Subjects are meant to work? Is the ID like a port number, intended to be transmitted out of band or perhaps be well-known?
    Daniel
    @dnorman
    Hey @SilverWingedSeraph you're not misunderstanding. Non-primary indices aren't implemented yet :)
    It requires a functioning policy system I think
    Leonora Tindall
    @NoraCodes
    Ok, gotchya! So for now the best thing to do is just transmit it out of band and ask the user to enter a known subject id number?
    Daniel
    @dnorman
    But: you can check out the index test case in /tests to get an idea of how it might work
    Or, you could manually use an index
    Leonora Tindall
    @NoraCodes
    Oh, ok! Makes sense.
    Daniel
    @dnorman
    If you're looking for some insight into where I'm going with all of this, you can check out the pubsub paper in the bibliography. A matching tree looks an awwwful lot like an index tree :smile:
    Leonora Tindall
    @NoraCodes
    Awesome, I definitely will.
    Daniel
    @dnorman
    (Added the beginnings of the bibliography to the website)
    David Roundy
    @droundy
    I'm wondering what the usability status of unbase is?
    Could I use it to easily implement a simple protocol?
    I'm trying to decide whether to do something centralized using TCP directly or whether to use a crate. I will have few nodes, and performance doesn't matter, but simplicity does.
    Daniel
    @dnorman
    Hey @droundy - it's highly in flux at this point.
    It bears mentioning that the concept is fairly antithetical to stateful protocols.
    There's a fairly detailed write up on the website about what it hopes to be
    Unba.se