Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Matias Rodriguez
    @rodrimati1992
    Here is an example of a manually written trait object ,the extern functions it uses are defined in abi_stable::erased_types::c_functions
    Now,Waker::will_wake and Waker::from_raw can only be called with the ffi wrapper as a parameter by re-wrapping it again into a RawWaker.
    Matias Rodriguez
    @rodrimati1992
    If you find it really inconvenient/hard to do this process manually you can use #[sabi_trait] like in the example
    (the reason I don't use it internally too much is because it impacts compile times)
    Matias Rodriguez
    @rodrimati1992
    Note that all parameters/return types to trait methods defined with #[sabi_trait] have to implement SharedStableAbi/StableAbi.
    jean-airoldie
    @jean-airoldie
    Cool, thx.
    jean-airoldie
    @jean-airoldie

    From my understanding, support for custom allocators still has a long way to go in rust. However there is currently a working group working on it.

    Based on the interface provided by this crate, do you think, do you think your stable ABI std_types could support custom allocators?

    I'm asking the question because I need to use shared memory underneath stable ABI types for user-space IPC communication. I'm debating whether I should try to add custom allocator support based on this wg crate, or if I should implement distinct types altogether.

    jean-airoldie
    @jean-airoldie
    And a secondly, what do you think of using an interface similar to std's RawVec for interval use of RVec? This would help reduce code duplication between a RVec and RVecDeque.
    Matias Rodriguez
    @rodrimati1992

    I don't understand the alloc-wg crate that well,but it looks possible with caveats.

    abi_stable could support custom allocators for RVec/RString/RBox/RBoxError with some work.

    RArc/RHashMap would require an implementation of Arc/HashMap with custom allocator support.
    I don't think it's a good idea to reimplement either Arc or HashMap in abi_stable.

    RawVec can work.adding a nested #[repr(C)]struct RawVec<T> would change the layout of RVec,so it'll have to go in an 0.8 branch.

    Those changes would warrant a new minor version (0.8) of abi_stable(even if RArc and RHashMap cannot have custom allocator support).

    Matias Rodriguez
    @rodrimati1992
    RSmallBox is another type that can support a custom allocator with as much work as RBox.
    jean-airoldie
    @jean-airoldie
    I'm not sure I understand why the RArc would require a custom allocator. Can't the RArc call the Drop implementation of the type it wraps? If the wrapped type implements a custom allocator, wouldn't it be called?
    Matias Rodriguez
    @rodrimati1992
    How does T know how to deallocate the ArcInner<T> that Arc<T> deallocates when it's dropped(and the reference count is 0)?
    Oh wait,by "the type it wraps" you mean Arc<T> itself?It doesn't have custom allocator support.
    If RArc supported custom allocators it would be through some other type that implements the same API that Arc<T> implements.
    jean-airoldie
    @jean-airoldie
    No I mean't the type T, I wasn't aware that RArc used Arc internally.
    Matias Rodriguez
    @rodrimati1992
    Yes,I am using virtual dispatch to call Arc<T>methods since I am not good at implementing stuff with atomics.
    That's why I mentioned that it would have to be a custom allocator supporting reimplementation of Arc<T>.
    jean-airoldie
    @jean-airoldie
    So for RArc to support custom allocators, the ideal solution would be for Arc<T> to support custom allocators in the first place.
    kk gotcha
    Matias Rodriguez
    @rodrimati1992
    That would be the ideal solution,but if a crate like crossbeam-channel appeared for Arc<T>with custom allocator support it could be made to work with RArc<T>
    (crossbeam-channel is why I am not wrapping the std::sync::mpsc types)
    jean-airoldie
    @jean-airoldie
    yeah the sync::mpsc types kinda suck anyway
    thats tricky. I kinda need a RArc<RMutex> in my case. Seems like my best bet would be to roll my own types with hardcoded allocators until the custom allocator supports for the std lib improves, which will probably take a while.
    jean-airoldie
    @jean-airoldie
    unless I do that exclusively for the Arc type.
    Matias Rodriguez
    @rodrimati1992
    It will be necessary to add another constructor function to RArc (likely will require declaring a trait),and implement the API of Arc that RArc uses.
    The trait should be something like this:
    trait ArcMethods<T>{
        fn into_raw(self)->*const T;
    
        unsafe fn destructor(*const T,only_deallocate:bool);
        fn clone(&self) -> Self;
        fn get_mut(&mut self) -> Option<&mut T>;
        fn try_unwrap(self) -> RResult<T, Self>;
        fn strong_count(&self) -> usize;
        fn weak_count(&self) -> usize;
    }
    jean-airoldie
    @jean-airoldie
    That would be an interesting approach. and if RArc<T = std::sync::Arc> then this wouldn't be a breaking change right?
    Matias Rodriguez
    @rodrimati1992
    It doesn't have to break anything because it would work the same as it works for Arc<T>,the only changes would be the constructor and adding another way to construct the RArc<T>'s vtable(it's still a pointer into static memory).
    jean-airoldie
    @jean-airoldie
    Seems like a good temporary solution until std::sync::Arc supports allocators, assuming it ever does.
    alright thx, i'll think about
    *it
    Matias Rodriguez
    @rodrimati1992
    Can you describe what you mean by user-space IPC?
    jean-airoldie
    @jean-airoldie
    i dont really know how to call it but basically i want a async channel between 2 rust binaires with no syscalls.
    so i would use a stable ABI channel allocated via shared memory (most likely via mmap)
    Matias Rodriguez
    @rodrimati1992
    Can you have static references into the binary(all the types that own heap memory in abi_stable have a vtable that is a pointer into read-only static memory)
    jean-airoldie
    @jean-airoldie
    its not a problem as long as I make sure that the peer still exists before using such reference. otherwise the static refs could point into garbage.
    so as long a the channel is opened, i know that its safe
    Matias Rodriguez
    @rodrimati1992
    Ok
    jean-airoldie
    @jean-airoldie
    now that i think about it, this could potentially be an issue. i would need to be certain that the read-only static memory pointed by the vtable can be read by another process.
    Matias Rodriguez
    @rodrimati1992
    Not only that,but if you pass an RMutex across processes you would have to know the globals that parking_lotuses when there is mutex contention.
    jean-airoldie
    @jean-airoldie
    right. that parking_lot mutex would have to be shared memory anyway.
    Matias Rodriguez
    @rodrimati1992
    abi_stable assumes that,once the dynamic library has been loaded,all the read-only data from it can be referenced forever.That is the primary reason why I don't promise that reloading of dynamic libraries will be supported.
    Changing that would be like adding first class support of garbage collection to the language.
    (with a Gc<T>type in the standard library)
    jean-airoldie
    @jean-airoldie
    yeah i'm starting to understand the problem. the type data / metadata would have to be stored in the type itself as opposed to within the binary because its possible that it might outlive the binary itself.
    which is basically a gc'd language
    jean-airoldie
    @jean-airoldie
    so essentially i can't use dynamic dispatch at all. i would have to hardcode the mmap allocator. i would also need a shared memory mutex.
    lastly i would have to heap allocated into shared memory the function pointers of the RawWaker
    Matias Rodriguez
    @rodrimati1992
    It looks like abi_stable isn't good for this sending data across processes thing.
    jean-airoldie
    @jean-airoldie
    yep, its not a good fit