Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Kevin Jahns
    @dmonad
    :wave:
    RangerMauve
    @RangerMauve
    Hello! I'm coming from the Dat community and was wondering how the y-dat stuff is going. 😁
    Duane Johnson
    @canadaduane
    <3 two of my fav projects
    Kevin Jahns
    @dmonad
    Hey @RangerMauve I haven't worked on it in
    In the past two months. But I hope to continue working on it at the end of the month when I have more time
    RangerMauve
    @RangerMauve
    @dmonad Sweet. Lemme know if there's anything I can help with. I've been working on the Dat SDK to make it easier to develop with it.
    Kevin Jahns
    @dmonad
    Your help would be very much appreciated! I'll definitely come back to you :)
    RangerMauve
    @RangerMauve
    Hopefully I'll have an example of using the SDK with multifeed by then
    Kevin Jahns
    @dmonad
    That would be very helpful. Btw, I love the dat project. When I read the multifeed paper - mentioning CRDTs - I immediately saw how Yjs would perfectly integrate into that. So I'm very excited about the possibilities.
    I will be afk now (literally got up from my bed to write back).
    RangerMauve
    @RangerMauve
    @dmonad. Awh 💜 I'm excited to talk to you about this stuff more when you're free.
    Kevin Jahns
    @dmonad
    This message was deleted
    Kevin Jahns
    @dmonad

    Hey @RangerMauve, I want to outline my current approach for y-dat to you.

    My approach is similar to the multiwriter approach described in https://www.datprotocol.com/deps/0008-multiwriter/. Each user would basically create their own feed for writing Yjs document updates. Users can invite/authorize other users by writing the public key to their feed. But there is no need for a clock-vector, because Yjs already handles that.

    My only concern is that the user-experience of giving other users write permission to the document would be pretty bad. I read an example on how that would work here https://blog.datproject.org/2018/05/14/dat-shopping-list/ I wonder if there are better approaches to grant someone write access. Ideally you could just share a link. Would it be possible to have a special metadata-feed so that other users can self-register by sharing the private key of that metadata-feed? I know that there is a chance that two users write to that feed concurrently, but what does actually happen when they write concurrently?

    Kevin Jahns
    @dmonad
    I just read up on union mounts, which will be an interesting solution to the problem.
    RangerMauve
    @RangerMauve
    Hey! @dmonad. I was AFK from IRC for the weekend. Just reading this now. 💜
    I like your approach! With regards to authorizing writers, you could look into using extension messages. Essentially, you define custom message types which either side can send over the replication stream.
    For instance, I send you the link to the document, with a special secret that says you can write to it, then you start replicating over the main feed and add your hypercore to the replication stream
    RangerMauve
    @RangerMauve
    Then you use your hypercore's key, and sign it with the secret or something and send it down an extension key over the main hypercore. Then all the other readers will register it as a valid writer and start replicating it (saving the key and the proof for later to send to other peers)
    Something like that
    Kevin Jahns
    @dmonad

    NP @RangerMauve Weekends without chat programs are very recommendable! In the meantime I have been looking into Dat SDK - the API looks much cleaner now - and it works in node, beaker, and legacy browsers. Really cool!

    The authentication solution you described definitely makes link-sharing easier. In terms of writing a note-taking app: My only concern about this extension-based solution is that it requires two clients (the client that registers itself and an authenticated client) to open the same document at the same time to actually persist the information that a feed was added.

    No matter how I look at it, it won't be possible to provide link-sharing functionality similar to other note-taking apps. Another solution to this problem might be to create ask-for-permission links. The user interaction would go somehow like this: Author sends read-only link to remote client. Remote client asks for permission by adding their Hypercore-key to the link and sending it back to the authenticated client. The remote user can immediately start working on the document. The authenticated client then can accept their changes by adding the remote Hypercore-key to the local Hypercore, or do nothing and ignore the remote changes.

    Another interesting solution would be to add a trusted peer (one that also persists) to the network. This would also integrate perfectly with your extension-based solution.

    At this time there doesn't seem to be a better solution to the authentication problem (none that I can adapt the protocol for), so I will probably go with my approach and just provide authentication functions for authenticating remote users. You should be able to implement the link-based approach and the extension-based approach on top of this.

    RangerMauve
    @RangerMauve
    Yeah, that all sounds great! One thing that I've thought of in the past is a crawler-like function where each peer will write the URLs of peers that they've seen to their cores, so if any of them adds a peer, the rest will auto add it too as they load the hypercore. That's basically your link sharing idea.
    RangerMauve
    @RangerMauve
    dmonad: Do you have any ideas for "checking out" an older version of a document? Does yjs handle that sort of use case out of the box?
    Kevin Jahns
    @dmonad
    In Yjs you can take snapshots of the current state of the document. If you have two snapshots, you can even show the differences by user. You can test versioning in this demo: https://yjs-demos.now.sh/prosemirror-versions/
    Note that snapshots are not stable yet. So there is a good chance that I'm going to change the API
    Winston Fassett
    @WinstonFassett
    @dmonad fyi I ran into an infinite loop problem with y-prosemirror's cursor-plugin. I was able to fix it for my app in my node_modules, but when I tried to make the fix in the repo, I was unable to get y-prosemirror itself to build at all after various attempts. It seems like Rollup is unable to resolve the files in lib0 and fails on importing things like lib0/testing.js. Have you ever run into that? Dunno what I'm missing. I tried building some older releases from source and had the same problem.
    Anyway, the fix I made was on line 104. https://github.com/yjs/y-prosemirror/blob/master/src/plugins/cursor-plugin.js#L104
    With the current code, if there is no local state, it falls back to an empty {}. In that case cursor is undefined, rather than null. If the 119 when we check if it is not null, it actually passes that test, and the plugin calls setLocalStateField, setting the cursor to null, which ends up triggering updateCursorInfo again, causing an infinite loop, it seems.
    The fix was to explicitly set the cursor property to null on line #104 so that the explicit comparisons to null would work correctly, i.e. const current = awareness.getLocalState() || { cursor: null }
    I also noticed an issue with the sync plugin that I was not able to fix in the source but worked around in the app. It seems like the sync plugin does not render until there is an update, so if the type is already loaded when the prosemirror instance is bound to it, it remains blank until an update comes in. I found I could work around the problem by running setTimeout(() => { prosemirrorView.state['y-sync$'].binding._forceRerender() }, 0)
    Kevin Jahns
    @dmonad

    Thanks for bringing this up @WinstonFassett
    I forgot to make an implicit comparison in the awareness cursor state. aw.cursor === undefined is a valid state, so I made sure to only make implicit checks (aw.cursor == null checks for undefined and null).
    For the other issue (rendering the initial state), I now force rerender after view creation. Sorry, I never stumbled on this issue because in the demos I always start with an empty document (the server syncs after view creation).

    Both issues are fixes in y-prosemirror@0.1.3.

    About local development: lib0, Yjs, and y-protocols are always mentioned as peerDependecies. I forgot to add lib0 as a devDependency in package.json. Thanks for the hint. Would it be helpful to you if I describe my dev-setup? Sometimes it is necessary to debug multiple packages. In this case I clone all y-* modules (including lib0 and yjs-demos), put them in a single repository and build with LOCALIMPORTS=1(an environment variable). I could describe this setup in more detail if this is helpful to you.

    Kevin Jahns
    @dmonad
    @/all A couple of webinars/interviews about Yjs / rich text editors / and CRDTs will be published to this channel: https://www.youtube.com/channel/UCOm1Cd-9coAGB-prBPyWWxg
    Winston Fassett
    @WinstonFassett
    thanks @dmonad! those fixes all worked for me.
    Winston Fassett
    @WinstonFassett
    @dmonad what might persistence look like for v13? The previous version I used (v11 I think) had support for client storage in IndexedDB and server storage on LevelDB. I never got the offline with IndexedDB to really work, but the persistence worked well and I was even able to use levelup to adapt it to use MongoDB and PostgreSQL. At the time I thought levelup was a great abstraction of the KV api for YJS persistence that would allow syncing with a ton of backends (https://github.com/Level/awesome/#stores). Although currently I can't seem to get leveldown to install. I'm very interested in the Dat stuff being explored, and I'm hoping that would also inform the design of a more general persistence API that might be adapted to other backends. Are there any particular challenges to doing persistence for v13? Do you think a levelup/IndexedDb-style store API would still work for it?
    Winston Fassett
    @WinstonFassett
    I'm reading the roadmap and searching around and it seems like you've been updating the existing stuff, it's just not clear to me which things work with v13. I see in y-websocket where it imports y-leveldb but I don't see any updates in y-leveldb to supply the methods being invoked on it (writeState, bindState).
    Kevin Jahns
    @dmonad

    Hey @WinstonFassett Sorry for the confusion. I do plan to add more persistence layers for v13. The new API makes it much easier to build custom persistence layers (I'm talking about this section). Someone just recently asked the same question about Leveldb here: yjs/yjs#170 - this thread might be interesting to you.

    For now the best approach would be to either build your own persistence, contribute to y-leveldb :slight_smile:, or wait for me to add it eventually. @calibr kindly shared his leveldb approach in the above ticket.

    I'm currently working on y-dat and I want to publish a simple demo app with y-dat to test if y-dat is viable in practice.