Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Alex Chumbley
    @chumbalayaa

    I also found a workaround to our issue - posting here in case anyone needs something similar

    We we pulled in our DB value and created a YDoc using prosemirrorJSONToYDoc, then looked at the Prosemirror equivalent of what was in our initial YDoc to see if it was empty. If it was, we did the following

              const stateVector1 = Y.encodeStateVector(ydoc)
              const stateVector2 = Y.encodeStateVector(dbDoc)
              const diff1 = Y.encodeStateAsUpdate(ydoc, stateVector2)
              const diff2 = Y.encodeStateAsUpdate(dbDoc, stateVector1)
              Y.applyUpdate(ydoc, diff2)
              Y.applyUpdate(dbDoc, diff1)
    kapil verma
    @kapv89
    what should be the type passed to the generic argument of Y.Map?
    kapil verma
    @kapv89
    nevermind found out
    Will Hawker
    @whawker
    thanks @chumbalayaa! What did you if it ydoc wasn't empty?
    Alex Chumbley
    @chumbalayaa
    @whawker we just loaded the ydoc data into our editor and used that as our source of truth
    Yonggoo Noh
    @ygnoh
    Is there a binding for input or textarea element?
    Namit Chadha
    @nc
    I had a question about Y.Map vs Presence for tracking something that's hard-realtime like dragging an element, what's the best data structure to use for those kind of situations.
    Eduard Ruzga
    @wonderwhy-er
    I was wondering, is there suggested strategy for cleaning up yjs doc. Have a hobby markdown/wiki where I am storing state of documents on server in sql database.
    For one often changed document with mostly links I am in situation where it contains 76 bytes of text but its base64 encoded yjs state is 65Kb and applyUpdate starts to take considerably more time.
    I was thinking about finding a condition under which I create a new doc on a server, insert text from old one, this way erase history so new connecting clients get that.
    But as far as I can say this may lead to occasional loss or duplication of text depending on how I handle case where there was client that still had state from before erasure.
    Did anyone encountered such a need? Issues with slow applyUpdate in almost empty document. Any strategies to clean things up occasionally?
    Kevin Jahns
    @dmonad
    Hey,
    so I'm back from vacation. I think it would be best if you used https://discuss.yjs.dev/ for questions for things that can't be answered in a single sentence. I'm slowly catching up to all the missed messages.
    Eduard Ruzga
    @wonderwhy-er
    Thanks, will ask there.
    Marcelo Luiz Onhate
    @onhate
    hello folks, simple question: Having a XML in string format <a>b</a> how do I create a XMLFragment with that string content?
    Marcelo Luiz Onhate
    @onhate
    or, having a XmlFragment how to I create another XmlFragment with the same content
    Marcelo Luiz Onhate
    @onhate
    I tried the following but the result is always an empty string
     const update = Y.encodeStateAsUpdate(fromDoc.doc);
     Y.applyUpdate(toDoc.doc, update);
     console.log('xml', toDoc.toJSON());
    Rhys Brett-Bowen
    @rhysbrettbowen
    Reading through the paper for YATA it mentions that it can be used for graphs. Is there an example of this? From what I could figure people usually have a map of nodes and a map of edges. With that that though, what happens if someone removes a node (and all connecting edges) in their client and another client adds an edge to that node? It seems like the new edge would still be added. This seems like it would be an issue with any piece of data that had multiple parents (as otherwise they could just be put in a map). I could imagine a garbage collector that would check and remove these as the changes came in, but is there a better way? Guessing that a graph like that (or tabular data) could be implemented as types...
    Nigel Gilbert
    @micrology
    @rhysbrettbowen There is an example of using yjs with vis-network to manage and display a graph at https://github.com/micrology/y-vis-network . This is just a skeleton example. For the full works, see https://prsm.uk and the code at https://github.com/micrology/prsm (for the interaction with yjs, see the top of https://github.com/micrology/prsm/blob/master/js/prsm.js
    Rhys Brett-Bowen
    @rhysbrettbowen
    @micrology Thanks for the links. I ran the example and can confirm that what I said is an issue. If you run 2 clients, and add 2 nodes, disconnect one, then add a link in one client and remove a node in another then reconnect you will end up with an extra edge in the edge data set. The client does not have an issue with this because the vis-network won't connect an edge where the nodes don't exist: https://github.com/visjs/vis-network/blob/master/lib/network/modules/components/Edge.js#L539. To fix this there would need to be extra logic when adding an edge to check that the node was not removed and not add it - or add it as a deleted node? I don't believe that's too bad though. For more context we've got some OT applications that do spreadsheets (which have a similar issue as a cell has a row and column - thus multiple parents) and this is handled in the transformers (because they know to look for a delete of either parent and so can remove the op). We've also got a graph application which is being done now. So the three options would be 1) leave it - not too bad as we'd just be marking them deleted anyway. It would be up to application code to make sure it deals with orphaned items, this would only be an issue if the values were large 2) change the bindings to check and fire a remove if the item is orphaned - adds extra logic to the bindings 3) have a process that garbage collects orphaned items - means a whole other process and the app will still have to deal with orphans
    Though it might be preferred to leave as is so an undo on removing the node will also bring back that edge...
    Nigel Gilbert
    @micrology
    I've just tried this and you are right - PRSM reports that the 'network is corrupt: links are connected to non-existent factors' (at least our error checking logic works!). For us this is a really edge (pardon the pun) case, as our users are typically always on line, but nevertheless, it is something we should deal with.
    Zixuan Chen (陈子轩)
    @zxch3n
    Hi all! I'm new to this awesome project. I have a few questions about the GC mechanism.
    In paper YATA "Garbage Collection" section, it says GC will be triggered after a period of time.
    And GC described in the paper seems to mean the removal of a operation, not just clearing the content field.
    However, I cannot find related code in yjs repo. If I insert 10000 sentences to a Y.Text and delete them afterwards,
    the store will still be large even if the remaining data is small. Because the store still holds the ContentDeleted/GC objects.
    Am I missing something here? Or is it implemented in somewhere else?
    kapil verma
    @kapv89

    Reading through the paper for YATA it mentions that it can be used for graphs. Is there an example of this? From what I could figure people usually have a map of nodes and a map of edges. With that that though, what happens if someone removes a node (and all connecting edges) in their client and another client adds an edge to that node? It seems like the new edge would still be added. This seems like it would be an issue with any piece of data that had multiple parents (as otherwise they could just be put in a map). I could imagine a garbage collector that would check and remove these as the changes came in, but is there a better way? Guessing that a graph like that (or tabular data) could be implemented as types...

    @rhysbrettbowen can a possible solution be to perform edge-garbage-collection each time an edge is created?

    Rhys Brett-Bowen
    @rhysbrettbowen
    @kapv89 Yep - that's what I was imagining. You could have a YGraph type that was made up of a nodes and edges map that would handle the changes. That would be preferable because if you look at the connector it fires when there are changes to the map. The issue there is that you would still fire the change and it would be picked up by the app that there was an orphaned edge so you would need to instead do the check on the edge before firing off those events. It's all do-able. Could also extract out a tabular data type which allows for columns/rows to be added/removed using similar logic. I think wrapping this logic in as types is probably the way to go. Still in exploration stage but if I get around to writing these I'll make sure to push back upstream
    Rhys Brett-Bowen
    @rhysbrettbowen
    On a different note - how sure are we about undo/redo in y.js? I know it was flawed in quilljs for a while but should have been fixed a couple of years ago: quilljs/quill#2500. When I go to the quill editor on https://docs.yjs.dev/getting-started/a-collaborative-editor I put in "ac" in the top, then in the 2nd one I add "b" int he middle. Then I got back to the first and undo. So far so good. Then redo... "acb".... not quite right though not the worst. However it gets worse if you keep undoing/redoing.
    Namit Chadha
    @nc
    hello has anyone tried writing a yjs connection provider for ... firebase! ?
    ah found this -> yjs/yjs#189
    Namit Chadha
    @nc
    and it works with firebase, pretty easy to setup. if anyone has questions can share
    separately if i have a queue of updates and want to flatten them to save space is there an easy way? and does that then become a snapshot other changes are based off of?
    Sean Zellmer
    @lejeunerenard

    Hey everyone! I have two Yjs docs connected via the webrtc provider sharing a Map type. When I set a property foo on device A, device B updates perfectly. However when I set foo on device B now, it is deleted on A (as shown by my observe event on the Map and the update on the doc). Note the order of changing foo on A & B doesn't make a difference.

    I have tried creating a simple example script but never can reproduce the desync. Any suggestions on how to debug how the two docs get out of sync? Are there any common scenarios that can cause this?

    Sean Zellmer
    @lejeunerenard
    Also note that the first change to foo is an add action according to the observer event's changes. basically whichever device sets it first is the only one that works.
    songispm
    @songispm
    image.png
    I need the source code about Drawing, where can i find it ?
    songispm
    @songispm
    image.png
    what different of these types
    kapil verma
    @kapv89

    separately if i have a queue of updates and want to flatten them to save space is there an easy way? and does that then become a snapshot other changes are based off of?

    @nc the way y-leveldb persistence does it is by merging the updates queue into a single update when fetching the updates from the db. I use the same mechanism in my relational db backed y-websocket backend https://github.com/kapv89/yjs-scalable-ws-backend/blob/main/src/setupWSConnection.ts#L133-L156

    3 replies
    songispm
    @songispm
    image.png
    my data is a JSON tree, what YType should I use ?
    Will Heslam
    @willheslam
    This message was deleted
    Will Heslam
    @willheslam

    Hi Everyone, I've been playing around with yjs and enjoying it so far :)

    I've found that a nested text inside a map behaves differently to a top level text, which is surprising to me.
    Here's the behaviour with two top level texts:

    aText current state aaaaaa
    bText current state bbbbbb
    insert into aText 'AAA" at 2
    insert into bText 'BBB' at 2
    apply updates to both
    adoc latest state bbBBBbbbbaaAAAaaaa
    bdoc latest state bbBBBbbbbaaAAAaaaa

    This makes sense to me - both texts end up with the combined inserts.

    Whereas with a nested text inside a map:

    aMap current state { 'my nested text': 'aaaaaa' }
    bMap current state { 'my nested text': 'bbbbbb' }
    insert into a nested text 'AAA" at 2
    insert into b nested text 'BBB' at 2
    apply updates to both
    aMap latest state { 'my nested text': 'bbBBBbbbb' }
    bMap latest state { 'my nested text': 'bbBBBbbbb' }

    They randomly converge on either the text from A, or B, instead of including the changes from both.

    This makes me feel like I've incorrectly set up my nested text inside my map. Happy to show my code, but I wanted to check as I couldn't find any conclusive statements in the docs beyond my intuition - the nested text state in my second example should end up looking just like the top level text in my first example, right? (otherwise there'd be little point nesting shared types)
    I used the docs for Y.Text that has an example of a nested text for my inspiration, but the docs don't actually show it being updated and being propagated across multiple document instances
    https://docs.yjs.dev/api/shared-types/y.text
    Are there any other examples of nested text in the docs or demos?

    Thanks :)

    Will Heslam
    @willheslam

    I've found that a nested text inside a map behaves differently to a top level text, which is surprising to me.
    Here's the behaviour with two top level texts:

    I solved my problem - it turns out that creating two Texts separately under the same key are treated as completely separate things - the key is not their identity, they are. The only safe way to collaboratively edit the same text is to create it on one document, and replicate it to the others by sending state as an update, then editing it afterwards!

    Thanks, that makes a lot of sense and is something I'll bear in mind for the future!

    Zach Daniel
    @zachdaniel
    Hello! I'm having a bit of trouble understanding the actual usage pattern of yjs from a macro standpoint. I'm not using nodejs for my server, but I could theoeretically call out to a managed nodejs process from my backend server, something I've set up for SSR already. I already have websocket connections set up from my clients to my server. Is there some way that I could use yjs on the clients, managed from a non-js server without having to involve nodejs server side?
    Zach Daniel
    @zachdaniel
    I guess WebRTC is an option, unsure how to go about sending down the result of multiple clients collaborating over WebRTC to the server...
    Will Hawker
    @whawker
    morning all, I have a few outstanding PRs on y-prosemirror, is the repo still maintained? https://github.com/yjs/y-prosemirror/pulls/whawker
    kapil verma
    @kapv89

    Hello! I'm having a bit of trouble understanding the actual usage pattern of yjs from a macro standpoint. I'm not using nodejs for my server, but I could theoeretically call out to a managed nodejs process from my backend server, something I've set up for SSR already. I already have websocket connections set up from my clients to my server. Is there some way that I could use yjs on the clients, managed from a non-js server without having to involve nodejs server side?

    @zachdaniel you can take a look at this repo https://github.com/kapv89/yjs-scalable-ws-backend and modify it to interact with an api instead of interacting directly with the db, and then have your actual api-backend in any language of your choice

    jessiesanford
    @jessiesanford
    Does anyone know why observe would not trigger for a delete call on a YMap?
    seb001
    @seb001:matrix.org
    [m]
    Hi all. I was wondering if anyone was using TipTap and found a way around the duplicate text upon initialization issue. I found this yjs/yjs-demos#16 but unfortunately my only option is to read and write the content as HTML.
    2 replies
    Ulion
    @ulion

    That would be surprising to me too! The hard drive is an SSD. Does anyone else have any data on this?

    Related to this, I am using both y-websocket and y-indexeddb. The README says

    y-websocket
    A module that contains a simple websocket backend and a websocket client that connects to that backend. The backend can be extended to persist updates in a leveldb database.
    y-indexeddb
    Efficiently persists document updates to the browsers indexeddb database. The document is immediately available and only diffs need to be synced through the network provider.

    from which I concluded that y-indexeddb (which uses the local client) should sync faster than y-websocket (the websocket server is in Ireland and I am in England). But I always find that the websocket provider syncs first, by a few milliseconds. Why?

    The code looks like this (exactTime() yields the time now to the nearest millisecond):

        const doc = new Y.Doc();
        const persistence = new IndexeddbPersistence(room, doc);
        persistence.once('synced', () => {
            console.log(exactTime() + ' local content loaded');
        });
        const wsProvider = new WebsocketProvider(websocket, 'prsm' + room, doc);
        wsProvider.on('sync', () => {
            console.log(exactTime() + ' remote content loaded');
        });
        yNodesMap = doc.getMap('nodes');
        yEdgesMap = doc.getMap('edges');

    Example output:

    09:49:47:513  remote content loaded
    09:49:47:516  local content loaded

    I believe y-websocket implemented with a bug which did not wait both side exchange the initial vector, then sent out the initial empty data to client, then client thought it is "synced" but indeed the server side did not load the yjs doc from leveldb yet.

    Nigel Gilbert
    @micrology
    Interesting idea, but I checked by inserting a debug message inside wsProvider.on('sync', () => {, which confirmed that at the time of the sync event, the remote data had indeed been loaded.
    CRIMX
    @crimx
    Hi I'd like to make a custom provider, but the docs seem unfinished. What should the interface be for a provider?
    Mezriss
    @Mezriss
    Hi, is there an easy way to add access control to yjs documents? I.e. certain fields can be modified only by certain users, other users can only read them.
    Sudhanshu Gautam
    @sudhanshu16
    Hey, I am using yjs with quill over websockets. I noticed that yjs is ignoring the initial content from Quill and overwriting it with <p><br/></p>, anybody know how to get around this?
    1 reply
    Reyhan Sofian
    @reyhansofian
    hi everyone. I'm using yjs for realtime dashboard using y-websocket. I've been hitting an issue whenever a client refreshes their page, the yjs server will send the last state before the page refresh. is there a way I can avoid this?