Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Alfredo Marrero
    @alfredoemr_twitter
    image.png
    RisbergAdam
    @RisbergAdam
    Hi, I'm having trouble understanding how document changes are synced between indirectly connected peers using the webrtc provider. Lets say we have client B which is connected to A and C, but A and C are not directly connected. From the code I could understand that when A updates the shared document it broadcasts a sync message with the encoded document update to all its connected peers (only B in this case). When B receives the sync message, is it relayed to C in some way?
    1 reply
    peter527
    @peter527
    I try to develop a collaborative system with Yjs and Java, there is one problem for me, how to initialize the full data when a new user enter the room?
    Kevin Jahns
    @dmonad

    @alfredoemr_twitter I don't think you are supposed to manipulate the HTML (that might lead to bugs in the editor you are using).

    @RisbergAdam please ask this question again in the discussion board and I will give a more exhaustive answer.

    1 reply
    @peter527 Please look for an answer in discuss.yjs.dev
    peter527
    @peter527
    @dmonad OK, thanks
    Mattan Ingram
    @mattaningram

    Building a multiplayer interactive text editor (using TipTap), but haven't found any good documentation or discussion around doing yjs migrations, particularly when people are actively in rooms editing. I found this question on the discussion board but it appears no one has answered it: https://discuss.yjs.dev/t/best-practices-for-data-migrations-with-yjs/766

    Anyone have any experience with this or links that can suggest some best practices?

    Alfredo Marrero
    @alfredoemr_twitter
    thanks @dmonad
    chaosprint
    @chaosprint:matrix.org
    [m]
    I suspect that eduroam can bring problems for yjs-codemirror. Two browsers in same computer can find each other but any two computers can not. Does anyone have this issue recently?
    Erik
    @erik:michelson.eu
    [m]
    Using y-webrtc?
    8 replies
    Ming
    @Mr-Ming

    Hi, I have a question =) regarding: https://github.com/yjs/y-webrtc#client-code

    How long does ydoc data gets stored in a webrtc room before it expires? (assuming there is an expiration time and if the assumption is true, if we can control that time) Thanks!

    Kevin Jahns
    @dmonad

    @alfredoemr_twitter @erik:michelson.eu If you are behind a proxy/firewall, it might very well be the case that you can't connect to certain other peers. For example, I have trouble connecting to my phone (LTE Vodafone) from my computer (DSL Vodafone)...

    This is a major drawback of WebRTC in practice. Often you see peers syncing randomly. This is because as long as all peers are (indirectly) connected, everyone will sync eventually. A joining client might connect to unsynced networks.

    When you want to use webrtc in practice, you should set up a TURN server (basically a server that forwards messages to other clients). This is part of the WebRTC spec. Videoconferencing solutions work because they usually set up turn servers (there are even commercial providers that you can pay to host TURN servers).

    Yjs uses simple-peer, you can supply a list of TURN servers to the simple-peer instance using the peerOpts parameter (see y-webrtc docu).


    @Mr-Ming The data in the room is destroyed once all clients disconnected. Rooms never expire, clients might just forget the content. If you set up a persistence provider (e.g. y-webrtc) you can retain it indefinitely. If that wasn't clear: there is no server holding the data in the y-webrtc room - it's all client-side.

    2 replies
    Joshua Onwuzu
    @joshuaonwuzu:matrix.org
    [m]
    Hello 👋 everyone, please i have question, is there a way i could implement e2ee on a collaborative editor using yjs and its y-websocket
    Kevin Jahns
    @dmonad
    There have been a lot of discussions on this topic in discuss.yjs.dev. The community already implemented a few nice solutions. https://github.com/SerenityNotes/naisho & https://github.com/YousefED/Matrix-CRDT
    In any case, you need to implement a custom provider, because y-websocket needs to understand the content (it can't support e2ee).
    Joshua Onwuzu
    @joshuaonwuzu:matrix.org
    [m]
    @dmonad: thanks , i really appreciate your response, i will check them out.
    MaxNoetzold
    @MaxNoetzold
    Hey, I have a question regarding the y-indexeddb. I would personally think that it is a bug, but I am not sure. I have built a very simple react app to check out y.js. I use my own express server and connect all clients via the y-websocket provider. This all worked very well. Last, I wanted to add the indexeddb persistence provider, which I did as explained here: https://docs.yjs.dev/ecosystem/database-provider/y-indexeddb. In Chrome, the synced log is triggered, but in Firefox, it is not. When I look into the devtools, I can see that both browsers store data in the correct store. It seems to actually still work anyway, since I can load the ydoc content offline. I just thought that it would be better practice to load the data after the indexeddb sync is completed?
    Jeremy Gordon
    @jeremysf

    Hello all! I'm brand new to yjs, but have a lot of experience with OT and CRDTs. I'm running into a problem with Y.Map. I am doing:

    const doc = new Y.Doc()
    const data = doc.getMap("data")
    data.set("generator", "")
    data.set("generator", "something")
    expect(data.get("generator")).toBe("something")
    const update = doc.encodeStateAsUpdate(doc)
    const doc2 = new Y.Doc()
    const data2 = doc2.getMap("data")
    data2.set("generator", "")
    Y.applyUpdate(doc2, update)
    expect(data2.get("generator")).toBe("something")

    The test sometimes passes, and sometimes fails on the second "expect" which checks to see if the update was applied. If I remove setting the "generator" entry to an empty string, the tests consistently pass.

    I have tried adding in a delay between applying the update to doc2 and the expect. I have tried waiting for on("update") to be called for doc2. I have tried dumping out the JSON of data2 (it sometimes also shows "something" and sometimes shows "" after the update). I have tried dumping out the bytes of the update (they change on every run, successful or unsuccessful).

    Any ideas?

    Jeremy Gordon
    @jeremysf
    Hmm, sounds like from this post, this is not a supported use case? https://discuss.yjs.dev/t/initial-offline-value-of-a-shared-document/465/13
    Jeremy Gordon
    @jeremysf

    Well, I tried the following, but no luck, still the same intermittent test failures on the second "expect":

    const doc = new Y.Doc()
    const data = doc.getMap("data")
    data.set("generator", "")
    data.set("generator", "something")
    expect(data.get("generator")).toBe("something")
    const update = doc.encodeStateAsUpdate(doc)
    const doc2 = new Y.Doc()
    Y.applyUpdate(doc2, update)
    const data2 = doc2.getMap("data")
    expect(data2.get("generator")).toBe("something")

    Jeremy Gordon
    @jeremysf
    Still having that issue, but also a little confused about the TypeScript types for Y.Map. Specifically looking at the class definition Y.Map<MapType>. I see a method definition on that class "set(key: string, value: MapType): MapType". Is the intent there that the MapType generic argument on Y.Map is for uniform value types in the map? Like a Y.Map<number> is a map that only contains number values? What is the canonical way to declare a map that can hold any values, Y.Map<any>? With the version and configuration of tsc I'm using, a naked instantiation of "new Y.Map()" (vs. a "new Y.Map<any>()") will not transpile.
    Jeremy Gordon
    @jeremysf
    Ok, looks like a bad cached test result or something. As described in the post I linked to, my first attempt where I initialize both documents "empty" and attempt to sync, is not really a supported case. The second case is now working. Please drive through :)
    Kevin Jahns
    @dmonad
    @MaxNoetzold So y-indexeddb doesn't fire a synced event in firefox? Can you please open a bug report in y-indexeddb?
    1 reply
    Iván Topp Sandoval
    @ivan-topp

    Hello, I wanted to comment that I have just published a package called "y-socket.io" which is a provider quite similar to y-websocket (since it is inspired by this package), but implemented with socket.io. If anyone wants to use it feel free to do so and also make suggestions.

    PS: This is my first package published on npm and it's a way to give something to the world in gratitude for the many packages I've used throughout my life as a software developer. Especially to the YJS community, as I used YJS to develop my title work.

    3 replies
    chaosprint
    @chaosprint:matrix.org
    [m]

    The y-codemirror demo stops working recently: https://demos.yjs.dev/codemirror/codemirror.html

    I think it's because of the broken link of the css from codemirror.net as they move to cm6

    I am putting up an personal/unofficial alternative and I will continue to test why sometimes Mac and Windows cannot see each other. This must be lasting for quite a long period. Did anyone else notice it?

    https://y-cm-demo.netlify.app/

    chaosprint
    @chaosprint:matrix.org
    [m]

    :point_up: Edit: The y-codemirror demo stops working recently: https://demos.yjs.dev/codemirror/codemirror.html

    I think it's because of the broken link of the css from codemirror.net as they move to cm6

    I am putting up an personal/unofficial alternative and I will continue to test why sometimes Mac and Windows cannot see each other. This must be lasting for quite a long period. Did anyone else notice it?

    https://y-cm-demo.netlify.app/

    Well. I have done the same thing for cm6.

    CM6 official demo is working for my windows+mac.
    https://demos.yjs.dev/codemirror.next/codemirror.next.html

    But when I deploy on netlify, windows and mac cannot see each other:
    https://y-cm6-demo.netlify.app/

    :point_up: Edit: The y-codemirror demo stops working recently: https://demos.yjs.dev/codemirror/codemirror.html

    I think it's because of the broken link of the css from codemirror.net as they move to cm6

    I am putting up an personal/unofficial alternative (https://y-cm-demo.netlify.app/) and I will continue to test why sometimes Mac and Windows cannot see each other. This must be lasting for quite a long period. Did anyone else notice it?


    UPDATE:

    Well. I have done the same thing for cm6.

    CM6 official demo is working for my windows+mac.
    https://demos.yjs.dev/codemirror.next/codemirror.next.html

    But when I deploy on netlify, windows and mac cannot see each other:
    https://y-cm6-demo.netlify.app/

    :point_up: Edit: The y-codemirror demo stops working recently: https://demos.yjs.dev/codemirror/codemirror.html

    I think it's because of the broken link of the css from codemirror.net as they move to cm6

    I am putting up an personal/unofficial alternative (https://y-cm-demo.netlify.app/) and I will continue to test why sometimes Mac and Windows cannot see each other. This must be lasting for quite a long period. Did anyone else notice it?


    UPDATE:

    Well. I have done the same thing for cm6.

    CM6 official demo is working for my windows+mac.
    https://demos.yjs.dev/codemirror.next/codemirror.next.html

    But when I deploy on Netlify after vite build, my Windows and Mac cannot see each other:
    https://y-cm6-demo.netlify.app/

    I had the same bug on CodeSandbox, using React. So I am not sure where the problem comes from.

    :point_up: Edit: The y-codemirror demo stops working recently: https://demos.yjs.dev/codemirror/codemirror.html

    I think it's because of the broken link of the css from codemirror.net as they move to cm6

    I am putting up an personal/unofficial alternative (https://y-cm-demo.netlify.app/) and I will continue to test why sometimes Mac and Windows cannot see each other. This must be lasting for quite a long period. Did anyone else notice it?


    UPDATE:

    Well. I have done the same thing for cm6.

    CM6 official demo is working for my windows+mac.
    https://demos.yjs.dev/codemirror.next/codemirror.next.html

    But when I deploy on Netlify after vite build, my Windows and Mac cannot see each other:
    https://y-cm6-demo.netlify.app/

    I had the same bug on CodeSandbox, using React. So I am not sure where the problem comes from:
    https://0t7bn.csb.app/

    chaosprint
    @chaosprint:matrix.org
    [m]

    :point_up: Edit: The y-codemirror demo stops working recently: https://demos.yjs.dev/codemirror/codemirror.html

    I think it's because of the broken link of the css from codemirror.net as they move to cm6

    I am putting up an personal/unofficial alternative (https://y-cm-demo.netlify.app/) and I will continue to test why sometimes Mac and Windows cannot see each other. This must be lasting for quite a long period. Did anyone else notice it?


    UPDATE:

    Well. I have done the same thing for cm6.

    CM6 official demo is working for my windows+mac.
    https://demos.yjs.dev/codemirror.next/codemirror.next.html

    But when I deploy on Netlify after vite build, my Windows and Mac cannot see each other:
    https://y-cm6-demo.netlify.app/

    I had the same bug on CodeSandbox, using React. So I am not sure where the problem comes from:
    https://0t7bn.csb.app/

    Any suggestions on where to debug next?

    MaxNoetzold
    @MaxNoetzold
    I got another question now :D
    I wanted to use the y-mongodb library. I looked into it and figured out that it is an incomplete, faulty, and unmaintained library. Since I need a MongoDB provider, I will write my own local fork now. I would be fine with uploading it somewhere, but I am not really sure where since there is already the y-mongodb provider. Does someone have an idea?
    Kevin Jahns
    @dmonad
    @chaosprint:matrix.org I have no idea where this message ends. I feel some of these questions have been answered. The codemirror issue was fixed, however I can't get the build running anymore on heroku (they are shutting down OSS projects anyway, so I need to switch).
    @MaxNoetzold Thanks for making the effort! Maybe you can ping the author to give you the package-name. If that doesn't work, we can ping the people at npm to take over the package (done that before). In the meantime, you can publish something under a different name (e.g. y-mongo or y-mongodb-provider)
    Tong Yuqiang
    @smallst

    i don't know if anyone ask this before. But why y-webrtc/y-websocket provider.awareness cannot get the first user when open editor ? (but not click the editor)

    for example using yjs-demo/quill , when i create the WebsocketProvider, and check provider.awareness.getStates() (as init state) and add a change listener for provider.awareness, then open the demo in first browser, i get the init state of 1 awareness, but no listener triggered.
    but when i open the demo in second browser, i get the init state of 1 awareness, and a change event to show now we have 2 awareness. Both demo didn't click anything.

    and if i click the first demo editor before second browser open, i can get the change event that 1 user connected.

    But if so, i cannot know how many awareness exactly when a new tab/browser opened. since the first 2 browsers both give me 1 awareness when open, and only second browser give me the event. if i using event to check users, only 2, 3, ... users can be found. if i use init state, i cannot detect the second browser..

    so without editor updates, why second browser can trigger the awareness event but first one can't?

    what i want to do is just, when only 1 user connected to the editor, do not show awareness, but only multiple users join, show each awareness. i think most of online CRDT webpage do these ? (google docs. slab .. and others)

    1 reply
    MaxNoetzold
    @MaxNoetzold
    Alright: I published my mongodb provider: https://www.npmjs.com/package/y-mongodb-provider.
    James Fox
    @jamesopti

    When using a YDoc on the client side that syncs with a YDoc on a Node server, is it possible to check whether or not these 2 YDocs are in sync or not? The use case would be checking if a client has applied certain updates that haven't yet been received by the server, and if they are not in sync for a prolonged period of time that there may be an issue.

    Would it be possible to compare the client YDoc's store.client data with the servers for a particular clientID?

    clientDoc.store.clients.get(clientDoc.clientID) compared to serverDoc.store.clients.get(clientDoc.clientID)

    James Fox
    @jamesopti
    Basically every 1min or so I want a client to be able to ask the server if it has received all the updates that the client has applied.
    Tong Yuqiang
    @smallst
    From my last post, i find i can use y-websocket and YDoc.observe to find first sync event with server. i think webrtc is p2p so ydoc won't sync with server, then maybe no event? but i still believe we need an event to make webrtc clients know if there're other peers or not. awarenesss.update event tells peers that there exists more than 1 peers. but how can i get event that there're no other peers ?
    MaxNoetzold
    @MaxNoetzold
    I was wondering if anyone ever implemented a graph shard type. I searched a lot and found some forum entries like: https://discuss.yjs.dev/t/graph-data-structures/745 or https://discuss.yjs.dev/t/deletion-of-things-with-multiple-parents-on-conflicts/685. Where no one really gave a solution but just mentioned problems and possible approaches. I was wondering if I want to share a graph with yjs, should I do what they proposed (object with two arrays for edges and nodes) and add logic to the onChange event where I check if I have to remove additional edges? or could it make more sense to build my own shared data type (without much experience)?
    Neftaly Hernandez
    @neftaly
    I'm trying to set and read values on a Y.Map but nothing is being written, am I correct in assuming that the Y.Map needs to be attached to a Y.Doc to work properly? I can't find this written in the docs
    const Y = require('yjs')
    const x = new Y.Map([ ['asd', true], [1232, 4234] ])
    x.set('aaa', 123)
    x.set('q', true)
    console.log(x.get('aaa'))
    console.log(x.toJSON())
    Kevin Jahns
    @dmonad

    @MaxNoetzold Sweet! Thank you <3

    If you want, we can add it to the list of providers in github.com/yjs/yjs - just open a PR with a description

    @smallst In peer-to-peer WebRTC, it is really hard to determine whether there are no peers. Every peer is always waiting for incoming connections. Sometimes it takes a longer time for two peers to find each other. The awareness state usually takes a longer time to populate.
    Kevin Jahns
    @dmonad

    @MaxNoetzold There is PRSM a really cool graph tool that you can take for inspiration. @micrology, the author, is active as well :)

    All shared types are basically a list-crdt. You can't the only difference is how you interpret the data. So I don't recommend building custom shared types.
    You should, however, think, and experiment a bit with the existing shared types. It might make sense build another abstraction abound one of the existing types. E.g., you could model the graph using two Y.Arrays and handle consistency (removing dead connections, and so on..) in a custom class that is the only entity that can manipulate the shared types. Your custom class (I'd call that a shared model) can serve as a nice wrapper for handling graph-like data. I've been trying to popularize this approach in the Jupyter project as well. This is what I'd do if I were to implement a data model for graphs.

    @neftaly It's explained in the introduction to shared types: https://docs.yjs.dev/getting-started/working-with-shared-types
    Neftaly Hernandez
    @neftaly
    Great, thanks!
    Is it safe to do new Y.Array([ 1, 2, 3, 4 ])? I dont see it mentioned in the docs and there isnt a test for it but its in the code
    i also didn't see new Y.Map([[ "a", 1 ], ["b", 2]) in docs but theres a test for it
    Nishant Mittal
    @nishantwrp
    const Y = require('yjs');
    
    const doc1 = new Y.Doc();
    const doc2 = new Y.Doc();
    
    const m1 = doc1.getMap('a');
    
    const m11 = new Y.Map();
    m1.set('b', m11);
    m11.set('c', 'd');
    
    const m2 = doc2.getMap('a');
    m2.set('c', {
        b: 'd'
    });
    
    const m21 = new Y.Map();
    m2.set('b', m21);
    m21.set('a', 'd');
    
    const doc2FullUpdate = Y.encodeStateAsUpdate(doc2);
    Y.applyUpdate(doc1, doc2FullUpdate);
    console.log(doc1.toJSON());

    I'm getting the following output.

    { a: { b: { c: 'd' }, c: { b: 'd' } } }

    When I merge doc1 with doc2. Should the value of the nested shared type be also be merged? Shouldn't the value of doc1 now be. { a: { b: { c: 'd', a: 'd' }, c: { b: 'd' } } }

    Kevin Jahns
    @dmonad

    @neftaly Yes, it's safe to use that

    @nishantwrp Shared Types merge, values don't. So that's the expected behavior. However, you can model JSON objects as shared types if you need them to be merged.

    Nishant Mittal
    @nishantwrp
    @dmonad can you please give me an example. In my example, I created two new shared maps m11 and m21 which I was expecting to get merged.