Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Jeremy Dombrowski
    @meatflavourdev
    Hey @dmonad , I'm wondering about the pratcical limitations of Yjs in a single doc-- if the server is behind a load balancer, containerized and each has an instance of redis... it should scale pretty far, no? (I'm using a combination of Websockets and WebRTC, only single ws server atm). I think the limitations of React may become problematic (at some point)? Each user has a live cursor throttled at 30 updates / sec.
    With gfx applications, is there a way to prioritize data that is on the screen or near the user? (I think "no") What kinds of things might you recommend to look at before a scale test, and what kind of things during?
    I suppose I'd have the database server acting as a persistent server that doesn't die? The rest would be ephemeral containers? or am I thinking about that the wrong way...
    Any others with advice, also appreciated.
    Kevin Jahns
    @dmonad
    @MatrizLoin1210_twitter Other users can only connect their own connection to that specific peer. Ideally, that other peer disconnects themselves (provider.disconnect())
    @evilfant if ydoc.store.pendingDs === null && ydoc.store.pendingStructs === null that you can be certain that you are missing some updates. Otherwise, you simply have to initialize a fresh sync.
    After losing connection, you must create a fresh sync to the other peer. You will only exchange the missing updates (it exchanges a few bytes in case there are no updates at all). You can see examples on docs.yjs.dev ("document updates")
    Kevin Jahns
    @dmonad
    @meatflavourdev That should scale pretty well. With redis you can basically scale indefinitely. But note that all peers will still receive all document updates / awareness updates. So something you could look into is to separate your (I assume - large document) into smaller fragments that users can subscribe to when they need to. This is what subdocuments are for. But you can also just create several Yjs documents to reflect your application-specific data.
    Jeremy Dombrowski
    @meatflavourdev
    oh yes, subdocuments is an elegant solution to load pieces of canvas-- I'd not have considered that, didn't think about them that way.
    the canvas is infinite so i think it prudent to control document size and transfer rate (by extension)
    Qs
    @MatrizLoin1210_twitter
    am I the only one facing this issue after updating y-webrtc?
    TypeError: Cannot read property 'encrypt' of undefined
    Module.encrypt
    
      49 |   return /** @type {PromiseLike<Uint8Array>} */ (promise.resolve(data))
      50 | }
      51 | const iv = crypto.getRandomValues(new Uint8Array(12))
    > 52 | return crypto.subtle.encrypt(
         | ^  53 |   {
      54 |     name: 'AES-GCM',
      55 |     iv
    Kevin Jahns
    @dmonad
    @MatrizLoin1210_twitter This is probably because your browser / environment updated. There haven't been. any changess to y-webrtc/crypto in a long time. Can you please create a. bug report (and share which browser + version you are using)?
    Namit Chadha
    @nc
    Hey I'm using awareness for cursor type information, but need to remember the last location across sessions. What's the best way to do that? Use a Y.Map?
    Kevin Jahns
    @dmonad
    Hi @nc, I created the Awareness CRDT because the Yjs document does not deal very well with cursor positions. When you move your cursor across the document, you could easily create thousands of changes per second that cannot be efficiently shared using the Yjs CRDT. If possible, I recommend to add at least some kind of timeout before writing to Yjs. If possible, you should store the cursor in a different data structure when other users are not concerned about it. E.g. localStorage might be perfect if you just want to be able to restore cursors across browser-sessions.
    Other users probably don't care about the "last" cursor positions from other users.
    Namit Chadha
    @nc
    Ah yeah local-storage is a good shout. That'll probably work fine the user-case i'm targeting.
    Christophe Benz
    @cbenz_gitlab
    Hi everyone, I'm just wondering why it is named Yjs. I'm having fun imagining that the author could say to himself (why do I have to code CRDTs in JS?) (lol that's kind of meta!)
    Or more plausibly, a reference to functional programming with an inverted lambda
    Kevin Jahns
    @dmonad

    Hey @cbenz_gitlab,
    that's really funny and is actually what I thought when I started the project. I was a huge fan of Haskell, but that's pretty useless if you want to create collaborative applications. But why did it have to be JavaScript?? WHYYY JS!!

    Now I really like JavaScript. It's a really fun language.

    The actual reason is that I named the underlying CRDT algorithm YATA - Yet Another Transformation Approach. So Yjs is the js implementation of the Y(ATA) CRDT. Also, the Y kinda looks like two branches merging, which is what Yjs does. So when I name a variable ytext or ymodel, I immediately know that the data is observable and automatically merges with other peers :)

    ilovedesert001
    @ilovedesert001

    am I the only one facing this issue after updating y-webrtc?

    TypeError: Cannot read property 'encrypt' of undefined
    Module.encrypt
    
      49 |   return /** @type {PromiseLike<Uint8Array>} */ (promise.resolve(data))
      50 | }
      51 | const iv = crypto.getRandomValues(new Uint8Array(12))
    > 52 | return crypto.subtle.encrypt(
         | ^  53 |   {
      54 |     name: 'AES-GCM',
      55 |     iv

    [encrypt] this library only work for https

    Kevin Jahns
    @dmonad
    That's valuable to know, thanks!
    @MatrizLoin1210_twitter
    ilovedesert001
    @ilovedesert001
    @dmonad hello , is there any counter like build in shared type ?
    Kevin Jahns
    @dmonad
    Nope @ilovedesert001 , there is not.
    ilovedesert001
    @ilovedesert001
    Or there any tutorial to create custom shared type ?
    image.png
    I create counter , but feels not so right .
    Kevin Jahns
    @dmonad
    Custom types are no longer supported. It is too complicated to consider all the cases. I wanted to create a new API to create custom state-based CRDTs with Yjs. However, this probably needs to wait until the rust port is finished.
    The best approach at the moment is probably to maintain a Y.Array and add all addititons & substractions to the Y.Array. You must not delete entries, as concurrent deletes would still decrement by only one.
    ilovedesert001
    @ilovedesert001
    thanks , good enough for now .
    Kevin Jahns
    @dmonad
    if you simply store integers you should get really good compression.
    ilovedesert001
    @ilovedesert001
    Yes, I'm implemented with YArray
    Jun-04-2021 20-17-20.gif
    ilovedesert001
    @ilovedesert001
    It seems there is no way to implement this with single integer. after synced , I need cumulative result
    Kevin Jahns
    @dmonad
    Counter CRDTs are super popular. However, I believe that in many cases, you want the replace semantics. If user1 incremented a counter to 4 - because they think that 4 is the correct number. And user2 also increments the counter to 4 (because that other user is making the same calculation), it might be confusing when the conter is incremented to 8 (for no apparent reason)
    ilovedesert001
    @ilovedesert001
    In some case , this might be very useful , like likes , query counters, update counters, etc.
    image.png
    Kevin Jahns
    @dmonad
    Yeah, there are definitely valid use-cases.
    Although, in that use-case, you definitely want to store who pressed the "like button". Otherwise, people could like the same post many times.
    So you also end-up with an array.
    I'm just saying that Y.Array and Y.Map should be preferred in a lot of cases. There are definitely valid use-cases.
    Will Bartlett
    @Bartlett-Will
    Hi All, I'm working with y-websocket right now, and i'm wondering if anyone has experience adding messages to the server. I want clients to be aware when someone adds another "room", so that an added room gets added to other connected clients. Initially I thought i could just use the same WebsocketProvider to wrap the other providers to sync the client state using custom message types. I've gotten started but i keep running into issues with the persistence layer and i feel stuck. Any ideas?
    Will Bartlett
    @Bartlett-Will

    Hi All, I'm working with y-websocket right now, and i'm wondering if anyone has experience adding messages to the server. I want clients to be aware when someone adds another "room", so that an added room gets added to other connected clients. Initially I thought i could just use the same WebsocketProvider to wrap the other providers to sync the client state using custom message types. I've gotten started but i keep running into issues with the persistence layer and i feel stuck. Any ideas?

    The issue is if i just send a regular message like Websocket.send("stateUpdate") the server crashes with an Index out of bounds error.

    Kevin Jahns
    @dmonad
    You can either implement your own protocol based on the binary protocol that Yjs uses (which is fairly complicated) or use subdocument which seem perfect for your usecase https://docs.yjs.dev/api/subdocuments ;)
    Will Bartlett
    @Bartlett-Will
    @dmonad thank you for the quick reply. I saw that in the documentation - but i was thinking it might be overkill. Something simple like the awareness CRDT makes more sense to me. I'm really just trying to maintain a list collaborative of docs, and if someone added a doc, then that gets propagated to everyone.... but now that i type that out it seems like exactly what subdocument documentation is describing haha
    Nigel Gilbert
    @micrology
    Suppose I have a Y.Map and have set up a Y.UndoManager to track updates to it. I now make a series of, for instance, 20 adds, deletes and updates to the Y.Map. Then I decide that I want to undo these most of these changes, back to, say, the 3rd change. I could do this by calling undoManager.undo 17 times, but is there any way to revert straight to the 3rd change (e.g. by saving a pointer to the undoManager state before each change)? The use case is that I want to give the user a menu of the last n changes, so that the user can revert straight back to a previous state, without having to unwind each action one at a time.
    ilovedesert001
    @ilovedesert001

    I'm just saying that Y.Array and Y.Map should be preferred in a lot of cases. There are definitely valid use-cases.

    Indeed , Y.Array is good for now

    Nick Odumo
    @nodumo
    I am working on a PubNub Connector for YJS.
    Where I am having trouble is that YJS , the "Official Connectors", and the Y protocol library making use of array buffers but, PubNub passes data as strings.
    How would you just working around this limitation?
    ilovedesert001
    @ilovedesert001
    you can convert buffer to base64 string ?
    like , interceptor , encode to base64 before data goes to PubNub ,
    after receiving data from PubNub , decode to Yjs's type structs ,
    Will Hawker
    @whawker
    I'm just playing around with YJS map, but I notice there is no clear method, (to empty the map completely) - Is that intentional, as it's not really a CRDT way of doing things?