Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    micrology
    @micrology
    Just to remind @dmonad when y-websocket is next revised: on April 13, @callum-atwal and I suggested code that could be included to allow y-websocket to be used with wss:// as well as ws://
    Kevin Jahns
    @dmonad
    Hey @micrology , where is the PR to allow wss in y-websocket? If that would be helpful to you, I might include it in y-websocket. But I generally advise to use a proxy (e.g. Nginx, haproxy, apache, ..) to handle connection requests. Nodejs is not that good at handling a lot of connections and it is generally advised to use a proxy for handling (secure) connections.
    Micah
    @micahscopes
    Are there any plans for a y-js port to rust? I don't mean this as a frivolous question... My team relies heavily on Python, and having a rust port of y-js could enable us to reuse core data structures across javascript and python environments
    Koma
    @thebata_twitter
    Hey everyone! I'm working on a collaborative app and I'm having some difficulties understanding how YDoc handles peer updates.
    I've started with example code for Quill editor that works fine, but if I replace const type = ydoc.getText('quill') with something like const mymap = ydoc.getMap("event"); const mytext = new Y.Text(); mymap.set("description", mytext); const type = mymap.get('description');, Quill won't work in collaborative mode. I see that messages are being exchanged through websocket, but editor value isn't updated in my case.
    Both versions of the code return YText as type, in original/demo version it is stored in the root of YDoc, while my modified version stores the YText inside a YMap. I see that YDoc object is being updated via websocket on clients by console logging on update action, but I can't figure out why the editor won't update the content. Any ideas?
    I'm trying to create a bit complex structure of the YDoc and have multiple editors/properties handled via single YDoc, so I'd like to avoid having all those editors inside YDoc root, I'd rather like them to be a part of my document structure. Is this doable in any way?
    Kevin Jahns
    @dmonad
    Hi @micahscopes There is a very good chance that Yjs will be ported to Rust this year. This has been requested several times and now I'm also pretty excited about the prospects of porting Yjs. I'm currently looking for funding, and I will find a way to make this possible.
    Kevin Jahns
    @dmonad

    @thebata_twitter Yes, this is possible. A client could, for example, create a new document as a Y.Map property and initialize it with some content. Another client could iterate through the map, find the document, and then start working on that document as well. This can be useful to create some form of directory structure.

    When you execute the above code for every client, each client will overwrite the description map and write a new Y.Text object. So they all work on different objects. Instead, only one client should create the document. Other clients can read it. Overwrite it only if this is your intended behavior. Also note that you can't bind data to a deleted Y.Text type.

    In your case, it probably makes more sense to create another top-level type `"event-description".

    If you still want to use the above approach, you should also wait for ydoc.on('synced', () => ..) to make sure that you received initial content.
    Koma
    @thebata_twitter

    Hey @dmonad, thanks for the help! I'd still like to use the mentioned approach with a bit complex structure, so the solution you provided sounds good. I've tried to implement it as per your suggestion ( ydoc.on('synced', () => {console.log('synced')}) ) , but 'synced' event seems not to be fired at any moment.

    Could you please tell me when 'synced' event should be fired and by which entity? I've searched through the source code, but couldn't figure that out...

    I'm using WebsocketsProvider if it matters, tried with local one as well as with the public one from the quill demo.

    Kevin Jahns
    @dmonad
    Or was it "sync"? I haven't standardized this and im not sure if i want to encourage people to use it. There are only few cases where you really need this event.
    In the future, especially when you start using several provider at
    especially when you start using several provider together, you will have an easier time with my initial suggestion. I really need to improve the docs on this topic.
    Koma
    @thebata_twitter
    Yes, you're right, its 'sync' event but on provider, not the ydoc. provider.on('sync', () => {console.log('sync')}) does the job
    Koma
    @thebata_twitter

    I'm working on a MVP now and single ydoc per object on my side at the moment makes more sense. I get the idea about multiple providers and top-level props, will try that approach also at some point.

    Still getting to know this awesome library that you've made, it will take some time and experiments to figure out all the possibilities :)

    Kevin Jahns
    @dmonad
    Right. Great to hear 😊
    Orkhan Alikhanov
    @OrkhanAlikhanov

    Hi guys and @dmonad, I have quick question.

    Let's say we have prosemirror document: <p>Hello World<p>
    in yjs, paragraph represented as YXmlElement the text 'Hello World' as YXmlText, right?

    Now, there are 2 users that change the document concurrently. One splits the paragraph at the end of the word 'Hello'. Another one appends !!! to the end of the word World.
    End result becomes: <p>Hello<p><p> World!!!<p>
    as Yjs doc: 2 YXmlElements and 2 YXmlTexts, right?

    My question is that, does yjs split the YXmlText into two? I seems it does not, then how in the world yjs is able to insert the !!! at the end of newly created second YXmlText? It somehow works perfectly, but I'm unable to understand how the position where !!! is inserted is tracked when the YXmlText was created newly?

    Kevin Jahns
    @dmonad
    You are right. I would actually expect thet the result would be <p>Hello!!!</p><p>world</p>. I don't know why this worked ^^
    Orkhan Alikhanov
    @OrkhanAlikhanov
    Lol. Do you have any solutions to preserve user intention in that case? What do you think about the possibility of introducing splitting single ystrucure into two ystructure-s?
    Kevin Jahns
    @dmonad
    That would require the ability to move things around - which is not yet supported in Yjs. And it won't be in the near future because of the complexity of this topic. If you want to be able to split paragraphs, you could use y-text and formatting attributes instead.
    Orkhan Alikhanov
    @OrkhanAlikhanov
    That's right, splitting would not suffice, we will need to move them around too.
    We would follow Quill's structure but that unfortunately won't support features that requires nested nodes, like tables
    Kevin Jahns
    @dmonad
    You can represent tables using the delta-format. But it's not as convenient - you are right about that
    Orkhan Alikhanov
    @OrkhanAlikhanov
    Just for information, When we start with <p>Hello World<p>, user A appends !!! and user B wraps the paragraph in <blockquote />, the end result should be <blockquote><p>Hello World!!!<p></blockquote>. However yjs loses appended !!! because wrapping in blockquote replaces the node making information about the insertion position for !!! gets marked as deleted.
    Kevin Jahns
    @dmonad

    Right, that's one of the downsides of the structured representation. From a certain perspective it may make sense. For example, userA transforms a paragraph to a Quote, adds comment markers and the author of the quote. And userB transforms the sentence to indirect speech. The expected result should be just either of the representations (the blockquote in Yjs), but with the possibility to see the differences (which you can do with snapshots).

    It is basically impossible to retain the intention of the user if there are concurrent actions. Yjs could be better at merging concurrent changes. In practice though, it doesn't really make a difference. In any case, concurrent changes that resulted from editing while offline should be reviewed using diffs & snapshots. Concurrent changes from live editing are resolved immediately. The user doesn't really care how these conflicts are resolved.

    Orkhan Alikhanov
    @OrkhanAlikhanov

    live editing are resolved immediately

    That's right, prosemirror author martin had also mentioned that in his blogpost.

    I see that flat representation seems to be more beneficial to get most out of yjs. If paragraph had a boolean attribute that telling this paragraph should be wrapped in blockquote then there is no need for nesting. Same for list items, those can be solved with a simple flag mostly.

    For tables, I have no robust solution in mind yet. Eventually, it would be really great to have a robust document structure representation encompassing most of these use cases like Quill's but a bit extended for tables, list items, blockquotes etc...

    Orkhan Alikhanov
    @OrkhanAlikhanov
    @dmonad What if the paragraph which was enclosed in blockquote by another user had a comment annotation assigned to a range in it?
    That information would be lost in that operation too in the structured document representation, wouldn't it?
    Kevin Jahns
    @dmonad
    Yeah, .. that's a fair point. Although I think that this will happen very rarely. Eventually, there will me move operations in Yjs. But for now we are just going to copy data because it is easier to reason about.

    I see that flat representation seems to be more beneficial to get most out of yjs. If paragraph had a boolean attribute that telling this paragraph should be wrapped in blockquote then there is no need for nesting. Same for list items, those can be solved with a simple flag mostly.

    I guess you could represent the prosemirror binding differently. If xml.nodeName would be an attribute, you could simply change xml.nodeName = 'blockquote'. But there are other examples when this might not be what you are looking for. The only true solution is to add move operations on ranges of types/text. If you assign a comment to a paragraph, and another users moves the same paragraph to a different location, you would expect that the comment will be retained.

    Orkhan Alikhanov
    @OrkhanAlikhanov
    I had read the thread in discuss.yjs about moving ranges and understand that it has challenges. I appreciate your work. Thank you for your reply :)
    Kevin Jahns
    @dmonad
    Sure thing!
    anjalikk14
    @anjalikk14

    @dmonad quick question, whats the right way to check when a YDoc has loaded if there is nothing to sync against? It looks like .observe and provider.on('sync') only fire when there's some data already in the YDoc.

    Context: when a user loads in, I want X to happen if there's nothing in the YDoc, Y to happen if there is something in the YDoc, and I can't consistently get an event to fire when the YDoc has loaded fully.

    I'm sure this has been answered somewhere but couldn't find anything
    (Using y-websocket in case it matters)
    Kevin Jahns
    @dmonad
    synced should be fired even when there is no content in the document. If it is not fired, could you please submit a PR that fixes the issue?
    anjalikk14
    @anjalikk14
    I had a bug on my end, looks like it's being fired correctly. Thanks for the quick response!
    anjalikk14
    @anjalikk14
    Quick follow up @dmonad : is there a 'isSynced' boolean or something equivalent that can be tracked on the provider or the ydoc? I'm building a react app and I suspect there may be cases where I miss the on('sync') event because of some react state management weirdness
    Aayush Bajaj
    @AayushBajaj2000
    @dmonad How can I use y-webRTC to make a p2p video chat im having some difficulties. Thanks
    Kevin Jahns
    @dmonad
    @anjalikk14 There is a provider.synced flag that does exactly that ;)
    @AayushBajaj2000 You could have a look at https://github.com/holtwick/peer2school and at many discussions in the github issues in the y-webrtc repositoryd. If you have any specific question let me know.
    Callum Atwal
    @callum-atwal
    @AayushBajaj2000 take a look at the simple-peer repository on GitHub. Y-webrtc uses this to handke webrtc stuff. It has an example on how to share video and audio feed.
    Harvey38
    @Harvey38
    hey !@dmonad I have just started working with yjs and am confused with how conflict resolution is done and what is the actual overall structure that is created.Can you help with or atleast provide a source from where i can read about this stuff. Thanks
    Kevin Jahns
    @dmonad
    Hi @Harvey38 , Internally, Yjs uses a CRDT (see https://crdt.tech/) to resolve conflicts automatically. The specific crdt algorithm is laid out here: https://github.com/yjs/yjs#yjs-crdt-algorithm Here, you get an overview of the codebase: https://www.youtube.com/watch?v=0l5XgnQ6rB4 . This documentation should cover all resources available about Yjs: https://docs.yjs.dev/api/internals (probably start here)
    Harvey38
    @Harvey38
    thanks a lot..i will go through these
    Damian Kaczmarek
    @Rush
    I am wondering, is it possible to have "dry run" of a state update? I have a client-server infrastructure and would like to verify the client updates before announcing them to the rest of the clients
    I have my own protocol and I am sending the updates manually. I simply encode state, and decode.
    I mean, encode and apply
    Ganesh Ravichandran
    @TheBatmanofButler
    Hi @dmonad! I've just started using yjs with levelDB as the persistence layer, and I'm having trouble figuring out how to peek at what's actually stored in levelDB. I found the Python lib plyvel, but do you know of an easier way to debug/view levelDB data?
    Ganesh Ravichandran
    @TheBatmanofButler
    Some additional context for why I'm asking, every so often my yMaps return as empty and I am unable to figure out why or how to restore the data. I keep garbage collection on for my yDocs so I am wondering if maybe that could be the problem, but it would help to look inside levelDB as well
    Kevin Jahns
    @dmonad
    Hi @Rush That might be a little hard at the moment. Right now you could create a copy of the current Yjs instance, apply the update on the copy, verify, and then apply the update on the original document as well. If you expect that most updates will be propagated, you can destroy the current Yjs document and reload it from persistence when the verification failed. The proper solution would be to implement a revert function based on snapshots.
    @TheBatmanofButler y-leveldb stores incremental document updates. a document update (see docs) is a compressed binary blob that stores the operation. There have been plans to implement a JSON decoder for update messages, but this is far down my priority list.
    Kevin Jahns
    @dmonad

    Some additional context for why I'm asking, every so often my yMaps return as empty and I am unable to figure out why or how to restore the data. I keep garbage collection on for my yDocs so I am wondering if maybe that could be the problem, but it would help to look inside levelDB as well

    It is unlikely that y-leveldb misbehaves. You would probably notice that you can't apply further updates (because of update dependency) when y-leveldb doesn't store a document update.

    I suggest that you try to reproduce the issue. You could implement a simple sync approach like this to reproduce the behavior. Then post it to the issues and I'll debug it.