Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Kevin Jahns
    @dmonad

    @yuanwei92 Usually, you don't want to do that. You should know beforehand which types you want to add to the Undo Manager. Then you can do new Y.UndoManager([type1, type2, ..])

    If you want to add all currently known types to an UndoManager, you could iterate over ydoc.share which is a Map of all known root-level types.

    Tei Yuan Wei
    @yuanwei92
    @dmonad Thank you for responding! Is there a way/helper function to determine what sharedType from the value in Map? or I would determine from the shape?
    Kevin Jahns
    @dmonad
    That is why you should know beforehand what types you expect in ydoc.share. By default, all types are simply Y.AbstractType until you define the shape using something like ydoc.getText('key'). There is never a reason to randomly create root-level types because they can never be garbage collected. If you need to create many types with different types, put them inside a Y.Map.
    You can do constructor checks or instanceof checks otherwise
    Tei Yuan Wei
    @yuanwei92
    Alright understood. Thanks again!
    trafnar
    @trafnar:matrix.org
    [m]
    I'm starting to build a project using YJS, it is an offline-first app that will sync to a server I control. I want to make sure I do the server storage part right. The official database adapters don't seem to cover any popular/typical databases like firebase/postgres etc, is that because leveldb has adapters for common databases? Should I prefer using leveldb instead of implementing my own connection to something like firebase?
    I'm also wondering if my server should be able to run its own code in order to calculate state vectors and exchange diffs with my clients? That is opposed to being "dumb" storage which can only dump the entire history to clients.
    trafnar
    @trafnar:matrix.org
    [m]
    Basically, I wish this page in the docs existed :) https://docs.yjs.dev/tutorials/persisting-the-document-to-a-central-database
    Jed Grant
    @jedgrant
    I'm using yjs successfully with TipTap (react), and saving a base64 encorded string to a database which I can successfully load back into the editor. However, now I'm trying to turn that base64 into an HTML string for use outside of the editor in the simplest way possible so I can compare one saved version to another using react-diff-viewer. I assuming there's something with y-prosemirror that I need to do to get the content, but I'm not sure where to start. Can anyone point me in the right direction?
    2 replies
    trafnar
    @trafnar:matrix.org
    [m]
    so you want the raw update from yjs as a string? if you have a reference to the Y.Doc, you can use y.on('update', (update) => console.log(update))
    however that update value is a binary array, but you can convert it to a string
    using import { fromUint8Array, toUint8Array } from "js-base64"
    although i could be totally misunderstanding what you're trying to do
    Jed Grant
    @jedgrant
    I put a reply in the previous message you responded to. I think I'm fundamentally missing the scope of what yjs provides. Your question made me think about where I'm getting the html content in the first place, and now that I think about it, it's probably always through tiptap/prosemirror providing a bit of a translation layer on the data structure prosemirror uses.
    trafnar
    @trafnar:matrix.org
    [m]
    if you describe your use case a bit, I may have a better answer
    1 reply
    trafnar
    @trafnar:matrix.org
    [m]
    If I'm storing a ydoc as binary, then merging incoming updates into that, but one of the updates is corrupt/garbage/malicious is it possible I mess up my whole stored ydoc?
    I'm worried that if I always merge the incoming data and overwrite the stored document data, I could get some broken update coming in that destroys the whole stored doc.
    Jimmie Goode
    @jimgoo
    I was curious was the backend for wss://demos.yjs.dev looks like. Is it just a single y-websocket server with LevelDB persistence like from here https://github.com/yjs/y-websocket? It seems very reliable even with tons of demo code pointing to it. I was planning to go the y-websocket + levelDB mounted to k8s persistent volume route for my drawing app but wasn't sure how well that would scale up whenever there's more than one such pod. I'm sure the demo wss takes way more abuse that mine ever will, which is why I ask here.
    Nishant Mittal
    @nishantwrp

    Hi, I'm running a y-webrtc signalling server that is included in the package. (https://github.com/yjs/y-webrtc). But two different machines are unable to establish webrtc connection b/w each other even when they're connected to the same signalling server. When I open two tabs in either of the machines it works i.e. two tabs in the same machine are running as expected.

    Am I doing something wrong?

    2 replies
    alibama
    @alibama
    hey ya'll = running tljh with github auth = is this a fairly standard use case?
    YarnBall
    @yarnball

    Hi folks, love this look of this project but need some help understanding some basics. I want to setup a 'middleware' (backend/sever) to verify each yjs change event (update) before writing to leveldb (or whatever gets used for sync).

    I have looked at y-socket but can't find a way to do that.

    Am I missing something here? Is it even possible?

    Kevin Jahns
    @dmonad

    @jimgoo The websocket server is located here and has been running for two years ^^ https://github.com/yjs/yjs-demos/tree/main/demo-server

    There's no backup. From time to time it runs out of memory and restarts - discarding all rooms that are not used anymore. It has overall ~500mb memory.

    And yeah, a crazy amount of people use the demo server. Today it restarted every second. I guess somebody is bombarding it with requests again. Seems to work out though because CRDTs are pretty resilient ^^

    @alibama I guess this is the wrong discussion board for this
    @yarnball What does it mean "to verify" to you? You can check whether an update is parseable by decoding it. You should not filter updates depending on its content.
    Ben Hohner
    @benhohner
    Hey there, is there any way to persist UndoManager state with the doc when persisting it, so the user can pick up where they left off in a new session with preserved editing history?
    YarnBall
    @yarnball

    @dmonad Thanks. By verify, I mean as in checking the JSON schema of the 'data' is still valid. Also, ideally checking at the time of the request so I can "filter" the data before sending it.

    Can you explain how I'd decode the data?

    I've tried several methods from the docs but keep getting {} eg: https://github.com/yjs/y-websocket/blob/master/bin/utils.js#L176

    My store is simple just:
    { todos: [], fragment: "xml" } (is xml the right thing here?)
    Inside todo is an array of: { completed: bool, title: string }

    Kevin Jahns
    @dmonad
    @benhohner You could disable garbage-collection (ydoc.gc = false) and store & restore the state stored in undomanager.undoStack / undomanager.redoStack after every session. There are probably quite a lot of pitfalls, but it is theoretically possible.
    Kevin Jahns
    @dmonad

    @yarnball You should not filter updates depending on its content. That is an antipattern and prevents buggy clients from ever syncing again. You can undo changes on the backend if a client performs changes that you didn't expect. You can use the UndoManager for this.

    You can't decode Yjs updates and read the changes as, for example, a diff. The updates contain CRDT operations that cannot be understood by mere humans (really scliency math stuff).

    If you want to read a Yjs document, use the appropriate methods. It's always: ydoc.getText('name').toString(). The ydoc.toJSON() is deprecated and should not be used. The server must know what it reads.

    YarnBall
    @yarnball
    UndoManager looks great.
    What is the suggested pattern on returning filtered content (ie only a subset of a Yjs “room”)? Eg my data is stored inside a Y.array()
    Jamalah
    @jaibeee

    Hey there. I'm using webrtc with the default setup and I'm receiving this error:

    WebSocket connection to 'wss://y-webrtc-signaling-us.herokuapp.com/' failed: 
    setupWS @ websocket.js:25
    WebsocketClient @ websocket.js:122
    SignalingConn @ y-webrtc.js:473
    (anonymous) @ y-webrtc.js:610
    setIfUndefined @ map.js:49
    (anonymous) @ y-webrtc.js:610
    connect @ y-webrtc.js:609
    WebrtcProvider @ y-webrtc.js:595
    loadDoc @ space.js?t=1664184681825:125

    Is there anything I'm doing wrong or is the signaling service down?

    Jamalah
    @jaibeee
    I'm also seeing the same error in the console on the yjs website https://yjs.dev/. The main demo subdomain is giving a heroku error https://demos.yjs.dev/.
    image.png
    Kevin Jahns
    @dmonad
    Hi @jaibeee That's definitely an issue with heroku. It's been getting very unreliable . One reason is that there is an insane amount of requests any the free instance can't handle it anymore ^^. I'm already looking to migrate to a different platform.
    1 reply
    @yarnball subdocuments
    YarnBall
    @yarnball

    Thanks, saw your comment @dmonad on y-websocket.

    However, the on('synced', ... does not 'fire'. You mention in your example it is 'hard to determine'. So how would I "return" the subdocument on my client?

    My example: https://dpaste.org/dPPbt

    YarnBall
    @yarnball

    Also, if my subdocuments are objects, is it best to JSON.stringify() them or use y.Set()?

    EG:

    const rootDoc = new Y.Doc()
    const folder = rootDoc.getMap()
    
    const subDoc = new Y.Doc()
    subDoc.insert(0,  new Y.Map().set({id:123, text:'foo'}))
    folder.set('123', subDoc)
    Kevin Jahns
    @dmonad

    If you need the granularity, use a Y.Map. If not, prefer a simple JSON object (FYI Yjs accepts all kinds of JSON objects, so no need for JSON.stringify).

    I'm not sure what the code on dpaste is supposed to do. The synced event only works if you use a provider. Please read the official documentation: https://docs.yjs.dev/api/subdocuments

    YarnBall
    @yarnball

    @dmonad Thanks. I've been through the docs but it still not quite clear

    1. For the relationship between a "subDocs" & "folder"- is it possible to get "subDocs" from multiple folders?

    2. You say "Yjs accepts all kinds of JSON objects" but how in my example would I do that? My attempt is below, which fails saying subDoc.set is not a function. I tried using const subDoc = new Y.Doc(); subDoc.getText().insert(0, {foo:123}) but that also returns blank unless I "JSON.stringify" it

        const folder = rootDoc.getMap()
        const subDoc = new Y.Doc()
        subDoc.getMap()
        subDoc.set(data.id, data)
        folder.set(data.id, subDoc)
    YarnBall
    @yarnball
    Here is a code example to make it more tangible https://playcode.io/971379
    Kevin Jahns
    @dmonad
    YText only accepts string. The reason should be quite obvious. Please start with a simple example and the build up something more complex. I'm really not sure what you want to achieve - the code doesn't make it clear to me.
    In Y.Map or YArray you can insert JSON objects: yarray.insert(0, [{x:1}])
    This is explained with code examples here: https://docs.yjs.dev/getting-started/working-with-shared-types
    There is more doc for each specific type
    Jed Grant
    @jedgrant
    I'm developing a react firebase app locally using the firebase emulator. I'm using Yjs with Tiptap. Everything has been working great until a few minutes ago and now the console is spamming this error every couple of seconds:
    WebSocket connection to 'wss://signaling.yjs.dev/' failed: Error during WebSocket handshake: Unexpected response code: 503
    Any suggestions?
    Ben Hohner
    @benhohner
    @dmonad I think I've figured out what's going on with the high traffic, the y-webrtc-signaling-us.herokuapp.com server is failing because it seems that the y-webrtc client auto retries the server every second even if it can connect to other servers, effectively ddosing the server whenever it is down or client-server round trip latency goes above 1000ms.
    Jonathan Clem
    @jclem
    Is there documentation on what the expectations are for server-side implementations of y-websocket? It looks like the demo server in the repo is all the documentation there really is (which is fine, it's a big project). Just trying to make sure I'm not missing anything, as I don't want to use a standalone server and want to integrate with one I already have.
    I'm also interested to know what the bare minimum requirements would be for implementing a y-websocket server in a language like Go—for example, I don't need server-side storage and just want a centralized server for authenticating user access to collab sessions and proxying operations, but does building something like that still require being able to use yrs bindings?
    Jonathan Clem
    @jclem
    Or, putting the question another way—I already have an extremely large-scale client-to-client message exchange implementation built on top of Kafka and WebSockets. Is it possible just to shuttle Yjs messages over a system like this, or is it a requirement to actually keep and store documents in-memory somewhere on the server?
    Jonathan Clem
    @jclem
    Has anyone implemented comments in CodeMirror 6 with Y.js? It's not clear exactly what the right way is to sync the comment positions. Right now, I'm syncing them separately in a Y.Map as JSON-encoded relative positions, but if I end up accidentally applying the comment decorations before the CM6 view has apparently been updated with the document content.