Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    calibr
    @calibr
    @dmonad what are you plans on updating UndoManager for the latest yjs version? I tried to implement a custom undo manager myself, but gave up because it seems core methods like isParentOf, AbstractItem.redo need to be modified as well and probably you already have plans on them.
    Kevin Jahns
    @dmonad
    I want to rework the undo manager. I will probably take a similar approach to snapshotting, but on a per-user setting. Both are not working at the moment. I put it on my bucket list for next month #99.
    crazypenguinguy
    @crazypenguinguy
    image.png
    crazypenguinguy
    @crazypenguinguy
    @dmonad It looks like there may still be an issue with Webpack on the most recent versions (yjs@13.0.0-87, y-websocket@1.0.0-5) . I'm trying to use React with your example code from README.v13.md.
    import React, {Component} from 'react';
    import * as Y from 'yjs'
    import { WebsocketProvider } from 'y-websocket'
    const doc = Y.Doc() 
    const provider = new WebsocketProvider('http://localhost:1234', 'roomname')
    provider.sync('doc')
    const ytext = doc.getText('my resume')
    class App extends Component {
      render() {
            return (<div>test</div>);
      }
    }
    export default App;
    Kevin Jahns
    @dmonad
    Could you please publish a demo application that I can debug? This could be related to your bundler configuration. The monaco example (build with webpack) still works https://github.com/y-js/yjs-demos/blob/master/monaco/index.js
    crazypenguinguy
    @crazypenguinguy
    Kevin Jahns
    @dmonad

    Thanks @crazypenguinguy I made some mistakes i the documentation. For example the url must be a ws:// or wss:// url. I corrected it. Let me know if something else is unclear.

    Here is the fixed code for your demo:

    import React, { Component } from "react";
    import ReactDOM from "react-dom";
    import * as Y from "yjs";
    import { WebsocketProvider } from "y-websocket";
    
    const doc = new Y.Doc();
    //const doc = new Y.Doc()
    const provider = new WebsocketProvider("ws://localhost:1234", "roomname", doc);
    const ytext = doc.getText("my resume");
    
    class App extends Component {
      render() {
        return <div>test</div>;
      }
    }
    export default App;
    
    const rootElement = document.getElementById("root");
    ReactDOM.render(<App />, rootElement);
    crazypenguinguy
    @crazypenguinguy
    That worked perfectly! Thanks for the help @dmonad!
    Duane Johnson
    @canadaduane
    Is there any documentation comparing yjs and automerge/hypermerge? I'd love to be able to more intelligently compare the systems. They seem very similar on the surface. I'm seeing a (possibly?) more advanced set of demonstration apps on the hypermerge side (e.g. trellis, pixelpusher, pushpin, farm) but more speed and compact size achievements on the yjs side. Is that fair? Any other points of interest?
    Duane Johnson
    @canadaduane
    I'd also appreciate any pointers anyone has to p2p Electron apps implemented with yjs under the hood.
    Duane Johnson
    @canadaduane
    Last but not least... are there any querying engines built in or otherwise connected to yjs? For example, if I need a read-only queryable index of documents, what is the go-to solution at this time?
    Kevin Jahns
    @dmonad

    Good questions.

    Comparing Yjs and Automerge:
    • Both projects provide easy access to manipulating the shared data. Automerge's design philosophy is that state is immutable. In Yjs, state is mutable and observable.
    • Automerge has more demo apps where state is shared to build some kind of application. But in theory, you could implement shared text editing with Automerge.
    • Yjs is more focused on shared editing (on text, rich text, and structured content). It has lots of demos for different editors (quill, ProseMirror, Ace, CodeMirror, Monaco, ..).
    • I spent a lot of time to optimize Yjs to work on large documents. While Automerge works fine on small documents, it has serious performance problems as the document grows in size. https://github.com/dmonad/crdt-benchmarks
    • Yes, Automerge has support for hypermerge/DAT. I am also looking into it, as it seems like a really cool idea. I'm currently exploring multifeed[https://github.com/kappa-db/multifeed) for that. On the other hand Yjs has support for IPFS.

    Yjs & Electron:
    No problem here in general. You'll need to polyfill WebSocket / WebRTC support in electron. There is ws for WebSocket support and node-webrtc for WebRTC.

    Query Index:
    There is nothing like that to my knowledge. But maybe you could have a look at GUN

    calibr
    @calibr
    @canadaduane if I understand the question about indexing documents correctly you still
    need some engine that will store that index. For example, in our project to provide full-text search feature, we extract all the texts from a document once it gets updated and put them into the full-text engine.
    Duane Johnson
    @canadaduane
    Awesome info, thank you, both. @calibr how write-intensive is the update to the full-text engine? Does it do streaming updates (i.e. each operation such as increment, insert, etc. gets its own corresponding operation) or does it write the whole document at once (i.e. clobbering the previous state in the full-text engine)?
    @dmonad Do you have any more detail on "In Yjs, state is mutable and observable"? I'm trying to fit that into my current understanding of CRDTs which is that operation logs are always immutable.
    Beautiful project, by the way. I am very impressed by how much work has gone into Yjs over the years. This is no small feat.
    BTW the Hypermerge folks found through testing that webrtc is not optimal for p2p electron apps. They ended up using utp (from bittorrent) to improve connection between peers.
    calibr
    @calibr
    @canadaduane we use Sphinx for full-text search and it expects submitting the whole document at once when the document changes, but as far as I know every full-text engine works like that. It is possible to throttle updates of the full-text index. A disadvantage of this is that the index can be stale for some time, but this approach can be totally appropriate for some projects.
    Kevin Jahns
    @dmonad

    @calibr I have no experience with document indexing. Thanks for sharing your experience.

    @canadaduane Thanks for your appreciation :) I was referring to the frontend of the shared editing framework. Yjs exposes mutable types (e.g. Y.Array). Automerge exposes immutable json-like objects.

    In Yjs, the operation log is not immutable. I.e. it may decrease in size when you delete content. I describe some optimizations I do in the v13 log, but let me know if you want to know more.

    About P2P electron apps: DAT is a very ambitious project that wants to share many, large files in a distributed network. Compared to WebRTC, UDP connections are initialized much faster and are better suited for their use-case (e.g. walking through peers of the DHT). However, If you only want to share a single document, WebRTC will work just fine and are also supported in the browser.

    Kevin Jahns
    @dmonad

    @/all

    The Quill Editor binding was just added for v13 including shared cursors and many additional consistency tests.

    Demo: https://yjs-demos.now.sh/quill/
    y-quill: http://github.com/y-js/y-quill

    Duane Johnson
    @canadaduane
    Nice!
    Quill is a nice editor, it's great to see it connected & working!
    Duane Johnson
    @canadaduane
    Possibly interesting database to consider for yjs: https://github.com/sirixdb/sirix
    "Sirix facilitates effective and efficient storing and querying of your temporal data through snapshotting (only ever appends changed database pages) and a novel versioning approach called sliding snapshot, which versions at the node level. Currently we support the storage and querying of XML- and JSON-documents in our binary encoding."
    Kevin Jahns
    @dmonad
    Thanks for the hint Duane. Internally Yjs just appends all document updates together and then stores them to a database. So an append-only log database might be interesting for Yjs Your proposed DB expects XML or JSON documents. Yjs already encodes updates in a binary format.
    Duane Johnson
    @canadaduane
    Ah, I see.
    Dmitry Kostenkov
    @dmikoss
    Hi. Thanks for y-js! Example editors (quill, prosemirror) use shared (for all remotes) history for undo and redo. How i can use local redo\undo operations?
    Kevin Jahns
    @dmonad
    Welcome Dmitry. The local undo/redo history does not work at the moment. I will work on it this week and let you know when I'm done
    Séverine Darlot
    @severinedarlot
    Hello, we would like to try Yjs with ace. But when I follow the tutorial for Yjs 13 on https://github.com/y-js/yjs/blob/master/README.v13.md, the y-textarea repository is empty. Is it too soon to use Yjs 13? Or the documentation is out-of-date?
    What is your point of view? Thank you.
    Kevin Jahns
    @dmonad
    Hi @severinedarlot
    I encourage you to use v13 (although it is still in beta). I need to work on the documentation - but the API documented in README.v13.md is stable and working.
    In the tutorial section I wanted to give an example of how to use Yjs. The y-textarea binding does not exist yet - currently we only have Monaco, QuillJS, and ProseMirror support. Next week I will add the missing bindings (the ones I already advertise in the README. I just haven't ported them to the new API).
    Séverine Darlot
    @severinedarlot
    @dmonad Thank you for these advice and these information
    Kevin Jahns
    @dmonad
    @dmikoss UndoManager just landed in Yjs https://github.com/y-js/yjs/blob/master/README.v13.md#YUndoManager
    I also added an undo plugin to y-prosemirror. https://github.com/y-js/yjs-demos/blob/master/prosemirror/index.js
    For the quill demo, you can actually just use their history plugin. For Quill, it really doesn't make a difference if you use Quills undo manager or Yjs undo manager. To enable selective undo in Quill, you basically just have to configure the undo plugin with userOnly: true (see https://github.com/y-js/yjs-demos/blob/master/quill/index.js).
    Dmitry Kostenkov
    @dmikoss
    @dmonad Thank you for your work. Can these demos work in edge and ie11? i got errors. See these issue: y-js/yjs-demos#1
    Kevin Jahns
    @dmonad
    They can - but at the moment I don't convert the source to es5 which is why it doesn't work.
    Dmitry Kostenkov
    @dmikoss
    @dmonad what about Edge? It support es6 as i know
    Kevin Jahns
    @dmonad
    I just debugged it with a friends computer - the problem with edge is that the url is created incorrectly. It tries to connect to wss:yjs-demos.now.sh instead of wss://yjs-demos.now.sh. Apparantly window.location works differently than in chrome and firefox.
    Thanks for noticing! I'd be happy to receive a PR for this. Otherwise I'm gonna fix it tomorrow. I'm off to my day-job.
    Dmitry Kostenkov
    @dmikoss
    @dmonad Hi Kevin. Find small bug in prosemirror demo (cursor and selection pops). https://youtu.be/OF_4_zSkuoI
    Kevin Jahns
    @dmonad
    Thanks @dmikoss You can just report it to https://github.com/y-js/y-prosemirror
    Kevin Forest
    @kevmegforest_gitlab
    @dmonad . Hi Kevin, is the persistence db server working with https://github.com/y-js/y-websocket? The https://github.com/y-js/y-leveldb doesn't seem to have been updated for yjs v13?
    Kevin Forest
    @kevmegforest_gitlab
    const LevelDbPersistence = require('y-leveldb').LevelDbPersistence in y-websocket does not exist. Is there a reference implementation for that persistence function?
    rafeautie
    @rafeautie
    Hi all, question about the 'y-websocket' package. Is this package still available through npm? I am trying to follow the tutorial but I am running into issues installing 'y-websocket'.
    Kevin Jahns
    @dmonad
    @kevmegforest_gitlab You can enable persistence PORT=1234 YPERSISTENCE=./dbDir node ./node_modules/y-websocket/bin/server.js (see y-websocket docs). y-leveldb only works with v12
    `
    @rafeautie Yes, it is available via npm. Can you give me a an error message.
    rafeautie
    @rafeautie
    username$ npm i y-websocket
    npm WARN y-websockets-client@8.0.16 requires a peer of yjs@^11.0.0 || ^12.0.0 but none is installed. You must install peer dependencies yourself.
    npm WARN client@1.0.0 No description
    npm WARN client@1.0.0 No repository field.
    
    npm ERR! path /Users/username/Desktop/Projects/YJS/react-yjs-test/node_modules/y-websocket/bin/server.js
    npm ERR! code ENOENT
    npm ERR! errno -2
    npm ERR! syscall chmod
    npm ERR! enoent ENOENT: no such file or directory, chmod '/Users/username/Desktop/Projects/YJS/react-yjs-test/node_modules/y-websocket/bin/server.js'
    npm ERR! enoent This is related to npm not being able to find a file.
    npm ERR! enoent
    Tim
    @timweiland
    @dmonad Hey Kevin, thanks for the great work on Yjs, it's a really cool project. I've got a question: What's the recommended way to achieve persistent storage with Yjs 13? I'm trying to add persistent storage to y-prosemirror, and y-leveldb is not working (getting the same error as @kevmegforest_gitlab ).
    Kevin Forest
    @kevmegforest_gitlab
    @timweiland I created my own persistence with a y-leveldb.js file. You can take a look there. https://pastebin.com/5SuefhUC . I don't know if it's the right solution and if I will be able to update the y-websocket version with new yjs version once it's updated. (is the encodeStateAsUpdate method final? @dmonad )
    calibr
    @calibr
    I slightly modified the old version of level db persistence lib in order to get it work with new yjs. Probably it should perform better than the lib proposed by @kevmegforest_gitlab because it doesn't write the whole document on every update, but only the update itself. Here you can get it https://gist.github.com/calibr/c8764d796db270c7c60460f254451580
    crazypenguinguy
    @crazypenguinguy
    @rafeautie it's trying to grab the package y-websockets-client which is different. Try the command npm i y-websocket@next