Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    questsin
    @questsin
    for v13
    Kevin Jahns
    @dmonad

    Right, I haven't published the textarea package. v13 is still lacking good demos. You can try out the https://github.com/y-js/yjs-demos and the prosemirror demo.

    No, v13 is not backward compatible.

    I haven't tested v13 with webpack. I can't figure out like this why it doesn't work.

    questsin
    @questsin
    Ok thanks. I’ll fiddle with both. Is it still possible to build custom providers?
    Kevin Jahns
    @dmonad
    It most definitely is. The
    The provider idea is a concept to manage document updates. It is not a class you need to implement. The v13 docs above explain how document updates work
    laurieherault
    @laurieherault
    I confirm the problem with webpack. https://codesandbox.io/s/r1mqzy4oo (Thank you for this amazing library :) )
    Kevin Jahns
    @dmonad
    Thanks @laurieherault for the link and the appreciation :)
    That helped me to figure out the issue. I'll fix it in the next release
    I'll try to release tonight
    laurieherault
    @laurieherault
    Thanks !
    laurieherault
    @laurieherault
    Hello :) Any news about a correction ?
    Kevin Jahns
    @dmonad

    Thanks to @NathanaelA pledge on Patreon we just received a wonderful addition:

    :sparkles: Monaco support (the editor that powers VS Code) :sparkles:

    Live Demo: https://yjs-demos.now.sh/monaco/
    Demo Code: https://github.com/y-js/yjs-demos/tree/master/monaco
    Binding repository: https://github.com/y-js/y-monaco

    Kevin Jahns
    @dmonad
    @laurieherault The issue was fixed in lib0@0.0.5 and Yjs@13.0.0-83 (yes I'm reposting this because I accidentally deleted the message). You need to update all y-* packages to get rid of lib0@0.0.4 (which was causing the issue).
    laurieherault
    @laurieherault
    It's work, tank you :)
    calibr
    @calibr
    @dmonad conguratulations with the great refactoring! I haven't got a chance to take a closer look at the latest version of yjs yet. Has encoded structures format changed since version 13.0.0-78? I mean binary format written by writeSyncStep2 and writeUpdate. Is format that was back in 13.0.0-78 compatible with 13.0.0-84?
    Duane Johnson
    @canadaduane
    This is a very cool project, thanks for all of your work @dmonad
    Does anyone know if it's possible to compose data types? e.g. an array of maps or map of arrays? Or even a map with some values as collaborative text?
    Kevin Jahns
    @dmonad

    @calibr Yes, I broke encoding. These are my final fixes to create a stable API to Yjs https://github.com/y-js/yjs/blob/master/README.v13.md#Document-Updates

    Thanks for your feedback @canadaduane :)
    You can absolutely do that in both v13 and v12. Shared types are just data types and of course you can nest them. But it may take some time to get used to the concept of shared data types. You should just start and observe the changes as you are doing them. Here is a template to create a file system using Y.Map as the directory and Y.Text as files. This will work in Yjs version 13 (in beta):

    const yMap = doc.getMap('my-directory')
    
    const file = new Y.Text()
    yMap.set('index.js', file)
    
    new TextareaBinding(file, document.querySelector('textarea'))
    Duane Johnson
    @canadaduane
    Would this example be for file contents, or file names?
    (Just trying to wrap my head around what a "file system" means in context of CRDTs :)
    Kevin Jahns
    @dmonad
    Fair enough. Bad example. This is definitely not a real file system. But the example creates a Y.Map (similar to Map) that maps from filename to the content of the file (in this case text only).
    I hoped this was the example you were looking for.
    You can add a new file by adding a new entry to the map yMap.set('index.js', file)
    The gist is that all changes to the shared types are automatically distributed to other clients.
    Kevin Jahns
    @dmonad
    I got an announcement for @all:
    I took a month vacation from my daytime job to work on the v13 release. v13 will be released very soon. I'll be hacking and documenting Yjs for the whole month of June. You can contribute by working with me on the website, documentation, testing, additional demos, or Yjs integrations (e.g. to other editors / frameworks). In exchange I'll be available for daily group meetings and answer any questions you have. The background is that I want more people involved in the project. Ping me directly if you are interested.
    Duane Johnson
    @canadaduane
    Awesome. I'm new here, but very interested in getting things working & helping where I can.
    calibr
    @calibr
    @dmonad what are you plans on updating UndoManager for the latest yjs version? I tried to implement a custom undo manager myself, but gave up because it seems core methods like isParentOf, AbstractItem.redo need to be modified as well and probably you already have plans on them.
    Kevin Jahns
    @dmonad
    I want to rework the undo manager. I will probably take a similar approach to snapshotting, but on a per-user setting. Both are not working at the moment. I put it on my bucket list for next month #99.
    crazypenguinguy
    @crazypenguinguy
    image.png
    crazypenguinguy
    @crazypenguinguy
    @dmonad It looks like there may still be an issue with Webpack on the most recent versions (yjs@13.0.0-87, y-websocket@1.0.0-5) . I'm trying to use React with your example code from README.v13.md.
    import React, {Component} from 'react';
    import * as Y from 'yjs'
    import { WebsocketProvider } from 'y-websocket'
    const doc = Y.Doc() 
    const provider = new WebsocketProvider('http://localhost:1234', 'roomname')
    provider.sync('doc')
    const ytext = doc.getText('my resume')
    class App extends Component {
      render() {
            return (<div>test</div>);
      }
    }
    export default App;
    Kevin Jahns
    @dmonad
    Could you please publish a demo application that I can debug? This could be related to your bundler configuration. The monaco example (build with webpack) still works https://github.com/y-js/yjs-demos/blob/master/monaco/index.js
    crazypenguinguy
    @crazypenguinguy
    Kevin Jahns
    @dmonad

    Thanks @crazypenguinguy I made some mistakes i the documentation. For example the url must be a ws:// or wss:// url. I corrected it. Let me know if something else is unclear.

    Here is the fixed code for your demo:

    import React, { Component } from "react";
    import ReactDOM from "react-dom";
    import * as Y from "yjs";
    import { WebsocketProvider } from "y-websocket";
    
    const doc = new Y.Doc();
    //const doc = new Y.Doc()
    const provider = new WebsocketProvider("ws://localhost:1234", "roomname", doc);
    const ytext = doc.getText("my resume");
    
    class App extends Component {
      render() {
        return <div>test</div>;
      }
    }
    export default App;
    
    const rootElement = document.getElementById("root");
    ReactDOM.render(<App />, rootElement);
    crazypenguinguy
    @crazypenguinguy
    That worked perfectly! Thanks for the help @dmonad!
    Duane Johnson
    @canadaduane
    Is there any documentation comparing yjs and automerge/hypermerge? I'd love to be able to more intelligently compare the systems. They seem very similar on the surface. I'm seeing a (possibly?) more advanced set of demonstration apps on the hypermerge side (e.g. trellis, pixelpusher, pushpin, farm) but more speed and compact size achievements on the yjs side. Is that fair? Any other points of interest?
    Duane Johnson
    @canadaduane
    I'd also appreciate any pointers anyone has to p2p Electron apps implemented with yjs under the hood.
    Duane Johnson
    @canadaduane
    Last but not least... are there any querying engines built in or otherwise connected to yjs? For example, if I need a read-only queryable index of documents, what is the go-to solution at this time?
    Kevin Jahns
    @dmonad

    Good questions.

    Comparing Yjs and Automerge:
    • Both projects provide easy access to manipulating the shared data. Automerge's design philosophy is that state is immutable. In Yjs, state is mutable and observable.
    • Automerge has more demo apps where state is shared to build some kind of application. But in theory, you could implement shared text editing with Automerge.
    • Yjs is more focused on shared editing (on text, rich text, and structured content). It has lots of demos for different editors (quill, ProseMirror, Ace, CodeMirror, Monaco, ..).
    • I spent a lot of time to optimize Yjs to work on large documents. While Automerge works fine on small documents, it has serious performance problems as the document grows in size. https://github.com/dmonad/crdt-benchmarks
    • Yes, Automerge has support for hypermerge/DAT. I am also looking into it, as it seems like a really cool idea. I'm currently exploring multifeed[https://github.com/kappa-db/multifeed) for that. On the other hand Yjs has support for IPFS.

    Yjs & Electron:
    No problem here in general. You'll need to polyfill WebSocket / WebRTC support in electron. There is ws for WebSocket support and node-webrtc for WebRTC.

    Query Index:
    There is nothing like that to my knowledge. But maybe you could have a look at GUN

    calibr
    @calibr
    @canadaduane if I understand the question about indexing documents correctly you still
    need some engine that will store that index. For example, in our project to provide full-text search feature, we extract all the texts from a document once it gets updated and put them into the full-text engine.
    Duane Johnson
    @canadaduane
    Awesome info, thank you, both. @calibr how write-intensive is the update to the full-text engine? Does it do streaming updates (i.e. each operation such as increment, insert, etc. gets its own corresponding operation) or does it write the whole document at once (i.e. clobbering the previous state in the full-text engine)?
    @dmonad Do you have any more detail on "In Yjs, state is mutable and observable"? I'm trying to fit that into my current understanding of CRDTs which is that operation logs are always immutable.
    Beautiful project, by the way. I am very impressed by how much work has gone into Yjs over the years. This is no small feat.
    BTW the Hypermerge folks found through testing that webrtc is not optimal for p2p electron apps. They ended up using utp (from bittorrent) to improve connection between peers.
    calibr
    @calibr
    @canadaduane we use Sphinx for full-text search and it expects submitting the whole document at once when the document changes, but as far as I know every full-text engine works like that. It is possible to throttle updates of the full-text index. A disadvantage of this is that the index can be stale for some time, but this approach can be totally appropriate for some projects.
    Kevin Jahns
    @dmonad

    @calibr I have no experience with document indexing. Thanks for sharing your experience.

    @canadaduane Thanks for your appreciation :) I was referring to the frontend of the shared editing framework. Yjs exposes mutable types (e.g. Y.Array). Automerge exposes immutable json-like objects.

    In Yjs, the operation log is not immutable. I.e. it may decrease in size when you delete content. I describe some optimizations I do in the v13 log, but let me know if you want to know more.

    About P2P electron apps: DAT is a very ambitious project that wants to share many, large files in a distributed network. Compared to WebRTC, UDP connections are initialized much faster and are better suited for their use-case (e.g. walking through peers of the DHT). However, If you only want to share a single document, WebRTC will work just fine and are also supported in the browser.

    Kevin Jahns
    @dmonad

    @/all

    The Quill Editor binding was just added for v13 including shared cursors and many additional consistency tests.

    Demo: https://yjs-demos.now.sh/quill/
    y-quill: http://github.com/y-js/y-quill

    Duane Johnson
    @canadaduane
    Nice!
    Quill is a nice editor, it's great to see it connected & working!
    Duane Johnson
    @canadaduane
    Possibly interesting database to consider for yjs: https://github.com/sirixdb/sirix
    "Sirix facilitates effective and efficient storing and querying of your temporal data through snapshotting (only ever appends changed database pages) and a novel versioning approach called sliding snapshot, which versions at the node level. Currently we support the storage and querying of XML- and JSON-documents in our binary encoding."
    Kevin Jahns
    @dmonad
    Thanks for the hint Duane. Internally Yjs just appends all document updates together and then stores them to a database. So an append-only log database might be interesting for Yjs Your proposed DB expects XML or JSON documents. Yjs already encodes updates in a binary format.