Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Apr 12 21:02
    aleksk commented #3
  • Mar 31 13:02
    dmonad unassigned #3
  • Mar 31 13:02
    dmonad assigned #3
  • Mar 31 12:29
    yarkoe labeled #3
  • Mar 31 12:29
    yarkoe assigned #3
  • Mar 31 12:29
    yarkoe opened #3
  • Mar 04 07:55
    alex521 commented #1
  • Mar 04 02:34
    alex521 commented #1
  • Mar 03 17:19
    aleksk commented #1
  • Mar 03 17:19

    aleksk on main

    Update package.json Explicitly… (compare)

  • Mar 03 11:49
    dmonad commented #1
  • Mar 02 05:57
    alex521 commented #1
  • Feb 22 20:24
    aleksk commented #1
  • Feb 22 20:24
    aleksk commented #1
  • Feb 22 20:24
    aleksk commented #1
  • Feb 22 18:47
    aleksk commented #1
  • Feb 21 05:46
    alex521 commented #1
  • Feb 21 05:43
    alex521 commented #1
  • Feb 18 21:01
    aleksk commented #2
  • Feb 18 21:01
    aleksk edited #2
Aleksey Kabanov
@aleksk
feel free to try that react-monaco + yjs + ycs sample in the repo, you need to open solution in VS2019 and run it as 'YcsSample' (not the default 'IIS Express')
image.png
only issue in the protocol that I found was with the clientId generation, that is random.uint32, while it's saved as varint32. It's read as int32 on c# side and if the original value was >=int.maxint, the value on the server-side would be negative and would cause uintoptrledecoder to try and read past the buffer as it expects the counter there
Kevin Jahns
@dmonad

Regarding client-ids: I plan to make 53bit unsigned integers a standard for user ids and eventually I even want to support 64bit integers (BigInts in JavaScript; this will be an optional feature). So I think it makes sense to use a 64bit uint in vsc as well. I named encoding.readVarUint specifically without a bit-range - theoretically it should support at least 53bits in JavaScript (this is how many integers a float64 can represent).

I have been pretty unclear on this in the documentation, so I say this now. There is a real chance when using 32 bit uints that two clients will get the same user-id (1 in 4 billion). The chance is much higher when using only 31 bits / int32 for representing client-ids. I implemented additional mechanisms that allow clients to detect that other users have the same client-id. But still, we should avoid this if possible.

Thanks for clearing up the reasons against Fluid and why you decided to publish Ycs as open source. Yeah that completely makes sense.

One of the motivations of the Rust port is to compile it to webassembly for increased performance. The current wasm bundle (very preliminary) uses a tenth of the memory resources and is much faster than the javascript implementation because it doesn't need to do automatic garbage collection. Still, .. the javascript implementation is very fast. Now with the Ycs port I'm evaluating how much the Rust port is still worth to me.

FYI This chat here is public.

I'm going to check out react-monaco + ycs tomorrow. I haven't found the time yet to set this up (I don't have VS2019 yet)
Aleksey Kabanov
@aleksk
I'm going to remove the unrelated topics from nov16 to keep the history clear. Edit: done.
If your concern is garbage collection, then .net implementation might not be the answer, but it could help with other ports. Go/Swift/Java should be pretty straightforward to do. It might even be worth trying to auto-generate ports to at least Java. With Roslyn compiler and new codegen features it might even make sense to build something custom for it too.
Aleksey Kabanov
@aleksk
there is still remaining work in ycs to reduce amount of collections created and use streams + offsets instead of byte arrays for serialization/deserialization for improved perf
you could also use GUIDs as clientIds, comparison of 128bits is optimized, and they will virtually never collide
Aleksey Kabanov
@aleksk
it also should be possible to compile and run that sample from *nix, but I haven't tried. In theory, all dotnet commands should work there
Aleksey Kabanov
@aleksk
'dotnet build' in the repo root, and 'dotnet run' in the samples/YcsReactMonacoSample did the trick for me. Just navigate to the https://localhost:5001/
you may also need to run the 'dotnet dev-certs https --trust' to add the local cert to trust store
Kevin Jahns
@dmonad

I see, you really ported everything :)

Other CRDT frameworks use guids and Yjs v10 did as welll. But guids are awkward to represent in JavaScript. It is very common to have millions of references to client-ids in a Yjs document. Representing them as strings, or worse ArrayBuffers, will create too many objects on the heap. The garbage collector won't be able to keep up. In many regards JavaScript is its own worst enemy. 53 bits (potentially 64 bits) integers add more than enough entropy. Furthermore, they can be directly stored in the hidden classes.

Generating ports to other languages based on Ycs is a really intriguing idea. Part of the reason for the Rust port is to improve performance further. Did you do some benchmarking on Ycs?

Aleksey Kabanov
@aleksk
don't take the code generation part as a serious suggestion, it could be a fun thing to do over the weekend to create a gen-1 version of the port, that code will have to be optimized per-language anyway down the road, but it's fun regardless! Don't want to distract you from the other stuff. Our usecase of the Yjs is very tolerable to the performance, so I did only initial acceptance. I'm going to do some benchmarks this weekend after I reduce the memory copies during serialization/deserialization. Don't expect a good numbers now, though, this stuff will definitely get improved.
Kevin Jahns
@dmonad
Great thanks :thumbsup:
Aleksey Kabanov
@aleksk
Hi, want to double-check on the nuget and copyright stuff. Copyright - I can replace the original copyright header in files with the one of your preference that includes MIT license and mentions author names (myself and Mike in this case). Nuget - I can sign it with either my personal cert or MSFT 3rd party redist cert, or you can be an owner of the nuget package, up to you. I'm yet to work on the perf optimizations for it, but what I currently have performs 2-3 times better than js implementation for our scenarios.
Kevin Jahns
@dmonad

Hey @aleksk , thanks for checking in. That sounds good!

For the nuget package: You should sign it. I don't want to complicate the publishing process. I read in the documentation that, in the case that you are unreachable, I can recover ownership: https://docs.microsoft.com/en-us/nuget/nuget-org/publish-a-package#managing-package-owners-on-nugetorg

If possible, maybe we can later add me as an owner as well. Just in case. (https://docs.microsoft.com/en-us/nuget/nuget-org/publish-a-package#managing-package-owners-on-nugetorg)