Support V2 update

[jstoup111]

https://github.com/yjs/yjs#using-v2-update-format

It'd be great to support the v2 format for better compression. Since https://github.com/y-crdt/yrb-actioncable is it's own provider, we could enhance to use v2 by default, or add a configuration to allow users to choose which version they'd like to use.

I'm happy to help with the Ruby side, but I won't be able to do much on the Rust side of things.

[eliias]

The biggest caveat is that we don't use binary encoded messages in the first place. ActionCable does not allow us to send binary messages directly, so they need to be Base64 encoded. v2 isn't the recommended default by now.

[jstoup111]

I'm still learning about yjs among other things, so forgive me if I'm asking silly questions.

Since the encoder/decoder is here, theoretically, we could change to whatever encoder you want to send data over the wire. Right now it's JSON.
https://github.com/y-crdt/yrb-actioncable/blob/main/gems/yrb-actioncable/lib/y/actioncable/sync.rb#L132

The bigger issue appears to be in the javascript where the JS library only supports sending JSON.
https://github.com/rails/rails/blob/main/actioncable/app/javascript/action_cable/subscription.js#L83
https://github.com/rails/rails/blob/main/actioncable/app/javascript/action_cable/connection.js#L24

Am I on the right track here encoding/decoding? If so does it seem worthwhile to handle transport this way vs using base64? If so I can poke around and see if I can get something to work.

[eliias]

Since the encoder/decoder is here, theoretically, we could change to whatever encoder you want to send data over the wire. Right now it's JSON.

Yeah, when I first looked at this, I thought exactly the same and wanted to use binary encoded data. I only learned later that the ActionCable message encoder is a somewhat different abstraction on top of the actual protocol. The only job of the encoder is to handle Ruby object encoding. Raw ActionCable messages always requires a JSON envelope for the message to be transmitted. You can put whatever you want into that JSON message, even when it's not JSON. In other words, you need to Base64 the binary and stuff it into the envelope as a message. You can't really change the underlying protocol without changing ActionCable itself (on the server as well as on the client-side).

In general, you are right. Both, the v1 and v2 binary encoding of yjs/yrs, would benefit from using actual binary transport (https://developer.mozilla.org/en-US/docs/Web/API/WebSocket/binaryType) at the WebSocket level. The y-websocket Node.js implementation has a nice example of how this can be achieved (https://github.com/yjs/y-websocket/blob/master/bin/utils.js#L238), but for Rails this requires a rewrite of ActionCable or at least some heavy modification. As much as I agree with you here, I did decide to not go down the route of trying to contribute this to Rails, due to many unknowns:

• Does the Rails community really want this? ActionCable is fairly simple and easy to understand. This would add lots of complexity.
• Eventually WebSocket will be replaced with WebTransport. This seems to be the right time to rethink the Channel/Cable concept IMO.

I believe, the bigger problem is getting towards a reliable transport mechanism. ActionCable w/ Redis is by design not suitable for the job. e.g., Redis PubSub is for “live” clients only, but we eventually want to sync to all clients at-least-once. You can find preliminary work on reliable transport is in this PR/branch y-crdt/yrb-actioncable#17.

[eliias]

btw… I can easily support v2 in yrb if you really need it. I can just add v2 companion methods for diff, sync, and state. It is relatively straightforward. But, what I recall from one of the y-crdt meetings, Kevin and Bartosz (Y.js/Yrs maintainers) are not sure about the future of v2. There might as well be a v3, or v4, before we settle on a new default.

[jstoup111]

Sorry for the delay in responding. Thanks for doing this for me, much appreciated!