Triplit / TipTap Demo?

[MentalGear]

Thank you very much for the triplit demo!

A tiptap & triplit demo would be a great addition in my opinion.

PS: it'd be great if you could also provide a comparison how your CRDT method differs from that of other libs, like Y.js.

[mweidner037]

ProseMirror/TipTap is a tough nut to crack. The existing websocket-prosemirror demo works by using a restricted schema (a linear sequence of blocks) and is still kind of flaky.

Essentially, the problem is that ProseMirror uses a tree data structure, while list-positions only directly supports a flat linear data structure. You can map a tree onto a flat data structure by encoding the start & end of each node as a special symbol (as ProseMirror does internally). However, then merging collaborative edits can lead to a state that doesn't correspond to a valid tree - e.g., <p><blockquote>Some content</p></blockquote>.

My current plan to work around this while supporting arbitrary schemas is:

• Keep a log of ProseMirror steps (atomic changes) annotated with list-positions metadata.
• Use a server to determine an (eventual) total order of steps.
• The current state is given by applying the steps in order, skipping steps that would result in an invalid state (according to ProseMirror).
  • Here the list-positions metadata is used by clients when applying steps, so that they still make sense in the current state. (E.g., "insert 'x' at index 3" -> "insert 'x' at index 4", if a concurrent step inserted a character at index 0.)

Is this architecture doable with Triplit? I guess you could store the log of steps and query it in some eventually consistent order. This will have a large overhead relative to storing the ProseMirror state directly, though, and it doesn't let you "inspect" the state in a Triplit-native way. (It is okay to replace a log of steps by the resulting ProseMirror state - forgetting the steps themselves - once you are sure that no future steps will be inserted in the middle of the log.)

PS: it'd be great if you could also provide a comparison how your CRDT method differs from that of other libs, like Y.js.

The Yjs-ProseMirror binding uses Yjs's XML types. My understanding (from the source code) is that these represent the XML tree as a literal tree of nodes, where each node has either a list of child nodes or some flat text content.

y-prosemirror itself then ignores ProseMirror's built-in steps/transactions. Instead:

• When the local ProseMirror state changes, y-prosemirror computes a diff between its own state and the ProseMirror state for each node, to figure out what operations to apply to the Yjs state.
• When the Yjs state changes, y-prosemirror computes the corresponding ProseMirror state and overwrites the editor's current state.

I suppose you could do a similar strategy with Triplit/list-positions: Store the tree in Triplit in a natural way, just using list-positions to decide the order of children/characters within each node. The main challenge is then figuring out how to map this state to ProseMirror and back, perhaps using y-prosemirror as a guide.

I had not considered a literal-tree approach like this before, because of a semantic quirk: If Alice types in a paragraph, while concurrently, Bob merges that paragraph with the previous one, then often Alice's edits will be lost. (Because: Bob's "merge" is actually "cut+paste 2nd paragraph's current content at the end of the first paragraph, then delete the 2nd paragraph".) Likewise for paragraph splits.

However, this disadvantage may be outweighed by the advantage of representing the state in a Triplit-native way - something that I had not considered until writing this response.

[MentalGear]

Thank you for the detailed response, very interesting indeed and happy if it got you thinking in novel ways as well.

I'll just add this triplit issue aspen-cloud/triplit#95, as they also seems to be working on Collab Text and it might be interesting to exchange.

[matlin]

Not as up to date on on the various editors but we've typically found state diffing to be the most straightforward path when integrating with some external system (ProseMirror, Monaco, Excalidraw, etc). The caveat is that there is often a performance penalty when you bulldoze the state with the new state and to remedy it, you skip applying the new state back to the editor and instead just routinely check that the editor state aligns with state in your data store. I think Yjs even does this with one of their bindings.

But happy to help model anything in Triplit if either approach makes more sense than the other. We are actually adding metadata that can provide total ordering but it's only used internally for now so I would have to think through the best way to expose that.

[mweidner037]

Here is a demo of the log-based approach I mentioned: https://github.com/mweidner037/list-demos/tree/master/websocket-prosemirror-log#readme . I'll keep thinking about an alternative literal-tree approach.

The caveat is that there is often a performance penalty when you bulldoze the state with the new state

With ProseMirror, this can also confuse some plugins/features that expect to see ProseMirror transforms, instead of a complete state overwrite. E.g., y-prosemirror needs to implement its own cursor tracking and undo/redo, because it breaks ProseMirror's built-ins; and it also triggers some issues with decorations and custom plugins (yjs/y-prosemirror#113). On the other hand, it seems to work well enough for TipTap.