Text wrapping support

[planger]

We'd like to have support for text wrapping within shapes (e.g. for comments in a diagram or long labels, etc.) and I think it could make sense to add support for text wrapping to Sprotty. But before we start working on this, I wanted to get your feedback on whether that's something we'd like to have in Sprotty at all and get your opinion which approach may be the best to implement that.

The reason this is not straightforward is because SVG itself doesn't really support text wrapping (maybe SVG 2.0 will eventually provide textArea at some point but this doesn't help us now). So to address this in Sprotty, we may need a bit more than. After a first round of research and I think we have the following options. Please let me know what you think and if you would agree to have this in Sprotty after all.

1. Use foreignObject to embed HTML in the SVG. This would be quite straightforward and would have the benefit that we could directly support also HTML-based "rich-text" without any additional work. The drawback is that IE11 doesn't support foreignObject (Chromium and Firefox do though). Also, there may be some glitches when it comes to selection, focus and click behavior. I'm not sure how browsers would behave when forwarding the mouse events from the foreign object to the underlying SVG, but I guess that's something that can be addressed.
2. Use a third-party library, such as d3plus-text. Essentially they dynamically split the text into separate SVG text and textSpan elements and compute their positions according to a text flow with a given width. However, this would pull in a dependency to Sprotty and require some dedicated implementations in Sprotty to take the wrapping into account for (micro-)layouting.
3. Implement something similar to (2) in Sprotty directly with a dedicated SModel label type (splitting text into textSpans), dedicated Layouter (to compute the height of an element which is affected by the text and its wrapping according to some specific layout options) and a IVNodeDecorator which positions elements according to the text flow. The layouter knows the size of the wrappable words based on the hidden SModel and somehow has to preserve this information so that the IVNodeDecorator can then compute the wrapping and apply the positioning. The reason I'd like to put the computation and positioning into the node decorator is to enable live wrapping while users e.g. change the shape size. Only doing that in the layouter, we'll just be able to rewrap after a model update. This approach would be Sprotty- and SVG-native, but would also be more work and more code that we'd have to maintain for very basic functionality (word-wrapping).
4. Use UIExtension to overlay HTML with text wrapping on top of the diagram. However, this may cause some focus issues, probably worse than approach (1), with the benefit of IE support, but the drawback that we'd need an additional workaround for the SVG export.

I'm leaning towards (1) or (3), but maybe I missed something. It'd thus be great if you could share your thoughts on this and let me know if you'd like to have something like that in Sprotty at all.

Thanks a lot in advance!

[tomvdbussche]

I currently solve this by splitting the text into multiple labels, and then have ELK take care of layouting. So that sounds similar to option (3) and probably the safest way to go about it. Though I really like the sound of option (1).

[planger]

Re approach (3):
After giving this a bit more thought, I think the most versatile approach would be to implement a FlowLayouter (alongside vbox and hbox) suitable to process any BoundsAware and split the text to be word-wrapped into separate labels already in the model source (or maybe the in the SModelFactory to keep the smodelschema more concise). This way, the flow layout can just be applied to the labels, each representing a single word.

The label edit feature would certainly have to be adapted and needs to be made aware that it isn't dealing with single labels but with wrapped text instead -- but I think we can manage that with a dedicated feature to be applied to the compartment containing the single labels. The rest, however, could be handled with standard Sprotty means and a very generic additional flow layouter, I guess. Having to create single SLabelElements for each word doesn't seem very elegant, but it would be a very unintrusive approach.

Let me know what you think! I may do a quick proof-of-concept of this approach in the next couple of days.

[planger]

@tomvdbussche Oh, interesting... just hit "Comment" and saw your comment after that. Thanks a lot for the feedback.

Approach 1 is for sure the most straightforward one. I'm just not sure how important the IE11 support is. I guess it isn't on the long run given that Edge uses Chromium, but it may impact some use cases where you currently use Sprotty e.g. in Eclipse SWTBrowser on Windows (where IE11 is certainly still used).

[planger]

@tomvdbussche Your approach is indeed similar to (3). The main difference is that I'd like to do that in the "micro-layouting" on the client-side with a dedicated layouter (similar to the existing hbox and vbox layouter).

[planger]

I just gave approach 1 a test run and with some namespace hacking of snabbdom it works fine in Chromium-based browsers and Firefox:

IE11 obviously will ignore the foreign objects. Whiel SVG export works fine in general, it will not be supported by plain SVG viewers that don't understand HTML too, obviously.

Since I already have it locally, I'll probably clean it up in the next days and provide a ForeignObjectNodeView that will virtualize a string attribute of the SModelElement into VNodes, fix their namespaces into a namespace given in the SModelElement too and then put it inside of a foreignObject element. With that it should be rather straightforward to use it for whatever foreignObjects you'd like to put into a Sprotty diagram.

[tomvdbussche]

@planger I was also thinking about how approach (1) would behave in other SVG viewers. Regardless it still seems like an interesting view to have available.

I'm still wondering how you would decide when to wrap the text in approach (3). Because widths of nodes for example are usually calculated after calculating the bounds of it's children.

[planger]

I'm still wondering how you would decide when to wrap the text in approach (3). Because widths of nodes for example are usually calculated after calculating the bounds of it's children.

Yeah, they'll have to have at least a fixed width. And therefore we'd need a dedicated layouter always returning the fixed width and -- if resizeContainer === ture -- computing the required height based on the wrapped text.

[tomvdbussche]

I just did some testing with foreignObject on Linux and it seems to break most image viewers (including Inkspace) making the exported SVG practically unusable. So I feel that realistically we would need an alternative to approach (1).

[planger]

I just did some testing with foreignObject on Linux and it seems to break most image viewers (including Inkspace) making the exported SVG practically unusable. So I feel that realistically we would need an alternative to approach (1).

Well, for me it doesn't really break the SVG export, but it would just ignore the foreignObject. So the box with the text would just be missing in the example that I've posted above -- the rest works just as usual. I tested with the native SVG viewer in Linux, Inkscape and Gimp import on Linux. Is it the same for you?

[tomvdbussche]

Well, for me it doesn't really break the SVG export, but it would just ignore the foreignObject. So the box with the text would just be missing in the example that I've posted above -- the rest works just as usual. I tested with the native SVG viewer in Linux, Inkscape and Gimp import on Linux. Is it the same for you?

Okay the SVG's don't really break, but if all the text if missing from the exported image to me that's enough to consider them broken. But yes the viewers will just ignore the foreignObject sections, the rest still looks fine.

[planger]

Okay the SVG's don't really break, but if all the text if missing from the exported image to me that's enough to consider them broken. But yes the viewers will just ignore the foreignObject sections, the rest still looks fine.

Yes, I agree, for use cases where we need a complete SVG export that works in plain SVG viewers it is not a suitable solution. However, at least we have use cases, where the text wrapping "just" needs to work in browsers and SVG export, if at all required, is sufficient if it can be viewed in a browser and if needed be translated by the browser into a PNG (e.g. with https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toDataURL). For those use cases the foreignObject support is sufficient. I've opened a PR for that, since imho the generic foreignObject support is useful anyway and it addresses our immediate requirement.

Nevertheless, approach 3 would be great to have too (flow layout and some automatic SLabel splitting). I'll try to work on it in near future. Or is that something you @tomvdbussche would like to take over?

Thanks for your feedback!

[tomvdbussche]

@planger I havn't looked into layouting that much so maybe I should do that first. Regarding the PNG exporting I have been experimenting with that and I'm probably going to create a PR for it some time soon.