[FEEDBACK] Interoperability concerns and normative-optional features

[sffc]

(split from #977)

I really want to avoid a situation where someone writes a message using some optional extensions, and the message works in ICU4C, and then when they go use an implementation that doesn't implement the same optional extensions (like ICU4X), their message doesn't work. Avoiding this kind of situation is, in my mind, the only job of a specification.

It would make me slightly happier if implementations with optional extensions required the client code to "opt in" to using them. For example, something like

let mf = new Intl.MessageFormat("de", { extensions: ["calendarOption"] })

And then messages with a :datetime calendar="..." option will work only if the stated extension is enabled. If someone tries building an ICU4X formatter with that extension, we'll know ahead of time in the constructor that the extension is not available, instead of waiting to see the message payloads.

[eemeli]

I don't really see what the user benefit might be of needing to explicitly opt-in to extensions that are supported by the formatter. Isn't that information equally well conveyed in the message source?

In your code example above, your constructor call is missing the second required argument, i.e. the message source. Recall that that's provided initially to the constructor, and so its information (e.g. whether it includes a :datetime with a calendar option) is already known at that time.

[sffc]

Opting-in the extensions makes it clearer what messages your formatter is capable of processing. Enabling extensions: ["calendarOption"] does not mean that the messages being processed must utilize that extension. But what it does mean is that if you migrate between implementations, the implementations need to support the same set of extensions.

[aphillips]

I don't get it?

I don't see a lot of benefit to making developers write into their code what optional-to-implement functionality is "enabled". Bear in mind that we would like to see user-defined and implementation-defined functions (ICU has a whole host of formatters/selectors that should be enabled in messages, for example, using the icu: namespace). Having to list these every time I use MF2 would be super inconvenient. (The inverse: the ability to query if functionality is installed might be an interesting implementation feature)

While total message portability would be great, we envision wide adoption by many languages, some of which will be seriously constrained. The REQUIRED default functions are necessarily minimal, as a result. MF2 would be kind of a failure if we were limited to those functions, since the ability to do inflection, person name formatting, duration handling, list management, and so forth might never appear in the (insert environment name here, e.g. awk or Lua or PHP) version, but be seriously useful... and since the default function set already is available via MF1.

Yes, this means that there will be "dialects" of MF2. But they should all work with translation tooling in a consistent way. Developing a machine-readable function description was a deliverable that we dropped along the way, but which could be resurrected in the post-47 world.

Full message portability isn't that common anyway. Your JS code doesn't typically use the same resource files as your Android app. The syntactical and conceptual consistency between frameworks/platforms is an important feature and the ability to port messages that use the most common features is important (it's the reason we have strong guidance to do these functions in a specific way). But there is some flexibility inherent in the design and that flexibility is a feature.

[sffc]

I wouldn't find it as elegant, but an alternative that aligns with my problem statement would be to basically have a MessageFormat Validator, like the W3C HTML Validator, that checks your syntax and returns which version / optional extensions are being used. There can be an ICU4X Profile that fails validation of messages that use optional extensions not implemented in ICU4X.

Then if clients want to use ICU4X, they can validate that their messages are conformant ahead of time.

[eemeli]

@sffc Do you mean including such a validator somehow in the spec, or just having it exist in general, possibly even outside Unicode?

I'm in the middle of refactoring Mozilla's Pontoon translation platform to internally store all source and target messages using an MF2-ish representation, and using that representation in the API between its server and client. As MF2 is more powerful than most of our supported formats, that'll require serverside validation that each new translation isn't using features that its source format doesn't support.

That sounds an awful lot like the validator you're talking about? The Python implementation work for that will be in moz.l10n, so it won't be limited to just Pontoon's use cases.

[sffc]

Validator doesn't need to be in the spec, but it is important for it to exist and be utilized.

Another thought: the message bundle format could have a header listing which optional features are required to consume the message bundle. Then the validator just needs to make sure the messages are consistent with the header, and ICU/ICU4X can check the header before parsing messages.

[sffc]

I guess the main theme through these various solutions is that there should ideally be a naming scheme for optional features. We can't talk about optional features unless they have names, we can't list them in a message bundle header, we can't list them in an API, and we can't configure them in a validator. That naming scheme should, I think, live in MF2.

Many of my concerns would be alleviated if, as an implementer, I could see a single centralized list of "these are the optional features, this is what they are called, and this is where you can read more about them."

[macchiati]

We could use one of the 25 reserved single-letter namespaces for that, without inventing any new syntax.

Say o:calendar, for example?

[macchiati]

And an optional value could also have an o:... prefix