Groundwork for cross-platform i18n with libintl for libghostty/macOS

[mitchellh]

This builds on @pluiedev's excellent #6004.

Background: The macOS (and libghostty consumer) Plan

Broadly, the decision I've come to is that for cross-platform translations (i.e. strings shared across libghostty), we will be using gettext and libghostty will export helper methods to call those (e.g. ghostty_translate in this PR for singular forms). To be clear, this only applies to strings owned by libghostty. For application-level strings such as macOS-specific menu items and so on, we still have choice but will likely using native features.

The reason for this is because converting gettext translations (po) to native formats (Xcode String Catalog, .strings/.stringsdict) is nightmare level, in particular for plural forms. I don't see a robust path to doing it. And if we don't convert and don't use gettext, then translators would have to maintain an identical translation in multiple locations. To make matters worse, the macOS translation formats require Apple-tooling for now unless you want to edit raw JSON.

Leveraging gettext lets us share translations across platforms and take advantage of proven tech.

PR Contents

pkg/libintl builds and statically links libintl for macOS. macOS doesn't ship libintl with the system while Linux generally does with libc, so we need to build this ourselves. This makes gettext available to macOS. libintl is LGPL and we remain in compliance with that despite static linking because our build process is fully open source, so downstream consumers can modify our build scripts to replace it if they wanted to.

src/os/locale.zig now sets the LANGUAGE environment variable on macOS based on the app's preferred languages. macOS lets you configure the system locale separate from preferred language. We previously relied solely on NSLocale.currentLocale, but this only represents the system locale. We now also look at NSLocale.preferredLanguages (a list in priority order) and if we support a given language we set LANGUAGE so gettext translates properly. Notably, the above lets us debug translations in Xcode by setting alternate languages for debug builds only. Removed this for a future PR since it was problematic.

build.zig unconditionally builds binary mo files since they're required for all apprts now.

The macOS app bundles the translation strings. This includes our GTK-specific translation strings but the size of these is so small it isn't worth the complexity of splitting up into multiple pots at this time, I think.

i18n APIs moved to src/os from src/apprt/gtk. Since these are now cross-platform/cross-apprt, they're a core API. The only notable change here is that _ now maps to dgettext and explicitly specifies our domain so that it's library-friendly. The GTK apprt calls initGlobalDomain so that blueprint translations still work.

Next Steps

This PR is all groundwork. The macOS app doesn't leverage any of this yet, although I've verified it all works (e.g. calling the ghostty_translate API from Swift).

For next steps, we need to have a use case for cross-platform translations and the first one I was looking at was configuration error messages and other core strings.

[pluiedev]

I need to ask my friends in Hong Kong and Taiwan and ask them if they like to see Mainland Chinese when we don't have translations for them :p

Jokes aside I wonder if this approach is too naïve. gettext actually comes with specific instructions about dealing with the myriad varieties of Chinese:

Special advice for Chinese users: Users who want to see translations with Simplified Chinese characters should set LANGUAGE to zh_CN, whereas users who want to see translations with Traditional Chinese characters should set LANGUAGE to zh_TW. Chinese users in Singapore will want to set it to zh_SG:zh_CN, Chinese users in Hong Kong will want to set it to zh_HK:zh_TW, and Chinese users in Macao will want to set it to zh_MO:zh_TW. Here zh_CN or zh_TW, respectively, acts as fallback, since only few packages have translations for zh_SG, zh_HK, or zh_MO.

Maybe we should implement this specific fallback logic as well?

[mitchellh]

Maybe we should implement this specific fallback logic as well?

I was trying to avoid this, but this is helpful and makes a lot of sense. I think we'll have to implement just a fully custom function we maintain then. A raw mapping has too many dimensions since we DO have access to their system locale region.

[mitchellh]

I'm going to add that I think implementing the full logic of that function I'll defer to a future PR/improvement, but I think properly mapping at least simplified vs traditional Chinese is something this PR can do. In the future, I think it'll be up to translators to help inform us of additional logic that may be required to fulfill their translations.

[pluiedev]

Yeah. Update: I just asked some of them, and they have very strong opinions against Simplified/Mainland Chinese lol

[mitchellh]

I want to add an additional note since it isn't clear: our LANGUAGE-setting behavior only happens for macOS app bundle launches, not CLI launches. For CLI invocations we expect the user to set the proper env vars. Additionally, it is specific to macOS and not libghostty in general.

I'm noting this because we can tailor this function specifically to known Apple language codes and formats rather than trying to make something super generic (for the generic case, we rely on well-formed environments or users of the library to handle it).

[mitchellh]

@pluiedev To simplify the PR for now, I've removed all the preferred languages handling. I think I can follow that up in a separate PR and I think do it all in the macOS app rather than libghostty, I think that clarifies the responsibilities better.

[pluiedev]

[mitchellh]

Thanks ❤️ Going to merge this to avoid future package conflicts. But still need to lay some groundwork to make this actually useful longer term...