Multilingual and internationalisation

[Ontopic]

Since it seems most of the issues I'm running into are related to multilingual and localisation, I thought it might be best to start one open discussion for all issues related to this. You yankees don't get what we have to deal with over here in Europe ;) Still, one of the reasons I'm betting on Payload for the long-term is for its already great localisation support and all the building blocks are there.

If you guys don't agree with an open forum like approach, please - knowing I'd fully understand - remove this discussion so I can open separate issues that are "real" issues. Not yet sure what the goal of discussions is on GitHub, but this seems to be one 🙃

Otherwise I'll elaborate on the issues listed below more;

1. Localised slugs and uniqueness
2. A relationship from a non-localised field, that is localised through other methods from higher up - like adding a language field - and needs to have a relationship to a localised field. Right now a relationship contains the collection and id, but not locale
3. Ability to clone a document, but ignore validation rules. Allowing f.e. to create a translated version of a post where not all required fields are ready, waiting on translation. Preferably without creating a drafts collection.

I'll continue to detail these issues below, directions I tried and what I felt worked best if this is the way to discuss this. Anyone that can burn everything - or just a little thing here or there - I did to the ground; please do.

Questions on how I did things are also more than welcome. Open forum code-review ftw.

[jmikrut]

You'd be surprised! The yankees at TRBL are actually currently localizing like 3 separate large-scale sites that were all built with headless WP before Payload was released. It's thrilling, let me tell you. One of the reasons we decided to build Payload was because of the industry's lack of support for proper localization, and even though our features can still be expanded, I think we have the requisite groundwork and infrastructure to do exactly that: expand those features.

That said, this discussion is great. I have some ideas bubbling already, but let me see if I can break down solutions / ask questions regarding each of your points:

1. Localized slugs and uniqueness
   Fields already have the ability to be set to unique, and that works on the locale level. So, you would be prevented against having two equal German slugs, but those German slugs would not affect any existing English slug values. If I am understanding your question correctly, that means that you could have a Pages collection with a slug field featuring the unique and localized properties set to true. See the demo All Fields collection for an example. Is this what you mean? Note - small UI bug where the slug field is not properly shown in an error state if you try and set a value that already exists. We are working on that but you can check the Network panel while you try and create a duplicate slug and see that the field indeed errors due to enforcing unique slugs per locale.

2. Localized relationships from non-localized fields

I'm not sure if this is possible in your case, but you could just request the top-level document in a specified locale, and the document returned from the nonLocalizedRelationship will be automatically returned in that same requested locale.

For example, if you are requesting all pages with language set to de, you know that your locale needs to be also set to de so you may as well just request the page with a locale set:

/api/pages?where[language][equals]=de&locale=de

Payload passes on top-level locales through all APIs so in most cases you wouldn't need to do anything there. I might be missing your goal here entirely though.

Alternatively, if for some reason you need to know the document's language and can't simply request with a specified locale, you could do something like the below field hook:

const pages = {
  slug: 'pages',
  fields: [
    {
      name: 'language',
      type: 'select',
      options: [
        {
          label: 'English',
          value: 'en',
        },
        {
          label: 'German',
          value: 'de',
        },
      ],
    },
    {
      name: 'nonLocalizedRelationship',
      type: 'relationship',
      relationTo: 'some-other-collection',
      hooks: {
        afterRead: [
          async ({ req, value, data }) => {

            // Manually "populate" the related document in the document's set language
            return req.payload.findByID({
              id: value, // the ID of the related document that is saved to db
              collection: 'some-other-collection',
              locale: data.language // the language from full document being read
            });
          }
        ]
      }
    },
  ],
}

I have not tested the above but the idea in principle should work and is basically just you being responsible for returning that relationship data however you need, based on the language set on the doc itself.

Note - this approach will lose out on a bit of Payload's internal performance optimizations we've built into the auto-population depth parameter. With depth, we memoize all relationship populations on a request basis... so even if you were retrieving 100 pages, you'll only actually load relationships one time each per request. Here, that field hook would run for every doc you retrieve, even if it's the same related doc. But that's getting into the weeds a bit.

3. Cloning and ignoring validation rules
   We are actively working on building out a draft / published system, where you can maintain versions of documents as a draft that are not yet published. This will mean that there is no need for a separate drafts collection, and it will also mean that you could then easily duplicate a page / post with incomplete fields, save it as a draft, and work on it over time. Would this solve your need?

Phew. Got carried away with that second example. Curious to hear if that is what you meant / if the solution I offered is relevant. Big chance I totally missed what you were going for. Let's keep this going!

[Ontopic]

Wow, thanks! I'm gonna carefully read all of that a few times :)

[Ontopic]

 const langField = {
    name: 'lang',
    label: 'Language',
    type: 'text',
    admin: {
      readOnly: true,
      position: 'sidebar',
      components: {
        Field: PlainValue,
      },
    },
    hooks: {
      beforeValidate: [
        ({
          value,
          operation,
          originalDoc,
          data,
          req: {
            locale,
          },
        }) => {
          const lang = typeof value === 'string'
            ? value
            : data?.lang || originalDoc?.lang || locale

          return lang
        }
      ],
    },
  }

I use this field to automatically set a lang field if I'm not using the built-in localisation. But as you mentioned, I'll be missing out on some features this way.

What would be the best way to show in the List overview page just the pages that have their lang field matching the current locale from payload? Seems to be possible, but feels like it'd then have to take out the entire Filter module

[jmikrut]

Hmmm... good question!

We could probably open up some type of collection-level prop that allows you to place a query constraint on the List view's fetch, where you could be provided with certain data like current page, locale, active filters, etc. and then return your own filters.

Maybe a potential pattern:

{
  // Collection config goes here....
  slug: 'pages',
  admin: {
    List: {
      // New property - takes current filters, locale, page
      modifyFilters: ({ filters, locale, page }) => {
        return {
          ...filters,
          language: {
            equals: locale
          }
        }
      }
    }
  }
}

Would that work?