Improve API parameter types

[Derugon]

Improve the types of API parameters, by adding types for more modules and parameters, infering stricter types, and fixing some type issues.

To simplify the implementation of most of these changes, the api_params type generator has been rewritten. The changes proposed in this PR are mostly backwards-compatible (using some deprecation annotations), but some type fixes may break existing stuff. All changes are introduced below, potentially breaking changes are annotated with ⚠.

Proposed changes

1. Extract modules from multiple APIs

Use multiple site APIs to generate types, allowing to generate types for extensions or modules that are not available on Wikipedia (including Flow and WikiBase).

This also provides a generic way for the type generator to detect which enum parameters are wiki-dependant (e.g. roles, namespaces, or languages). If incompatible types are detected, their type can be automatically generalized to string, so types are not over-specific. The proposed type generator queries module data on all sites independently, then merges the generated API trees together.

The set of queried APIs can be extended with the SOURCES dict. In this PR it includes most english (and international) MediaWiki sites: MediaWiki, Wikipedia, Wikidata, Wikifunctions, Wikibooks, Wikimedia API Portal, Wikimedia Commons, Meta-Wiki, Wikinews, Wikiquote, Wikisource, Wikiversity, Wikivoyage, Wiktionary, and some test sites.

2. Expose types

Expose all interface and enumeration types within the mw.Api/mw.Api.Params namespaces.

This part follows PR #40, in which API parameter types were left unchanged to prevent changing them twice with this proposal.

3. Change type names

Change the interface naming scheme to be based on the actual API path, and not on PHP class names.

This prevents the interface names from being changed when PHP files are renamed or moved (which will happen with MW 1.43, where various API files have changed name and been moved to the MediaWiki\Api namespace), and fixes some parameter interfaces sharing the same name and being merged together. In practice, the same API module can appear multiple times with different parameter prefixes, depending on the modules it is used with. By using API paths, a single module can generate multiple interfaces, preventing this issue.

API paths are not extension-dependant, which allows to search for specific type interfaces more easily with auto-completion. For example:

• all format=… interfaces are in the mw.Api.Params.Format namespace,
• all action=query&list=… interfaces in the mw.Api.Params.Action.Query.List namespace.

Note that this makes most interfaces have longer names (e.g. ApiQueryAllLinksParams fully qualified becomes mw.Api.Params.Action.Query.List.AllLinks), which may be mitigated by using namespace aliases.

In this PR, current type names are still accepted and deprecated, using a (hardcoded) list of replacements.

4. Generate types for templated parameters ⚠

Various interfaces use templated parameters to let users specify a variable number of structured arguments. These are generally defined by using a slots parameter to specify a set of identifiers, then by using parameters multiple times, suffixed by one of the previously defined identifiers. For example, the action=compare API uses 2 set of templated parameters (from… and to…) to specify additional properties of one or multiples sources and targets.

As a first approach, templated parameters are generated using template literal types in this PR. This only ensures properly-prefixed parameters have the correct types. For example, the following parameters are generated for the action=compare interface:

interface Compare extends Params {
    ...
    fromslots?: string | string[];
    [k: `fromtext-${string}`]: string;
    [k: `fromsection-${string}`]: string;
    ...
    toslots?: string | string[];
    [k: `totext-${string}`]: string;
    [k: `tosection-${string}`]: string;
    ...
}

This could be improved further, as this approach does not ensure parameter name suffixes are synchronized with each other.

5. Make API parameters stricter in types

Make parameters required, when mentioned on the API module declaration.

action, format, and similar parameter types are narrowed in sub-module interfaces. For example, using mw.Api.Params.Action.Block | mw.Api.Params.Action.Unblock only allows action: "block" | "unblock". ⚠

For compatibility, exported (and deprecated) type aliases still have all their parameters optional.

6. Include a basic JSdoc

Generate JSdoc comments along with the module interfaces and parameters, to include information that can not be expressed with TS types.

In this PR, this includes:

• default parameter values,
• sensitive parameters,
• deprecated modules and parameters,
• online documentation links.

This does not include:

• which parameters can be used with GET and which ones with POST,
• required read/write rights,
• deprecated or internal parameter values,
• value bounds, like byte limits for strings or min/max values for numbers.

Other changes

• Remove JSON-format specific parameters from the root Params interface. They may not be used with other formats, so Params.Format.Json can be used instead. If a user wants to typecheck Params.Action.Move with JSON-specific parameters, Params.Action.Move & Params.Format.Json can be used instead.
• Sort interfaces. The proposed system builds a type hierarchy, then folds it in breadth-first order.
• Disable tslint rules on specific lines and not for the entire file, so unintended error cas still be detected.

[AnYiEE]

Some methods of mwn (e.g. parseWikitext) provide a default value for action, but ApiParseParams treats action as a required property, which is an incompatible issue.

E.g. AnYiEE/AwesomeGadgets@edc450c#diff-e5e7386f4d0511dd5de28802496acf9bdf305877daae5388f87443c1f8450779R492

[AnYiEE]

At the same time, the "variant" property is missing. qiuwenbaike/QiuwenGadgets@6bd8000#diff-ad3705a25402295b5651e9eb78f318e7c91737361988b391165eeff64948fb32R12

[AnYiEE]

Some websites may still be using Flow, but all types related to Flow have been removed in this change (including optional values ​​of the *contentmodel property of some interfaces). Is it necessary to keep them?

The three interfaces ReadingListsApi* extend from ReadingListsApiQueryReadingListsParams -> ApiQueryParams, but their list property type is number, which is incompatible with OneOrMore<string>, causing lint errors.

Some properties should not strictly limit optional values (such as useskin) because some websites do not use certain skins.

[Derugon]

Thank you very much @AnYiEE for your feedback and comments, I took my time with this. :)

I've put aside the changes with issues for which I don't have a solution at the moment. I used MWN and QiuwenGadgets for usage reference, tried to fix everything I could find, feedbacks on the design or anything are more than welcome.

The three interfaces ReadingListsApi* extend from ReadingListsApiQueryReadingListsParams -> ApiQueryParams, but their list property type is number, which is incompatible with OneOrMore<string>, causing lint errors.

The issue is that the same interface or different ones can be used in different places with parameters of the same name, but with different prefixes. To avoid collisions, the proposed script duplicates interfaces according to where they are used. I proposed a different interface naming scheme to make it (I hope) more intuitive where to find the types with the right prefixes.

Some websites may still be using Flow, but all types related to Flow have been removed in this change (including optional values ​​of the *contentmodel property of some interfaces). Is it necessary to keep them?

This should be fixed now, Flow is still used on other WMF sites than Wikipedia, so the script retrieves types from different APIs and merges the generated trees.

Some properties should not strictly limit optional values (such as useskin) because some websites do not use certain skins.

This should be fixed too, in the same way as the previous issue: enumerated parameters whose values differ between sites are generalized back to a string, this seems to fix most similar issues.

For skins specifically, I tried to use a wiki with different available skins (so among the most visited MediaWiki sites outside WMF). I included Graces Guide, I also considered a Fandom site, but they seem to have a bunch of undocumented modules (or with broken/missing parameter types) so I didn't include any. The site list can be expanded, so if there are other modules to add, we can simply add a site that uses them.

[AnYiEE]

This is the current patches made by QiuwenGadgets regarding API types, for your reference. It seems we encountered the same issue as #55. I chose to directly introduce ApiParams as a union type of UnknownApiParams to avoid errors.

[Derugon]

directly introduce ApiParams as a union type of UnknownApiParams to avoid errors.

UnknownApiParams is an object with unknown properties, but that only allows value types that the API can handle. The reason ApiParams currently can not be used to narrow UnknownApiParams is because it does allow any value for unspecified keys, while UnknownApiParams only allow non-object and non-function values (i.e. strings, numbers, booleans, or arrays of these).
If we instead decide to allow any value type, using UnknownApiParams = ApiParams | Record<string, ...> would be more or less the same as using UnknownApiParams = Record<string, unknown>.
If we want to ensure non-object and non-function values, then we would have to restrict ApiParams instead (which is the proposed approach here).

[AnYiEE]

A use case. We globally declare some types like ApiQueryUserContribsParams to constrain query parameters with more specific types like this. At this point, if it is not handled as I mentioned, then ApiQueryUserContribsParams will not even be accepted by the get() method, it only accepts UnknownApiParams, but the two types do not completely overlap (or can be narrowed down).

[Derugon]

I apologize for the late reply, and I assume also for my poor communication skills.
To rephrase the previous message more clearly, I agree that ApiParams cannot be narrowed down to UnknownApiParams (and that is an issue). What I'm questioning is which approach to use, since we've both proposed a different one:

• making ApiParams more specific, i.e. only allowing non-object and non-function values for string keys as with UnknownApiParams, e.g.

  interface ApiParams extends UnknownApiParams {
    ...
  }
  // or more explicitely
  interface ApiParams {
    ...
    [k: string]: boolean | number | string | ...; // and other accepted value types
  }

• making UnknownApiParams less specific, i.e. allowing any value for string keys as with ApiParams, e.g.

  type UnknownApiParams = {};
  // or
  type ApiParams = Record<string, unknown>;