add recommendation for client side filters

[tomkralidis]

Note: sections marked with * are required.

Overview of proposal*

A key concept of a WCMP2 record is "actionable links"; this means being able to access a dataset or data granule without any further interactions. For real-time data, a WCMP2 record provides linkages to the WIS2 Global Broker via the MQTT protocol. At its core, MQTT has two key components:

• topic: the topic to subscribe to
• message payload: the message provided as part of a notification to a given topic

WIS2 defines the WIS2 Topic Hierarchy (WTH) and WIS2 Notification Message (WNM) standards which provide a standards-based GeoJSON payload/message.

A typical MQTT link in a WCMP2 document is defined as follows:

{
  "rel"  : "items",
  "type" : "application/geo+json",
  "title": "WIS2 notification service",
  "href" : "mqtts://example.org",
  "channel": "cache/a/wis2/ca-eccc-msc/data/core/weather/surface-based-observations/synop"
}

Given WCMP2, WTH and WNM, a user can subscribe to topics related to data of interest for download and access.

In some cases, a dataset may be organized in a manner which requires additional further "filtering" such that a data consumer is only interested in a certain subset of the data granules being advertised by a given WNM.

Amendment details*

Given WCMP2, WTH and WNM, a user can subscribe to topics related to data of interest for download and access.

In some cases, a dataset may be organized in a manner which requires additional further "filtering" such that a data consumer is only interested in a certain subset of the data granules being advertised by a given WNM. Some examples include (but are not limited to), where a data consumer may be only be interested in:

• surface weather observations from a certain station, or
• numerical weather prediction forecast data for a certain timestep or weather parameter

To implement this behaviour, add additional properties to both WCMP2 and WNM as follows:

{
  "rel"  : "items",
  "type" : "application/geo+json",
  "title": "Real-time notifications",
  "href" : "mqtts://globalbroker.meteo.fr:8883",
  "channel": "cache/a/wis2/ca-eccc-msc/data/core/weather/surface-based-observations/synop",
  "properties": {
    "wigos_station_identifier": {
       "type": "string",
       "title": "WIGOS station identifier"
    }
  }
}

Surface weather observations: WNM additional properties

{
  "properties": {
    "wigos_station_identifier": "0-20000-0-71628"
}

When implemented by a data producer, a data consumer can:

• subscribe to real-time notifications to the given topic
• perform client side filtering by against all incoming WNMs with properties.wigos_station_identifier = "0-20000-0-71628"

Requestor(s)*

WMO ET-W2IT

Consultations

Discussion in https://wmo-teams.atlassian.net/wiki/spaces/WIS2/pages/1025507329/2025-01-27-ET-W2IT+Monday+meeting (4c).

Comments

This proposal has been documented in the WIS2 Cookbook for the time being, and is being proposed here to support interoperability for those wishing to implement this optional feature consistently.

[tomkralidis]

TT-WISMD 2025-02-18:

• @wmo-im/tt-wismd to review/provide feedback
• @amilan17 to invite TT-NWPMD for feedback
• add/update example in WNM additional properties
• update WCMP2/WNM examples (which render into the Manual)

[josusky]

I agree to the overal concept, clarifying which (additional) properties can be used as filters is useful. Now let's get rid of a trifle:

• The proposal has an editorial issue. Chapter "Overview of proposal" ends with text "may be only be interested in:" and then the chapter "Amendment details" follows which is confusing.

More important is to be able to define the filtering properties in a machine readable way. A "string" property is OK for WSI and the users know that they can apky a regex on it. The cookbook also shows example of model_run which is also a string/enum (so the user knows there are two possible values), and forecast_hour again as a string (e.g. 004). That is, er, suboptimal. The forecast_hour should be in the worst case treated as an enum (with about 120 possible values) or better as a "duration" with some range deifinition. A possible value in the WNM would then P4H (https://json-schema.org/understanding-json-schema/reference/type).

[sebvi]

This looks promising.

I have few questions:

• as part of a domain expert team, we may want to define a set of controlled properties and we want to associate to these properties some code lists of allowed values, is that possible? if yes, could you give an example of how to do it? give a href?
  example:
  property is "nwp_parameters" and the possible values are 'u', 'v', 't', 'r', etc. and these allowed values could be published on codes.wmo.int (see next question as well for an example)

• is it allowed to have nested properties?
  example to specify a list of parameters on pressure levels (the values are related to the next question):

"properties ": {
    "type_of_level" : {
        "type" : "string",
        "title" : "type of level",
        "href" : "codes.wmo.int/type_of_level",
        "value" : "pressure",
        "parameters" : {
             "type" : "array",
             "title" : "list of parameters",
             "href" : "codes.wmo.int/parameters",
             "values" : [ "u", "v", "t" ]
         },
        "pressure_levels" : {
            "type" : "array",
            "title" : "pressure levels",
             "values" : [ 700, 800, 850, 1000 ]
         }
    }
}

• as a data producer I may want to publish in the discovery metadata the list of possible value a property can take because I am declaring the full dataset in one discovery record but publish the data using more than one notification message

[tomkralidis]

I agree to the overal concept, clarifying which (additional) properties can be used as filters is useful. Now let's get rid of a trifle:

* The proposal has an editorial issue. Chapter "Overview of proposal" ends with text "may be only be interested in:" and then the chapter "Amendment details" follows which is confusing.

Good catch -- fixed.

More important is to be able to define the filtering properties in a machine readable way. A "string" property is OK for WSI and the users know that they can apky a regex on it. The cookbook also shows example of model_run which is also a string/enum (so the user knows there are two possible values), and forecast_hour again as a string (e.g. 004). That is, er, suboptimal. The forecast_hour should be in the worst case treated as an enum (with about 120 possible values) or better as a "duration" with some range deifinition. A possible value in the WNM would then P4H (https://json-schema.org/understanding-json-schema/reference/type).

Any example of what this would look like, using JSON Schema?

[tomkralidis]

This looks promising.

I have few questions:

* as part of a domain expert team, we may want to define a set of controlled properties and we want to associate to these properties some code lists of allowed values, is that possible? if yes, could you give an example of how to do it? give a href?
  example:
  property is "nwp_parameters" and the possible values are 'u', 'v', 't', 'r', etc. and these allowed values could be published on codes.wmo.int (see next question as well for an example)

You could reference codelists on codes.wmo.int, but we are trying for a pure JSON schema implementation of this addition. We could use JSON schema referencing for an enumeration on codes.wmo.int, but its representation (RDF, JSON-LD, etc.) will be incompatible for a JSON schema parser, say. Of course we can depart from JSON Schema, which is fine, but that would introduce more custom implementation and tooling.

* is it allowed to have nested properties?
  example to specify a list of parameters on pressure levels (the values are related to the next question):

Yes, this is possible given JSON Schema.