Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions guide/sections/part2/global-services.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,8 @@ In WIS2, Global Caches provide access to WMO core data for data consumers. This
* A Global Discovery Catalogue should connect to and subscribe to more than one Global Broker to ensure that no messages are lost in the event of a Global Broker failure. A Global Discovery Catalogue will discard duplicate messages as needed.
* A Global Discovery Catalogue will verify that a discovery metadata record identifier’s centre-id token (see Manual on WIS, Volume II – Appendix F. WMO Core Metadata Profile (Version 2)) matches the centre-id level of the topic from which it was published (see Manual on WIS, Volume II – Appendix D. WIS2 Topic Hierarchy) to ensure that discovery metadata are published by the authoritative organization.
* A Global Discovery Catalogue will validate discovery metadata records against the WCMP2. Valid WCMP2 records will be ingested into the catalogue. Invalid or malformed records will be discarded and reported to the Global Monitor against the centre-id associated with the discovery metadata record.
* A Global Discovery Catalogue may perform quality assessment of discovery metadata against the WCMP2 Key Performance Indicators (KPIs) footnote:[https://wmo-im.github.io/wcmp2/kpi/wcmp2-kpi-STABLE.html], providing the following metrics as defined in the WIS2 Metric Hierarchy footnote:[https://github.com/wmo-im/wis2-metric-hierarchy/blob/main/metrics/gdc.csv]: ``wmo_wis2_gdc_kpi_percentage_total``, ``wmo_wis2_gdc_kpi_percentage_average``, ``wmo_wis2_gdc_kpi_percentage_over80_total``.
* A single Global Discovery Catalogue perfoming quality assessment is sufficient for WIS2.
* A Global Discovery Catalogue will only update discovery metadata records to replace links for dataset subscription and notification (origin), with their equivalent links for subscription at Global Brokers (cache).
* A Global Discovery Catalogue will periodically assess discovery metadata provided by NCs and DCPCs against a set of key performance indicators (KPIs) in support of continuous improvement. Suggestions for improvement will be reported to the Global Monitor against the centre identifier associated with the discovery metadata record.
* A Global Discovery Catalogue will remove discovery metadata that are marked for deletion as specified in the data notification message.
Expand Down
20 changes: 4 additions & 16 deletions guide/sections/part3/sensor-centres.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -67,21 +67,9 @@ For such a centre operated by Météo-France the centre-id would be fr-meteofran

===== 3.3.3.1 Technical considerations

A list of metrics has been defined for this Sensor Centre (see https://github.com/wmo-im/wis2-metric-hierarchy/blob/main/metrics/scgc.csv):

[cols="3*", options="header"]
|=============================================================================================================================================================
wmo_wis2_scgc_cache_delay_seconds,global_cache|centre_id|report_by,Delay between origin and cache message,-
wmo_wis2_scgc_messages_cached_total,global_cache|centre_id|report_by,Number of data files cached for centre_id,-
wmo_wis2_scgc_messages_cached_delay_total,global_cache|centre_id|delay_threshold|report_by,Number of data files cached for centre_id within defined delay threshold (120s 300s 600s),-
wmo_wis2_scgc_messages_duplicates_total,centre_id|report_by,Number of messages with identical data_id and no rel=update,-
wmo_wis2_scgc_messages_published_total,centre_id|report_by,Number of cacheable data files published,-
wmo_wis2_scgc_messages_missed_total,global_cache|centre_id|report_by,Number of cacheable data not in global cache,-
wmo_wis2_scgc_messages_missed_all_total,centre_id|report_by,Number of cacheable data not in any cache,-
wmo_wis2_scgc_messages_cache_override_total,global_cache|centre_id|report_by,Number of cacheable data with override cache directive,-
|=============================================================================================================================================================

The processing of this Sensor Centre is as follow:
A list of metrics has been defined for this Sensor Centre at https://github.com/wmo-im/wis2-metric-hierarchy/blob/main/metrics/scgc.csv.

The processing of this Sensor Centre is as follows:

- Subscribe to a given `origin/a/wis2/...`, it could be `#` or a particular centre-id on at least one Global Broker
- Subscribe to a given `cache/a/wis2/...`, it could be `#` or a particular centre-id on at least one Global Cache - The subscription must be done on the broker of the Global Cache (unlike normal subscription to be made only on the Global Broker)
Expand All @@ -100,7 +88,7 @@ For each of the subscription to the various Global Caches:
- Compare this value with the three threshold defined in the metric table above. Increase by 1 `wmo_wis2_scgc_messages_cached_delay_total` using the threshold as a label (so less than 120s, less that 300s, less than 600s)
- If no Notification Message for the `data_id` is received after the highest threshold (here 600s), increase by 1 `wmo_wis2_scgc_messages_missed_total`

As a Global Cache can decide not to cache a data granule, and in order to assess if a Global Cache has decided to do so, the Sensor Centre will compare the `links` array published on `origin/a/wis/#` and in the Notification Message received on `cache/a/wis2/#` if the two `links`are identical, then the Global Cache has decided not to cache the data. In this case, increase by 1 `wmo_wis2_scgc_messages_cache_override_total `.
As a Global Cache can decide not to cache a data granule, and in order to assess if a Global Cache has decided to do so, the Sensor Centre will compare the ``links`` array published on `++origin/a/wis/#++` and in the Notification Message received on `++cache/a/wis2/#++` if the two ``links`` are identical, then the Global Cache has decided not to cache the data. In this case, increase by 1 ``wmo_wis2_scgc_messages_cache_override_total``.

If no Global Cache has cached the data, increase by 1 `wmo_wis2_scgc_messages_missed_all_total`

Expand Down