Pilot implementation of Global Replay (GRep) service

[6a6d74]

Prior to introducing this new "Supporting Service" to INFCOM-4, we should run a Pilot to validate the service (and give us time to make changes if needed).

[6a6d74]

@tomkralidis reports that a reference implementation of the GRep service is deployed at ECCC/MSC - albeit only with a 6-hour message retention.

@golfvert suggests connecting the pilot deployment to the WIS2-dev environment.

@6a6d74 offers to build a simple client to validate the GRep service.

• ACTION (@tomkralidis) Connect the ECCC/MSC deployment of the GRep reference implementation to the WIS2 Dev environment.
• ACTION (@6a6d74) Build a simple test client to validate the GRep service functions.

[tomkralidis]

Note: wis2-grep is dusted off/connected to WIS2 OPS, but can connect to DEV as needed.

[6a6d74]

ET-W2IT meeting 2-Feb-2026

@tomkralidis said that the GRep instance deployed in Canada is working in pilot mode, connected to the operational Global Brokers.
@golfvert asked this instance be modified so that it listened to the operational Global Brokers for WIS2 Notification Messages, but published its replay messages on the WIS2Dev Global Broker.
@tomkraldis agreed - but noted that this would require a small software change.
@6a6d74 is yet to begin work on the GRep test client.

• ACTION (@tomkralidis) modify the GRep software to allow listening and publishing to different Global Brokers; and redeploy the instance, configured to listen to operational GBs and publish to the WIS2Dev test GB.
• ACTION (@6a6d74) Build a simple test client to validate the GRep service functions.

[6a6d74]

ET-W2IT meeting 16-Feb-2026

@6a6d74 noted that development of the test client remains pending.

@tomkralidis provided an update on changes to the reference implementation: it now subscribe to multiple GBs and de-duplicates WNMs. Pilot implementation remains at 6-hr message caching. Operational implementation will be updated cache messages for longer, as outlined in the spec.

[6a6d74]

ET-W2IT meeting 16-Mar-2026

• @tomkralidis noted that the Canadian implementation of GRep now includes 72-hours worth of messages and is connected to all Global Brokers, with message de-duplication implemented.
• @golfvert is incorporating GRep into a component for Meteo France; which is providing some useful testing.
• @6a6d74 has not yet completed the GRep test client.

[6a6d74]

Test harness for GRep developed (see below). GRep implementation is currently not returning all the expected messages. @tomkralidis investigating.

+++

Test Harness Description

MQTT Subscriber Web

GitHub repo: 6a6d74/mqtt-subscriber-web

A web application for testing WIS2 (World Meteorological Organization Information System) Global Broker coverage by comparing messages received live against those available via the GRep (Global Replay) service.

For more information see the project README.

What it does

1. Live subscription — connects to one or more WIS2 Global Brokers via MQTT and collects messages for a user-defined duration
2. Pubtime windowing — filters messages by properties.pubtime (not wall-clock time) to handle broker relay lag; only messages within [START_PUBTIME, END_PUBTIME] are counted
3. Deduplication — unique messages are stored in Redis keyed by message ID
4. GRep HTTP fetch — after a configurable delay, queries the GRep service HTTP API for the same time window and topic to get a reference message count
5. GRep MQTT replay — triggers the GRep service to replay messages via MQTT, reconciles each replayed message against Redis, and reports any messages not received live (orphans)
6. Summary — prints a full reconciliation report to the event log and shows key metrics (received, GRep matched, coverage %) in the Results panel

---

Architecture

Layer	Technology
Backend	Python / Flask 3.1, served by Gunicorn (1 worker, 8 threads)
Frontend	Vue 3 SPA (CDN, no build step), communicates via SSE + REST
Broker client	paho-mqtt, one thread per broker (up to 5 simultaneous)
State / dedup	Redis
Deployment	Docker Compose, linux/arm64, on jztnet bridge network

Single Gunicorn worker is required — broker threads, SSE listener queues, and job state are all shared in-memory. SSE streams live events (messages, logs, tick countdowns) to the browser in real time.

[6a6d74]

ET-W2IT meeting 27-APR-2026

@6a6d74 gave an update on behalf of @tomkralidis: The lost message issues for the GRep service (i.e., not all messages are available from the pilot deployment) seem to be hardware bound. Updating the concurrency/queue in wis2-grep may help implementation. This remains work in progress.

[tomkralidis]

Update: based on ET-W2IT 2026-05-11 discussion, retention period for notification messages updated to 1 day (associated PR in wmo-im/wis2-guide#251).

[6a6d74]

@tomkralidis noted that limited progress has been made in resolving the issue with the pilot deployment. He reitereated that the problem appears to be hardware bound. Based on local tests, the implementation (code) seems fine. @tomkralidis is currently investigating increasing the hardware allocation for the pilot deployment.

@davidpodeur asked if there's a way to configure the speed of message replay - e.g., for low-bandwidth consumers.

This is not currently possible. However, it's important to note that there are both HTTP and MQTT methods provided to fetch "replayed" messages. The HTTP method pages resources thereby avoiding overloading client applications. However, with MQTT streaming it is possible to overwhelm subscribers on low-bandwidth connections.

@tomkralidis suggests that this might be avoided by forcing users to request "manageable" numbers of messages in a single (asynchronous) request. For example, when a request is to big, the server should respond with a HTTP 400 (Bad Request) or 413 (Content too large), along with some helpful diagnostics to help the user submit a more appropriate request.

@tomkralidis notes that this would require a change to the specification.

• ACTION (@davidpodeur): Open a new issue in the wmo-im/wis2-grep repo describing the concern. @tomkralidis to respond.

@hhaddouch noted that as Global Caches only store 24-hours of data, there seems little point in allowing users to request notifications that are more than 24-hours old - these notifications would point to data URLs that may no longer resolve.

Following discussion, ET-W2IT agree that the retention period for all WIS2 Notification messages (origin, cache) is set to 24 hours. Retention period for WIS2 Monitoring Event messages remains 7-days.

@tomkralids created a new PR for this change: wmo-im/wis2-guide#251.