Skip to content

Push notifications: launch tracking #72251

Description

@dmarchuk

Tracking issue for shipping push notifications in workflows to GA. There are enough moving pieces across several teams (product/eng, client libraries, billing, observability, comms) that a single place to see what's done and what's left is useful.

Goal

Reach feature + billing parity with email for workflow push steps (function_push), then launch — first as a free channel to gather feedback, then GA with pricing once the billing dependency lands.

Launch approach

  • Free first. Ship push with pricing dormant so we can gather real usage/feedback before GA. On the app side this is already safe: push enforcement no-ops until the billing service sets a workflow_push limit (an unset limit is never over-limit).
  • GA with billing once the billing-side product grouping and the usage-reporting/limiting dependency (below) are ready. Per-plan billing_start_date means pre-GA free usage won't count toward bills once pricing turns on.

✅ Done (merged)

PostHog app (our side)

  • Merge MCP discoverability + building-workflows skill docs for function_push - chore(workflows): document push actions in MCP tools and skill #71537
  • Unregister endpoint (DELETE /api/push_subscriptions) for SDK logout/reset() - feat(messaging): unregister push subscriptions via DELETE endpoint #72490
  • Mark push as Beta in the workflow build panel — chore(workflows): add beta tag to push notifications action #72498
  • Configure identity verification in the channel setup modals (disabled / optional / required), plus APNs .p8 file upload. Also fixes two backend gaps: the serializer dropped the config key entirely, and credential rotation wiped it — feat(messaging): configure push identity verification in channel setup #74143 (open, in review). Review also surfaced three authorization/concurrency fixes now in the PR: setting the policy requires project admin, the setup form no longer resubmits disabled over a stored policy, and concurrent writers for one integration serialize on an advisory lock.
  • Billing page: surface push as a line item under the workflows product (alongside workflow email + destinations), not as a standalone product ⛔ depends on billing product grouping + refactor below
  • Usage / spend dashboards: show push usage (useful during the free period even before it's billed)
  • Store sent push notifications so customers can see what a user received, mirroring how sent emails appear on the person view — feat(workflows): capture sent push notifications as message assets #74555 (draft). Reuses message_assets with kind: 'push' (no ClickHouse migration; kind is LowCardinality(String)). The per-person endpoint stays filtered to kind = 'email' in that PR, so surfacing push on the person view is the follow-up.
  • "Push opened" analytics. Approach decided, see below. Correlation ids are injected by feat(workflows): capture sent push notifications as message assets #74555 (draft); the metric consumer is the remaining piece.
  • optional mode has no customer-visible signal. The verified/unverified outcome only increments an internal Prometheus counter, so a customer running optional cannot tell from PostHog whether their app is actually sending tokens yet. That makes it a weak rollout step: the only check available today is their own backend's minting logs. Worth surfacing per-integration before we tell customers to stage through optional.

Open tracking: decided approach

Listen to the event the SDKs already capture. No webhook. A webhook would be a second unauthenticated public endpoint carrying a claim about which user did something, so it would inherit the same identity problem identity_token exists to solve, and it would ask customers to send us data we already receive.

Materialize a push_opened app-metric rather than querying events at read time, so opens sit alongside push_sent / push_skipped / push_failed in the Metrics tab and Sent → Opened is one query.

The SDK-side contract, verified against the source on feat/push-notifications in both repos (PostHogSDK.capturePushNotificationOpened / PostHog.capturePushNotificationOpened):

Payload we send a single posthog entry in data, holding workflow_id, invocation_id, action_id
Encoding JSON string. FCM's data map only accepts string values; both SDKs parse this entry from a dictionary or a JSON string, unconditionally rather than by delivery route, so one encoding covers FCM and direct APNs
Event captured $push_notification_opened
Properties $notification_<key>, so $notification_workflow_id, $notification_invocation_id, $notification_action_id, plus $notification_title / _body / _subtitle / _action

Only that nested entry is read. Sibling keys are ignored, which is what made an earlier revision of #74555 wrong: it sent flat posthog_workflow_id keys that would have been dropped on the device, so opens would have arrived unattributable and the metric would have read zero with no error anywhere. Fixed in that PR.

Remaining work for the metric: consume $push_notification_opened off KAFKA_EVENTS_JSON, resolve the hog flow from $notification_workflow_id, and queueAppMetric({ metric_kind: 'push', metric_name: 'push_opened', app_source_id: parentRunId ?? flowId, instance_id: actionId || invocationId }) — the same HogFunctionMonitoringService seam EmailTrackingService.trackMetric already uses for email_opened. Note the direction is inverted from email: for email the app-metric is primary and the customer event is derived from it, whereas for push the event is primary and the metric is derived. Then add push_opened to the metric-name union and to the Metrics tab tiles.

#74555 also left status as the seam for per-message open state: an open writes a higher-version message_assets row with status: 'opened' and the ReplacingMergeTree collapses to it, so no schema change is needed there either.

Security / abuse hardening

A push token binds a delivery destination, and the public project token can't prove the caller owns a given distinct_id. So this is the surface where an attacker could try to redirect (takeover) or remove (silence) a victim's notifications.

  • Identity verification mechanism — feat(messaging): optional identity verification for push subscriptions #72488
  • Make it usable: no UI could set the mode until feat(messaging): configure push identity verification in channel setup #74143, so in practice every integration is still disabled today
  • Context — takeover is not closed by default. Two routes: (a) a forged token planted via a raw $set capture is already prevented (the send path only delivers tokens it can decrypt with our key, so a plaintext forgery is ignored); but (b) takeover via the endpoint itself is open in the default disabled mode — the endpoint encrypts whatever token is submitted, so an attacker with the public token can register their device under a victim's distinct_id. Route (b) is closed only by the opt-in required mode. So takeover is fully prevented only once a customer enables required, which needs both feat(messaging): configure push identity verification in channel setup #74143 and the SDK releases below.
  • Residual: silencing via a raw $unset capture is not blocked by endpoint auth. Deferred fix is ingestion-level protection of the $device_push_subscription_* property namespace. Self-healing (the device re-registers), so it's a DoS rather than a takeover.

Client libraries

Contract confirmed aligned with the backend: token sent as identity_token on register POST and unregister DELETE; host app mints via a pushIdentityProvider(distinctId, appId, completion) hook; on 401 the SDK drops its per-(distinctId, appId) cache, re-mints, retries once, then halts for the session. Both SDK PRs additionally added a bounded timeout on the provider callback and guards against a stale registration snapshot overwriting a newer register/unregister while a mint is in flight.

Docs (posthog.com repo)

  • Workflows side: the push action, connecting FCM / APNs channels, payload fields, and the identity verification modes
  • Client libraries side: registering and unregistering device tokens, open tracking, and implementing the pushIdentityProvider hook (what the backend mints and how it's signed)
  • Changelog entry at GA

Both pages are drafted. The backend half (JWT claims, exp required, current/backup key acceptance, app_id resolution, the $device_push_subscription_<app_id> person property) is verified against the code, as is the open-tracking contract above.

Billing (separate billing repo)

  • Reuse existing test/prod Stripe prices — confirmed live tiers match the agreed pricing
  • Add the partner plan variant (2x free limits)
  • posthog.com pricing page already documents the push rate
  • Product/plan config so push meters under the workflows product (email + destinations + push) ⛔ see dependency
  • Set the workflow_push limit (this is what wakes up the app-side enforcement at GA)
  • Set per-plan billing_start_date so pre-GA free usage is excluded

Observability

  • Business metrics (push_sent / push_skipped / push_failed) surfaced in the Metrics tab
  • Ops counters: push_notification_sent_total, _failed_total (by reason), _skipped_total, _rescheduled_total, _token_pruned_total, push_notification_send_duration_ms
  • Grafana dashboard for push send health (volume, failure rate by provider/reason, send duration, reschedule rate)

Alerts

  • Alert on sustained push failure rate (by provider + normalized reason)
  • Alert on sustained push_notification_rescheduled_total for a single provider (a customer's FCM project / APNs topic being rate-limited)
  • Alert on credential/auth failures (expired APNs key, bad FCM service account)

Key dependency & sequencing

Grouping push under the workflows product makes it the third usage metric on that product (email + destinations + push). Turning on limiting for a 3-metric product depends on an in-progress usage-reporting/limiting refactor. Notes:

  • This blocks billing enforcement at GA, not the free launch. The app-side push code is already in its final shape and ships free today.
  • No PostHog-app code change is needed to "move push under workflows" — on the app side workflow_push is already a peer resource of email and destinations; the grouping is decided in the billing repo.

The other sequencing constraint is the SDK release: identity verification is only reachable by customers once the iOS and Android releases carrying it are published, so the docs and any "turn on required" guidance should follow those releases rather than lead them.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions