Fetch a path's metadata over REST when the delta stream never carries it

[mairas]

Why

A path that materialises after Skip is already connected can lose its metadata for the whole session. The Signal K server's WebSocket hook resolves meta from a registry that gains entries at runtime, but marks a path as sent before checking whether the lookup found anything — so a producer that emits a value before registering the path's meta costs that connection its one chance. The course calculations do exactly this: they exist only while a course is active, and @signalk/course-provider sends values first, meta second.

The user sees a Numeric widget on the bearing to the next waypoint reading radians instead of degrees, with a unit menu offering every conversion group rather than the path's own. A page reload fixes it, because spark.sentMetaData starts empty again.

How

REST re-reads the same registry per request, so it can still answer after the WebSocket has given up. When a displayed self path produces a value with no metadata cached, Skip fetches vessels/self/<path>/meta.

Verified on a HaLOS device (signalk-server 2.30.0) with a course active: that endpoint returns 200 with units: "rad" and full displayUnits, for the exact path the issue names.

The request is subject to the same race — too early and the server has nothing either — so it retries rather than spending its one chance. What bounds it:

• Self paths only. The endpoint resolves against vessels/self. Self-ness is recorded where the delta states it, because the cache key cannot be inverted back to it: a foreign context named self.<something> produces a key indistinguishable from a self path's.
• Displayed paths only. A value registration or an observed meta subject — the tree API and the electrical widgets take meta without a value registration.
• One request per path outstanding, and the attempt counted when it completes. Counting at issue time made the budget measure deltas rather than answers: a 10 Hz path spent all three inside 300 ms, concurrently, each racing the registry state the first had already lost.
• Three answers per path, reset whenever endpoint discovery re-runs — a reconnect is both a fresh chance and the moment the stream most often loses the meta.
• A 5 s timeout, and a response accepted only if it carries a recognisable metadata key. Anything cached here is permanent for the page, so a gateway's JSON error envelope must not become the path's units.

Tests

18 cases in data.service.spec.ts. Reverting the delta-time call site fails 9; reverting the registration call site fails 1. Mutation-checked individually: dropping the in-flight guard, and narrowing the self-context guard to reject any delta that states its context, each fail exactly one named test.

Not addressed here

The server-side defect is the root cause and is tracked separately.

Fixes #545

[mairas]

Code review — 7 personas, findings applied

Reviewers: correctness, testing, maintainability, project-standards (always-on) + security (URL built from stream-supplied path segments), reliability (outbound request on the delta hot path, swallowed errors), adversarial (59 changed lines in the central data hub).

The finding that mattered

The premise in the original PR description was wrong, and three reviewers caught it independently. I claimed the server's meta hook reads only the static @signalk/path-metadata schema, so a path outside it never gets meta over the stream — making "a value with no meta" proof that the stream was done.

getMetadata is backed by a MetadataRegistry that gains entries at runtime via addMetaData, documented as "Called when meta deltas are received". And @signalk/course-provider sends its calcValues values first, then its meta delta on the next statement. So the real mechanism is a race: if the first value beats the metadata registration, handleValuesMeta marks the path sent before checking whether the lookup found anything, and that connection never gets the meta. A reload works because spark.sentMetaData starts empty.

The consequence for this PR was serious: a 404 means "not yet" as often as "never", and the original code spent its single attempt on it — reproducing the exact defect it set out to fix, in Skip. Fixed: the attempt is now retried on later values up to a ceiling of 3.

One P1 raised and refuted

The adversarial reviewer argued (adv-1, confidence 0.72) that rest.js:216 calls the same getMetadata singleton as ws.ts:1046, so REST can never know anything the stream did not — which would make the whole fix inert. Tested against halpi.hurma with a course active:

GET /signalk/v1/api/vessels/self/navigation/course/calcValues/bearingTrue/meta
HTTP 200
{"description":"The bearing of a line between the vessel's current position and nextPoint...",
 "units":"rad",
 "displayUnits":{"category":"angle","targetUnit":"degree","formula":"value * 57.29577951308231",...}}

The value endpoint also returns meta inline with "$source":"course-provider", which refutes the supporting claim that v2 deltas never reach the v1 full model. The fix works for the path in the issue.

Applied

Finding	Reviewers	Fix
One attempt spent on a "not yet" 404 reproduces the defect	correctness, reliability, adversarial	Retry on later values, ceiling of 3; success stops it
Path segments unencoded — %2e%2e survives the split, #/? escape the trailing /meta, session cookie attached	security P2	encodeURIComponent per segment
self. prefix on the cache key does not prove a self context	security P3	Self-ness taken from the delta's context, not the composed key
No timeout(...), unlike every sibling in core/services	reliability P2	5s timeout
httpServiceUrl concatenated assuming a trailing slash the server does not guarantee	maintainability P2	Normalized
In-flight comment stated the merge precedence backwards	project-standards, maintainability, testing	Response now dropped outright if a delta landed meanwhile — the precedence question is gone
Tree API and electrical widgets take meta without a value registration, so never backfill	correctness P3	Guard accepts an observed meta subject too
Widget placed on an already-cached idle path waits for a delta that may never come	correctness, adversarial	Backfill also attempted from subscribePath
Empty/array/non-object body could poison the cache and stop retries	testing, security	Rejected before setMeta
Every failure swallowed with no diagnostic	reliability, maintainability	console.debug with status and attempt number
Optional chain on a non-nullable BehaviorSubject emission	project-standards	Removed
Doc comment narrated the pre-fix symptom; spec comment duplicated it	project-standards, maintainability	Rewritten; spec comment cut to one line
CLAUDE.md enumerates the DI edges, data ← connection was missing	maintainability P3	Added

Tests went from 5 to 13 in the backfill block, covering the retry ceiling, retry-then-success, encoding, null endpoint mid-session, no-trailing-slash root, unusable bodies, the mid-flight delta race, and both new reachability paths. Suite: 2043 passing.

Not applied

• resetSignalKData has no callers (flagged by reliability, testing, maintainability, adversarial), so its clear() never runs, and isResetService's consumer in path-discovery.service.ts is unreachable with it. Pre-existing and out of scope here; filed as #581. The clear() stays because it is correct if reset is ever wired.
• Extracting a SignalKMetaClientService (maintainability P2). The trigger genuinely lives in updatePathData — only DataService knows "a value arrived with no meta" — and the reviewer acknowledged the counter-argument. Deferred with #563, which already tracks consolidating base-URL derivation.
• Widget-config unit dropdown for a path no live widget holds (correctness, residual risk). path-control-config reads getConversionsForPath synchronously and creates no registration, so that symptom in A path that appears after connect never gets units: meta is delta-only and one-shot #545 is unfixed for an unbound path. Narrower than the reported bug; not expanded here.

[mairas]

Code review — 7 personas, findings applied

Reviewers: correctness, testing, maintainability, project-standards, reliability, api-contract, adversarial. Each ran in a fresh context against an isolated worktree.

The premise holds; the P1 against it is refuted

The api-contract reviewer traced dispatchDelta in signalk-server and concluded that v2 deltas never reach the v1 registry, so this endpoint cannot serve navigation.course.calcValues.* — the exact path issue #545 names — and the fix is inert. Their own device had no course active, which they flagged as inconclusive.

Tested against a HaLOS device (signalk-server 2.30.0) with a course active:

GET .../vessels/self/navigation/course/calcValues/bearingTrue      -> 200
  value 5.0849, "$source":"course-provider", meta inline: units "rad",
  displayUnits {category angle, targetUnit degree, symbol °}
GET .../vessels/self/navigation/course/calcValues/bearingTrue/meta -> 200
  {"description":…, "units":"rad", "displayUnits":{…}}

The v1 model does carry course-provider meta on 2.30.0. The source reading is incomplete. The adversarial reviewer independently reached the same conclusion from the server source: it is the same MetadataRegistry singleton, but handleValuesMeta marking sentMetaData before the lookup is a TOCTOU, and REST re-reads the registry later — so "same singleton" is why the fix works.

The retry design did not survive measurement

Three reviewers, two with scratch specs run against the branch, found the same defect from different angles. Measured:

• one subscribed path + 10 rapid deltas → 3 concurrent in-flight GETs
• stalled server → 3 in flight, all 3 aborted on the same 5 s deadline, 0 requests afterwards, meta null for the page lifetime
• 25 paths × 5 deltas → 75 concurrent in-flight GETs

The counter incremented at issue time with nothing tracking an outstanding request, so the budget measured deltas rather than answers. The doc comment's claim that "the next value delta supplies both the retry and its spacing" holds only when the delta interval exceeds the round trip — false for any 5–10 Hz path.

Reworked: one request per path outstanding, attempt counted on completion. Three attempts now mean three answers.

The budget also never re-armed — its only reset lived in resetSignalKData, which has no callers (#581). A reconnect or server restart is both a fresh chance at the metadata and the moment the stream most often loses it, so the map now clears when endpoint discovery re-runs.

Four reviewers on the same forged-context hole

backfillMetaOverRest took the context from the delta precisely because the cache key can be forged, and a spec pins that. The registration call site then passed context: undefined and re-derived the path by slicing the key, so a delta with context: 'self.evil' was correctly declined at delta time and queried at registration time. Self-ness is now recorded where the delta states it, in a _selfPathKeys set, because setPathContext cannot be inverted safely.

Also applied

• Response validation. The old check accepted any non-empty object. Anything cached here is permanent for the page — the pathItem.meta guard then blocks every further attempt — so a gateway's JSON error envelope would have become the path's units. Now requires a recognisable metadata key.
• setMeta key. It re-derived the key from the current _selfUrn, so a self-URN change mid-flight filed the response under a phantom context. Now written against the self form directly.
• Call ordering. The backfill sat ahead of the delta's registration fan-out, and encodeURIComponent throws URIError on a lone surrogate — measured at 3 dropped widget updates on such a path. It now runs last.
• Error parameter typed unknown and narrowed (timeout() emits a TimeoutError, which has no status); console prefix matched to the file's.
• The PR body described the superseded one-attempt design. Rewritten.
• The upstream server defect was noted as "not filed yet" — now #584.

Tests

Four cases added, two mutation-verified individually: dropping the in-flight guard fails "keeps one request per path outstanding"; narrowing the self-context guard to reject any stated context fails "fetches for a delta whose context is the vessel's own self URN". That second one matters — every prior case used context: undefined, so the branch that actually fires in production was untested and a mutation removing it left the whole file green.

Not actioned

Percent-encoded segments are a permanent 404 (verified live: speedOverGr%6Fund/meta → 404 where the unencoded form → 200), and single-segment self paths like name and mmsi always 404 because rest.js gates on path.length > 4. Both cost a bounded number of wasted requests and neither is fixable client-side. No perf-harness numbers were taken; the added hot-path cost is a property read for paths that have meta and one string compare for foreign contexts, so it is an evidence gap rather than a suspected regression.

Gate on the head commit: 2054 tests pass, snc clean, lint clean.