[dborup-agent]

Relates to #490

Summary

StoreObs is the store's most numerous struct. Its lazy timestamp cache (Kpa-clawbot#1481) was a sync.Once, a time.Time and a bool: 40 bytes, which made the struct 200 bytes and put every observation in the 208-byte heap size class. The cache is now a single atomic.Uint64. StoreObs is 168 bytes, in the 176-byte size class: 32 bytes less heap per observation. ParsedTime's contract is unchanged, and wireRoute (#76) keeps its padding byte.

On a 1M-observation store the retained heap after load drops from 864.6 to 841.0 MiB (−23.6 MiB, −2.7 %). First parses cost the same. A cached UTC read goes from about 1.5 ns to 3.1 ns, because it now builds the time.Time from an integer.

Plan (two commits)

1. Red test(store): … (2dea959c): the differential, cache-state, concurrency, layout and accounting tests, plus a benchmark and a gated heap measurement. The benchmark and the measurement use only public behaviour, so they also produce the "before" numbers on master. The commit does not build, because the cache field and its sentinels don't exist yet. On master's code the layout and accounting guards fail with real assertions (see Tests).
2. Green perf(store): … (8b208f36): the one-word cache, parseObsTimestamp, and storeObsBaseBytes set to the actual size class. It also adds a fuzz target that checks the same differential property.

There are no schema, config, API, workflow or deployment changes. cmd/server stays read-only.

Change

cmd/server/store.go:

cache word	meaning	what ParsedTime returns
0 (tsCacheNotParsed)	zero value, not parsed yet	parses, stores the word, returns the parse
1 (tsCacheInvalid)	empty or not a time	time.Time{}, false
2 (tsCacheReparse)	parsed, but not held exactly as UTC nanoseconds	parses Timestamp again on each call
anything else	uint64(UnixNano) ^ 1<<63	time.Unix(0, n).UTC(), true

• The encoding flips the sign bit of UnixNano. So the epoch (0), negative times and the zero point of every other scheme are ordinary cached values. Only the three instants whose encoding is a sentinel collide: 1677-09-21T00:12:43.145224192Z plus 0, 1 or 2 ns, the first three instants UnixNano can hold. Those three are stored as tsCacheReparse, so they are always correct. They are never misread.
• A time is cached only if the word gives it back exactly. Its location must be time.UTC (pointer comparison, so Local and fixed +hh:mm zones are excluded), and time.Unix(0, t.UnixNano()) must equal it. The second check is exact at both range ends: it catches the silent wrap of UnixNano outside 1677–2262, with no conservative year cut-off. All other parsed times are reparsed, which keeps their location, offset and precision.
• parseObsTimestamp is the old parse chain unchanged: empty → false, then RFC3339Nano → RFC3339 → "2006-01-02 15:04:05".
• Concurrency: racing first readers each parse and Store the same word. Reads are a single atomic Load. StoreObs.Timestamp is never assigned after construction (checked: the only .Timestamp = assignments in cmd/server are on other types), so a reparse reads the same input each time.
• storeObsBaseBytes 192 → 176. The old constant was already 16 B under the 208-byte class it charged for. A test now pins the constant to the runtime's own size class for unsafe.Sizeof(StoreObs{}).

Layout and accounting (64-bit)

	master 9080b6aa	this PR
unsafe.Sizeof(StoreObs{})	200 B	168 B
heap size class	208 B	176 B
storeObsBaseBytes	192 B	176 B
estimateStoreObsBytes (TestEstimateStoreObsBytes observation)	300 B	284 B
wireRoute offset	after tsParsedOK (192)	after tsCache (160)

amd64 and arm64 give the same layout. I checked this with a compile-time [168-Sizeof]byte assertion built for both GOARCHes; it is not committed. The layout tests skip on 32-bit, as before.

Coordination with #489 (PR #551, observer string pool, still open): that branch adds sharedStrs uint8 to the same padding block. With both changes StoreObs is still 168 B / 176 B, because the bytes after tsCache hold wireRoute and sharedStrs. The two branches will conflict textually in the struct and in heard_by_test.go's layout guard, and in storeObsBaseBytes if #551 touches it. If #551 lands first, I'll merge master into this branch (no rebase).

Measurements

Environment: linux/amd64, 4 vCPU, go1.27.1. Before = origin/master 9080b6aa with only obs_time_cache_490_bench_test.go copied in. After = this branch.

First-parse and cache-hit cost

go test -cpu 1 -bench StoreObsParsedTime_490, before and after interleaved, 3 rounds × 4 runs, benchstat with n=12:

case	before	after
UTC …Z, first call	66.8 ns	65.6 ns	~ (p=0.48)
SQL YYYY-MM-DD hh:mm:ss, first call	519.6 ns	501.7 ns	~ (p=0.48)
+01:00 offset, first call	69.6 ns	66.4 ns	−4.6 %
invalid, first call	314.9 ns	309.3 ns	~ (p=0.67)
UTC, cached	1.55 ns	3.09 ns	+1.5 ns: time.Unix(…).UTC() from the word
SQL, cached	1.45 ns	3.12 ns	same
offset, cached	1.49 ns	54.2 ns	reparse on every call (trade-off below)
invalid, cached	1.59 ns	1.22 ns	−23 %

Allocations are unchanged in every case: SQL first parse 240 B / 6 allocs, invalid first parse 336 B / 9 allocs, all others 0.

Heap after load

CORESCOPE_MEASURE_490=<n> go test -run TestStoreObsHeapAfterLoad_490 -v. The test measures heap growth across NewPacketStore + Load with every observation's time parsed, after closing the DB and running GC twice:

data	before	after
synthetic, 400k transmissions / 1,000,000 observations (UTC timestamps, as Load produces them)	862.7 / 863.0 / 864.6 / 866.2 / 868.2 MiB, median 864.6	838.4 / 841.0 / 841.1 MiB, median 841.0
per observation	904.6–910.4 B	879.2–882.0 B
trackedBytes	1062.24 MiB	1046.99 MiB (−16 B × 1M)
test-fixtures/e2e-fixture.db (499 observations)	11.71 / 11.72 MiB	11.70 MiB

The measured saving is about 25 B per observation, against the 32 B size-class difference. The rest is within the run-to-run spread of about ±3 MiB on this machine. The fixture is too small to show the change: 499 × 32 B is 16 KB. TestTrackedBytesTracksTheHeap_113's 0.85–1.15 band still passes.

Trade-offs

• Cached reads cost about 1.5 ns more. For /api/observers/{id}/analytics' 60k-observation loop that is roughly 0.1 ms per request, in exchange for 32 B per observation.
• Non-UTC timestamps are reparsed on every call (~54 ns, no allocation). Times outside 1677–2262 and the three colliding 1677 instants are too. The store never produces them on the v3 schema: Load, the chunked load and IngestNewFromDB select strftime('%Y-%m-%dT%H:%M:%fZ', o.timestamp, 'unixepoch'), which is always UTC Z. Only a legacy-schema text timestamp with an explicit offset would take this path, and it stays correct.
• For offsets that aren't whole hours (e.g. -05:30), each reparse returns a new *time.Location, so == between two calls can differ while Equal, Zone() and Location().String() match. UTC and Local values stay ==-identical, and the tests assert this. Whole-hour offsets reuse Go's cached fixed zones. No caller compares ParsedTime results with ==.

Tests

New in cmd/server/obs_time_cache_490_test.go, plus the updated guards in heard_by_test.go and eviction_test.go:

• TestStoreObsParsedTimeMatchesLegacyParser_490: 31 inputs, three calls each, against the replaced sync.Once body copied verbatim as the oracle. It checks ok, instant, Location().String(), Zone() name and offset, RFC3339Nano rendering, and == for UTC/Local. Inputs: RFC3339 and Nano at 0/1/3/9 fractional digits, SQL, SQL epoch, epoch, ±1 ns around the epoch, 1900, +01:00, -05:30, nanosecond offset, +00:00 (yields Local), the zero time, year 0, 9999, the exact UnixNano max and max+1 ns, min−1 ns, the three sentinel-colliding instants and the first non-colliding one, empty, space, garbage, date only, month 13, leap second, and no zone.
• TestStoreObsParsedTimeCacheStates_490: the word each input leaves after the first call (sentinel or the exact encoding), and that a cached call leaves it unchanged.
• TestStoreObsParsedTimeCachedCallDoesNotReparse_490: replaces Timestamp after the first call and checks that a cached answer (including "invalid") doesn't change.
• TestStoreObsParsedTimeConcurrentFirstReads_490: 8 readers released together, 20 rounds over all inputs, first and repeated calls. Run under -race.
• TestStoreObsLayoutTimeCacheIsOneWord_490, TestStoreObsLayoutFitsWireRouteInPadding_76 (now 168 B, wireRoute at tsCache+8), and TestStoreObsBaseBytesIsTheHeapSizeClass_490 (constant equals the runtime size class, and 4096 allocations land in that class).
• FuzzStoreObsParsedTime_490: the differential property, seeded with the table. 60 s of -fuzz locally (563k execs) found no divergence.

Red before: on master's code the layout guard reports unsafe.Sizeof(StoreObs{}) = 200, want 168, and the accounting guard reports storeObsBaseBytes = 192, want 208: the heap size class of a 200-byte StoreObs. The differential and race tests use the old parser as their oracle, so they pass on master by construction. The mutants below show what they catch.

Mutants (each applied to the green tree, run, reverted)

#	mutant	result
M1	no sign-bit bias (uint64(n))	killed: UTC time misdecoded on the cached call
M2	tsCacheInvalid reparses instead of returning false	killed: CachedCallDoesNotReparse
M3	invalid stored as tsCacheReparse	killed: cache state 2, want 1
M4	no sentinel-collision guard	killed: …145224193Z comes back ok=false
M5	no exact range check	killed: 0001-01-01 comes back as 1754
M6	no location check	killed: +01:00 loses its location
M7	conservative year range 1678–2261 instead of the exact check	killed: UnixNano max not cached
M8	never cache (always reparse)	killed: cache state
M9	cached read without .UTC()	killed: location Local vs UTC
M10	cached read returns ok=false	killed
M11	SQL layout dropped	killed
M12	storeObsBaseBytes back to 192	killed: TestEstimateStoreObsBytes, size-class guard
M13	an extra word in StoreObs	killed: Sizeof 176
M14	wireRoute moved out of tsCache's padding	killed: offset guard
M15	non-atomic cache word	killed: -race DATA RACE
M16	tsCacheReparse decoded as a time	killed: offset time comes back as 1677
M17	RFC3339 tried before RFC3339Nano	survived, equivalent: since Go 1.20 time.Parse sends both layouts through the same parseRFC3339 fast path, and the fallback accepts fractional seconds for either. It also survived 60 s of fuzzing.
M18	Load drops wireRoute	killed: heard_by_test.go (route semantics)

Validation

• cd cmd/server && go test ./...: ok. cd cmd/ingestor && go test ./...: ok.
• go test -race -run 'ParsedTime|490|Retransmission|ObserverAnalytics|HeardBy|Route|Evidence': ok. The 490|ParsedTime tests also pass with TZ=Europe/Copenhagen and TZ=Etc/GMT-1, where +01:00 resolves to Local.
• sh test-all.sh: 265 files passed. node test-frontend-helpers.js: 719 passed.
• E2E against a local Go server on a scratch copy of e2e-fixture.db, prepared as CI does (freshen, inline seeds, corescope-migrate, seeds 2073/199/245/122), then stopped by pid: test-e2e-playwright.js 154/157 (3 skips hard-coded in the suite), test-issue-76-heard-by-e2e.js 8/8, test-issue-122-compare-reference-many-e2e.js 11/11, test-issue-199-inactive-observer-e2e.js 3/3, test-issue-1486-collapse-reopens-detail-e2e.js 4/4, test-issue-147-packets-url-obs-e2e.js 12/12, test-issue-1639-observers-sort-e2e.js, test-issue-497-rx-airtime-e2e.js and test-observer-iata-1188-e2e.js all passed.
• scripts/check-xss-sinks.sh --diff origin/master: no frontend changes. No new map[string]interface{}. No .github changes; the fork guards are unchanged (9 in deploy.yml, 1 in release-fast-path.yml).

🤖 Generated with Claude Code

[dborup-agent]

Rapport — CS-pve-agent1 PR#555 #490 — head 8b208f3

Status: Implemented and locally green. CI is red on one job, Go Server Tests (3/3), from a pre-existing start-up/SIGTERM race in TestServerProcess_NormalStartDoesNotWait that this PR does not touch. It is not the known flaky test #271, so I have not re-run the job. The PR is still a draft.

Evidence: [T] test or command run, with its result · [A] analysis/measurement by the author · [K] source code reference

Requirements

#	Requirement	Test (red → green)	Mutant(s) killed	Evidence
1	Replace sync.Once + time.Time + bool with one atomic 64-bit word with unambiguous sentinels	TestStoreObsParsedTimeCacheStates_490 (does not build on master: no tsCache), TestStoreObsLayoutTimeCacheIsOneWord_490 (master: Sizeof = 200, want 168)	M1 no sign-bit bias, M3 invalid stored as reparse, M8 never cache, M13 extra word, M16 reparse sentinel decoded as a time	[T] [K] cmd/server/store.go tsCache, tsCacheEncode
2	Parse order, location/offset, precision, zero value, success/failure	TestStoreObsParsedTimeMatchesLegacyParser_490 (31 inputs × 3 calls vs the verbatim old parser), FuzzStoreObsParsedTime_490 (60 s, 563k execs, no divergence)	M6 location check, M9 cached read without .UTC(), M10 ok=false on hit, M11 SQL layout dropped. M17 (RFC3339 before RFC3339Nano) survived as equivalent: time.Parse sends both layouts through parseRFC3339	[T] [K] Go time/format.go Parse
3	Epoch, negative and zero times, values outside the UnixNano range, no sentinel collisions	Same differential table: epoch, ±1 ns around it, 1900, 0001-01-01, year 0, 9999, UnixNano max / max+1 ns / min−1 ns, the three colliding 1677 instants and the first non-colliding one	M4 collision guard dropped, M5 exact range check dropped, M7 conservative year range instead of the exact check	[T]
4	A cached call does not reparse; concurrent first reads	TestStoreObsParsedTimeCachedCallDoesNotReparse_490, TestStoreObsParsedTimeConcurrentFirstReads_490 under -race	M2 invalid sentinel reparses, M15 non-atomic word (DATA RACE)	[T]
5	Keep wireRoute and every fork-specific field	TestStoreObsLayoutFitsWireRouteInPadding_76 updated (168 B, wireRoute at tsCache+8). heard_by_test.go covers route semantics	M14 wireRoute moved, M18 Load drops wireRoute	[T]
6	Size guards and trackedBytes follow the actual layout	TestStoreObsBaseBytesIsTheHeapSizeClass_490 (master: storeObsBaseBytes = 192, want 208), TestEstimateStoreObsBytes (300 → 284)	M12 storeObsBaseBytes back to 192	[T] [A]
7	Measure Sizeof, size class, accounted bytes, first-parse and hit cost, heap after load	Measurements in the PR description	—	[A]

Measurements

	master 9080b6aa	this PR
unsafe.Sizeof(StoreObs{}) (amd64 and arm64, compile-time assertion)	200 B	168 B
heap size class	208 B	176 B
storeObsBaseBytes	192 B	176 B
heap after load, 1M observations (median)	864.6 MiB	841.0 MiB (−23.6 MiB, ≈ −25 B/obs)
trackedBytes, 1M observations	1062.24 MiB	1046.99 MiB
first parse (UTC / SQL / offset / invalid)	66.8 / 519.6 / 69.6 / 314.9 ns	65.6 / 501.7 / 66.4 / 309.3 ns (no significant change)
cached hit: UTC / offset / invalid	1.55 / 1.49 / 1.59 ns	3.09 / 54.2 (reparse) / 1.22 ns

[A] Offset times reparse on every call. The store doesn't produce them on the v3 schema, because every load path selects strftime(...'Z') [K].

Local validation

Check	Result	Evidence
cmd/server go test ./...	ok	[T]
cmd/ingestor go test ./...	ok	[T]
-race on ParsedTime/490/Retransmission/ObserverAnalytics/HeardBy/Route/Evidence	ok	[T]
TZ=Europe/Copenhagen and TZ=Etc/GMT-1 for the ParsedTime/490 tests	ok	[T]
sh test-all.sh	265 files passed	[T]
node test-frontend-helpers.js	719 passed	[T]
E2E on a local Go server with a scratch e2e-fixture.db prepared as in CI (freshen, seeds, migrate), stopped by pid	test-e2e-playwright.js 154/157 (3 skips hard-coded in the suite), heard-by #76 8/8, #122 11/11, #199 3/3, Kpa-clawbot#1486 4/4, #147 12/12, Kpa-clawbot#1639, #497 and Kpa-clawbot#1188 passed	[T]
scripts/check-xss-sinks.sh --diff origin/master	clean (no frontend changes)	[T]
new map[string]interface{}	0	[T]
fork guards	unchanged (9 in deploy.yml, 1 in release-fast-path.yml; no .github diff)	[T]
cmd/server read-only	no DB writes added	[K]

CI per job (run 40088547043)

Job	Result
Go Server Tests (1/3)	pass
Go Server Tests (2/3)	pass
Go Server Tests (3/3)	fail: TestServerProcess_NormalStartDoesNotWait
Go Ingestor Tests	pass
Go Libraries & CLI	pass
Verify Go Test Assignments	pass
JavaScript Unit Tests & Lint	pass
Script & Container Unit Checks	pass
Validate Docker Image	pass
Playwright E2E Tests (1/3, 2/3, 3/3)	pass
Verify E2E Shards & Coverage	pass
Go Build & Test (aggregate)	fail: needs Go Server 3/3
Build & Publish Docker Image	fail: needs the aggregate
Publish Badges, Release Artifacts, Deploy Staging	skipped

Failure analysis:

• [T] The test sent SIGTERM right after /api/healthz answered. The process exited with −1 (killed by the signal), and the log has no [server] received terminated line.
• [K] cmd/server/main.go:551–553 installs signal.Notify inside a goroutine, and ListenAndServe (main.go:648) can serve healthz before that goroutine runs. A SIGTERM in that window takes the default action.
• [A] The cause is a pre-existing race in start-up ordering. This PR does not change main.go or the test.
• [T] Locally the test passed 40/40 with -count=40 on this branch.
• It is not flaky: #180 packets URL E2E 'reload shows the same list' depends on fixture age (default 15-min window) #271, which is the fix(packets): leaving with the SlideOver open rewrites the new page's URL; Clear Filters on a detail URL re-filters on reload #180 packets-URL E2E, so under the agreed rule I have not re-run the job.

Remaining items

• The CI re-run of Go Server Tests (3/3) is the maintainer's call, since the failure is not flaky: #180 packets URL E2E 'reload shows the same list' depends on fixture age (default 15-min window) #271. The race itself (signal.Notify installed before the listener binds) deserves its own issue. I haven't opened one.
• perf(store): evaluate bounded sharing of observer metadata strings #489 (PR perf(store): bounded sharing of observer metadata strings #551) adds sharedStrs uint8 to the same padding block. With both changes StoreObs stays 168 B / 176 B. Expect a textual conflict in the struct and in heard_by_test.go. If perf(store): bounded sharing of observer metadata strings #551 lands first, I'll merge master into this branch (no rebase).
• Equivalent mutant M17 is documented and not killable (see row 2).
• Only amd64 was measured at run time. The arm64 layout was checked at compile time only.

[dborup]

Coordinator note: #551 (#489) is merged. git merges this PR cleanly, but the combined tree fails TestStoreObsLayoutFitsSharedStrsInPadding_489 (unsafe.Sizeof(StoreObs{}) = 168, want 200). #551's guard pins the old 200 B layout and this PR shrinks the struct to 168 B. Round 2 must merge master and update that guard to the combined layout: sharedStrs still in padding, and the size equal to this PR's 168 B, not larger. Check trackedBytes/estimateStoreObsBytes for the combined layout too.

[dborup]

Review — CS-Minimax PR#555 — head 8b208f3

Dom: REQUEST CHANGES

The change itself is correct, and I could not break the ParsedTime contract. The single blocker is new: #551 was merged into master during this review (157e7831). Merged into current master, this PR fails one cmd/server test, so it needs a master merge plus a one-constant update before it can land. Everything else would be an APPROVE with nits.

Evidence: [T] test or command I ran, with its result · [A] my analysis or measurement · [K] source reference

Findings

#	Severity	Finding	Evidence
1	Blocker (merge)	Merged with current master (157e7831, which includes #551), TestStoreObsLayoutFitsSharedStrsInPadding_489 fails: obs_strings_pool_489_test.go:311: unsafe.Sizeof(StoreObs{}) = 168, want 200: sharedStrs must use existing padding. There is no textual conflict: git merge-tree is clean, contrary to the PR's expectation. The break is semantic. Changing that assertion to 168 makes the whole cmd/server suite green. The comment at store.go:121 in the merged tree ("…so StoreObs stays 200 bytes") also goes stale and should say 168.	[T] merged tree 4f5187b9, [K]
2	Info	Combined layout (64-bit, arm64 natively and amd64 under emulation): Timestamp@136, tsCache@152 (8 B), wireRoute@160, sharedStrs@161, Sizeof = 168, size class 176, storeObsBaseBytes = 176. Both fork bytes share the tail padding, and nothing grows.	[T]
3	Nit	The collision guard tsCacheEncode(n) > tsCacheReparse is stricter than needed. Storing word 0 or 2 is self-correcting (a reparse); only word 1 would misreport. My mutants >= tsCacheReparse and > tsCacheInvalid survive as equivalent, and != tsCacheNotParsed is killed. The guard is correct; no change is required.	[T] mutants MX1, MX2, MX2b
4	Nit (documented)	Non-UTC and out-of-range times reparse on every call (about 38 ns on arm64). For offsets that aren't whole hours, the *time.Location is no longer == between calls. No v3 load path produces such timestamps (all of them select strftime(...'Z')), and no caller compares with ==.	[T] bench, [K]
5	Info (pre-existing)	TestRepeaterRelayInfoMap_ConcurrentIngestAndEviction failed once in my full-suite run. Run with -count=150 it fails 66/150 on master 5af6129c and 61/150 on the merged tree (darwin/arm64). It is unrelated to this PR (no ParsedTime use) and deserves its own issue.	[T]

Answers to the review questions

ParsedTime contract.

• [K] parseObsTimestamp keeps the old chain: empty → false, then RFC3339Nano → RFC3339 → "2006-01-02 15:04:05".
• [K] A cached UTC value is rebuilt with time.Unix(0,n).UTC(). That has the same wall/ext/loc=nil representation that time.Parse gives for Z and for zoneless SQL input, so the result is ==-identical, and not merely Equal.
• [K] Non-UTC locations (Local, fixed zones) and anything that time.Unix(0, t.UnixNano()) doesn't return exactly are stored as tsCacheReparse. That keeps the location, offset and precision, and it is exact at both range ends with no year cut-off.
• [K] The three 1677 instants that encode to 0/1/2 also reparse.

Tests on this:

• [T] I ran the PR's differential and state tests under TZ = UTC, Europe/Copenhagen, Etc/GMT-1, Asia/Kolkata and America/St_Johns: all green.
• [T] I ran the fuzzer for 120 s: 1.56M execs, no divergence.
• [T] My own differential test: 600k inputs covering random instants across the full UnixNano range, ±2000 ns around both ends, ±6000 years outside the range, and around the epoch. Each was rendered as RFC3339Nano, RFC3339 and SQL in UTC, +01:00, −05:30 and +00:45, with three calls each against the legacy oracle and an == check for UTC. All green.

Concurrency.

• [K] The cache is one atomic.Uint64, read with one Load and written with one Store. There is no multi-word state, so a torn read is impossible. Racing first readers store the same deterministic word.
• [K] StoreObs.Timestamp is never reassigned in cmd/server. The two .Timestamp = hits are on other types (routes.go:4288, ping_score_history_convert.go:203).
• [T] go test -race passes over ParsedTime/490/489/76/Retransmission/ObserverAnalytics/HeardBy/Route/Evidence/EstimateStoreObs/TrackedBytes on head and on master + this PR.
• [T] My mutant that stores a placeholder tsCacheInvalid before parsing is killed by TestStoreObsParsedTimeConcurrentFirstReads_490 20/20, with and without -race.
• [T] go vet (copylocks) is clean.

Sizeof and trackedBytes.

• [T] The PR alone measures 168 B, class 176, wireRoute@160 (= tsCache+8). That holds on arm64 natively and on an amd64 build run under emulation.
• [T] The size-class guard, TestEstimateStoreObsBytes (284) and TestTrackedBytesTracksTheHeap_113 all pass.
• [T] Heap after load, 1M observations, arm64, 2 runs each: master 866.0 / 864.9 MiB, PR 837.2 / 833.2 MiB, about −30 MiB (≈ −31 B/obs). trackedBytes goes 1062.24 → 1046.99 MiB, which matches the author's amd64 numbers.
• [A] trackedBytes falls by 16 B/obs while the heap falls by about 32 B/obs, because the old constant under-charged by 16 B. The tracked/heap ratio on this synthetic set moves from 1.23 to 1.25.

wireRoute and fork fields.

• [K] Only the three cache fields were replaced. wireRoute and every other field are unchanged.
• [T] The heard-by and route tests pass. The Node "Heard By" card: decide whether to show direct-RF observers (defer) #76 E2E passes 8/8 on both merged trees.

#551 conflicts and merge order.

• [T] git merge-tree origin/master <#551 53682d4f> <#555 8b208f36> reports no conflicts. The only break is semantic: finding 1.
• The merge order has now been settled by events: perf(store): bounded sharing of observer metadata strings #551 landed first.
• Recommendation: merge master into this branch (as the PR body plans, no rebase). Then update the _489 layout assertion to 168 and the store.go comment, and re-run CI.
• [T] With exactly that change, the following are green: the full cmd/server suite, cmd/ingestor, test-all.sh (266 files), test-frontend-helpers.js (719), the targeted -race set, and 13 E2E suites.

CI failure TestServerProcess_NormalStartDoesNotWait.

• [T] Attempt 1 of run 40088547043 failed with exit code -1 after SIGTERM (Go Server Tests 3/3). Attempt 2 is green on every job.
• [T] In the last 100 CI runs (all branches, all attempts), this was the only "Go Server Tests" job failure, so the test is rare but not new.
• [K] main.go:551–553 calls signal.Notify inside a goroutine. /api/healthz can answer before that goroutine runs, and a SIGTERM in that window takes the default action and kills the process.
• [T] A probe on a scratch copy of master (a 3 s sleep before signal.Notify) reproduces the exact CI signature 3/3: exit −1, and no "received" line.
• [T] It did not reproduce naturally: master and head each passed 40/40 with GOMAXPROCS=1.
• Conclusion: this is a pre-existing start-up race in master's code and unrelated to this PR. It deserves its own issue (register the signal handler before the listener binds). I have not opened one.

Hygiene

• [T] No closing keywords in the PR body or commits ("Relates to perf(store): evaluate compact, concurrency-safe observation timestamp cache #490").
• [T] Both commits are authored and committed by dborup <kontakt@meshview.dk>.
• [T] No .github diff. The fork-guard string occurs 9× in deploy.yml and 1× in release-fast-path.yml, the same as master.
• [T] scripts/check-xss-sinks.sh --diff is clean (no frontend files). The PR adds 0 map[string]interface{} and no hardcoded colours.
• [K] cmd/server stays read-only (no DB writes in the diff).
• [T] Red before: with the PR's guards on master's code, TestEstimateStoreObsBytes fails (300 vs 284) and the Sizeof guard fails (200). The state and race tests don't compile on master, by design.
• [A] The differential tests pass on master by construction, because they use the old parser as the oracle. Their strength comes from the mutants. That is acceptable for a pure refactor.

Acceptance criteria (#490)

Criterion	Status
Differential tests (RFC3339/Nano, SQL, offsets, empty/invalid, epoch, negative/zero, range ends, sentinel collisions)	met [T]
Concurrent first and repeated reads under -race; retransmission/observer analytics/route-evidence tests green	met [T]
Sizeof, size class, accounted bytes, first-parse and hit cost on the supported architectures	met [T]: amd64 by the author, arm64 by me (bench and heap)
Real retained-heap saving, plus the trade-offs of fallback timestamps	met on a synthetic 1M set [T]; the fixture is too small to show it [A]
Mutants catch wrong sentinel handling and lost offset/route semantics	met [T] (author's M1–M18, plus mine below)

Tests and mutants (mine)

Tests on the merged trees.

• master 5af6129c + #555 (41ad0290):
  • cmd/server passes except the pre-existing finding-5 flake.
  • cmd/ingestor ok; test-all.sh 266/266; test-frontend-helpers.js 719/719.
  • E2E (fixture prepared as in CI: freshen, inline seed SQL, corescope-migrate, seeds 2073/199/245/122; servers stopped by pid looked up by port): test-e2e-playwright.js 154/157 (3 skips built into the suite), Node "Heard By" card: decide whether to show direct-RF observers (defer) #76 8/8, Feature request: Reference-to-many observer comparison with complete-window packet overlap #122 11/11, Active observers vanish as nodes after retention.nodeDays without an advert; observer link dead-ends on 'Node not found' #199 3/3, fix(packets): updatePacketsUrl drops ?obs= and ?viewPath= on load and on filter changes #147 12/12, Packets page collapse button in the left column of the table opens the dialog. Kpa-clawbot/CoreScope#1486 4/4, bug: observers table is not sortable (regression-equivalent — same class as #679) Kpa-clawbot/CoreScope#1639, feat(observers): show RX 1h airtime (channel load) next to TX 1h, and both on the observer detail page #497, port(upstream#2208): rolling 1 h observer TX airtime and duty-cycle warning #479, feat: show observer IATA on packets (table, observation rows, detail pane) + add to filter grammar Kpa-clawbot/CoreScope#1188, [feat] node detail, separate flood-adv from zero-hop adv Kpa-clawbot/CoreScope#2073 10/10, feat(nodes): estimated advert intervals (flood / zero-hop) per node, then mesh-wide settings analytics #245 7/7, node-analytics 6/6.
  • /api/observers/{id}/analytics returns identical JSON on master and this PR for all 31 fixture observers × 4 days values (124 responses).
• master 157e7831 + #555 (4f5187b9): finding 1. With the one-constant update, everything listed in the perf(store): bounded sharing of observer metadata strings #551 answer above is green.

Mutants.

#	Mutant (applied to store.go, then restored from a snapshot)	Result
MX1	guard >= tsCacheReparse	survived, equivalent (word 2 means reparse)
MX2	guard > tsCacheInvalid	survived, equivalent (same reason)
MX2b	guard != tsCacheNotParsed	killed: "collides with invalid" (differential and state)
MX3	store a tsCacheInvalid placeholder before parsing	killed: concurrent first reads, 20/20
MX4	range check Equal replaced with n != MinInt64	killed: far future, min−1 ns (state and fuzz seeds)
MX5	decode off by 1 ns	killed: differential and fuzz seeds
MX6	the reparse path returns t.UTC()	killed: offset cases

What I did not verify

• amd64 at run time on real hardware. I ran amd64 only under emulation, for the layout dump; the amd64 performance numbers are the author's.
• The heap saving on real fork/production data. I used only the synthetic generator and the fixture, with no staging or prod access.
• 32-bit builds (the layout tests skip there).
• Whether the finding-5 flake also fails on Linux CI runners at a meaningful rate.

[dborup]

Review feedback addressed by the coordinator: master (with #551) merged in as b409b4a9, and the size guard and store comment moved from 200 B to 168 B in bb158123. This is the only blocker from Minimax' review. go test ./... in cmd/server passes locally on the merged tree (346 s).