M2: continuously PTP-disciplined SV timeline, truthful smpSynch and IED sync-alarm acceptance #154

Desired physical outcome

In an isolated IEC 61850 lab, ARStack Studio → ESP32-P4 emits hardware-timestamped L2 PTP + canonical IEC 61850 Sampled Values (SV) such that an approved IED can:

1. discover/select the correct PTP source and obtain accepted time synchronization;
2. subscribe to the intended canonical SMV stream;
3. measure valid sample phase relative to the same timebase as PTP;
4. show no unexpected PTP/SV synchronization alarm during steady state and tested transitions.

Never fake source clockClass/timeTraceable or force smpSynch to clear alarms. A free-running class 248 clock may correctly be rejected by an IED configured to require traceable time. Certified/microsecond timing needs independent hardware evidence and potentially better oscillator/external reference.

Baseline, dependencies

• Native Qt Studio (not legacy web app); main@9e70a4937874.
• M1 software gating/first coarse epoch arm: draft PR M1: standards-aligned PTP source defaults and fail-closed SV epoch startup #153.
• Standards / related existing issues: P3: standards-first SCL-driven SV profiles and IEC/IEEE 61850-9-3 PTP synchronization #24, P2: deterministic ESP32-P4 Sampled Values publisher and timing evidence #21, P3-A2: observed SV wire fingerprint and configured-vs-observed resolver #35, P3-B0: PTP Grandmaster Clock foundation for the SCL-driven SMV Injector #56; IEC/IEEE 61850-9-3:2016, IEC 61850-9-2 AMD1:2020, IEC 61869-9 applicable variant.
• Preserved P0: 4I+4V/4000fps/1 ASDU, fixed buffer, nominal 250us GPTimer, accepted 1h run and Start/Stop, no dynamic allocations/locks/IOCTL in ISR or 4000fps task.

Ordered implementation (each own exact-head CI)

M2a — portable source-of-truth model

• Canonical SvTimelineAuthority combines exact explicitly resolved sample-counter/epoch rule, nominal frame cadence, selected PTP clock source provenance, independent timeline alignment, uncertainty, freshness, and conditional smpSynch eligibility.
• Do NOT create a second PTP servo; reuse PtpClockDiscipline/existing SmpSynchPolicy and immutable SvPublisherProfile.
• Define wrap, phase-step, GM switch, PTP outage, holdover and atomic generation transitions for 4000/4800+ profile variants; no universal rule inferred from SmpRate alone.

M2b — ESP32-P4 bounded clock bridge

• Correlate EMAC IEEE1588 PTP clock and monotonic/GPTimer via timestamp pair off realtime path, with documented uncertainty and stale-source invalidation.
• Hardware-safe bounded phase/frequency steering of GPTimer deadlines; no catch-up bursts; no arbitrary smpCnt discontinuity; holdover behavior explicit.
• Start PTP before SV (or fail closed/controlled rearm); source LAB_SOURCE local and receiver TIME_RECEIVER are distinct. Maintain local-source clockClass=248 until proven otherwise.

M2c — hardware evidence

• Correlate profile/sample sequence, intended PTP epoch, deadline, task wake, TX submit, and actual EMAC hardware TX time. Investigate driver timestamp throughput/queue contention before sampling every SV TX, otherwise use bounded sampling/external analyzer.
• Canonical tagged/multicast stream captured independently; default broadcast mirror disabled during timing acceptance.
• Report time error min/mean/max/P95/P99, drift, missed slots, TX failures; acceptance tolerances sourced from selected product/IED profile, not guessed.

M2d — portable native Studio UX

• Compact status showing PTP role/source/clockClass/domain, Pdelay readiness, timebase FREE-RUN/PTP-ANCHORED, sample-phase evidence, AUTO smpSynch and why blocked. Keep the validated Qt desktop workflow.

M2e — physical IED matrix

• PTP domain, transportSpecific, L2 mode P2P, 1Hz intervals, VLAN/PCP/switch path, BMCA/source priority, clockClass/timeProperties, rx Pdelay, correct source selection, no multiple-GM conflict.
• Match SV MAC/APPID/svID/confRev/DataSet/quality/rate/counter reset and smpSynch to IED SCL & standard.
• Test cold start, START mid-second, PTP first / SV first, GM loss/change, holdover timeout, PHY drop, forced lab anomaly (must be labelled), 4000/4800 and 1h retention.
• Save exact IED model/firmware, alarm text/state, full canonical + PTP .pcapng, device firmware commit, hardware/PHY, network topology and time measurement provenance.
• Don't claim universal alarm-free or IEC conformance until independent bench signatures exist.

Required CI / release gate

GCC/Clang/MSVC + ESP-IDF5.5.5 source/receiver builds + Qt Studio packaging + sanitizer/protocol regression green, then physically measured acceptance. Draft PR only before those gates.

Questions for bench operator: target IED model/firmware? IED enabled PTP profile and exact alarm text? Is GNSS/traceable clock required by that IED?

M2B — Combined PTP/SV wire-evidence checkpoint (2026-10-14)

• Reuse existing Ethernet, PTP, SV codecs and SampledValuesStreamSupervisor (no duplicate parser).
• Add host CLI ariec61850_ptp_sv_wire_audit for one classic PCAP, explicit APPID/svID/counter modulus/PTP domain, source identity, counts of Announce/Sync/FollowUp/PdelayResp and smpSynch.
• Positive/negative in-memory PCAP self-test: valid wrap, foreign stream ignored, counter gap detected.
• Add bounded-memory PcapReader::for_each_packet; preserve read_all() by calling the SAME streaming parser and moving packets into the legacy vector; tool processes one frame at a time, suitable for multi-hour high-rate capture.
• Update CI routing on branch so unrelated IED Qt Draft builds skip; one scoped checkpoint, no full Qt/ESP32/Windows build requested.
• NOT YET VERIFIED: actual C++ compilation/CTest on current exact PR head (Draft checks intentionally skip); run targeted checks when ready.
• Physical MAC TX timestamps, real IED PTP lock, phase error and alarm-free claims require bench evidence and are NOT inferred from host PCAP.
• Cross-vendor/independent protocol decoder comparison remains future acceptance.
  Documentation: docs/PTP_SV_WIRE_AUDIT_M2B.md. Current M2B code checkpoint: 3370c7dbc7e38729ae78b56c6a8ba45254d8ef94.

M2B verified targeted checkpoints — 2026-10-14

• Combined read-only PTP+SV PCAP auditor with bounded-memory streaming, SmpCnt continuity validation, selected source and SV identity, PTP L2 message visibility, and conservative/unknown hardware status.
• Two-step PTP Follow_Up must match both the selected PTP source and Sync sequence ID; missing or orphan Follow_Up is a failed software-wire condition.
• Linux targeted C++ builds and synthetic CTest PASS at initial audit and hardened Follow_Up pairing, last code commit 0c8372b01883ed80ce523e590026b8ce8763179b.
• Only one opt-in Ubuntu C++ job ran for each checkpoint; heavyweight GCC/Clang/MSVC full matrix and installers deliberately skipped during Draft. Temporary opt-in removed from PR body to prevent repeated builds.
• Still NOT verified: continuous PTP-disciplined SV on silicon, hardware TX timestamp, inter-vendor IED alarm-free behavior, full standards conformance. M1/M2A remain Draft; no public release.

M2A.1 — loss-aware phase authority, verified checkpoint (2026-10-14)

• Implementation: canonical publisher missed slots, Ethernet link failures, and failed canonical SV frames increment a fault generation and revoke phase tracking; the low-priority observer invalidates its previous PI state and must reacquire clean epochs. No per-frame extra atomic operation on the normal successful SV send path, and no PTP IOCTL in the ISR.
• The fault generation is rechecked within serialized phase-command handoff to reject stale corrections across STOP/START or new publishing faults.
• Portable regression: 360-second fault injection with repeat losses, session change, bounded ±1-tick steering, and reacquisition at 4000/4800 fps.
• CI verified on exact branch head 986ca6c1567e70827b56e88a0ad79fc4037a4349: targeted run #38537488220 — Portable strict phase regression: success, Compile guarded ESP32-P4 SOURCE firmware: success; full two-role matrix skipped as intended.
• Temporary Draft checkpoint marker has been removed. Firmware feature CONFIG_AR_SMV_PTP_PHASE_SERVO_EXPERIMENTAL still defaults OFF. The evidence proves portable algorithm regression + buildability of firmware SOURCE, not IED lock, clock accuracy, actual EMAC SV egress timestamps, or absence of alarms.

M2C — sparse canonical SV hardware TX descriptor evidence (2026-10-14)

• Implemented, commit 68151cafa43478ff411576bb61aa74d4e2c41813: optional (default OFF) ESP32-P4 SOURCE diagnostic that timestamps the actual canonical SV frame at one out of every 16 smpCnt=0 sample epochs. Normal 4k/4.8kHz frames retain esp_eth_transmit; the sparse selected frame uses ESP-IDF v5.5.4 esp_eth_transmit_ctrl_vargs with a MAC DMA-descriptor timestamp.
• Hardware timestamp polling is deliberately sparse because Espressif's pinned driver busy-waits for descriptor completion; the feature cannot be enabled alongside broadcast mirror or diagnostic probes. It does not run in the GPTimer ISR and does not alter public firmware defaults.
• Portable fixed-size SvTxEgressEvidence records measured/missing/failed/rejected frames, smpCnt, sample epoch and scheduled GPTimer deadline ticks, PTP egress seconds/phase, and CPU TX submit blocking. Failed or missing hardware timestamps are explicitly UNAVAILABLE (never replaced with software timing). A descriptor wait exceeding the scheduled sample interval is marked unqualified to expose instrumentation disturbance.
• The collector reports low-rate diagnostic min/mean/max in an existing 1Hz telemetry task, only on a new observation, with no smpSynch AUTO promotion. Correct PTP whole-second phase does NOT prove GM traceability, per-frame SV scheduling or relay sync alarm clearance.
• Strict negative/positive regression in tests/test_sv_tx_egress_evidence.cpp and existing phase authority suite. Verified PASS on exact commit: M2C one-job portable tests and guarded ESP32-P4 SOURCE firmware build. The full two-role matrix is skipped as intended.
• Source/driver contract documented in docs/PTP_SV_HARDWARE_TX_EVIDENCE_M2C.md, including pinned ESP-IDF DMA wait semantics and the possible inability to timestamp non-PTP SV EtherType on physical silicon.
• Temporary Draft checkpoint marker removed after PASS. Remaining hard gates: ESP32-P4 physical SV descriptor timestamp support, measured runtime jitter and missed slots under instrumentation, independent clock/IED capture, source quality/GM and actual alarm-free acceptance. Draft PR, no merge/release.

M2D — timestamp continuity & PTP SOURCE restart reliability (2026-10-14)

Verified code checkpoint: e871b54261e39f1468a030d72c2e35351395ad0c; single targeted ESP32-P4 + portable regression run — SUCCESS (portable strict phase/evidence CTest and ESP32-P4 SOURCE firmware), unrelated two-role matrix SKIPPED.

• Truthful sparse MAC TX proof: SvTxEgressEvidence no longer treats one near-zero-phase MAC descriptor timestamp as proof of a good window. First observation is baseline only. A measured value requires the next selected frame to advance exactly 16 SV sample-zero epochs AND the nearest integer PTP second by 16, with unchanged SV session, publisher fault-generation and PTP SOURCE generation. A missing TX timestamp, missed slot, failed canonical TX, unexpected epoch jump, PTP 1-second step or source restart invalidates/rebases observations and is not included in accepted min/mean/max.
• SOURCE restart signal: A monotonic PTP SOURCE generation increments when verified Announce + Sync + Follow_Up returns to READY; ar_ptp_lab_source_generation() is zero when SOURCE is not ready. Both sparse egress audit and ongoing software phase observer reject PTP source transitions and clear stale PI/phase evidence before reacquiring. No extra per-sample EMAC IOCTL, heap or locking in the GPTimer ISR.
• Portable negative test vectors cover PTP time jumps ±1/2 seconds, SOURCE restart while integer seconds still look continuous, SV publisher restart, canonical loss between 16-second probes, descriptor missing/timeout, initial-baseline-only and 32-bit SV epoch generation rollover.
• Limitations: No real ESP32-P4 silicon timestamp or real IED available; clock changes that cancel between sparse measurements still require continuous hardware observation. No IEC/IEEE certification, no automatic smpSynch=1/2, no claim that a relay has locked or sync alarms disappeared.
• Feature gates remain default OFF. Draft branch only; no public release or merge.

The one-time Draft CI opt-in was removed after the successful exact-head checkpoint. Subsequent Draft pushes do not automatically schedule expensive firmware builds.

M2E — PTP SOURCE health and fixed-cadence power profile (2026-10-14)

Checkpoint: GitHub Actions run #38545619492 SUCCESS, exact code SHA a035744c6ab87e0fec31648ff13050a1eb9b708a. Exactly one targeted job ran: portable strict C++ phase/egress/PTP runtime/source readiness tests SUCCESS and ESP32-P4 SOURCE build (ESP-IDF 5.5.4) SUCCESS. Full SOURCE/RECEIVER matrix and unrelated Draft jobs skipped as intended.

• Fixed a real false-readiness bug: previously any successful Pdelay/Announce/Sync TX could restore g_ptp_ready after a failed Sync/FollowUp because lifetime Announce/Sync/FollowUp counters remained nonzero. New PtpSourceReadinessGate requires fresh Announce AND complete hardware-timestamped Sync/Follow_Up since the most recent source/peer-delay TX fault; Pdelay success alone cannot qualify SOURCE. A new PTP SOURCE generation only begins after the fresh gate is satisfied.
• Fixed source cadence drift: PtpPublisherRuntime::poll_due() advances from the previous monotonic deadline, not from delayed task wake time; a 7ms late poll does not phase-shift subsequent nominal one-second Announce/Sync deadlines. Long stalls skip missed periods, never send burst catch-up messages.
• Standards-first admission: source defaults require 1s Announce, 1s Sync, and hardware peer-delay response. Nonstandard SOURCE cadence can only be explicitly opted into in LAB Kconfig (CONFIG_AR_PTP_ALLOW_NONSTANDARD_LAB_CADENCE=y). Domain and VLAN remain configurable for the target IED. Portable positive/negative tests cover readiness on Pdelay/Sync/Announce failures, recovery, 1Hz delayed polls, long missed intervals, and independent 250ms generic cadence.
• Documentation: docs/PTP_SOURCE_HEALTH_M2E.md. Source time quality remains truthful (clockClass=248, unknown accuracy, internal oscillator). This does not assert IEC/IEEE 61850-9-3 certification, GM traceability, a measured physical SV/PTP clock, or absence of IED synchronization alarms.
• Temporary Draft checkpoint marker removed. Default-off experimental phase/egress modes and public release remain unchanged. PR remains Draft until relevant ready-PR integration checks and eventual physical acceptance.

M2F — fail-closed SOURCE freshness and truthful timescale, verified 2026-10-14

Code commit: 90062f391a9c97cb1ff20d072b7f15ba4a777fac. Exact-head verified CI: ESP32-P4 PTP Lab Profile, one-job run #38551623186 — portable strict phase/power-profile/runtime/TX-evidence regression SUCCESS, guarded ESP32-P4 SOURCE firmware build SUCCESS, two-role matrix SKIPPED. Checkpoint opt-in was removed from PR immediately after initiation; ordinary Draft pushes do not trigger heavy jobs.

• A previously successful SOURCE could remain READY indefinitely when its FreeRTOS PTP task stalled. New zero-allocation portable PtpSourceEvidenceFreshness checks both a recent successfully transmitted Announce and a recent complete hardware-TX-timestamped Sync/FollowUp pair using monotonic microseconds. Default 1Hz power profile expires after 3 seconds. Stale timestamps, zero timestamps, backwards clock and STOP/START all fail closed.
• Public SOURCE status, initial startup readiness and ar_ptp_lab_source_generation use the same live, age-qualified health check. On a stale SOURCE loop, previous readiness is revoked before fresh sends, and a newly verified cycle starts a new generation, preventing PTP/SV phase and MAC TX evidence from inheriting a stale clock authority.
• M2F removes the previous overclaim of ptpTimescale from a plausible RTC alone: both ptpTimescale and currentUtcOffsetValid flags now require an actual EMAC seed and explicit operator verification of TAI–UTC offset/RTC epoch. Otherwise both remain false. timeTraceable and frequencyTraceable remain false; clockClass=248, unknown accuracy and internal oscillator unchanged. The strict IED may reject a lab clock with unknown provenance, which is appropriate.
• Portable negative tests check stale/missing Announce/Sync, timeout boundary, backwards clock, unverified TAI offset, and raw Ethernet wire flags 0x0000 versus 0x000c. Documentation: docs/PTP_SOURCE_FRESHNESS_M2F.md.
• Remaining gates: the M2 branch is 13 commits behind main at this checkpoint. It has not been rebased/merged; Ready PR full selected regression and actual ESP32-P4/IED clock evidence remain necessary. This checkpoint is a software build/test result only, NOT an IEC/IEEE 61850-9-3 conformance certificate, physical PTP lock, relay acceptance, or absence of IED sync alarms. Feature gates remain default OFF. No installer or public release was modified.