M2A: bounded continuous PTP–SMV phase authority and guarded ESP32-P4 observer

[masarray]

Milestone M2A: Continuous software PTP-to-SV phase authority

Stacked on Draft PR #153 (base branch: feature/esp32p4-ptp-sv-epoch-m1-20261014); not an independent merge to main. Work toward #154.

What actually changed

• Portable allocation-free SvPtpPhaseAuthority: 1 MHz GPTimer / EMAC PTP cross-clock bracket, PTP phase-error normalization, source epoch validation, 32-bit wrap handling, clock-step rejection, bounded PI correction.
• SvBoundedSlewCursor can distribute +/-400 µs per second as no more than +/-1 GPTimer tick per interval (4000/4800 fps), no abrupt smpCnt reset or catch-up burst.
• Guarded ESP32-P4 CONFIG_AR_SMV_PTP_PHASE_SERVO_EXPERIMENTAL=n by default. When explicitly enabled in LAB_SOURCE (started before SV), a CPU0 background observer captures PTP/GPTimer time once per epoch 250 ms after ISR sample zero and publishes a bounded correction for the remaining intervals. Existing realtime publisher stays canonical; no PTP ioctl/mutex/heap/log in 4kHz ISR.
• ISR deadline-slew cancellation on invalid evidence and corrected publisher lateness telemetry.
• Host regression: 480 s simulation at +80 ppm oscillator bias for 4k/4.8k fps; discontinuity, stale read, overflow and timed slew distribution; GCC/Clang/MSVC CI. ESP32-P4 source CI explicitly builds experimental adapter; public profile remains unchanged.
• Clear safety and evidence scope in docs/PTP_SV_PHASE_AUTHORITY_M2A.md.

Acceptance and limits

• Software-only, no physical IED/timing analyzer attached. Tracking is based on scheduled sample-zero DEADLINES rather than MAC TX timestamps and does not promote smpSynch AUTO or claim IEC/IEEE conformance.
• ClockClass 248 remains honest. Any IED requiring high-precision traceable GM may still alarm; that is not papered over.
• Requires EXACT head GCC/Clang/MSVC + SOURCE/RECEIVER ESP-IDF CI green before this can be considered mergeable into M1. No public release until further physical phase/egress benchmark and interoperability acceptance.

M2B — Combined PTP/SV wire-evidence checkpoint (2026-10-14)

• Reuse existing Ethernet, PTP, SV codecs and SampledValuesStreamSupervisor (no duplicate parser).
• Add host CLI ariec61850_ptp_sv_wire_audit for one classic PCAP, explicit APPID/svID/counter modulus/PTP domain, source identity, counts of Announce/Sync/FollowUp/PdelayResp and smpSynch.
• Positive/negative in-memory PCAP self-test: valid wrap, foreign stream ignored, counter gap detected.
• Add bounded-memory PcapReader::for_each_packet; preserve read_all() by calling the SAME streaming parser and moving packets into the legacy vector; tool processes one frame at a time, suitable for multi-hour high-rate capture.
• Update CI routing on branch so unrelated IED Qt Draft builds skip; one scoped checkpoint, no full Qt/ESP32/Windows build requested.
• NOT YET VERIFIED: actual C++ compilation/CTest on current exact PR head (Draft checks intentionally skip); run targeted checks when ready.
• Physical MAC TX timestamps, real IED PTP lock, phase error and alarm-free claims require bench evidence and are NOT inferred from host PCAP.
• Cross-vendor/independent protocol decoder comparison remains future acceptance.
  Documentation: docs/PTP_SV_WIRE_AUDIT_M2B.md. Current M2B code checkpoint: 3370c7dbc7e38729ae78b56c6a8ba45254d8ef94.

M2B targeted validation evidence — 2026-10-14

• PASS: C++ wire audit checkpoint (first run): built with -Werror, positive/negative synthetic PCAP and streaming/read_all parity.
• PASS: Follow_Up sequence matching checkpoint at code SHA 0c8372b01883ed80ce523e590026b8ce8763179b: same one-job Linux compile + CTest, mismatched Follow_Up sequence and missing Follow_Up both rejected.
• Current portable audit includes one outstanding two-step Sync sequence, explicit orphan/missing counters and PTP source identity checks. No host capture timestamps promoted to 1588 hardware accuracy.
• COST CONTROL: removed temporary one-job opt-in immediately after the successful checkpoint. Linux matrix and Windows MSVC remain skipped for Draft. The opt-in job is retained in the existing C++ workflow for future intentional checkpoints; it does not launch for ordinary Draft pushes.
• M2B remains Draft pending full selected checks and physical PTP/SMV bench acceptance. No main merge or installer release.

M2A.1 — loss-aware phase authority, verified checkpoint (2026-10-14)

• Implementation: canonical publisher missed slots, Ethernet link failures, and failed canonical SV frames increment a fault generation and revoke phase tracking; the low-priority observer invalidates its previous PI state and must reacquire clean epochs. No per-frame extra atomic operation on the normal successful SV send path, and no PTP IOCTL in the ISR.
• The fault generation is rechecked within serialized phase-command handoff to reject stale corrections across STOP/START or new publishing faults.
• Portable regression: 360-second fault injection with repeat losses, session change, bounded ±1-tick steering, and reacquisition at 4000/4800 fps.
• CI verified on exact branch head 986ca6c1567e70827b56e88a0ad79fc4037a4349: targeted run #38537488220 — Portable strict phase regression: success, Compile guarded ESP32-P4 SOURCE firmware: success; full two-role matrix skipped as intended.
• Temporary Draft checkpoint marker has been removed. Firmware feature CONFIG_AR_SMV_PTP_PHASE_SERVO_EXPERIMENTAL still defaults OFF. The evidence proves portable algorithm regression + buildability of firmware SOURCE, not IED lock, clock accuracy, actual EMAC SV egress timestamps, or absence of alarms.

M2C — sparse canonical SV hardware TX descriptor evidence (2026-10-14)

• Implemented, commit 68151cafa43478ff411576bb61aa74d4e2c41813: optional (default OFF) ESP32-P4 SOURCE diagnostic that timestamps the actual canonical SV frame at one out of every 16 smpCnt=0 sample epochs. Normal 4k/4.8kHz frames retain esp_eth_transmit; the sparse selected frame uses ESP-IDF v5.5.4 esp_eth_transmit_ctrl_vargs with a MAC DMA-descriptor timestamp.
• Hardware timestamp polling is deliberately sparse because Espressif's pinned driver busy-waits for descriptor completion; the feature cannot be enabled alongside broadcast mirror or diagnostic probes. It does not run in the GPTimer ISR and does not alter public firmware defaults.
• Portable fixed-size SvTxEgressEvidence records measured/missing/failed/rejected frames, smpCnt, sample epoch and scheduled GPTimer deadline ticks, PTP egress seconds/phase, and CPU TX submit blocking. Failed or missing hardware timestamps are explicitly UNAVAILABLE (never replaced with software timing). A descriptor wait exceeding the scheduled sample interval is marked unqualified to expose instrumentation disturbance.
• The collector reports low-rate diagnostic min/mean/max in an existing 1Hz telemetry task, only on a new observation, with no smpSynch AUTO promotion. Correct PTP whole-second phase does NOT prove GM traceability, per-frame SV scheduling or relay sync alarm clearance.
• Strict negative/positive regression in tests/test_sv_tx_egress_evidence.cpp and existing phase authority suite. Verified PASS on exact commit: M2C one-job portable tests and guarded ESP32-P4 SOURCE firmware build. The full two-role matrix is skipped as intended.
• Source/driver contract documented in docs/PTP_SV_HARDWARE_TX_EVIDENCE_M2C.md, including pinned ESP-IDF DMA wait semantics and the possible inability to timestamp non-PTP SV EtherType on physical silicon.
• Temporary Draft checkpoint marker removed after PASS. Remaining hard gates: ESP32-P4 physical SV descriptor timestamp support, measured runtime jitter and missed slots under instrumentation, independent clock/IED capture, source quality/GM and actual alarm-free acceptance. Draft PR, no merge/release.

M2D — timestamp continuity & PTP SOURCE restart reliability (2026-10-14)

Verified code checkpoint: e871b54261e39f1468a030d72c2e35351395ad0c; single targeted ESP32-P4 + portable regression run — SUCCESS (portable strict phase/evidence CTest and ESP32-P4 SOURCE firmware), unrelated two-role matrix SKIPPED.

• Truthful sparse MAC TX proof: SvTxEgressEvidence no longer treats one near-zero-phase MAC descriptor timestamp as proof of a good window. First observation is baseline only. A measured value requires the next selected frame to advance exactly 16 SV sample-zero epochs AND the nearest integer PTP second by 16, with unchanged SV session, publisher fault-generation and PTP SOURCE generation. A missing TX timestamp, missed slot, failed canonical TX, unexpected epoch jump, PTP 1-second step or source restart invalidates/rebases observations and is not included in accepted min/mean/max.
• SOURCE restart signal: A monotonic PTP SOURCE generation increments when verified Announce + Sync + Follow_Up returns to READY; ar_ptp_lab_source_generation() is zero when SOURCE is not ready. Both sparse egress audit and ongoing software phase observer reject PTP source transitions and clear stale PI/phase evidence before reacquiring. No extra per-sample EMAC IOCTL, heap or locking in the GPTimer ISR.
• Portable negative test vectors cover PTP time jumps ±1/2 seconds, SOURCE restart while integer seconds still look continuous, SV publisher restart, canonical loss between 16-second probes, descriptor missing/timeout, initial-baseline-only and 32-bit SV epoch generation rollover.
• Limitations: No real ESP32-P4 silicon timestamp or real IED available; clock changes that cancel between sparse measurements still require continuous hardware observation. No IEC/IEEE certification, no automatic smpSynch=1/2, no claim that a relay has locked or sync alarms disappeared.
• Feature gates remain default OFF. Draft branch only; no public release or merge.

The one-time Draft CI opt-in was removed after the successful exact-head checkpoint. Subsequent Draft pushes do not automatically schedule expensive firmware builds.

M2E — PTP SOURCE health and fixed-cadence power profile (2026-10-14)

Checkpoint: GitHub Actions run #38545619492 SUCCESS, exact code SHA a035744c6ab87e0fec31648ff13050a1eb9b708a. Exactly one targeted job ran: portable strict C++ phase/egress/PTP runtime/source readiness tests SUCCESS and ESP32-P4 SOURCE build (ESP-IDF 5.5.4) SUCCESS. Full SOURCE/RECEIVER matrix and unrelated Draft jobs skipped as intended.

• Fixed a real false-readiness bug: previously any successful Pdelay/Announce/Sync TX could restore g_ptp_ready after a failed Sync/FollowUp because lifetime Announce/Sync/FollowUp counters remained nonzero. New PtpSourceReadinessGate requires fresh Announce AND complete hardware-timestamped Sync/Follow_Up since the most recent source/peer-delay TX fault; Pdelay success alone cannot qualify SOURCE. A new PTP SOURCE generation only begins after the fresh gate is satisfied.
• Fixed source cadence drift: PtpPublisherRuntime::poll_due() advances from the previous monotonic deadline, not from delayed task wake time; a 7ms late poll does not phase-shift subsequent nominal one-second Announce/Sync deadlines. Long stalls skip missed periods, never send burst catch-up messages.
• Standards-first admission: source defaults require 1s Announce, 1s Sync, and hardware peer-delay response. Nonstandard SOURCE cadence can only be explicitly opted into in LAB Kconfig (CONFIG_AR_PTP_ALLOW_NONSTANDARD_LAB_CADENCE=y). Domain and VLAN remain configurable for the target IED. Portable positive/negative tests cover readiness on Pdelay/Sync/Announce failures, recovery, 1Hz delayed polls, long missed intervals, and independent 250ms generic cadence.
• Documentation: docs/PTP_SOURCE_HEALTH_M2E.md. Source time quality remains truthful (clockClass=248, unknown accuracy, internal oscillator). This does not assert IEC/IEEE 61850-9-3 certification, GM traceability, a measured physical SV/PTP clock, or absence of IED synchronization alarms.
• Temporary Draft checkpoint marker removed. Default-off experimental phase/egress modes and public release remain unchanged. PR remains Draft until relevant ready-PR integration checks and eventual physical acceptance.

M2F — fail-closed SOURCE freshness and truthful timescale, verified 2026-10-14

Code commit: 90062f391a9c97cb1ff20d072b7f15ba4a777fac. Exact-head verified CI: ESP32-P4 PTP Lab Profile, one-job run #38551623186 — portable strict phase/power-profile/runtime/TX-evidence regression SUCCESS, guarded ESP32-P4 SOURCE firmware build SUCCESS, two-role matrix SKIPPED. Checkpoint opt-in was removed from PR immediately after initiation; ordinary Draft pushes do not trigger heavy jobs.

• A previously successful SOURCE could remain READY indefinitely when its FreeRTOS PTP task stalled. New zero-allocation portable PtpSourceEvidenceFreshness checks both a recent successfully transmitted Announce and a recent complete hardware-TX-timestamped Sync/FollowUp pair using monotonic microseconds. Default 1Hz power profile expires after 3 seconds. Stale timestamps, zero timestamps, backwards clock and STOP/START all fail closed.
• Public SOURCE status, initial startup readiness and ar_ptp_lab_source_generation use the same live, age-qualified health check. On a stale SOURCE loop, previous readiness is revoked before fresh sends, and a newly verified cycle starts a new generation, preventing PTP/SV phase and MAC TX evidence from inheriting a stale clock authority.
• M2F removes the previous overclaim of ptpTimescale from a plausible RTC alone: both ptpTimescale and currentUtcOffsetValid flags now require an actual EMAC seed and explicit operator verification of TAI–UTC offset/RTC epoch. Otherwise both remain false. timeTraceable and frequencyTraceable remain false; clockClass=248, unknown accuracy and internal oscillator unchanged. The strict IED may reject a lab clock with unknown provenance, which is appropriate.
• Portable negative tests check stale/missing Announce/Sync, timeout boundary, backwards clock, unverified TAI offset, and raw Ethernet wire flags 0x0000 versus 0x000c. Documentation: docs/PTP_SOURCE_FRESHNESS_M2F.md.
• Remaining gates: the M2 branch is 13 commits behind main at this checkpoint. It has not been rebased/merged; Ready PR full selected regression and actual ESP32-P4/IED clock evidence remain necessary. This checkpoint is a software build/test result only, NOT an IEC/IEEE 61850-9-3 conformance certificate, physical PTP lock, relay acceptance, or absence of IED sync alarms. Feature gates remain default OFF. No installer or public release was modified.

[masarray]

CI integration checkpoint (2026-10-15): synchronized Draft routing from merged PR #158 (main merge e6b798c) onto this stacked M2/M3 branch in a single fast-forward commit 83a531a1d1bce036d86b4e7fcc2cbbf931fb3f26. Only CI workflow YAML, CI-routing checker and documentation changed; no embedded firmware, PTP runtime, Qt implementation or sampled-value source was modified. Retained the special one-job Draft wire-audit-checkpoint and phase-guard-checkpoint with narrow exact-marker policy exceptions; full build matrices continue to be skipped while Draft. Main PR #158 Ready checkpoint passed 23/23 workflows before merge. M3 branch CI Routing Policy run 38645571001 was queued when this note was written; do not infer its result until complete. Keep the PR Draft until separate M2/M3 functional gates are passed.