[masarray]

Field-reproduced failure on #269 (2026-10-21) — physical acceptance FAILED

Real board build d21d66e2872de4cd was flashed, verified, and captured with Studio. PCAP ARStack_SV_20261021_004403_675.pcap + report/CSV prove SV active ~00:44:06–00:44:32Z, then SV ceased until ~00:44:51Z while source PTP continued (354 valid PTP L2 frames). Studio independently reports RUNNING=false at 00:44:33Z and operator restarted. After restart smpSynch went 0→1 at 00:45:50.259Z (~59s), which is still too slow. PCAP 417,881 SV frames, no duplicated counters, one discontinuity across the actual STOP/START gap (1189 counter delta; does NOT prove 1189 packets missed while RUNNING). The only provided Studio serial log is a trailing 12,000-character snapshot from 00:46:10 onward, and cannot establish the precise physical STOP trigger at 00:44:32. Do not falsely claim actual cause is proven.

Corrections (on top of #269)

• Remove concurrent ESP-IDF one-shot timer stop/rearm previously called from both USB-serial HEARTBEAT task and timer callback. Replace with 4Hz periodic watchdog started once at boot, using a tight FreeRTOS critical section for heartbeat timestamp vs watchdog's STOP decision. 2.5-second fail-closed cutoff is unchanged and UART disconnect still stops physical output. No QML auto-restart, no bypass of explicit STOP. The old watchdog's races are a plausible failure path, not yet conclusively established as this incident's exact cause.
• Persist board-owned last SV STOP reason/counter/uptime/heartbeat age: SVLIFE state=... stop_cause=... stop_seq=... ..., emitted on every SHOW, surviving log truncation and STOP→START. Categories OPERATOR_STOP, LEASE_EXPIRED, RUNTIME_FAULT. This makes the next unexpected stop causally attributable without an early short serial log.
• Native Studio uses the same board SVLIFE readback for svStopReason and surfaces an explicit red "SV stopped · lease expired/runtime fault" instead of misleading READY green on unrequested physical STOP. While awaiting the board's cause display a neutral pending explanation. Status clears on verified restart/board RUNNING. PTP SOURCE remains logically independent.
• Add deterministic GUI session test for genuine RUNNING→STOPPED, board-cause readback after truncated logs, operator vs firmware STOP and clean restart; structural invariants assert periodic watchdog, protected timeout, and board/UI authority.

Acceptance gates

Build ESP32-P4 / Qt Studio / Windows native combined Studio+firmware exact-head preview; all CI green required before installer is offered. Repeat physical capture with installed matching firmware, prove no unexplained STOP over 1h and enforce continuity/heartbeat fault injection, and investigate slow smpSynch 60-second acquisition separately. Do not merge main, release publicly, or label hardware PASS from CI. Stack: #270 → #269 → #268.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-21T01:02:41.788297Z	ea447c1	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ea447c1a19

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Mark lease expiry as a stopped state

When the board emits Control session lease expired, firmware has already disabled SV output, but this branch only records the reason and leaves running_ true. Because WorkflowBar.unexpectedSvStop also requires !device.running, Studio continues showing green Injection running until a later SHOW response updates the state; that poll can be delayed or suppressed during other transactions. Transition to stopped when this authoritative lease-expiry event is received.

AGENTS.md reference: AGENTS.md:L74-L86

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Preserve operator-stop intent through the state transition

For a normal operator STOP while running_ is true, clearing operatorStopPending_ before calling setRunning(false) makes that method classify the transition as unsolicited and set svStopReason_ to AWAITING_BOARD_CAUSE. The UI therefore shows a red unexpected-stop warning after an intentional STOP until a later SVLIFE readback clears it; perform the running transition while the operator intent is still set.

AGENTS.md reference: AGENTS.md:L74-L86

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Reset STOP sequence tracking when the boot ID changes

The firmware's stop_seq restarts at zero on every boot, but a same-COM soft reboot leaves lastSvStopSeq_ intact and keeps deviceVerified_ true. If the previous boot reached a higher sequence, this comparison rejects every new boot's STOP cause until its counter catches up, leaving Studio with a stale reason or AWAITING_BOARD_CAUSE; scope the sequence to the boot ID or reset it in the existing new-boot path.

AGENTS.md reference: AGENTS.md:L74-L86

Useful? React with 👍 / 👎.

[masarray]

2026-10-21 field-failure follow-up — verified P0 code checkpoint

GitHub HEAD: 7f30f99a1fe432281a063f1ac934043fb315f9da | PR #270 stacked atop #269, #268. No merge to main, no public release.

Independent physical evidence, from the operator's new #269 board build

• Verified both installed firmware and bundled Studio firmware d21d66e2872de4cd.
• PCAP window 2026-10-21 00:44:03–00:46:08Z; 417,881 SV, 354 PTP L2 (118 each ANNOUNCE/SYNC/FOLLOW_UP). On-wire SV was active from ~00:44:06, ceased ~00:44:32, returned ~00:44:51 after operator re-START, while PTP continued. Studio readback RUNNING=false by 00:44:33Z, next RUNNING=true at ~00:44:50Z.
• PTP/SV AUTO smpSynch 0→1 occurred 00:45:50.258909Z, ~59 seconds AFTER the second on-wire START: still unacceptable acquisition delay. 1189 counter delta across 18.3s stopped session is NOT 1189 continuous-RUNNING packet drops.
• The provided Copy diagnostics includes only the trailing ~00:46:10+ console lines, so original STOP cause cannot be deduced with confidence. Old firmware had a plausible race: multiple tasks competed to stop/rearm the same 2.5s one-shot lease timer. Do not call it proven physical root cause.

Actual repairs at HEAD

1. ESP-IDF control lease uses one boot-lifetime 4Hz periodic watchdog, no esp_timer_stop / start_once on every heartbeat or timeout callback. Short critical section serializes heartbeat timestamp update and physical STOP decision. 2.5s missing-heartbeat fail-safe unchanged; no fake RUNNING or unsafe auto-restart.
2. Board persists SVLIFE state=... stop_cause=OPERATOR_STOP|LEASE_EXPIRED|RUNTIME_FAULT stop_seq=... stop_uptime_ms=... stop_hb_age_ms=... on every SHOW, surviving serial log rollover and operator restart.
3. Native Studio parses board SVLIFE, exposes current stop condition AND last board-cause history separately in Copy diagnostics, preserves last board cause after a verified restart, and surfaces visible red physical STOP status in the actual Main.qml footer plus operator alert, never green Ready on unexplained loss.
4. Deterministic native session regression for unsolicited RUNNING→STOPPED, cause retrieval, restart and operator STOP; structural safety contracts. No QML autonomous START or STOP watchdog.

Exact SHA CI: 8/8 workflow success

Embedded Profile CI; Qt Studio full harness, headless and Hi-DPI UI; C++ CI Linux+MSVC; Windows Lifecycle; Naming Guard; ESP32-P4 SMV Injector; PTP SOURCE+RECEIVER profile; Windows Native Preview. Windows native built and checked GUI + matched ESP32-P4 firmware, firmware contract, portable+Inno installer, clean install/upgrade/uninstall, real public NSIS v0.1.1 migration.

Matched native Windows Studio package: https://github.com/masarray/arstack61850/actions/runs/39525598664/artifacts/12313397185 (~78 MB, GitHub Actions temporary artifact). Build ID 7f30f99a1fe43228. Matched native preview CI https://github.com/masarray/arstack61850/actions/runs/39525598664.

Physical acceptance still FAILED / outstanding

Flash exact build from Studio GUI and verify build ID before test. Capture >=1h without operator STOP: board must maintain true 4kfps SV, distinguish PTP physical state, classify any stopped output using SVLIFE from Copy diagnostics, prove heartbeat when serial/capture CPU under load, test safe stop after actual USB unplug, no spontaneous STOP under connected control; measure warm-vs-cold click-to-LOCK, MAC+phase proof. The 59s AUTO synchronization acquisition is NOT fixed/certified by this P0 watchdog change and requires a separate timing investigation with full early SV/PTP phase telemetry. Do not merge to main or release until bench gates pass.