Skip to content

Commit b8e5a4e

Browse files
authored
feat: M1-M3 runtime and operator closeout with M4 ARStack wire screening
Consolidates tested SV/PTP correctness, bounded high-rate processing, coalesced WPF presentation and local M4 PCAP+SCL acceptance tooling. 106/106 Windows tests, runtime stability, CodeQL and dependency review passed on PR head. Hardware TAP/SPAN, 60-minute soak and independent IED acceptance remain separately tracked in #30; no field-ready claim.
1 parent 8f88482 commit b8e5a4e

43 files changed

Lines changed: 2907 additions & 662 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎ProcessBusSuite.sln‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ProcessBus.Iec61850.Raw", "
1010
EndProject
1111
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ProcessBus.Tests", "tests\ProcessBus.Tests\ProcessBus.Tests.csproj", "{2F4B9C6E-7A31-4E8D-9B5A-6C1D82E4A0F7}"
1212
EndProject
13+
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ProcessBus.InteropCli", "tools\ProcessBus.InteropCli\ProcessBus.InteropCli.csproj", "{6F21A42B-A4EE-4901-8336-05A85E40D15F}"
14+
EndProject
1315
Global
1416
GlobalSection(SolutionConfigurationPlatforms) = preSolution
1517
Debug|Any CPU = Debug|Any CPU
@@ -50,6 +52,14 @@ Global
5052
{2F4B9C6E-7A31-4E8D-9B5A-6C1D82E4A0F7}.Release|Any CPU.Build.0 = Release|Any CPU
5153
{2F4B9C6E-7A31-4E8D-9B5A-6C1D82E4A0F7}.Release|x64.ActiveCfg = Release|x64
5254
{2F4B9C6E-7A31-4E8D-9B5A-6C1D82E4A0F7}.Release|x64.Build.0 = Release|x64
55+
{6F21A42B-A4EE-4901-8336-05A85E40D15F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
56+
{6F21A42B-A4EE-4901-8336-05A85E40D15F}.Debug|Any CPU.Build.0 = Debug|Any CPU
57+
{6F21A42B-A4EE-4901-8336-05A85E40D15F}.Debug|x64.ActiveCfg = Debug|x64
58+
{6F21A42B-A4EE-4901-8336-05A85E40D15F}.Debug|x64.Build.0 = Debug|x64
59+
{6F21A42B-A4EE-4901-8336-05A85E40D15F}.Release|Any CPU.ActiveCfg = Release|Any CPU
60+
{6F21A42B-A4EE-4901-8336-05A85E40D15F}.Release|Any CPU.Build.0 = Release|Any CPU
61+
{6F21A42B-A4EE-4901-8336-05A85E40D15F}.Release|x64.ActiveCfg = Release|x64
62+
{6F21A42B-A4EE-4901-8336-05A85E40D15F}.Release|x64.Build.0 = Release|x64
5363
EndGlobalSection
5464
GlobalSection(SolutionProperties) = preSolution
5565
HideSolutionNode = FALSE
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# M1/M2 receiver hardening — 17 October 2026
2+
3+
Scope: Process Bus Insight / DigSubAnalyzer receive-only application. This record
4+
describes the bounded software tranche proposed in PR #29. Physical interoperability
5+
and timestamp accuracy require separate evidence.
6+
7+
## Non-negotiable IEC 61850 boundaries
8+
9+
- The ARStack injector's canonical SCL-derived multicast/VLAN/APPID/svID identity and
10+
broadcast `AR_DIAG_SV1` mirror are distinct streams. A mirror observation cannot
11+
establish canonical multicast/VLAN acceptance.
12+
- One frame may contain multiple SV ASDUs; each ASDU contributes to sample sequence,
13+
but the frame has only one host receive timestamp. Do not invent intra-frame jitter.
14+
- `SmpRate/SmpMod` are sampling metadata, not universal sample-counter modulo rules.
15+
Counter wrap conclusions remain bounded by observed and profile-supported evidence.
16+
- The eight (INT32 instMag, UINT32 q) pairs are a 64-byte 4I+4V *profile-specific*
17+
decoding authority. Never apply their quality interpretation to arbitrary SCL
18+
layouts solely because the payload is 64 bytes.
19+
- Quality with unusable validity/detail flags immediately invalidates displayed
20+
engineering measurements; wait for a new complete coherent window before resuming.
21+
A TEST or operator-blocked quality remains visible with its truthful context.
22+
- PTP observations are scoped to domain/sourcePortIdentity/transport; observation
23+
does not imply clock lock, grandmaster traceability, power-profile compliance,
24+
or reliable software receive timing.
25+
26+
## Data plane and UI rules
27+
28+
- One dedicated capture pump preserves input ordering. Do not introduce independent,
29+
unordered parallel SV workers or silent lossy queues merely to increase thread count.
30+
- Npcap owns raw frame memory only until its next API read. The source transfers
31+
a newly allocated owned frame to the analyzer. The latest payload is retained for
32+
debug-on-demand; indiscriminate ArrayPool return would corrupt this evidence.
33+
- Per-ASDU projections use direct bounded INT32 reads and stack-local eight-channel
34+
values. SCL eligibility and explanatory labels refresh when mapping/sequence state
35+
changes, not for each SV frame.
36+
- Rolling jitter maximum uses a bounded monotonic ring rather than scanning the
37+
entire window for each sample. Retire the unused counter dictionary/linear scan.
38+
- The active per-stream coherent waveform buffer is the single snapshot authority.
39+
RMS/phasor/waveform projections of the same generation reuse that window; an
40+
invalid quality state must invalidate prior measurements, never show stale values
41+
as live.
42+
- User-visible SCL comparisons may be coalesced to 500 ms, with explicit user SCL
43+
reload forcing immediate recomputation.
44+
- Stream history has a 256-identity guard. Reclaim only stale unselected state;
45+
when at active capacity, report rejection rather than silently evicting a stream
46+
currently being inspected.
47+
- PTP source history is bounded and stale sources are labeled STALE instead of
48+
green OBSERVED; the interface should emphasize the fault and next diagnostic
49+
action rather than flooding raw details.
50+
51+
## Automated acceptance
52+
53+
`dotnet build ProcessBusSuite.sln -c Release` and `dotnet test
54+
ProcessBusSuite.sln -c Release` plus repeated RuntimeStability acceptance must pass.
55+
56+
New regressions include:
57+
58+
- O(1) rolling jitter reference equivalence under a 30,000-sample run;
59+
- invalid-quality suppression, complete coherent recovery, and TEST labeling;
60+
- multi-ASDU continuity with a single frame timestamp and independent frame count;
61+
- per-domain PTP source isolation and stale Announce;
62+
- strict PTPv2 length/transportSpecific parsing;
63+
- bounded stream identity churn and preserved selected stream;
64+
- rejection of `SmpRate` as an unverified counter modulus.
65+
66+
## Evidence still required before field/product claims
67+
68+
1. Trusted independent canonical multicast/VLAN capture from ARStack ESP32-P4.
69+
2. Same-run canonical SV (4000/4800 samples/s) plus Announce, Sync, Follow_Up,
70+
Pdelay and `smpSynch` transition stimuli.
71+
3. Npcap/adapter drop counters, host CPU/GC/memory, sustained capture throughput,
72+
and 60+ minute bench soak with no cross-stream contamination.
73+
4. Hardware timestamp validation for any measurement-grade jitter/offset statement.
74+
5. Multi-vendor profile/SCL fixtures, arbitrary `nofASDU` and vendor DataSet layouts,
75+
and receiver/IED behavior with real devices.
76+
77+
The software tranche avoids claiming those field results. A green CI result means
78+
the bounded software regressions passed, not that an IED accepted the traffic.
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# M3 operator workspace and M1/M2 closeout — 17 October 2026
2+
3+
This engineering handoff describes the implemented changes on PR #29, and
4+
distinguishes a successful software acceptance run from hardware interoperability.
5+
6+
## Operating contract
7+
8+
Process Bus Insight remains a **receive-only** IEC 61850 SV/GOOSE/PTP analyzer.
9+
The operator workflow is:
10+
11+
1. Select the physical Ethernet/Npcap adapter.
12+
2. Press **Start** to observe traffic.
13+
3. Choose an SV publisher in the explorer to inspect waveform, RMS, phasors,
14+
stream identity, quality and counter continuity.
15+
4. Read the one-line context-aware status for the current situation.
16+
5. Open **Diagnostics** for capture-driver drops vs SV sequence evidence, and
17+
**Timing/PTP** for passive source-specific PTP evidence.
18+
6. Open **SCL/Validation** only to compare engineering-file expectations with
19+
actual wire-observed identities; **Advanced** holds low-level raw evidence.
20+
21+
No value inferred from the diagnostic broadcast mirror is presented as proof
22+
of canonical multicast, APPID, VLAN, PCP or SCL profile acceptance.
23+
24+
## M3 implementation
25+
26+
- Main capture-status strip displays one concise contextual message instead of
27+
five overlapping status chips. The persistent footer is short and conditional:
28+
it surfaces diagnostic-mirror, invalid-quality and stale-stream concerns.
29+
- Real SV measurements are never tweened/interpolated between capture snapshots.
30+
Dead 30 FPS composition-loop/interpolation code was removed from the waveform
31+
control: WPF paints the latest decoded, coherent snapshot atomically.
32+
- `LiveRenderCoalescer` compares **every numeric waveform sample** plus
33+
series/shape/status metadata and channel RMS/angle/instant/units. It suppresses
34+
equivalent redraw notifications while preserving tiny real transients, shape
35+
changes, quality invalidations, stream selections and cleared values.
36+
- Phasor lists and per-stream UI workspace objects are reused while unchanged.
37+
Periodic SelectedStream/IsRunning re-notifications were eliminated to prevent
38+
selection churn.
39+
- Diagnostic navigator refresh is capped to the passive UI cadence;
40+
managed-memory sampling runs at 1 Hz; SCL binding joins are coalesced.
41+
- Engineering tables use recycling/row virtualization and shorter minimum row
42+
heights. Static flow instructions were removed from the main content.
43+
- PTP two-step sequence correlation is passive and displayed only in the Timing
44+
workspace; there is **no** claim of measured PTP lock or UTC traceability.
45+
46+
## M1/M2 additional correctness/efficiency
47+
48+
- PTP packet-rate tracking uses fixed 5-second buckets, not an unbounded list
49+
of packet arrival timestamps per source.
50+
- Announce/Sync/Follow_Up evidence is isolated by source/domain/transport;
51+
Sync+Follow_Up pairing additionally requires matching sequence and bounded
52+
freshness. Source history and pending correlators have hard capacity limits.
53+
- Diagnostic/event collection indexes persist across UI refreshes, and remain
54+
bounded at 200 visible rows; reset removes both rows and indexes.
55+
- Stable SV explorer rows are retained rather than replaced on every update.
56+
- Npcap native drop statistics are polled at 1 Hz off the capture hot path.
57+
Driver drop counters are displayed as **capture-path evidence**, and are never
58+
folded into IEC 61850 sample-counter packet-loss inferences.
59+
- Full IEEE 61850 profile compliance, PTP servo discipline, calibrated host
60+
jitter, and physical IED subscription are not asserted here.
61+
62+
## Automated regression gates
63+
64+
Windows .NET 8 full build and complete xUnit suite must pass together with
65+
repeated RuntimeStability, CodeQL, and Dependency Security Review workflows on
66+
the **same final commit**.
67+
68+
New coverage includes render-coalescing correctness, multi-ASDU timing,
69+
quality invalidation/recovery, rate-window boundedness, cross-source two-step
70+
correlation, stream catalog pressure and reset-safe UI event deduplication.
71+
Do not merge if final-commit CI is red or incomplete.
72+
73+
## Physical and measured acceptance still necessary
74+
75+
The following must be collected on real hardware before an
76+
**interoperable/field-ready** release claim:
77+
78+
- independent preserved-VLAN TAP/SPAN capture of ARStack canonical multicast
79+
alongside the separately labeled diagnostic mirror;
80+
- 4000/4800 SV samples/s and concurrent L2 PTP source / receiving profile;
81+
- verified observed APPID, VLAN, MAC, confRev, svID, quality, smpCnt rollover,
82+
two-step matching, source expiry and smpSynch transition evidence;
83+
- actual Npcap drop counters, process CPU, allocation rate, Gen0/Gen1/Gen2 GC,
84+
heap/working set, UI frame responsiveness during sustained mixed traffic;
85+
- 60+ minute hardware soak and multi-vendor IED capture/acceptance matrix.
86+
87+
**Receiver-worker decision:** maintain one ordered dedicated capture worker until
88+
independent profiling demonstrates its insufficiency. Unbounded independent
89+
workers or lossy frame queues would violate sample continuity. A multi-worker
90+
dispatcher requires bounded queues, source-preserving ordering and explicit
91+
drop accounting and remains a future measured-design gate, not a hidden
92+
"optimization" in this PR.
93+
94+
The software milestone can be reviewed independently of that bench evidence.
Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
# M4 — ARStack Injector ↔ DigSubAnalyzer interoperability acceptance
2+
3+
Date: 17 October 2026
4+
Repositories: `masarray/DigSubAnalyzer` (consumer, **receive-only**) and
5+
`masarray/arstack61850` (ESP32-P4 SMV Injector).
6+
7+
## Three different evidence levels
8+
9+
1. **Software replay gate (automated):** anonymous synthetic PCAPs and SCL-intent
10+
fixtures exercise the **same production parser** used by live reception.
11+
This demonstrates decoder behavior but not a physical process bus.
12+
2. **Independent observed-wire gate (operator bench):** capture on an independent
13+
TAP or correctly configured SPAN port that preserves multicast and 802.1Q
14+
tags. Compare observed frames with the actually deployed SCL profile.
15+
3. **Device/IED gate (external equipment):** verify subscriber/IED acquisition,
16+
quality, synchronization and sustained operation with equipment-approved
17+
tools. DigSubAnalyzer alone cannot certify relay subscription, functional
18+
safety, protection action or hardware PTP synchronization.
19+
20+
The M4 software utility intentionally reports **WIRE_FIELDS_MATCH** instead
21+
of “compliant”, “certified”, “locked” or “accepted by IED”.
22+
23+
## Run the offline acceptance inspector
24+
25+
Windows with .NET 8 SDK installed:
26+
27+
```powershell
28+
dotnet build .\ProcessBusSuite.sln -c Release
29+
dotnet test .\ProcessBusSuite.sln -c Release --no-build --filter Category=M4Interop
30+
dotnet run --project .\tools\ProcessBus.InteropCli -c Release -- --pcap C:\lab\arstack-canonical.pcap --scl C:\lab\current-profile.scd --max-frames 20000000 --json C:\lab\interop-report.json
31+
```
32+
33+
The tool supports classic Ethernet PCAP 2.4 (microsecond/nanosecond). If the
34+
capture application creates PCAPNG, export/copy the packet recording as
35+
**classic PCAP** with full Ethernet headers before inspecting it.
36+
37+
`--scl` is optional for wire fingerprint only; mandatory for automated
38+
expected-vs-observed verdicts. The CLI prints a short operator summary,
39+
keeps raw details in optional local JSON, and returns:
40+
41+
- **0**: screening executed and no specified SCL comparison failed; **not**
42+
certification and **not** proof of capture-path visibility. For a capture
43+
without SCL, there are no configured fields to validate.
44+
- **3**: observed/configured mismatch, unresolved SCL identity or expected
45+
stream not observed; this includes **MIRROR_ONLY**.
46+
- **2**: malformed arguments, unreadable/unsupported PCAP, or no useful
47+
successfully decoded traffic.
48+
49+
Bench files, identities, MAC addresses, PCAP, JSON logs and original SCL stay
50+
on the authorized **local machine**; do not push them to GitHub.
51+
52+
### Verdicts
53+
54+
| Verdict | Interpretation | Operator action |
55+
| --- | --- | --- |
56+
| `WIRE_FIELDS_MATCH` | Visible fields match configured SCL stream | Verify TAP/SPAN provenance and independent IED behavior |
57+
| `MIRROR_ONLY` | Only ARStack diagnostic broadcast identified | Fix capture visibility; do not infer canonical multicast/VLAN |
58+
| `NOT_OBSERVED` | Configured svID absent | Check VLAN membership, multicast filtering, SPAN/TAP and NIC |
59+
| `CONFLICT` | APPID/MAC/VID/PCP/confRev/nofASDU differs | Inspect deployed SCL and exact wire identity; do not normalize away |
60+
| `INCOMPLETE_INTENT` | Some observed fields match but SCL lacks authority | Load the deployed full SCL/profile; do not auto-fill defaults |
61+
| `UNRESOLVED` | No trustworthy SCL svID | Correct/resolve engineering intent before claiming match |
62+
63+
Capture **VLAN absence on a Windows USB-NIC is not proof of untagged wire TX**.
64+
A mirrored `AR_DIAG_SV1`, APPID `0x4F01`, broadcast/untagged stream is diagnostic
65+
evidence only, always separate from the canonical SCL stream.
66+
67+
## Physical bench procedure
68+
69+
1. Record ARStack Injector firmware build/manifest, selected immutable profile,
70+
source identity, configured DataSet order, 4I+4V payload authority, APPID,
71+
multicast MAC, VLAN ID/PCP, confRev, nofASDU, sample cadence, smpCnt modulus
72+
policy and intended `smpSynch`. The SCL is **configured intent**, not
73+
automatically the deployed firmware identity.
74+
2. Connect a trusted independent TAP or SPAN that demonstrably retains VLAN
75+
tags, **without trusting a diagnostic mirror as a canonical substitute**.
76+
Capture timestamp source, precision, NIC/driver version and driver drops.
77+
3. With an ARStack `LAB_SOURCE` release-candidate profile, observe L2 PTPv2,
78+
domain 0, Announce ~1/s, two-step Sync ~4/s, matching Follow_Up and peer
79+
delay response **when requested**. clockClass 248 and clockAccuracy Unknown
80+
do **not** constitute external UTC traceability; AUTO `smpSynch=0` remains
81+
truthful for undisciplined source mode.
82+
4. With the same capture path, observe independent canonical SV and diagnostic
83+
mirror identities and intentionally verify **different** APPID, destination
84+
MAC, VLAN visibility and counters. Check `nofASDU`, channel layout,
85+
instMag/q quality and `smpCnt` continuity for the selected profile.
86+
5. Test nominal **4000 and 4800 samples/s**, including appropriate
87+
profile-validated counter rollover, TEST/bad-quality stimuli, stream restart
88+
and multi-source isolation. A PCAP's host-arrival jitter is
89+
screening only, not on-wire timestamp quality.
90+
6. Run mixed SV/PTP for at least **60 minutes** per acceptance scenario.
91+
A 4000 sample/s stream produces ~14.4m SV frames in 60 minutes if
92+
`nofASDU=1`; canonical plus mirror doubles the SV frames. Ensure
93+
`--max-frames` covers all traffic, or split recordings into explicitly
94+
indexed windows and inspect each.
95+
7. Record capture-driver counters alongside publisher firmware telemetry
96+
(TX failure, missed slots and timestamp quality). A zero sequence-gap result
97+
on the reader does **not** imply an independent NIC dropped zero frames.
98+
8. Independently verify receiving IED/subscriber behavior. A matched stream
99+
in DigSubAnalyzer is not proof the IED subscribed or acted on samples.
100+
101+
## Performance and UX acceptance
102+
103+
While the live analyzer displays mixed traffic, record every 5 seconds:
104+
105+
```powershell
106+
.\\scripts\\measure-m4-bench.ps1 -DurationMinutes 60 -IntervalSeconds 5 -OutputPath C:\\lab\\m4-local-metrics.csv
107+
```
108+
109+
This is a local process CPU/working-set sampler, **not** a substitute for
110+
`dotnet-counters`, native Npcap driver statistics or hardware wire timestamps.
111+
112+
113+
- CPU utilization and process memory/working set;
114+
- .NET allocation rate and GC collections (`dotnet-counters` or equivalent);
115+
- NIC driver/interface drops where supported;
116+
- SV sample/counter anomalies, rejected frames and identity counts;
117+
- selection-to-coherent-snapshot latency and scope redraw responsiveness.
118+
119+
Inspect the main workspace for these user-facing invariants:
120+
121+
- changing selected SV publisher never carries waveform, RMS, phasor or
122+
engineering scaling from the previous publisher;
123+
- TEST and invalid quality are labeled; invalid data never appears as
124+
valid calibrated measurements;
125+
- canonical and mirror traffic remain distinct;
126+
- PTP observation, pairing, domain and freshness are not misrepresented as
127+
servo LOCKED;
128+
- the one-line status remains calm, and detailed raw evidence lives under
129+
Timing/Diagnostics/Advanced, not in an overwhelming initial screen.
130+
131+
There is no synthetic CPU/heap reduction percentage quoted here. Measure
132+
**before/after** on the same hardware/NIC/driver/traffic scenario and record
133+
absolute values and drop counters. Without such measurements, resource
134+
optimization is implementation-level, not performance-certified.
135+
136+
## Release gate and signed-off evidence
137+
138+
The engineering reviewer should retain, locally or in an authorized evidence
139+
store: sanitizer-approved capture checksum, SCL version, firmware manifest,
140+
analyzer commit/build, report JSON, port/TAP diagram and VLAN settings,
141+
NIC/drop counters, 60-minute performance measurements and IED independent
142+
results. Publish only sanitized summaries.
143+
144+
M4 cannot be called **physically complete** without this independently
145+
collected evidence. The automated replay suite is suitable for a software
146+
checkpoint and for detecting compatibility regressions.

0 commit comments

Comments
 (0)