Skip to content

v2026.09.24.001: device_id survives the engine; report sent as UTF-8 - #1

Open
cdburgess75 wants to merge 2 commits into
mainfrom
fix/checkin-identity-utf8
Open

cdburgess75 wants to merge 2 commits into
mainfrom
fix/checkin-identity-utf8

Conversation

@cdburgess75

Copy link
Copy Markdown
Owner

Do not merge until the branch has had one real Windows run. This follows the v2026.09.15.001 rule: "A parse is not a test." Merging to main puts this on every endpoint, running as SYSTEM, within about 8 hours. Test steps are below.

Two silent field failures

1. Devices "ignored" since 2026-09-08.

  • v2026.09.08.001 (0f8b292) removed the Defender catch { $defSigs = 'Unknown' }. On a box where every Defender probe fails, $defSigs is never assigned. Typical cases:
    • Get-MpComputerStatus throws under SYSTEM, and MSFT_MpComputerStatus is missing or has no signature date;
    • a third-party AV owns the box;
    • Defender has been removed from Server 2019.
  • Reading $defSigs in the MachineInfo literal then throws under StrictMode 2. The whole Assessment Engine is skipped and device_id goes out null.
  • Battlefield falls back to host:<name>, which a UUID-enrolled device doesn't match. Enrollment is frozen, so the server answers 200 ignored.
  • This affects RLG-HOST1, RLG-DCFS, RLG-JANE-PC, MAN-VPEN-HOST, PCH-LT-JTFCXL3, DESKTOP-T5T7J8S and others.
  • It is reproduced with the real engine code under mocks: origin/main aborts and v2026.07.30.001 does not.

2. 400 body is not valid JSON since 2026-09-17 (RLG Windows 11).

  • Windows PowerShell 5.1 encodes a string -Body as ISO-8859-1 when no charset is given. Any U+0080..U+00FF character becomes an invalid UTF-8 byte.
  • Characters above U+00FF are best-fitted. For example “ ” become ", which breaks the JSON outright.
  • Two machines recovered exactly 7 days after breaking, which fits content from the 7-day Event 7045 window.
  • Sources:
    • PowerShell#18219, which changed the default from ISO-8859-1 to UTF-8 in 7.4;
    • the Full-CLR web cmdlet source.

Changes

  • Device identity has its own Invoke-SafeBlock. It runs ahead of the engine and regardless of AssessmentEngine_Enabled.
    • It computes the same values as before: hardware UUID, then MachineGuid, then host:<name>.
    • $Script:DeviceId starts at host:<name>, so it can never be null.
    • MachineGuid is used only when WMI answers, matching the old behaviour.
  • Missing values no longer abort the engine. $defSigs and $wuStr start as 'Unknown' (an empty Windows Update history also used to abort it).
  • The report is POSTed as UTF-8 bytes with application/json; charset=utf-8.
  • Version bumped to v2026.09.24.001, with entries in both changelogs. Network inventory stays disabled.
  • New tests/Test-DeviceIdentity.ps1 runs the engine code verbatim under StrictMode 2 with mocks across 11 scenarios. It fails 20 assertions against origin/main and passes on this branch; CI runs it. The AST parse shows 0 errors, and no non-ASCII bytes were added.

Review: three independent reviewers, covering PS 5.1/StrictMode, fleet safety and diff correctness. Their follow-ups are in 3aa3256. Known engine hazards that were already there are left for a follow-up: quser, a null LastBootUpTime, and WMI -ErrorAction Stop. None of them can null device_id any more. The child-scope bug is also left: antivirus, edr and defender always report defaults.

Before merging: one real run on an affected machine

Run this on RLG-JANE-PC or RLG-DCFS, in an elevated Windows PowerShell 5.1:

$f = "$env:windir\Temp\ShellKnight-test.ps1"
Invoke-RestMethod 'https://raw.githubusercontent.com/cdburgess75/ShellKnight/fix/checkin-identity-utf8/ShellKnight.ps1' -OutFile $f
schtasks /create /tn SK-Test /tr "powershell.exe -NoProfile -ExecutionPolicy Bypass -File $f" /sc once /st 23:59 /ru SYSTEM /f
schtasks /run /tn SK-Test

When it finishes, check the newest C:\ProgramData\ShellKnight\Logs\ShellKnight_*.log:

  • it has no Device identity skipped and no Assessment Engine skipped line;
  • it has Battlefield push OK - run_id: <n>.

In Battlefield, the device should show a v2026.09.24.001 run under its existing UUID. Then remove the test task with schtasks /delete /tn SK-Test /f.

🤖 Generated with Claude Code

cdburgess75 and others added 2 commits September 24, 2026 20:41
…report sent as UTF-8

Two field failures at one client site, both silent on the endpoint.

Devices "ignored". v2026.09.08.001 dropped the Defender catch that set
$defSigs = 'Unknown'. Where every probe fails (Get-MpComputerStatus throws
under SYSTEM, and MSFT_MpComputerStatus is missing or has no signature date
because a third-party AV owns the box or Defender is removed), reading the
unset $defSigs in the MachineInfo literal threw under StrictMode 2.
Invoke-SafeBlock logged it and moved on, MachineInfo stayed empty, and
device_id went out null. Battlefield fell back to host:<name>, which frozen
enrollment does not know for a UUID-enrolled device, so every POST got
200 "ignored" and nothing was stored. This is very likely the 2026-09-09
reporting drop that v2026.09.15.001 could not explain.

$defSigs, and $wuStr (unset on an empty Windows Update history), now start
as 'Unknown'. Device identity (UUID -> MachineGuid -> host:<name>, same
values) moved out of the engine into its own block that runs first and
regardless of AssessmentEngine_Enabled. $Script:DeviceId starts at the
host:<name> fallback, is never null, and feeds both MachineInfo and the
payload. An engine failure now costs machine details, never the check-in.

HTTP 400 "body is not valid JSON". Windows PowerShell 5.1 encodes a string
-Body as ISO-8859-1 when -ContentType has no charset, so one character in
U+0080..U+00FF (e.g. in an Event 7045 service name) became an invalid UTF-8
byte and the whole report was rejected until the event left the 7-day
window. The report is now POSTed as UTF-8 bytes with
'application/json; charset=utf-8'.

tests/Test-DeviceIdentity.ps1 runs the Phase 2 code verbatim under
StrictMode 2 with mocked cmdlets. It fails on origin/main (engine aborts on
$defSigs and $wuStr, device_id null) and passes here. It is a mock test,
not a Windows run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review follow-ups on v2026.09.24.001:

- Inside the engine, the identity code only ran once Win32_OperatingSystem
  had answered, so with WMI down the report went out as host:<name>. The
  new standalone block would have fallen through to MachineGuid instead,
  re-enrolling a UUID-known machine under a new id that neither adoption
  nor Battlefield's hostname fallback can match. MachineGuid is now used
  only when WMI answers, as before; a wmi-down scenario covers it.
- Changelogs say plainly that passive network inventory stays disabled,
  and CHANGELOG.md points at the in-file history for .09.08.x-.09.15.001.
- The test restores $env:COMPUTERNAME, which is process-wide in CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant