From 34de16e9727f9d39feba52ded486efaf61372b3b Mon Sep 17 00:00:00 2001 From: cdburgess75 <508435+cdburgess75@users.noreply.github.com> Date: Fri, 25 Sep 2026 04:32:07 -0500 Subject: [PATCH] v2026.09.25.001: engine results reach the payload and the score Invoke-SafeBlock runs its block with & $Block, which is a child scope. The Assessment Engine set $avProduct, $edrProduct, $defStatus, $bitlockerWarn, $osEolWarn and $wuLastWarn with bare assignments, so each one made a local that was discarded when the block returned. Since v1.002 the payload has reported antivirus 'NONE DETECTED', edr 'None detected' and defender 'Unknown' on every device, and the BitLocker, OS EOL and Windows Update penalties have never applied. MachineInfo, built inside the block, was right throughout. The payload now reads antivirus, edr and defender from MachineInfo, so they are null when the engine did not run. The three warn flags are $Script:-scoped. The script-level defaults are gone, and $avProduct is assigned on every branch. Scoring: BitLocker off (-15), an end-of-life build (-20) and no Windows Update install in over 30 days (-15) now apply for the first time. Each flag is set only by a positive detection. The Defender DISABLED rule (-20) is removed rather than switched on. Live, it would take 20 from every box whose third-party AV turns Defender off, and on a box with no AV it would count twice alongside the -25 rule. The same bug kept two Persistence Engine counters at zero, so their "none found" summary lines appeared even after a removal. This is log text only. tests/Test-EngineScope.ps1 runs Phase 2, the scoring and the payload fields verbatim under StrictMode 2 with mocks, across 13 scenarios. It also AST-checks the whole script for any variable set bare inside an Invoke-SafeBlock and read outside it. It fails 41 assertions against v2026.09.24.001 and passes here. It is a mock test, not a Windows run. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 8 + ShellKnight.ps1 | 112 ++++++++----- tests/Test-EngineScope.ps1 | 318 +++++++++++++++++++++++++++++++++++++ 3 files changed, 402 insertions(+), 36 deletions(-) create mode 100644 tests/Test-EngineScope.ps1 diff --git a/CHANGELOG.md b/CHANGELOG.md index bfe9703..cd35cc0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,13 @@ # ShellKnight Changelog +## [v2026.09.25.001] - 2026-09-25 + +- **Assessment Engine results now reach the report and the score (critical):** `Invoke-SafeBlock` runs its block with `& $Block`, which is a child scope. The engine set `$avProduct`, `$edrProduct`, `$defStatus`, `$bitlockerWarn`, `$osEolWarn` and `$wuLastWarn` without a `$Script:` prefix, so each assignment made a local copy that was discarded when the block returned. The payload and the scoring read the script-level defaults instead, and have done since v1.002. **Every device reported `antivirus: "NONE DETECTED"`, `edr: "None detected"` and `defender: "Unknown"`**, and the BitLocker, OS end-of-life and Windows Update penalties never applied. `MachineInfo` and the log had the real values throughout. The payload now reads `antivirus`, `edr` and `defender` from `MachineInfo`, like the other machine fields, so they are null when the engine did not run instead of a default reported as fact. The three warn flags are `$Script:`-scoped. The three script-level defaults are gone: those values are now local to the engine, and `$avProduct` is assigned on every branch. +- **Scoring change: BitLocker, OS end-of-life and Windows Update now count.** A device loses 15 points if BitLocker is off on C:, 20 if its build is past the end-of-life date in the engine's table, and 15 if the last Windows Update install was over 30 days ago, up to 50 in all. These rules have been in the script since v1.002 but never fired. Each flag is set only by a positive detection; a probe that fails or returns nothing leaves it off, so a collection failure cannot lower a grade (ADR 0009). **Expect grades to drop fleet-wide on the first run of this version.** That is a measurement correction; nothing changed on the endpoints. +- **Scoring change: the Defender DISABLED rule (-20) is removed.** It never fired either. Live, it would take 20 points from every box whose third-party AV has turned Defender off, which Windows does by design, and on a box with no AV at all it would stack with the -25 "no active AV" rule. That -25 rule already scores a missing AV, and scores it once. +- **Persistence Engine summary lines:** the same bug made "no malware Run keys found" and "no browser policy hijacks found" appear in the log even after a removal, because their counters were incremented inside an `Invoke-SafeBlock`. The counters are now `$Script:`-scoped. This affects log text only: `run_keys_removed`, `ioc_alerts` and the score were already right. +- **Regression test:** `tests/Test-EngineScope.ps1` runs the Phase 2 code, the security scoring and the payload's machine fields verbatim, under StrictMode 2 with mocked Windows cmdlets. It covers BitLocker off (both probes), an end-of-life build, a stale Windows Update, all three together, third-party AV with Defender off, an EDR, no AV at all, and an engine that aborts or is disabled. It also checks the whole script, via the AST, for a variable assigned bare inside an `Invoke-SafeBlock` and then read outside it, which is this bug in general form. It fails against v2026.09.24.001 and passes here. + ## [v2026.09.24.001] - 2026-09-24 - **Check-ins restored: device identity no longer depends on the Assessment Engine (critical):** v2026.09.08.001 replaced the Defender catch that set `$defSigs = 'Unknown'` with fallbacks that set it only on success. Where every probe fails (`Get-MpComputerStatus` throws under SYSTEM, and `MSFT_MpComputerStatus` is missing or has no signature date because a third-party AV owns the box or Defender has been removed), reading the unset `$defSigs` in the `MachineInfo` literal threw under `Set-StrictMode -Version 2`. `Invoke-SafeBlock` logged it and moved on, `MachineInfo` stayed empty, and **`device_id` was sent as null**. Battlefield fell back to `host:`, which frozen enrollment does not recognise for a device enrolled by hardware UUID, so every POST was answered `200 {"status":"ignored"}` and nothing was stored. This is very likely the 2026-09-09 reporting drop that v2026.09.15.001 could not explain. `$defSigs`, and `$wuStr` (unset on an empty Windows Update history), now start as `'Unknown'`. Device identity (hardware UUID, then MachineGuid, then `host:`; same values as before) is now computed in its own block ahead of the engine and regardless of `AssessmentEngine_Enabled`. The result, `$Script:DeviceId`, starts at the hostname fallback so it is never null, and both `MachineInfo['Device ID']` and the payload `device_id` read it. An engine failure now costs machine details, never the check-in. diff --git a/ShellKnight.ps1 b/ShellKnight.ps1 index facb452..7442e07 100644 --- a/ShellKnight.ps1 +++ b/ShellKnight.ps1 @@ -2,7 +2,7 @@ #Requires -RunAsAdministrator <# .SYNOPSIS - ShellKnight v2026.09.24.001 - Enterprise Endpoint Security & Remediation Tool + ShellKnight v2026.09.25.001 - Enterprise Endpoint Security & Remediation Tool .DESCRIPTION Automated endpoint security remediation, threat detection, hardening, and @@ -18,9 +18,9 @@ C. David Burgess - PTech LLC .VERSION - Version : v2026.09.24.001 - Released : 2026-09-24 - Prior : v2026.09.15.001 + Version : v2026.09.25.001 + Released : 2026-09-25 + Prior : v2026.09.24.001 .ENGINES Phase 1 - Intel Engine : Threat intelligence download and cache @@ -33,6 +33,33 @@ Phase 8 - Reporting Engine : Reporting, trending, and extended checks .CHANGELOG + v2026.09.25.001 - Assessment Engine results now reach the report and the + score. Invoke-SafeBlock runs its block as a child scope + (& $Block). The engine set $avProduct, $edrProduct, $defStatus, + $bitlockerWarn, $osEolWarn and $wuLastWarn with bare + assignments, and each one made a local copy that was discarded + when the block returned. So the payload and the scoring saw the + script-level defaults on every run since v1.002: antivirus + 'NONE DETECTED', edr 'None detected' and defender 'Unknown' on + every device, and the BitLocker (-15), OS EOL (-20) and Windows + Update (-15) penalties never applied. MachineInfo and the log + were right all along. The payload now reads antivirus, edr and + defender from MachineInfo, so when the engine did not run they + are null rather than a default reported as fact. The three flags + are $Script:-scoped. + SCORING CHANGE. Devices with BitLocker off, an end-of-life build, + or no Windows Update install in over 30 days lose 15, 20 and 15 + points respectively, up to 50 in all. Each flag is set only by a + positive detection, so a probe that fails costs nothing. The + Defender DISABLED rule (-20) is removed rather than switched on. + Live, it would hit every box whose third-party AV turns Defender + off, and on a box with no AV it would stack with the -25 no-AV + rule. Expect grades to drop on the first run of this version. + That is a measurement correction; nothing changed on the + endpoints. + The same bug made the Persistence Engine log "no malware Run keys + found" and "no browser policy hijacks found" even after a + removal. Those counters are $Script:-scoped too (log text only). v2026.09.24.001 - Check-ins restored; two silent field failures fixed. (1) Devices "ignored" by Battlefield. v2026.09.08.001 dropped the Defender catch that set $defSigs = 'Unknown', so where every probe @@ -431,7 +458,7 @@ param() # ============================================================================== -# SHELLKNIGHT v2026.09.24.001 CONFIGURATION +# SHELLKNIGHT v2026.09.25.001 CONFIGURATION # All settings are configured here. No external config files required. # Each engine can be independently enabled or disabled. # ============================================================================== @@ -624,7 +651,7 @@ try { # Runtime Config Object - single source of truth for all engines $Script:Config = [PSCustomObject]@{ - Version = 'v2026.09.24.001' + Version = 'v2026.09.25.001' # Intel Engine IntelEngine_Enabled = $SK_IntelEngine_Enabled IntelEngine_CheckUpdates = $SK_IntelEngine_CheckForUpdates @@ -1014,7 +1041,7 @@ $Script:UseNewPSFeatures = $Script:PSVer -ge 5 # Banner $bannerWidth = 78 -$version = 'ShellKnight v2026.09.24.001' +$version = 'ShellKnight v2026.09.25.001' $hostname = $env:COMPUTERNAME $timestamp = Get-Date -Format 'yyyy-MM-dd HH:mm:ss' $psver = "PS $($PSVersionTable.PSVersion.Major).$($PSVersionTable.PSVersion.Minor)" @@ -1208,12 +1235,15 @@ Write-PhaseProgress -PhaseNum 2 -PhaseName 'Assessment Engine' Log-Info '--- Phase 2: Assessment Engine ---' $Script:MachineInfo = [ordered]@{} -$bitlockerWarn = $false -$osEolWarn = $false -$wuLastWarn = $false -$avProduct = 'NONE DETECTED' -$edrProduct = 'None detected' -$defStatus = 'Unknown' +# Read by the scoring after the engine, so the engine sets them with $Script:. +# Invoke-SafeBlock runs the engine as a child scope (& $Block), where a bare +# assignment makes a local that is gone when the block returns. Until +# v2026.09.25.001 these, and the payload's antivirus/edr/defender, were bare +# script-level variables, so the scoring and the report only ever saw their +# defaults. The payload now reads those three from MachineInfo. +$Script:BitLockerWarn = $false +$Script:OsEolWarn = $false +$Script:WuLastWarn = $false # Scored separately from $avProduct so a *failed* detection can never be scored # as "unprotected" - that mistake has cost the whole fleet 25 points twice now # (v2026.07.30.001's aborted engine, and the Defender-excluded-from-AV bug). @@ -1286,12 +1316,12 @@ if ($Script:Config.AssessmentEngine_Enabled) { try { $bl = Get-BitLockerVolume -MountPoint 'C:' -ErrorAction Stop $blStatus = $bl.ProtectionStatus - if ($blStatus -ne 'On') { $bitlockerWarn = $true; $blStatus = 'Off' } else { $blStatus = 'On' } + if ($blStatus -ne 'On') { $Script:BitLockerWarn = $true; $blStatus = 'Off' } else { $blStatus = 'On' } } catch { try { $blWmi = Get-CimInstance -Namespace 'Root\CIMV2\Security\MicrosoftVolumeEncryption' ` -ClassName 'Win32_EncryptableVolume' -Filter "DriveLetter='C:'" -ErrorAction Stop - $blStatus = if ($blWmi.ProtectionStatus -eq 1) { 'On' } else { 'Off'; $bitlockerWarn = $true } + $blStatus = if ($blWmi.ProtectionStatus -eq 1) { 'On' } else { 'Off'; $Script:BitLockerWarn = $true } } catch { } } @@ -1311,7 +1341,7 @@ if ($Script:Config.AssessmentEngine_Enabled) { } $eolDate = $eolDates[$osBuild] $eolStr = if ($eolDate) { - if ((Get-Date) -gt $eolDate) { $osEolWarn = $true; "END OF LIFE (since $($eolDate.ToString('yyyy-MM-dd')))"} + if ((Get-Date) -gt $eolDate) { $Script:OsEolWarn = $true; "END OF LIFE (since $($eolDate.ToString('yyyy-MM-dd')))"} else { "Supported until $($eolDate.ToString('yyyy-MM-dd'))" } } else { 'Unknown' } @@ -1413,9 +1443,12 @@ if ($Script:Config.AssessmentEngine_Enabled) { # Resolve the reported AV product and whether the box is actually # protected. Defender registered with Security Center counts as # protection unless we positively know real-time protection is off. + # Every branch assigns: $avProduct is local to the engine now, so an + # unassigned one would throw in the MachineInfo literal (StrictMode). if ($avProducts.Count -gt 0) { $avProduct = $avProducts -join ', ' } elseif ($defStatus -eq 'Active') { $avProduct = 'Windows Defender' } elseif ($defenderRegistered) { $avProduct = "Windows Defender (status $defStatus)" } + else { $avProduct = 'NONE DETECTED' } $Script:HasActiveAv = ($avProducts.Count -gt 0) -or ($defStatus -eq 'Active') -or ($defenderRegistered -and $defStatus -ne 'DISABLED') $Script:AvDetectionRan = $true @@ -1432,7 +1465,7 @@ if ($Script:Config.AssessmentEngine_Enabled) { $wuDate = $history.Item(0).Date $wuDaysAgo = ([datetime]::Now - $wuDate).Days $wuStr = "$($wuDate.ToString('yyyy-MM-dd')) ($wuDaysAgo days ago)" - if ($wuDaysAgo -gt 30) { $wuLastWarn = $true } + if ($wuDaysAgo -gt 30) { $Script:WuLastWarn = $true } } } catch { $wuStr = 'Unknown' } @@ -1496,13 +1529,13 @@ if ($Script:Config.AssessmentEngine_Enabled) { # Screen summary Write-Host " Hostname: $($env:COMPUTERNAME) | OS: $osName | RAM: $ramGB GB | Disk: $diskFreeGB GB free" -ForegroundColor White - if ($osEolWarn) { Log-Warn "OS EOL: $eolStr" } - if ($bitlockerWarn){ + if ($Script:OsEolWarn) { Log-Warn "OS EOL: $eolStr" } + if ($Script:BitLockerWarn) { Log-Warn "BitLocker: C: drive is NOT encrypted" Add-Finding -Severity Medium -Title 'BitLocker not enabled on C:' -Action 'Enable BitLocker (required for HIPAA/CJIS clients; escrow recovery key in AD/RMM)' } if ($pcAgeYrs -gt 5){ Log-Warn "Aging hardware: PC is $pcAgeYrs years (BIOS: $($biosDate.ToString('yyyy-MM-dd')))" } - if ($wuLastWarn) { Log-Warn "Windows Update: last install was $wuDaysAgo days ago" } + if ($Script:WuLastWarn) { Log-Warn "Windows Update: last install was $wuDaysAgo days ago" } # Hyper-V detection Invoke-SafeBlock -Label 'Hyper-V detection' -Block { @@ -1928,7 +1961,7 @@ if ($Script:Config.PersistenceEngine_Enabled) { 'HKLM:\SOFTWARE\WOW6432Node\Microsoft\Windows\CurrentVersion\Run' ) - $runKeysRemoved = 0 + $Script:RunKeysFound = 0 # $Script: - incremented inside Invoke-SafeBlock (child scope) foreach ($keyPath in $runKeyPaths) { if (-not (Test-Path $keyPath)) { continue } Invoke-SafeBlock -Label "Run key $keyPath" -Block { @@ -1944,14 +1977,14 @@ if ($Script:Config.PersistenceEngine_Enabled) { Log-Success "Removed Run key: $name" $Script:Counters.RunKeysRemoved++ $Script:Counters.IOCsFound++ - $runKeysRemoved++ + $Script:RunKeysFound++ } else { Log-Info " [RUN] $name = $val" } } } } - if ($runKeysRemoved -eq 0) { Log-Summary "Persistence Engine - no malware Run keys found" } + if ($Script:RunKeysFound -eq 0) { Log-Summary "Persistence Engine - no malware Run keys found" } # Per-user Run / RunOnce keys via HKEY_USERS. # Running as SYSTEM, HKCU above is SYSTEM's own hive - real users' Run keys @@ -2054,7 +2087,7 @@ if ($Script:Config.PersistenceEngine_Enabled) { 'HKCU:\SOFTWARE\Policies\Google\Chrome', 'HKCU:\SOFTWARE\Policies\Microsoft\Edge' ) - $policyRemoved = 0 + $Script:PolicyRemoved = 0 # $Script: - incremented inside Invoke-SafeBlock (child scope) foreach ($policyPath in $browserPolicyPaths) { if (-not (Test-Path $policyPath)) { continue } Invoke-SafeBlock -Label "Browser policy $policyPath" -Block { @@ -2064,12 +2097,12 @@ if ($Script:Config.PersistenceEngine_Enabled) { Log-IOC "Suspicious browser policy: $($_.Name) = $($_.Value)" Remove-ItemProperty -Path $policyPath -Name $_.Name -Force -ErrorAction SilentlyContinue Log-Success "Removed browser policy key: $($_.Name)" - $policyRemoved++ + $Script:PolicyRemoved++ $Script:Counters.IOCsFound++ } } } - if ($policyRemoved -eq 0) { Log-Summary "Persistence Engine - no browser policy hijacks found" } + if ($Script:PolicyRemoved -eq 0) { Log-Summary "Persistence Engine - no browser policy hijacks found" } # Defender exclusion audit Invoke-SafeBlock -Label 'Defender exclusions' -Block { @@ -3268,10 +3301,17 @@ $Script:SecurityScore = 100 if ($Script:Counters.IOCsFound -gt 0) { $Script:SecurityScore -= [math]::Min(50, $Script:Counters.IOCsFound * 15) } if ($Script:Counters.Failed) { $Script:SecurityScore -= 10 } if ($Script:AvDetectionRan -and -not $Script:HasActiveAv) { $Script:SecurityScore -= 25 } -if ($defStatus -eq 'DISABLED') { $Script:SecurityScore -= 20 } -if ($osEolWarn) { $Script:SecurityScore -= 20 } -if ($bitlockerWarn) { $Script:SecurityScore -= 15 } -if ($wuLastWarn) { $Script:SecurityScore -= 15 } +# Defender DISABLED is not scored on its own. That rule (-20) was here from +# v1.002 but never fired, because it read a script-level $defStatus the engine +# never wrote to (see Phase 2). If it were live it would take 20 from every box +# whose third-party AV has turned Defender off, which Windows does by design. +# On a box with no AV at all it would stack on the -25 above. "No working AV" +# is scored once, above. +# These three are live from v2026.09.25.001. Each is set only by a positive +# detection; a probe that fails leaves it $false. +if ($Script:OsEolWarn) { $Script:SecurityScore -= 20 } +if ($Script:BitLockerWarn) { $Script:SecurityScore -= 15 } +if ($Script:WuLastWarn) { $Script:SecurityScore -= 15 } if ($inactiveAccounts.Count -gt 0) { $Script:SecurityScore -= [math]::Min(15, $inactiveAccounts.Count * 5) } try { $smb1Sc = Get-SmbServerConfiguration -ErrorAction Stop | Select-Object -ExpandProperty EnableSMB1Protocol if ($smb1Sc) { $Script:SecurityScore -= 20 } } catch { } @@ -3328,7 +3368,7 @@ $freeAfterGB = if ($diskAfter) { [math]::Round($diskAfter.FreeSpace / 1GB, 1) } $sepLine = '=' * 80 Log-Info $sepLine -Log-Info " ShellKnight v2026.09.24.001 - Report" +Log-Info " ShellKnight v2026.09.25.001 - Report" Log-Info " Hostname : $($env:COMPUTERNAME)" Log-Info " Run Date : $(Get-Date -Format 'yyyy-MM-dd HH:mm:ss')" Log-Info " Runtime : $runtime seconds" @@ -3341,7 +3381,7 @@ Log-Info $sepLine $bannerWidth2 = 78 Write-Host '' Write-Host " $sepLine" -ForegroundColor Cyan -Write-Host " ShellKnight v2026.09.24.001 - Report" -ForegroundColor Cyan +Write-Host " ShellKnight v2026.09.25.001 - Report" -ForegroundColor Cyan Write-Host " Hostname : $($env:COMPUTERNAME)" -ForegroundColor White Write-Host " Run Date : $(Get-Date -Format 'yyyy-MM-dd HH:mm:ss')" -ForegroundColor White Write-Host " Runtime : $runtime seconds" -ForegroundColor White @@ -3613,7 +3653,7 @@ $jsonStamp= Get-Date -Format 'yyyy-MM-dd_HHmm' $jsonPath = "$jsonDir\ShellKnight_${jsonStamp}_$($env:COMPUTERNAME).json" $jsonData = [ordered]@{ - version = 'v2026.09.24.001' + version = 'v2026.09.25.001' device_id = $Script:DeviceId hardware_type = $Script:MachineInfo['Hardware Type'] site_name = $SK_SiteName @@ -3633,9 +3673,9 @@ $jsonData = [ordered]@{ disk_free_gb = $freeGB disk_free_after = $freeAfterGB bitlocker = $Script:MachineInfo['BitLocker'] - antivirus = $avProduct - edr = $edrProduct - defender = $defStatus + antivirus = $Script:MachineInfo['Antivirus'] + edr = $Script:MachineInfo['EDR'] + defender = $Script:MachineInfo['Defender'] defender_sigs = $Script:MachineInfo['Defender Sigs'] last_wu_install = $Script:MachineInfo['Last WU Install'] domain = $Script:MachineInfo['Domain/Workgroup'] diff --git a/tests/Test-EngineScope.ps1 b/tests/Test-EngineScope.ps1 new file mode 100644 index 0000000..ad344bc --- /dev/null +++ b/tests/Test-EngineScope.ps1 @@ -0,0 +1,318 @@ +<# +.SYNOPSIS + Regression test: what the Assessment Engine detects reaches the payload and + the security score. + +.DESCRIPTION + Invoke-SafeBlock runs its block with '& $Block', which is a child scope. Up + to v2026.09.24.001 the engine set $avProduct, $edrProduct, $defStatus, + $bitlockerWarn, $osEolWarn and $wuLastWarn with bare assignments. Each one + made a local copy that was discarded when the block returned, so the + payload reported antivirus 'NONE DETECTED', edr 'None detected' and + defender 'Unknown' on every device, and the BitLocker, OS EOL and Windows + Update penalties never applied. MachineInfo, built inside the block, was + right the whole time, which is why nobody noticed from the log. + + This runs the whole of Phase 2, the security scoring, and the payload's + machine fields, all verbatim from ShellKnight.ps1. They run under the + script's own StrictMode 2 / SilentlyContinue settings, with the Windows + cmdlets replaced by mocks, so the test runs on the CI Linux runner. It does + not replace a real Windows run. + + It also parses the whole script and fails on the general form of the bug: + a variable assigned bare inside an Invoke-SafeBlock body and then read + somewhere that body does not enclose. + + ShellKnight.ps1 is a monolith that executes on load, so the code is + extracted textually rather than dot-sourced. +#> +Set-StrictMode -Version 2 +# The test's own logic stops on any error, so a broken assertion fails loudly +# instead of being skipped. Only the extracted ShellKnight code runs under the +# script's own 'SilentlyContinue' (see the scenario loop). +$ErrorActionPreference = 'Stop' + +$scriptPath = Join-Path (Split-Path $PSScriptRoot -Parent) 'ShellKnight.ps1' +$source = Get-Content -LiteralPath $scriptPath -Raw + +function Get-Section { + param([string]$Pattern, [string]$What) + $m = [regex]::Match($source, $Pattern) + if (-not $m.Success) { throw "$What not found in ShellKnight.ps1 - did it get renamed or moved?" } + $m.Value +} + +$safeBlock = Get-Section '(?ms)^function Invoke-SafeBlock \{.*?^\}' 'Invoke-SafeBlock' +$biosDate = Get-Section '(?ms)^function ConvertTo-BiosDate \{.*?^\}' 'ConvertTo-BiosDate' +# Phase 2 from the MachineInfo reset to the end of the engine's if/else. +$phase2 = Get-Section ('(?ms)^\$Script:MachineInfo = \[ordered\]@\{\}\s*$.*?' + + '^ Log-Info "Assessment Engine - disabled"\s*^\}') 'Phase 2 (the Assessment Engine)' +$scoring = Get-Section ('(?ms)^\$Script:SecurityScore = 100\s*$.*?' + + '^\$Script:SecurityScore = \[math\]::Max\(0, \$Script:SecurityScore\)') 'Security scoring' +# The payload's machine fields, evaluated as a hashtable of their own. +$fields = [regex]::Matches($source, '(?m)^ (bitlocker|os_eol|antivirus|edr|defender)\s+=.*$') +if ($fields.Count -ne 5) { throw "expected 5 payload fields (bitlocker, os_eol, antivirus, edr, defender), found $($fields.Count)" } +$payloadSrc = "[ordered]@{`n" + (($fields | ForEach-Object { $_.Value }) -join "`n") + "`n}" + +# --- Mocks. Functions take precedence over cmdlets of the same name. --------- +function Say { param([string]$m, [string]$c = 'Gray') Microsoft.PowerShell.Utility\Write-Host $m -ForegroundColor $c } +function Write-Host { } +$Script:Logged = New-Object 'System.Collections.Generic.List[string]' +$Script:Findings = New-Object 'System.Collections.Generic.List[object]' +function Log-Info { param([string]$m) $Script:Logged.Add($m) } +function Log-Warn { param([string]$m) } +function Log-Summary { param([string]$m) } +function Add-Finding { param($Severity, $Title, $Action) $Script:Findings.Add([pscustomobject]@{ Severity = $Severity; Title = $Title }) } +$Script:Config = [pscustomobject]@{ AssessmentEngine_Enabled = $true } +$Script:Counters = @{ IntelSource = 'test'; IOCsFound = 0; Failed = 0 } +$Script:HWInfo = @{ IsServer = $false; IsHyperVHost = $false } +$Script:PSFullVer = '5.1.22621.5697' +$origComputerName = $env:COMPUTERNAME # process-wide: restored at the end +$env:COMPUTERNAME = 'SK-TEST-PC' + +# The scenario being run. Every mock reads it. +$Script:S = $null + +function Get-CimInstance { + param($ClassName, $Namespace, $Filter, $ErrorAction) + $s = $Script:S + switch ($ClassName) { + 'Win32_OperatingSystem' { + if ($s.Engine -eq 'aborts') { throw 'Invalid class (WMI repository damaged)' } + return [pscustomobject]@{ Caption = $s.Caption; BuildNumber = $s.Build + OSArchitecture = '64-bit'; LastBootUpTime = (Get-Date).AddDays(-2) } + } + 'Win32_ComputerSystem' { + return [pscustomobject]@{ PartOfDomain = $true; Domain = 'corp.local'; Workgroup = $null + UserName = 'CORP\user'; TotalPhysicalMemory = 17179869184 } + } + 'Win32_BIOS' { return [pscustomobject]@{ ReleaseDate = [datetime]'2023-03-01' } } + 'Win32_LogicalDisk' { return [pscustomobject]@{ FreeSpace = 100GB; Size = 250GB } } + 'AntiVirusProduct' { return @($s.AvList | ForEach-Object { [pscustomobject]@{ displayName = $_ } }) } + 'MSFT_MpComputerStatus' { + if ($s.Defender -eq 'removed') { throw 'Invalid namespace' } + return [pscustomobject]@{ AMServiceEnabled = $true; RealTimeProtectionEnabled = ($s.Defender -eq 'active') + AntivirusSignatureLastUpdated = (Get-Date).AddHours(-5) } + } + 'Win32_ComputerSystemProduct' { return [pscustomobject]@{ UUID = '4C4C4544-0042-5810-8052-B4C04F4B4C33' } } + 'Win32_SystemEnclosure' { return [pscustomobject]@{ ChassisTypes = @(3) } } + 'Win32_EncryptableVolume' { + if ($s.BitLocker -eq 'unavailable') { throw 'Invalid namespace' } + $ps = if ($s.BitLocker -eq 'On') { 1 } else { 0 } + return [pscustomobject]@{ ProtectionStatus = $ps } + } + default { throw "unmocked CIM class $ClassName" } + } +} +function Get-MpComputerStatus { + param($ErrorAction) + if ($Script:S.Defender -eq 'removed') { throw 'Get-MpComputerStatus: module could not be loaded (SYSTEM, -NoProfile)' } + [pscustomobject]@{ AMServiceEnabled = $true; RealTimeProtectionEnabled = ($Script:S.Defender -eq 'active') + AntivirusSignatureLastUpdated = (Get-Date).AddHours(-3) } +} +function Get-Service { + param($Name, $ErrorAction) + if ($Name -eq 'WinDefend') { + if ($Script:S.Defender -eq 'removed') { return $null } + return [pscustomobject]@{ Name = 'WinDefend'; Status = 'Running' } + } + if ($Script:S.Services -contains $Name) { return [pscustomobject]@{ Name = $Name; Status = 'Running' } } + return $null +} +function Get-BitLockerVolume { + param($MountPoint, $ErrorAction) + # 'Off-cim' and 'unavailable': the BitLocker module is not there, so the + # engine falls back to Win32_EncryptableVolume. + if ($Script:S.BitLocker -in 'Off-cim', 'unavailable') { throw "The term 'Get-BitLockerVolume' is not recognized" } + [pscustomobject]@{ ProtectionStatus = $Script:S.BitLocker } +} +function Get-ItemProperty { + param($Path, $Name, $ErrorAction) + if ("$Path" -match 'Cryptography') { return [pscustomobject]@{ MachineGuid = 'b1e2c3d4-0000-1111-2222-333344445555' } } + if ("$Path" -match 'Real-Time Protection') { throw 'Property DisableRealtimeMonitoring does not exist' } + if ("$Path" -match 'Control\\Lsa') { return [pscustomobject]@{ LmCompatibilityLevel = 5 } } + return $null +} +function New-Object { + param($TypeName, $ComObject, $ArgumentList, $ErrorAction) + if ($ComObject) { + $days = $Script:S.WuDays + $hist = [pscustomobject]@{ Count = 1 } + $hist | Add-Member ScriptMethod Item { param($i) [pscustomobject]@{ Date = (Get-Date).AddDays(-$days) } }.GetNewClosure() + $srch = [pscustomobject]@{} + $srch | Add-Member ScriptMethod QueryHistory { param($a, $b) $hist }.GetNewClosure() + $sess = [pscustomobject]@{} + $sess | Add-Member ScriptMethod CreateUpdateSearcher { $srch }.GetNewClosure() + return $sess + } + Microsoft.PowerShell.Utility\New-Object -TypeName $TypeName +} +# The engine's nested checks, and the scoring's own probes, all answer "fine", +# so the only deductions left are the ones under test. +function Get-WindowsOptionalFeature { param([switch]$Online, $FeatureName, $ErrorAction) $null } +function Get-LocalUser { param($Name, $ErrorAction) @() } +function net { 'Minimum password length 14' } +function Get-SmbServerConfiguration { param($ErrorAction) [pscustomobject]@{ EnableSMB1Protocol = $false } } +function Get-NetFirewallProfile { param($ErrorAction) @([pscustomobject]@{ Profile = 'Domain'; Enabled = $true }) } + +Invoke-Expression $safeBlock +Invoke-Expression $biosDate + +# --- Scenarios -------------------------------------------------------------- +# The machine each mock describes, and what the payload and the score must say. +# Penalty: points the four rules under test must take off 100 (AV -25, OS EOL +# -20, BitLocker -15, Windows Update -15). $null for Av/Edr/Def means the +# engine did not run, so the payload has no value to report. +$healthy = @{ Engine = 'runs'; Caption = 'Microsoft Windows 11 Pro'; Build = '22631'; BitLocker = 'On'; WuDays = 6 + AvList = @('Windows Defender'); Defender = 'active'; Services = @() } +function New-Scenario([string]$Name, [hashtable]$Machine, [hashtable]$Expect) { + $m = $healthy.Clone(); foreach ($k in $Machine.Keys) { $m[$k] = $Machine[$k] } + $e = @{ Av = 'Windows Defender'; Edr = 'None detected'; Def = 'Active'; Penalty = 0; Finding = $false } + foreach ($k in $Expect.Keys) { $e[$k] = $Expect[$k] } + $m.Name = $Name; $m.Expect = $e; $m +} +$scenarios = @( + New-Scenario 'healthy' @{} @{} + New-Scenario 'bitlocker-off' @{ BitLocker = 'Off' } @{ Penalty = 15; Finding = $true } + New-Scenario 'bitlocker-off-cim' @{ BitLocker = 'Off-cim' } @{ Penalty = 15; Finding = $true } + # Neither probe answers: unknown, so no penalty (ADR 0009). + New-Scenario 'bitlocker-unavailable' @{ BitLocker = 'unavailable' } @{} + New-Scenario 'os-eol' @{ Caption = 'Microsoft Windows 10 Pro'; Build = '19043' } @{ Penalty = 20 } + New-Scenario 'wu-stale' @{ WuDays = 45 } @{ Penalty = 15 } + New-Scenario 'all-three' @{ BitLocker = 'Off'; Caption = 'Microsoft Windows 10 Pro'; Build = '19043'; WuDays = 45 } @{ Penalty = 50; Finding = $true } + # Windows turns Defender off when a third-party AV registers. Protected: + # no penalty (the old Defender DISABLED rule would have taken 20). + New-Scenario 'third-party-av' @{ AvList = @('Windows Defender', 'Bitdefender Endpoint Security Tools'); Defender = 'off' } @{ Av = 'Bitdefender Endpoint Security Tools'; Def = 'DISABLED' } + New-Scenario 'edr' @{ Services = @('SentinelAgent', 'CSFalconService') } @{ Edr = 'CrowdStrike Falcon, SentinelOne' } + New-Scenario 'no-av' @{ AvList = @(); Defender = 'removed' } @{ Av = 'NONE DETECTED'; Def = 'Unknown'; Penalty = 25 } + # Defender off and nothing else: -25 once, not -25 and -20. + New-Scenario 'defender-off-no-av' @{ Defender = 'off' } @{ Av = 'Windows Defender (status DISABLED)'; Def = 'DISABLED'; Penalty = 25 } + # The engine produced nothing, so none of the four rules may fire. The 20 + # taken here is the password-length rule, which reads 0 when the engine's + # password check never ran. That is an existing flaw and not under test. + New-Scenario 'engine-aborts' @{ Engine = 'aborts'; BitLocker = 'Off'; WuDays = 45 } @{ Av = $null; Edr = $null; Def = $null; Penalty = 20 } + New-Scenario 'engine-disabled' @{ Engine = 'disabled'; BitLocker = 'Off'; WuDays = 45 } @{ Av = $null; Edr = $null; Def = $null; Penalty = 20 } +) + +$failures = 0 +function Fail([string]$Label, [string]$Why) { + Say " FAIL $Label - $Why" Red + $script:failures++ +} +function Show($v) { if ($null -eq $v) { '' } else { "'$v'" } } + +Say '' +Say ' Assessment Engine results reach the payload and the score (StrictMode 2)' +Say ' ------------------------------------------------------------------------' + +foreach ($sc in $scenarios) { + $Script:S = $sc + $x = $sc.Expect + $label = $sc.Name + $before = $failures + $Script:Config.AssessmentEngine_Enabled = ($sc.Engine -ne 'disabled') + $Script:Logged.Clear() + $Script:Findings.Clear() + + $ErrorActionPreference = 'SilentlyContinue' # as ShellKnight.ps1 runs + try { + Invoke-Expression $phase2 + Invoke-Expression $scoring + $payload = Invoke-Expression $payloadSrc + } finally { $ErrorActionPreference = 'Stop' } + + $skipped = @($Script:Logged | Where-Object { $_ -match 'skipped' }) + if ($sc.Engine -eq 'runs' -and $skipped.Count) { Fail $label "a block aborted: $($skipped -join ' | ')" } + if ($sc.Engine -eq 'aborts' -and -not @($skipped | Where-Object { $_ -match '^Assessment Engine skipped' }).Count) { + Fail $label 'expected the engine to abort in this scenario (test harness check)' + } + + foreach ($f in @(@('antivirus', $x.Av), @('edr', $x.Edr), @('defender', $x.Def))) { + $got = $payload[$f[0]] + if ($got -ne $f[1] -or ($null -eq $got) -ne ($null -eq $f[1])) { + Fail $label "payload $($f[0]) = $(Show $got), expected $(Show $f[1])" + } + } + if ($sc.Engine -eq 'runs') { + # The payload's evidence must agree with what was scored. + $wantBl = if ($sc.BitLocker -eq 'On') { 'On' } elseif ($sc.BitLocker -eq 'unavailable') { 'Not available' } else { 'Off' } + if ($payload['bitlocker'] -ne $wantBl) { Fail $label "payload bitlocker = $(Show $payload['bitlocker']), expected '$wantBl'" } + $eol = "$($payload['os_eol'])" -like 'END OF LIFE*' + if ($eol -ne ($sc.Build -eq '19043')) { Fail $label "payload os_eol = $(Show $payload['os_eol'])" } + } + + $score = 100 - $x.Penalty + if ($Script:SecurityScore -ne $score) { Fail $label "security score $($Script:SecurityScore), expected $score" } + + $blFinding = @($Script:Findings | Where-Object { $_.Title -like 'BitLocker not enabled*' }).Count -gt 0 + if ($blFinding -ne $x.Finding) { Fail $label "BitLocker finding present: $blFinding, expected $($x.Finding)" } + + if ($failures -eq $before) { + Say " ok $label - score $($Script:SecurityScore); antivirus=$(Show $payload['antivirus']) edr=$(Show $payload['edr']) defender=$(Show $payload['defender'])" Green + } +} + +# --- Static: no engine result is stranded in a child scope ------------------ +# Invoke-SafeBlock does '& $Block', so a bare '$x = ...' inside its body is a +# local that is gone when the body returns. Flag every read whose most recent +# write (in source order) is such a local, in a body that does not enclose +# the read. A $Script: write reaches everywhere; a $Script: read always sees +# the script-level variable. Code in functions is out of scope: it runs when +# called, not where it sits. +$tokens = $null; $parseErrors = $null +$ast = [System.Management.Automation.Language.Parser]::ParseFile($scriptPath, [ref]$tokens, [ref]$parseErrors) +if ($parseErrors.Count) { throw "ShellKnight.ps1 does not parse: $($parseErrors[0].Message)" } +$L = 'System.Management.Automation.Language' +$bodies = @($ast.FindAll({ param($n) + $n -is [System.Management.Automation.Language.ScriptBlockExpressionAst] -and + $n.Parent -is [System.Management.Automation.Language.CommandAst] -and + $n.Parent.GetCommandName() -eq 'Invoke-SafeBlock' }, $true)) +$functions = @($ast.FindAll({ param($n) $n -is [System.Management.Automation.Language.FunctionDefinitionAst] }, $true)) +function Test-Within($Node, $Outer) { + $Node.Extent.StartOffset -ge $Outer.Extent.StartOffset -and $Node.Extent.EndOffset -le $Outer.Extent.EndOffset +} +function Get-Body($Node) { # innermost Invoke-SafeBlock body holding $Node + $best = $null + foreach ($b in $bodies) { if ((Test-Within $Node $b) -and (-not $best -or (Test-Within $b $best))) { $best = $b } } + $best +} +$ignore = @('null', '_', 'psitem', 'true', 'false', 'matches', 'lastexitcode', 'args', 'input', 'this', 'error', + 'erroractionpreference', 'progresspreference', 'warningpreference', 'verbosepreference', 'confirmpreference') +$writes = @{}; $reads = New-Object 'System.Collections.Generic.List[object]' +foreach ($v in $ast.FindAll({ param($n) $n -is [System.Management.Automation.Language.VariableExpressionAst] }, $true)) { + $vp = $v.VariablePath + if (-not ($vp.IsUnqualified -or $vp.IsScript)) { continue } + $name = ($vp.UserPath -replace '^(?i)script:', '').ToLower() + if ($name -in $ignore) { continue } + if (@($functions | Where-Object { Test-Within $v $_ }).Count) { continue } + $p = $v.Parent + if ($p -is "$L.ConvertExpressionAst" -and $p.Child -eq $v) { $v2 = $p; $p = $p.Parent } else { $v2 = $v } + $isWrite = ($p -is "$L.AssignmentStatementAst" -and $p.Left -eq $v2) -or + ($p -is "$L.ForEachStatementAst" -and $p.Variable -eq $v) -or + ($p -is "$L.UnaryExpressionAst" -and "$($p.TokenKind)" -match 'PlusPlus|MinusMinus') + $body = if ($vp.IsScript) { $null } else { Get-Body $v } + $rec = [pscustomobject]@{ Name = $name; Line = $v.Extent.StartLineNumber; Offset = $v.Extent.StartOffset; Body = $body; Bare = -not $vp.IsScript } + if ($isWrite) { if (-not $writes[$name]) { $writes[$name] = New-Object 'System.Collections.Generic.List[object]' }; $writes[$name].Add($rec) } + else { $reads.Add($rec) } +} +$stranded = New-Object 'System.Collections.Generic.List[string]' +foreach ($r in $reads) { + if (-not $writes[$r.Name]) { continue } + $last = $writes[$r.Name] | Where-Object { $_.Offset -lt $r.Offset } | Select-Object -Last 1 + if (-not $last -or -not $last.Bare -or -not $last.Body) { continue } + $sees = $r.Body -and ($r.Body -eq $last.Body -or (Test-Within $r.Body $last.Body)) + if (-not $sees) { $stranded.Add("`$$($r.Name) read at line $($r.Line), last set at line $($last.Line) inside an Invoke-SafeBlock that does not enclose the read") } +} +if ($stranded.Count) { + foreach ($s in ($stranded | Select-Object -Unique)) { Fail 'scope' $s } +} else { Say ' ok scope - nothing set inside an Invoke-SafeBlock is read from outside it' Green } + +$env:COMPUTERNAME = $origComputerName + +Say '' +if ($failures -gt 0) { + Say " FAILED - $failures assertion(s)" Red + exit 1 +} +Say ' PASS - all assertions' Green +exit 0