v2026.09.25.004: the Intel Engine loads threat intel; every intel match is report-only

CHANGELOG.md

@@ -1,5 +1,32 @@
 # ShellKnight Changelog
 
+## [v2026.09.25.004] - 2026-09-25
+
+- **The Intel Engine loads threat intel for the first time (critical):** since v1.002 the engine's `Invoke-SafeBlock` read `$Script:Config.IntelEngine_PrimarySource`, which `$Script:Config` did not have; only `$SK_IntelEngine_PrimarySource` existed. Under `Set-StrictMode -Version 2` that threw in the `$consolidated` literal, before any download, cache write or `IntelSource`, and with no cache written the next run took the same path. **Every device on every run reported `intel_source: "Hardcoded fallback"` and 0 hash, filename and C2 IOCs** (Battlefield backtest, 2026-07-03 to 2026-09-25), so the detection engines ran on their hard-coded lists only. The only trace was one INFO line in the log: `Intel Engine skipped  -  The property 'IntelEngine_PrimarySource' cannot be found on this object. Verify that the property exists.` The property is now in `$Script:Config`.
+- **The parser keeps what can match:** it kept each whole trimmed line, so a hash entry was `hash;comment` and a filename entry `regex;score`. No computed hash or file name could ever equal one, so hash and filename intel could not have matched even with the Config fix. The new `ConvertFrom-IntelFeed` follows each file's own header. From `hash-iocs.txt` it keeps the SHA256; the MD5s and SHA1s are dropped, because the scan computes SHA256 only. From `c2-iocs.txt` it keeps the domain or IPv4. From `filename-iocs.txt` it keeps `regex;score[;false-positive regex]` and drops the Unix paths. A line that does not fit is dropped, never guessed at. It trims each line before testing it: a CRLF file's blank lines are `"\r"`, and the old code would have turned them into an empty entry that matched every Run value and every hosts line. On the September 2026 lists it keeps 3,707 Windows filename patterns, 1,260 SHA256s and 1,863 C2 entries.
+- **Filename IOCs are regexes over full paths:** `Find-IntelFilenameMatch` applies them as LOKI does: a case-sensitive regex searched for in the full path, unless the entry's false-positive regex also matches. Before, the consumers compared them with exact names (`Contains($proc.Name)`) or as escaped literal substrings. The regexes are compiled once, with a 250 ms match timeout. Only entries scored 60 or more load (`$SK_IntelEngine_MinFilenameScore`), LOKI's warning level; below it a match is a LOKI "notice". That is 2,184 of the 3,707.
+- **C2 matching by whole labels:** `Find-IntelC2Match` matches a listed domain and its subdomains (`x.evil.example` for `evil.example`, as LOKI's substring test does), and an address only exactly. The hosts file check used unanchored substrings, so `earn.fm` would have matched `learn.fm`. It now checks each address and name on a line, ignoring comments. A C2 name pointed at `0.0.0.0`, loopback or `::`/`::1` is a block that a blocklist added, and is logged as such. The DNS cache check also checks what a name resolved to (a C2 address, or a CNAME to a C2 name).
+- **Every intel match is report-only (fleet safety):** intel has never loaded in the field. With the parser fixed and nothing else changed, a feed match would have killed a process outside Windows and Program Files, removed a Run value, deleted a startup shortcut, or deleted a file in a redirected folder. Each would also have been an IOC: -15 points, exit code 2, and a Critical alert in Battlefield. Even the Config fix alone would have raised IOCs, from C2 substring matches in the hosts file. Every intel match now goes through `Add-IntelHit`, which logs and counts it. The first 50 go into the new payload object `intel.matches`, each with its source (process, Run value, startup shortcut, redirected folder, scanned file, hosts file, DNS cache), target, indicator, score, what a hard-coded match there would do (`would_have`), and, for a file, its SHA256 and Authenticode signer. The first 20 per run become Low findings titled `Intel match (report-only): ...`. None of it is an IOC: not in `ioc_alerts`, the score or the exit code, and no Battlefield alert (Low severity, and the title does not start with "IOC"). Nothing is killed, stopped or deleted. Matches against the hard-coded lists act exactly as before. Intel matches stay report-only until a release's worth of `intel.matches` has been reviewed. Against the September 2026 lists, 3 of a hand-picked 52 common Windows paths match at score 60 or more: `\\tmp\.exe;60`, `\\new\.exe;60` and `\\k7sysmon\.exe;60` (the name of a K7 antivirus component).
+- **Guards against a bad upstream list:** the feed is a third-party GitHub repository, and one bad line would reach every device.
+  - A list over 5 MB, or with under 100 or over 20,000 usable entries, is treated as an error page or the wrong file and is not used.
+  - An entry that matches a known-good value is left out: a filename regex that matches a core Windows binary where Windows keeps it (so `.`, `\\` or `(?i)c:`), the empty-file SHA256, or a top domain such as `microsoft.com`. None of the September 2026 entries does.
+  - A regex that times out is switched off for the rest of the run.
+  - Matching is capped at 3,000 paths and 30 seconds a run. The Detection Engine now scans users' Downloads, Temp and Roaming folders before `C:\Users\Public`, `C:\ProgramData` and `C:\Windows\Temp`, so the cap and the hash scan's first 100 files are spent there.
+- **Cache:** it is trusted only if SYSTEM or Administrators own it. ProgramData lets any local user create a file there and own it, which would let them choose the intel SYSTEM loads, so any other owner's cache is deleted. It must also still parse to 100 or more entries per list; an empty, truncated or corrupt cache no longer passes for current. A modified time in the future counts as stale. The cache is replaced only after all three lists download, so a list that keeps failing never looks current. A failed write is logged, and a cache in the old whole-line format is read correctly. `IntelSource` also reports `Live (Neo23x0, 2 of 3 lists)` and `Cache (download failed)`.
+- **Downloads:** they use `-UseBasicParsing`. Without it, Windows PowerShell 5.1 hands a text response to the Internet Explorer engine, which fails under SYSTEM wherever IE's first-run setup was never completed. The progress bar is off in the block. The hash scan skips hashing files when no hash intel is loaded.
+- **Measured in each report:** the payload's `intel` object has `hits`, `matches`, `paths_checked`, `paths_skipped`, `match_seconds`, `regex_timeouts`, `list_date` and `min_filename_score`, and the log's METRICS SUMMARY carries the same numbers. Phase 1 takes about 0.5 s plus a 0.65 MB download once a week. Filename matching takes about 1.3 ms a path on PowerShell 7 on Apple Silicon. Windows PowerShell 5.1 is expected to be several times slower, which is what the 30-second cap bounds; the first Windows run should read `match_seconds`.
+- **Known limits (misses only, never actions):** matching is case-sensitive as LOKI's is, and `Win32_Process` often reports `C:\WINDOWS\...`, so some process paths will not match. Hash scores are not used; the 86 SHA256 entries scored 55 or 60 (vulnerable libraries and drivers) are `.jar` and `.sys` files, which the hash scan does not hash.
+- **Regression test:** new `tests/Test-IntelEngine.ps1` runs Phase 1 verbatim under StrictMode 2, with `Invoke-WebRequest` mocked to serve lists in the real Neo23x0 formats, across 17 scenarios. They cover:
+  - a fresh download, and CRLF and whitespace lines;
+  - a current, aged, future-dated, user-owned, empty, `{}`, corrupt, partial and legacy cache;
+  - one and all lists failing;
+  - an error page, over 20,000 entries and over 5 MB;
+  - a disabled engine.
+
+  It asserts `IntelSource`, the loaded counts and what was left out, the cache, and `-UseBasicParsing`. It tests both matchers, including timeouts and the caps, and `Add-IntelHit`'s evidence and caps. It runs every intel consumer verbatim against mocked cmdlets, and asserts each match reported (kind, source, target, would_have), each action taken, and the IOC count. It also checks the whole script's AST for any `$Script:Config.<Name>` that the Config literal does not define.
+
+  With the Config fix reverted it fails 22 assertions, and the AST check names the line. Of 23 mutations to the new code, it catches all but one, which the code's other guards make harmless.
+
 ## [v2026.09.25.003] - 2026-09-25
 
 - **OS end of life is Microsoft's date for the build and the edition:** the Assessment Engine looked up `os_eol` by build number only, with one date per build, and several dates were years past Microsoft's. 19045 (Windows 10 22H2) read 2030-10-14 for 2025-10-14; 22621 and 22631 (Windows 11 22H2 and 23H2) read 2027-10-12 and 2028-10-10, later than even their Enterprise dates; 26100 read 2029-10-14. One date per build also cannot be right: Home/Pro and Enterprise/Education reach end of servicing on different days, and 14393, 17763, 19044 and 26100 are also LTSB/LTSC releases or Windows Server 2016/2019/2025, which run for years longer. The new `Get-OsEolDate` takes the edition family from `Win32_OperatingSystem.Caption` (Home/Pro, Enterprise/Education, LTSB/LTSC, IoT Enterprise LTSC, Server) and holds every date from Microsoft Learn's release-health and lifecycle pages. A caption it cannot place, such as a localized one, gets a date only when that date holds for every edition the machine could be; otherwise `os_eol` is `Unknown`, which is not scored (ADR 0009). New builds: 25398 (Server 23H2), 26200 (Windows 11 25H2) and 28000 (Windows 11 26H1). `os_eol` keeps its three forms, so Battlefield needs no change.

CONTEXT.md

@@ -61,6 +61,17 @@ Exactly one of:
 
 A Finding Class is a property of the finding *type*, not of the host it was found on.
 
+### Intel Match
+
+Something on a device that matches the threat-intel feed the Intel Engine downloads (Neo23x0
+signature-base: filename regexes, SHA256 hashes, C2 domains and addresses). It can be a process,
+a Run value, a startup shortcut, a file, a hosts file entry or a DNS cache entry. Report-only:
+it is logged and counted in the Run Report's `intel` object, whose `matches` hold the first 50
+with evidence; the first 20 in a Run are also Low findings titled
+`Intel match (report-only): ...`. It is NOT an IOC alert: it does not count in `ioc_alerts` or
+the Device Security Score, it raises no Battlefield alert, and nothing is killed or removed
+because of it. A match against ShellKnight's own hard-coded lists is an IOC, handled as before.
+
 ### Device Security Score
 
 The per-device score ShellKnight computes during a Run, 0 to 100, published on the Fleet Grid as