diff --git a/CHANGELOG.md b/CHANGELOG.md index f3afee5..28d28fd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,96 @@ +# **v0.7.6.2 — Import Table Structural Validator, and VERSIONINFO projection** +**Released: 2026‑09‑09** + +## Added +- Import table structural validator: new `pe_imports` parser + (`iocx/parsers/pe_imports.py`) and `imports` validator + (`iocx/validators/imports.py`) emitting `IMPORT_DIRECTORY_INVALID_HEADER`, + `IMPORT_TABLE_TRUNCATED`, `IMPORT_DESCRIPTOR_INVALID`, + `IMPORT_DLL_NAME_INVALID`, `IMPORT_ENTRY_INVALID`. +- Public `version_info` projection (`version_info_projection.py`), + present in every result at every analysis level; closed 8-key + shortlist by default (`CompanyName`, `FileDescription`, `FileVersion`, + `InternalName`, `LegalCopyright`, `OriginalFilename`, `ProductName`, + `ProductVersion`), open key set under `-a full`. +- New reason codes: `RESOURCE_DIRECTORY_ENTRY_UNREADABLE`, + `RESOURCE_STRING_TABLE_UNREADABLE`. +- Branded `--version` CLI output (ASCII art shown only on an interactive + terminal) including `python` and `pefile` dependency versions. +- Static tag-contract CI check (`tests/contract/tag_contract.py`, + `test_tag_contract.py`) statically verifying every parser tombstone + tag has a validator consumer, closing a class of silent-drop bugs. +- `HIGHADJ` relocation-entry pairing support: new `adjustment` field on + paired entries, `highadj_missing_adjustment` tombstone for an unpaired + trailing entry. +- Bounds on resource-tree output (`_MAX_RESOURCE_STRINGS`, + `_MAX_RESOURCE_ENTRIES`, `_MAX_RESOURCE_DEPTH`) and on version-info + blob size (1 MB) / child count (256), so hostile input can no longer + scale parser output or recursion depth. +- New VS_VERSIONINFO test fixture generator and + matching C/RC fixture pair for end-to-end version-info coverage. + +## Changed +- `build_version_info` renamed to `build_version_info_structure`; now + called unconditionally in the engine pipeline instead of only under + `-a full`. +- CLI: `-m/--min-length` moved from Detector Options to Engine Options; + `-a/--analyse` help text now documents what each analysis level + unlocks; `--list-transformers`/`--list-enrichers` help text made + consistent with `--list-detectors`. +- Delay-import and export-forwarder validation split into distinct + empty / non-printable / too-long checks instead of a single boolean, + each with its own reason-code sub-reason. +- `RELOCATION_TABLE_TRUNCATED` region values split into + `relocation_entries_exceed_directory` (declared size clamped to the + directory window) vs. `relocation_entries_truncated` (the clamped + read itself came back short) — previously conflated under one tag. +- `_MAX_ENTRIES_PER_BLOCK` corrected to 8,192 (previously documented and + sized for 2,048) to account for `HIGHADJ` occupying two WORD slots + per relocation. +- Machine-specific relocation type 9 renamed `MIPS_JMPADDR16` → + `MACHINE_SPECIFIC_9` for architecture-neutral naming. + +## Fixed +- `NameError` on `_RELOC_TYPE_HIGHADJ`, reachable from decoding *any* + relocation entry (not only `HIGHADJ` ones), capable of aborting an + entire analysis on a single malformed entry. +- `_parse_data_directories_raw` read PE32+ images at the PE32 (96-byte) + rather than the correct 112-byte `DataDirectory` offset, silently + misreading every data directory on a 64-bit image; now also tolerates + a missing/empty `__data__` and a truncated optional header without + raising. +- Export forwarder-string decode errors were discarded rather than + surfaced on the entry; `name_rva` was hard-coded `None` instead of + being resolved via the name-pointer cross-reference; two name + pointers resolving to the same EAT index silently overwrote one + another (now tagged `ordinal_index_duplicate`). +- Forwarder regex accepted over-long ordinals (e.g. + `Dll.#99999999999`) as ordinary symbol names. +- `pe_resources.build_resource_structure` and `pe_parser._parse_resources` + could propagate an exception, or silently drop entries, on a malformed + subtree; both now tombstone the failure (`entry_decode_failed`, + `directory_entries_unavailable`, `string_table_walk_failed`, + `resources_unavailable`, `resources_map_read_failed`) and never raise. +- Per-index parser tags (`*_unpack_failed_at_{index}`) that never + matched any validator priority list, because the embedded index made + every occurrence unique, deduplicated to a stable form. + +## Documentation +- `docs/specs/reason-codes.md`: new **Import Anomalies** section, new + **Resource Directory Entry Unreadable** section, expanded TLS / + delay-import / export / relocation / debug sub-reason tables. +- `docs/specs/structural-validation-deterministic-heuristics.md`: new + **§2.16 Imports Validator** section; documents the tag-contract check. + +## Testing +- Test suite: **2,136 → 2,802 tests.** Major new coverage for the + import parser/validator, the version-info projection, resource-tree + robustness, relocation `HIGHADJ` handling, and the tag-contract + checker itself (including deliberate regression fixtures for eight + previously-found silent-drop bugs). + +--- + # **v0.7.6.1 — Exception Directory Validator, and a Silent Output Defect** **Released: 2026‑08‑27** diff --git a/README-pypi.md b/README-pypi.md index 46e266a..8e199f8 100644 --- a/README-pypi.md +++ b/README-pypi.md @@ -40,6 +40,17 @@ If you need predictable, automatable IOC extraction — IOCX is built for you. --- +## Version highlights + +### v0.7.6.2 — Import Table Validator + +- New deterministic **import table structural validator** (`IMPORT_*` reason codes). +- `version_info` now parsed and surfaced at **every** analysis level (not just `-a full`), via a new bounded public projection. +- Rebuilt CLI: branded `--version` output, clearer `--help` text, reorganised argument groups. +- Fixed a relocation-parser crash reachable from any entry, a PE32+ data-directory offset bug, and several silent export/resource error drops. +- New static CI check that prevents parser error tags from silently going unconsumed by validators. +- Test suite: 2,136 → 2,802 tests. Coverage: 100%. + ### v0.7.6.1 — Exception Directory Validator - Adds deep semantic validation of the PE exception (`.pdata`) directory; 14 new reason codes; 15 validators total. @@ -47,13 +58,6 @@ If you need predictable, automatable IOC extraction — IOCX is built for you. - **Output-visible:** findings previously suppressed or mislabelled will now appear. - Tests: 1620 → 2136. Coverage: 100%. -## Version highlights (v0.7.6) - -- Added new PE structural validators for relocations and debug directories -- WIN_CERTIFICATE and tls validators now have pefile-independent struct parsers -- Never crashes on malformed input - byte-level parsing with structured error tombstones -- 1620 tests at 100% coverage - deterministic output, snapshot-stable - --- ## **Performance** diff --git a/README.md b/README.md index ed6108f..9d5c64e 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@

- + @@ -202,6 +202,17 @@ Fast path — no PE parsing.

Show Version History
+### v0.7.6.2 — Import Table Validator + +- New deterministic **import table structural validator** (`IMPORT_*` reason codes). +- `version_info` now parsed and surfaced at **every** analysis level (not just `-a full`), via a new bounded public projection. +- Rebuilt CLI: branded `--version` output, clearer `--help` text, reorganised argument groups. +- Fixed a relocation-parser crash reachable from any entry, a PE32+ data-directory offset bug, and several silent export/resource error drops. +- New static CI check that prevents parser error tags from silently going unconsumed by validators. +- Test suite: 2,136 → 2,802 tests. + +--- + ### v0.7.6.1 — Exception Directory Validator - Adds deep semantic validation of the PE exception (`.pdata`) directory; 14 new reason codes; 15 validators total. diff --git a/docs/specs/reason-codes.md b/docs/specs/reason-codes.md index fb25391..6178cd3 100644 --- a/docs/specs/reason-codes.md +++ b/docs/specs/reason-codes.md @@ -1,8 +1,8 @@ # **PE Structural Reason Codes** > **Truncation detail keys are not uniform.** `EXPORT_TABLE_TRUNCATED`, -> `DELAY_IMPORT_TABLE_TRUNCATED` and `EXCEPTION_TABLE_TRUNCATED` name the -> affected sub-table in a **`table`** key. `DEBUG_TABLE_TRUNCATED`, +> `DELAY_IMPORT_TABLE_TRUNCATED`, `IMPORT_TABLE_TRUNCATED` and `EXCEPTION_TABLE_TRUNCATED` +> name the affected sub-table in a **`table`** key. `DEBUG_TABLE_TRUNCATED`, > `RELOCATION_TABLE_TRUNCATED` and `TLS_DIRECTORY_TRUNCATED` use **`region`** > instead. Both are stable; consumers handling truncation generically must > read either. @@ -133,7 +133,6 @@ They are separable by a discriminating key: | **DATA_DIRECTORY_NOT_MAPPED_TO_SECTION** | Directory is in range but does not fall inside any section | RVA = 0x9000, Size = 0x200, no section covers it | Per‑directory *(suppressed for empty, zero‑RVA, zero‑size, out‑of‑range and zero‑length‑section directories)* | | **DATA_DIRECTORY_SPANS_MULTIPLE_SECTIONS** | Directory range overlaps more than one section | RVA = 0x1800, Size = 0x1000 spans .text → .rdata | Per‑directory | | **DATA_DIRECTORY_OVERLAP** | Two directories’ RVA ranges overlap | Import and IAT overlap | Global | -| **IMPORT_RVA_INVALID** *coming soon* | Import RVA does not map to a valid import table structure (import validator) | Import RVA = 0x9000 | Per‑directory | > Prior to the `raw_offset` guard fix, `DATA_DIRECTORY_NOT_MAPPED_TO_SECTION` was additionally suppressed for any directory in a file carrying an overlay, because the raw-mapping guard skipped the section-mapping checks entirely. Files analysed before that fix may under-report it. @@ -145,7 +144,7 @@ They are separable by a discriminating key: |------------|------------------|-----------------|--------| | **TLS_CALLBACK_OUTSIDE_RANGE** | Callback RVA not within the TLS directory’s `(start, end)` range | Callback = `0x5000`, TLS range = `0x4000–0x4100` | Per‑file | | **TLS_MULTIPLE_DIRECTORIES** | More than one TLS directory is present in the PE | Two `tls_directory` entries in `extended` | Per‑file | -| **TLS_INVALID_RANGE** | TLS directory has `start >= end` (structurally impossible) | Start = `0x6000`, End = `0x6000` | Per‑file | +| **TLS_INVALID_RANGE** | TLS directory has `start >= end` (structurally impossible). The parser independently records `tls_raw_data_end_before_start` in its errors list and sets `raw_data_size` to `None` for this case; the validator derives the finding from the VA fields directly rather than from that tag, so the two never double-count. | Start = `0x6000`, End = `0x6000` | Per‑file | | **TLS_ZERO_LENGTH_DIRECTORY** | TLS directory exists but `start == end` (zero‑length region) | Start = `0x7000`, End = `0x7000` | Per‑file | | **TLS_CALLBACKS_MISSING** | TLS directory is non‑empty but callback pointer is `0` | Start = `0x4000`, End = `0x4100`, Callbacks = `0` | Per‑file | | **TLS_CALLBACK_NOT_MAPPED_TO_SECTION** | Callback RVA does not fall inside any section’s VA range | Callback = `0x90000000` (no section covers it) | Per‑file | @@ -162,9 +161,31 @@ They are separable by a discriminating key: | Sub‑reason | Meaning | |------------|---------| -| header_decode | The fixed IMAGE_TLS_DIRECTORY could not be read or unpacked; unrecoverable, all later checks are skipped | +| header_decode | The fixed IMAGE_TLS_DIRECTORY could not be read or unpacked; unrecoverable, all later checks are skipped. The `errors` key lists the parser tags that triggered it | | callback_array | A parser truncation tag surfaced while walking the callback array (the `region` key names the tag) | +#### `header_decode` — contributing parser tags + +Listed in the issue's `errors` key. Any one of these short-circuits every +later TLS check: + +| Parser tag | Meaning | +|------------|---------| +| tls_directory_read_failed | `pe.get_data` raised when reading the fixed struct | +| tls_directory_truncated | The read returned fewer than the full struct size (24 bytes PE32 / 40 bytes PE32+) | +| tls_directory_unpack_failed | `struct.unpack` failed on the struct bytes (defensive; unreachable past the length guard) | + +#### `callback_array` — `region` values + +One issue is emitted per tag, so a single directory may raise several: + +| region value | Meaning | +|--------------|---------| +| tls_callbacks_read_failed | `pe.get_data` raised while reading a callback slot | +| tls_callbacks_truncated | A callback slot returned fewer than the pointer width (4 bytes PE32 / 8 bytes PE32+) | +| tls_callbacks_unpack_failed | `struct.unpack` failed on a slot (defensive; unreachable past the length guard) | +| tls_callbacks_max_exceeded | The walk hit the parser's hard limit (4096 callbacks) without finding a NULL terminator | + ### TLS_CALLBACK_RVA_INVALID Parser resolution tombstones (callback array unresolvable, `callbacks = []`): @@ -183,6 +204,11 @@ always in `invalid_callback_count`): | below_image_base | A callback VA lies below ImageBase, yielding a negative RVA | | not_mapped | A resolved callback RVA falls inside no section | +Emission is capped at 16 per-target issues; `invalid_callback_count` always +carries the true total. The two tombstone sub-reasons above are emitted at +most once each and are mutually exclusive with the per-target list, since the +parser returns `callbacks = []` in both cases. + --- ## **SIGNATURE ANOMALIES** @@ -226,7 +252,7 @@ always in `invalid_callback_count`): |------------|------------------|-----------------|--------| | **RESOURCE_DIRECTORY_OUT_OF_BOUNDS** | A resource directory's `rva + size` does not lie wholly inside the `.rsrc` section. Two cases reach this: the **root** directory lies outside `.rsrc` (`depth` = 0), or a **subdirectory** starts inside `.rsrc` but its extent overflows the end (`depth` ≥ 1). A subdirectory lying wholly outside is reported by the parent as `RESOURCE_ENTRY_OUT_OF_BOUNDS` instead, so the two never double-count. `SizeOfImage` is not consulted — `.rsrc` bounds are authoritative here | Root directory RVA = `0x90000000` while `.rsrc` spans `0x1000–0x3000`; or a Name directory at `0x2FF8` with size 24 | Per‑directory | | **RESOURCE_DIRECTORY_LOOP** | Recursive directory traversal detects a cycle (malformed or malicious resource tree) | Directory A → B → A | Per‑file | -| **RESOURCE_DIRECTORY_ZERO_LENGTH** | A resource directory exists but has zero length or no valid entries | RVA = `0x3000`, size = `0` | Per‑file | +| **RESOURCE_DIRECTORY_ZERO_LENGTH** (*reserved, not emitted*) | A resource directory exists but has zero length or no valid entries | RVA = `0x3000`, size = `0` | Per‑file | ### Resource Hierarchy Anomalies | Reason Code | What Triggers It | Example Pattern | Scope | @@ -241,26 +267,97 @@ always in `invalid_callback_count`): | **RESOURCE_ENTRY_OUT_OF_BOUNDS** | A resource directory entry points to a **subdirectory** whose RVA lies outside the `.rsrc` section. Out-of-bounds *data* entries are reported as `RESOURCE_DATA_OUT_OF_BOUNDS`, not here. The target's own size is not considered at this point — a subdirectory that starts inside `.rsrc` but overflows the end is caught by `RESOURCE_DIRECTORY_OUT_OF_BOUNDS` when it is descended into | Type directory entry points to a Name directory at RVA `0x80000000` | Per‑file | | **RESOURCE_DATA_OUT_OF_BOUNDS** | Resource data block lies outside the file or outside the `.rsrc` section | Data offset = `0x1F0000`, file size = `0x1E0000` | Per‑file | | **RESOURCE_DATA_OVERLAPS_OTHER_DATA** | A resource data blob spans the overlay start, or its raw or virtual extent intersects a section other than `.rsrc`. Blob-versus-blob comparison is **not** performed | Data at raw `0x2000–0x2400` intersects `.text` raw range | Per-file *(one issue per check; the raw-overlap and VA-overlap loops each stop at the first intersecting section, so a blob crossing several sections reports once per check, not once per section)* | + **RESOURCE_DIRECTORY_ENTRY_UNREADABLE** | A directory's entry list, or one entry within it, could not be decoded. The entry is skipped rather than aborting the walk, so the directory reports fewer entries than its declared size implies | An entry that is neither a subdirectory nor a data leaf | Per‑directory *(priority-resolved sub-reason)* | + +#### RESOURCE_DIRECTORY_ENTRY_UNREADABLE + +Priority‑resolved; an unreadable entry *list* subsumes any per-entry failure, +since no entry was reached at all: + +| Sub‑reason | Meaning | +|------------|---------| +| directory_entries_unavailable | The directory's `.entries` was missing or raised; no entry was decoded | +| entry_decode_failed | One or more individual entries were unreadable and skipped; `failed_entry_count` gives the total | + +Details carry `declared_size` and `decoded_entries`; their difference is the loss, and neither conveys it alone. ### Resource Version‑Info Anomalies | Reason Code | What Triggers It | Example Pattern | Scope | |-------------|------------------|-----------------|-------| -| **RESOURCE_VERSIONINFO_INVALID_HEADER** | The VS_VERSIONINFO envelope is malformed: placement outside `.rsrc`, `szKey` not equal to "VS_VERSION_INFO", or `wLength` inconsistent with the buffer size | szKey = "VS_VERSION_BAD" instead of "VS_VERSION_INFO" | Per‑file -| **RESOURCE_VERSIONINFO_INVALID_FIXEDINFO** | The embedded VS_FIXEDFILEINFO has an incorrect `dwSignature` (expected `0xFEEF04BD`) or `dwStrucVersion` (expected `0x00010000`), or fails to parse | dwSignature = `0xDEADBEEF` instead of `0xFEEF04BD` | Per‑file -| **RESOURCE_VERSIONINFO_INVALID_STRINGFILEINFO** | A StringFileInfo, StringTable, or String child is malformed: invalid length field, non‑hex lang_codepage key, or truncated string entry StringTable | key = "ENGLISHX" instead of 8‑hex‑char form | Per‑file -| **RESOURCE_VERSIONINFO_INVALID_VARFILEINFO** | A VarFileInfo or Var child is malformed, or the Translation array's length is not a DWORD multiple Var. | wValueLength = 6 (not divisible by 4) for a Translation array | Per‑file +| **RESOURCE_VERSIONINFO_INVALID_HEADER** | The VS_VERSIONINFO envelope is malformed: placement outside `.rsrc`, `szKey` not equal to "VS_VERSION_INFO", or `wLength` inconsistent with the buffer size | szKey = "VS_VERSION_BAD" instead of "VS_VERSION_INFO" | Per‑file | +| **RESOURCE_VERSIONINFO_INVALID_FIXEDINFO** | The embedded VS_FIXEDFILEINFO has an incorrect `dwSignature` (expected `0xFEEF04BD`) or `dwStrucVersion` (expected `0x00010000`), or fails to parse | dwSignature = `0xDEADBEEF` instead of `0xFEEF04BD` | Per‑file | +| **RESOURCE_VERSIONINFO_INVALID_STRINGFILEINFO** | A StringFileInfo, StringTable, or String child is malformed: invalid length field, non‑hex lang_codepage key, or truncated string entry | StringTable key = "ENGLISHX" instead of 8‑hex‑char `` form | Per‑file / Per‑table | +| **RESOURCE_VERSIONINFO_INVALID_VARFILEINFO** | A VarFileInfo or Var child is malformed, or the Translation array's length is not a DWORD multiple | Var wValueLength = 6 (not divisible by 4) for a Translation array | Per‑file | *Note: absence of an RT_VERSION resource is not treated as a structural anomaly — many legitimate binary types (kernel drivers, MSI helpers, cross‑compiled artefacts) omit version‑info entirely.* +> **Projection flags are not reason codes.** The public `version_info` +> output carries its own `truncated` list — `tables`, `strings`, +> `keys_filtered` — which shapes what is *emitted* rather than describing +> what is *wrong* with the file. A blob that trips all three can still be +> perfectly well-formed and produce no issue here. The two vocabularies +> are disjoint and must not be conflated: +> +> | `truncated` flag | Meaning | Structural? | +> |------------------|---------|-------------| +> | tables | More string tables than the public cap (16); the remainder were not projected | No | +> | strings | More keys in a table than the public cap (64) after key filtering; the remainder were not projected | No | +> | keys_filtered | Keys outside the default closed key set were dropped. Expected on any binary carrying non-shortlist keys, and absent under `full` | No | +> +> Structural truth lives in `structural_error_count` and the +> `RESOURCE_VERSIONINFO_*` codes above; `truncated` records only that the +> public view is narrower than the parsed one. + ### RESOURCE_VERSIONINFO_INVALID_HEADER sub‑reasons +The first four describe the VS_VERSIONINFO envelope itself. The last four +describe the child-dispatch walk and are appended to the top-level `errors` +list *after* `decoded` is set, so they are read by their own branch rather +than by the `undecoded` forward. + | Sub‑reason | Meaning | |------------|---------| -| placement | The VS_VERSIONINFO blob does not lie wholly inside `.rsrc` | -| undecoded | The parser could not decode the envelope; short-circuits the FIXEDINFO / STRINGFILEINFO / VARFILEINFO checks | +| placement | The VS_VERSIONINFO blob was read and decoded, but does not lie wholly inside `.rsrc`. A validator-side comparison of the blob's extent against section bounds — distinct from `leaf_placement_implausible` below, which is a parser-side refusal to read at all | +| undecoded | The parser could not decode the envelope; short-circuits the FIXEDINFO / STRINGFILEINFO / VARFILEINFO checks. The `errors` key lists the contributing parser tags | | szkey_mismatch | `szKey` is not "VS_VERSION_INFO" | | length_inconsistent | `wLength` disagrees with the buffer size | +| child_max_exceeded | The child walk hit the parser's hard limit (256) and stopped. Children beyond that point were not examined | +| child_header_unpack | A child's 6-byte header could not be unpacked; the walk stopped there | +| child_length_invalid | A child's `wLength` was below the 6-byte minimum or extended past the envelope; the walk stopped there | +| unknown_child | A child whose `szKey` is neither "StringFileInfo" nor "VarFileInfo". Does **not** stop the walk, so it may repeat — the issue's `errors` key carries every occurrence, bounded by the child cap | + +The four child sub-reasons are priority-resolved, in the order listed: +a fault that terminated the walk outranks one that did not, because the +remaining children were never examined. At most one issue is emitted per +blob, with the full matching set carried in `errors`. + +#### `undecoded` — contributing parser tags + +Listed in the issue's `errors` key. Any one of these means no VS_VERSIONINFO +envelope was decoded, so the FIXEDINFO / STRINGFILEINFO / VARFILEINFO checks +are skipped: + +| Parser tag | Meaning | +|------------|---------| +| leaf_struct_unpack | The RT_VERSION leaf's `OffsetToData` / `Size` could not be read from the resource entry | +| leaf_placement_implausible | The leaf's declared placement is structurally impossible: a negative RVA, a size of zero or less, or a size exceeding the 1 MB blob cap. The blob is **not** read — the guard precedes `pe.get_data` — so an attacker-controlled `Size` is bounded before any allocation occurs | +| read_failed | `pe.get_data` raised when reading the blob at a placement that passed the guard | +| too_short | The blob was read but is under the 6-byte VS_VERSIONINFO header minimum | +| header_unpack | `struct.unpack` failed on the 6-byte header (defensive; unreachable past the length guard) | + +*Placement is reported verbatim in `rva` and `size` even when the guard +refuses the read, so the declared values remain diagnosable. A tombstoned +structure is returned rather than `None`: absence of an RT_VERSION resource +is not a defect and produces no issue at all, whereas a resource that exists +but cannot be trusted must stay visible.* + +> **Zero-size leaves moved tag.** A leaf declaring `Size = 0` previously +> reached `pe.get_data`, returned an empty buffer and surfaced as +> `too_short`. It is now refused by the placement guard and surfaces as +> `leaf_placement_implausible`. Consumers keying on `too_short` for that +> case must be updated; `too_short` now means only that a non-empty blob +> was read and fell short of the 6-byte header. ### RESOURCE_VERSIONINFO_INVALID_FIXEDINFO sub‑reasons @@ -274,11 +371,53 @@ always in `invalid_callback_count`): `RESOURCE_VERSIONINFO_INVALID_VARFILEINFO` carry no sub‑reason; the parser's tags are passed through verbatim in an `errors` list. +#### RESOURCE_VERSIONINFO_INVALID_STRINGFILEINFO — parser tags + +Passed through verbatim in the issue's `errors` list. StringFileInfo-level +and StringTable-level tags are emitted as separate issues: the first has a +`tables` count in details, the second a `lang_codepage`. + +| Parser tag | Level | Meaning | +|------------|-------|---------| +| string_table_header | StringFileInfo | A StringTable's length field could not be unpacked; the walk stopped there | +| string_table_length | StringFileInfo | A StringTable's `wLength` was below the 6-byte minimum or extended past the StringFileInfo; the walk stopped there | +| lang_codepage_key | StringTable | The key is not 8 hex characters in `` form | +| string_header | StringTable | A String entry's length field could not be unpacked; the walk stopped there | +| string_length | StringTable | A String entry's `wLength` was below the 6-byte minimum or extended past the table; the walk stopped there | + +#### RESOURCE_VERSIONINFO_INVALID_VARFILEINFO — parser tags + +| Parser tag | Meaning | +|------------|---------| +| var_header | A Var child's length fields could not be unpacked; the walk stopped there | +| var_length | A Var child's `wLength` was below the 6-byte minimum or extended past the VarFileInfo; the walk stopped there | +| translation_not_dword_aligned | `wValueLength` is not a multiple of 4, so the Translation array cannot be a whole number of LANGID+codepage pairs | +| translation_unpack | `struct.unpack` failed on a translation pair (defensive; the slot count is derived from the value extent) | + ### **Resource String‑Table Anomalies** | Reason Code | What Triggers It | Example Pattern | Scope | |------------|------------------|-----------------|--------| | **RESOURCE_STRING_TABLE_CORRUPT** | String table length, offsets, or UTF‑16 entries are malformed or out of bounds | String count = 32 but table only contains 10 entries | Per‑file | +| **RESOURCE_STRING_TABLE_UNREADABLE** | The RT_STRING traversal raised before completing, so the string-table list is empty or partial and its absence carries no meaning | Malformed Name or Language directory beneath RT_STRING | Per‑file | +| **RESOURCE_TABLE_UNAVAILABLE** | The resource entry table could not be built at all: `pe.get_memory_mapped_image` was absent, or raised when called. No entry was decoded, so an empty `resources` list carries no meaning. Distinct from a binary with no resource directory, which produces no issue | A pe object lacking `get_memory_mapped_image`; or a memory-map read raising on a malformed image | Per‑file | + +#### RESOURCE_STRING_TABLE_UNREADABLE + +| Sub‑reason | Meaning | +|------------|---------| +| walk_failed | The RT_STRING walk raised; `string_tables` may be empty or partial. Distinct from a binary that genuinely carries no string resources, which produces no issue at all | + +#### RESOURCE_TABLE_UNAVAILABLE + +Mutually exclusive — the capability check precedes the call, so a missing method never reaches the raising branch: + +| Parser tag | Meaning | +|------------|---------| +| resources_unavailable | `get_memory_mapped_image` was not present on the pe object | +| resources_map_read_failed | The method was present but raised; the exception is swallowed and the walk abandoned | + +Both appear in the resource truncation list. Unlike `resources` and `resource_strings` in that same list, this is a capability tombstone rather than a cap: no entry was truncated because none was decoded. A tombstone and `resources` are mutually exclusive by construction, since both branches return before the entry loop. --- @@ -392,7 +531,9 @@ Priority‑resolved; the first matching tag wins: |------------|---------| | name_rva_missing | Parser did not capture the entry's name RVA | | name_rva_zero | RVA was explicitly zero | +| rva_zero | `_read_asciiz` was called with a zero RVA (defensive; currently unreachable, since the zero case is caught earlier) | | read_failed | pe.get_data raised when reading the name string | +| empty_read | The read returned zero bytes | | unterminated | No NUL terminator found within the maximum scan length | ### EXPORT_NAME_NOT_ASCII @@ -410,6 +551,7 @@ Priority‑resolved: |------------|---------| | missing | Parser could not read the EOT entry | | out_of_range | Ordinal index >= NumberOfFunctions | +| duplicate | Two or more name pointers resolve to the same EAT index; only the last is reflected in the function view, so an export name is silently unreachable through the resolved-function list | ### EXPORT_ORDINAL_OUT_OF_RANGE @@ -501,9 +643,12 @@ Priority‑resolved; the first matching tag wins: |------------|---------| | dll_name_rva_zero | DLL name RVA was explicitly zero | | read_failed | pe.get_data raised when reading the DLL name string | +| empty_read | The read returned zero bytes | | unterminated | No NUL terminator found within the maximum scan length (512 bytes) | -| dll_name_not_printable | Decoded successfully but contains bytes outside 0x20–0x7E | | non_ascii | Decode produced Unicode replacement characters | +| dll_name_empty | The string terminated immediately — a zero-length DLL name | +| dll_name_not_printable | Contains bytes outside 0x20–0x7E | +| dll_name_too_long | Exceeds 255 characters, the NTFS filename component limit | ### DELAY_IMPORT_INT_IAT_MISMATCH @@ -527,10 +672,114 @@ Priority‑resolved: | hint_unpack_failed | Could not unpack the WORD hint | | name_unterminated | Name string had no NUL terminator within the maximum scan length | | name_non_ascii | Name decode produced Unicode replacement characters | +| name_empty | The symbol name terminated immediately — a zero-length import name | | name_not_printable | Name decoded successfully but contains non‑printable bytes | --- +## **IMPORT ANOMALIES** + +*Added in v0.7.6.2 (validator §2.16). Backed by the `pe_imports` struct-level +decoder over the 20-byte `IMAGE_IMPORT_DESCRIPTOR` array. Directory placement +and bounds remain owned by the RVA-graph backbone — both the import directory +(index 1) and the IAT directory (index 12) are plain RVAs, so this validator +defers entirely rather than double-counting. The bound-import directory +(index 11) is a separate structure and is not decoded. Absence of an import +directory is not a defect.* + +> **`OriginalFirstThunk == 0` is legal here.** Unlike the delay-load INT, a +> zero `OriginalFirstThunk` is common: older linkers emit only `FirstThunk`, +> which then holds INT-style thunks on disk. The parser falls back to it and +> records `thunk_source: "iat_fallback"` **without** raising an anomaly. Only +> when the descriptor is *also* old-style bound — so `FirstThunk` holds +> resolved addresses rather than thunks — are names genuinely unrecoverable. + +| Reason Code | What Triggers It | Example Pattern | Scope | +|------------|------------------|-----------------|--------| +| **IMPORT_DIRECTORY_INVALID_HEADER** | Top-level decode failure; short-circuits every later import check | Descriptor array unreadable at the declared RVA | Per‑file | +| **IMPORT_TABLE_TRUNCATED** | A parser truncation tag surfaced while walking the descriptor array or a thunk array | Descriptor array reaches the declared end with no zero terminator | Per‑file *(one issue per tag; `table` names the cause)* | +| **IMPORT_DESCRIPTOR_INVALID** | A descriptor identifies a module but has no readable source of imported symbol names | Old-style bound with `OriginalFirstThunk = 0`; or both thunk RVAs zero | Per‑descriptor *(priority-resolved sub-reason)* | +| **IMPORT_DLL_NAME_INVALID** | A descriptor's DLL name RVA is zero, unreadable, unterminated, empty, non-printable, or exceeds the filename limit | Name RVA = 0x0, or name = `"kernel32\x01dll"` | Per‑descriptor *(priority-resolved sub-reason)* | +| **IMPORT_ENTRY_INVALID** | A per-import entry is malformed: a zero ordinal, or an `IMAGE_IMPORT_BY_NAME` that is unreadable, too short, unterminated, empty or non-printable | Thunk with the high bit set but ordinal = 0 | Per‑entry *(priority-resolved; emission capped, see below)* | + +## IMPORT SUB‑REASONS + +### IMPORT_DIRECTORY_INVALID_HEADER + +| Sub‑reason | Meaning | +|------------|---------| +| top_level_decode | The parser could not complete top-level decoding; the `errors` key lists the contributing parser tags | + +### IMPORT_TABLE_TRUNCATED + +The `table` field (not `sub_reason`) identifies the truncation cause. Thunk +tags are prefixed by the array actually read, so a consumer can tell whether +the INT or the fallback IAT was short: + +| table value | Meaning | +|-------------|---------| +| import_descriptor_truncated | A descriptor's 20-byte structure came back short | +| import_descriptor_read_failed | `pe.get_data` raised while reading a descriptor | +| import_descriptor_unterminated | The declared directory size was reached with no zero descriptor | +| import_descriptor_max_exceeded | Hit the hard descriptor limit (4096) without finding a terminator | +| int_truncated / iat_fallback_truncated | A thunk read came back shorter than the pointer width | +| int_read_failed / iat_fallback_read_failed | `pe.get_data` raised while reading a thunk | +| int_unpack_failed / iat_fallback_unpack_failed | `struct.unpack` failed on a thunk (defensive; unreachable past the length guard) | +| int_max_exceeded / iat_fallback_max_exceeded | Hit the imports-per-descriptor limit (16384) without a NULL thunk | + +### IMPORT_DESCRIPTOR_INVALID + +Priority‑resolved; the first matching tag wins. Mutually exclusive in +practice — the parser returns immediately after recording either: + +| Sub‑reason | Meaning | +|------------|---------| +| names_unrecoverable_bound_no_int | The descriptor is old-style bound (`TimeDateStamp` neither 0 nor 0xFFFFFFFF) and `OriginalFirstThunk` is zero, so `FirstThunk` holds resolved addresses and no name table exists | +| no_thunk_array | Both `OriginalFirstThunk` and `FirstThunk` are zero — the descriptor names no imports at all | + +Details carry `bound_state`, `original_first_thunk` and `first_thunk` so the +reason a name source is unavailable is visible without re-reading the file. + +### IMPORT_DLL_NAME_INVALID + +Priority‑resolved; the first matching tag wins: + +| Sub‑reason | Meaning | +|------------|---------| +| dll_name_rva_zero | The descriptor's Name RVA was explicitly zero | +| rva_zero | `_read_asciiz` was called with a zero RVA (defensive; the zero case is caught earlier) | +| read_failed | `pe.get_data` raised when reading the name string | +| empty_read | The read returned zero bytes | +| unterminated | No NUL terminator within the maximum scan length (512 bytes) | +| non_ascii | Decode produced Unicode replacement characters | +| dll_name_empty | The string terminated immediately — a zero-length DLL name | +| dll_name_not_printable | Contains bytes outside 0x20–0x7E | +| dll_name_too_long | Exceeds 255 characters, the NTFS filename component limit | + +### IMPORT_ENTRY_INVALID + +Priority‑resolved; the first matching tag wins: + +| Sub‑reason | Meaning | +|------------|---------| +| ordinal_zero | High bit set on the thunk but the ordinal value is zero | +| name_rva_zero | The thunk's `IMAGE_IMPORT_BY_NAME` RVA was zero | +| name_read_failed | `pe.get_data` raised when reading the hint+name structure | +| name_too_short | The buffer was fewer than 3 bytes (WORD hint plus at least one name byte) | +| hint_unpack_failed | Could not unpack the WORD hint (defensive) | +| name_unterminated | No NUL terminator within the maximum scan length (1024 bytes) | +| name_non_ascii | Decode produced Unicode replacement characters | +| name_empty | The symbol name terminated immediately — a zero-length import name | +| name_not_printable | Contains bytes outside 0x20–0x7E. Length is not constrained: the 1024-byte read is the only bound, so mangled C++ symbols are accepted | + +Emission is capped at 32 issues **per descriptor**; `invalid_entry_count` +always carries the true total for that descriptor. The cap is per-descriptor +rather than per-file, so a heavily malformed first module does not silence +later ones. Note the count is of *invalid* entries, not of the descriptor's +whole import list. + +--- + ## **RELOCATION ANOMALIES** *Added in v0.7.6 (validator §2.13). Backed by the `pe_relocations` struct-level decoder over `IMAGE_BASE_RELOCATION` blocks. Directory placement/bounds remain owned by the RVA-graph backbone; these codes cover block-stream and per-entry structural truth. `IMAGE_REL_BASED_ABSOLUTE` (type 0) padding entries are never flagged, and absence of a relocation directory is not a defect.* @@ -538,7 +787,7 @@ Priority‑resolved: | Reason Code | What Triggers It | Example Pattern | Scope | |------------|------------------|-----------------|--------| | **RELOCATION_DIRECTORY_INVALID_HEADER** | Top-level decode failure: the directory placement could not be resolved or the first block header was unrecoverable | Directory at an RVA `pe.get_data` cannot resolve | Per‑file | -| **RELOCATION_TABLE_TRUNCATED** | A block's declared `SizeOfBlock` extends past the directory's declared end, or the entry region could not be fully read | Block claims 0x200 bytes but only 0x40 remain before directory end | Per‑file | +| **RELOCATION_TABLE_TRUNCATED** | A block header, entry region, or the block walk itself could not be fully read within the declared directory | Block claims 0x200 bytes but only 0x40 remain before directory end | Per‑file | | **RELOCATION_BLOCK_MALFORMED** | A block is structurally invalid: `SizeOfBlock` below the 8-byte header minimum, not aligned to the 2-byte entry stride, or a non-advancing size that would stall the walk | `SizeOfBlock = 0`, or `SizeOfBlock = 7` | Per‑block *(priority-resolved sub-reason)* | | **RELOCATION_ENTRY_RVA_INVALID** | A non-ABSOLUTE entry's target (`page_rva + offset`) does not map to any section | Entry target = 0x9000 with no covering section | Per‑entry *(count always reported in details even when emission is capped)* | @@ -563,7 +812,40 @@ Priority‑resolved; the first matching tag wins: ### RELOCATION_TABLE_TRUNCATED The `region` field (not `table`, and not `sub_reason`) names the truncated -region. +region: + +| region value | Meaning | +|---|---| +| relocation_block_header_truncated | The 8-byte block header did not fit the declared directory window, or came back short | +| relocation_block_read_failed | `pe.get_data` raised while reading a block header | +| relocation_entries_exceed_directory | A block's declared entry region extends past the directory's declared end; the readable portion is clamped and the remainder is not decoded | +| relocation_entries_truncated | The entry array was clamped to the directory end, or came back short | +| relocation_entries_read_failed | `pe.get_data` raised while reading the entry array | +| relocation_block_max_exceeded | The walk hit the 65536-block hard limit | + +*Note `relocation_entries_exceed_directory` and `relocation_entries_truncated` +are distinct facts and may both fire for the same block: the former means the +declared size was clamped to the directory window; the latter means the +physical read then came back shorter than even that clamped window.* + +### Entry-level: HIGHADJ pairing + +`IMAGE_REL_BASED_HIGHADJ` (type 4) occupies two WORD slots per the PE spec — +the type+offset word, followed by a raw 16-bit adjustment value. The parser +threads a pairing state across the entry walk (reset per block, never shared +across blocks) rather than decoding every word independently: + +| Entries field | Meaning | +|----------------|---------| +| `adjustment` | Present only on a HIGHADJ entry whose pairing succeeded; holds the raw 16-bit value from the following WORD, verbatim, not decoded as type+offset | + +If a HIGHADJ entry is the last word in a block's entry region (no adjustment +word follows), the entry is still recorded, `adjustment` is absent, and the +block carries: + +| Sub‑reason (block-level `errors`) | Meaning | +|------------------------------------|---------| +| highadj_missing_adjustment | A HIGHADJ entry had no following word to pair with — the block ended, or was truncated, mid-pair | --- @@ -601,10 +883,19 @@ Priority‑resolved; the first matching tag wins: | pdb_path_unterminated | The PDB path had no NUL terminator within the scan length | | pdb_path_non_ascii | The PDB path decoded but contains non-printable bytes | +*On `pdb_path_unterminated`, pdb_path is still populated — with the region truncated at the 512-byte scan cap — rather than left None. Consumers must check errors before trusting the path.* + ### DEBUG_TABLE_TRUNCATED The `region` field (not `table`, and not `sub_reason`) names the truncated -region. +region: + +| region value | Meaning | +|--------------|---------| +| debug_directory_size_not_entry_aligned | Declared directory size is not a whole multiple of the 28-byte entry stride; the partial trailing entry is not decoded | +| debug_directory_entry_count_exceeds_max | Declared entry count exceeded the parser's hard limit (256) and was clamped | +| debug_entry_read_failed | `pe.get_data` raised while reading an entry | +| debug_entry_truncated | An entry read returned fewer than 28 bytes | --- @@ -668,8 +959,6 @@ Priority‑resolved; the first matching tag wins: | Sub‑reason | Meaning | |------------|---------| -| entry_truncated | The entry's fixed-size structure was short | -| entry_read_failed | pe.get_data raised when reading the entry | | entry_unpack_failed | struct.unpack failed on the entry bytes | | begin_rva_zero | BeginAddress was zero | | end_rva_zero | EndAddress was zero (amd64) | diff --git a/docs/specs/structural-validation-deterministic-heuristics.md b/docs/specs/structural-validation-deterministic-heuristics.md index 73019b0..d7429d5 100644 --- a/docs/specs/structural-validation-deterministic-heuristics.md +++ b/docs/specs/structural-validation-deterministic-heuristics.md @@ -43,7 +43,7 @@ Some structural metadata extracted by parsers is **producer-facing**: it exists > validator self-contained. Their codes are disjoint identifiers, so a directory > overrunning `SizeOfImage` surfaces under both the backbone's `DATA_DIRECTORY_OUT_OF_RANGE` > and the subsystem's own code — two distinct labels for one fact, not one code -> emitted twice. Relocations, debug and the security directory defer entirely. +> emitted twice. Relocations, debug and imports defer entirely, emitting no placement code of their own — though a directory placed outside the image will still surface in those subsystems as a truncation, since the parser cannot read it. The security directory defers for a different reason: its VirtualAddress is a file offset, not an RVA, so the backbone's checks do not apply at all. --- @@ -355,8 +355,9 @@ The debug parser is implemented as a pure `struct`-level decoder over both `pe.g - CodeView blobs are read via `PointerToRawData` (raw file offset) first, with a fallback to `AddressOfRawData` (RVA), so extraction is deterministic regardless of which addressing field the producer populated. - The RSDS (PDB 7.0) and NB10 (PDB 2.0) records are decoded against their fixed header layouts; the GUID is formatted in the canonical mixed-endian symbol-server form (Data1/2/3 little-endian, Data4 big-endian) by fixed arithmetic, not library formatting. - The PDB path scan is bounded (512 bytes); an absent terminator emits a deterministic tombstone tag rather than an unbounded read, and non-ASCII bytes are reported rather than silently normalised. +- The CodeView blob read is itself bounded (4096 bytes). The entry's declared `SizeOfData` is used where present, but is clamped to that ceiling, and a declared size of zero falls back to it — so neither an absurd declared size nor a missing one can drive an unbounded read. The debug directory's entry count is bounded on the same principle (256 entries); a declared count above it is clamped and reported rather than walked. -The validator then maps these structural states to a small, well-defined set of reason codes (`DEBUG_DIRECTORY_INVALID_HEADER`, `DEBUG_TABLE_TRUNCATED`, `DEBUG_DIRECTORY_ENTRY_MALFORMED`, `DEBUG_ENTRY_RVA_INVALID`), which downstream heuristics and IOC consumers can rely on as a stable contract. Per-entry malformations are priority-resolved so an entry carrying several defects emits one deterministic sub-reason. The PDB path is a high-signal forensic surface; build paths routinely leak project names, usernames, and toolchain layout, so deterministic extraction is a prerequisite for treating it as a reliable triage signal. +The validator then maps these structural states to a small, well-defined set of reason codes (`DEBUG_DIRECTORY_INVALID_HEADER`, `DEBUG_TABLE_TRUNCATED`, `DEBUG_DIRECTORY_ENTRY_MALFORMED`, `DEBUG_ENTRY_RVA_INVALID`), which downstream heuristics and IOC consumers can rely on as a stable contract. Per-entry malformations are priority-resolved so an entry carrying several defects emits one deterministic sub-reason. The PDB path is a high-signal forensic surface; build paths routinely leak project names, usernames, and toolchain layout, so deterministic extraction is a prerequisite for treating it as a reliable triage signal. On an unterminated PDB path the extracted value is retained rather than discarded — the path is truncated at the scan cap and the entry carries `pdb_path_unterminated`, so a partial build path remains available for triage while the fault stays explicit. Consumers must therefore check the entry's `errors` before treating `pdb_path` as complete. --- @@ -386,6 +387,40 @@ The validator then maps these structural states to a well-defined set of reason --- +# **2.16 Imports Validator** + +### Validates the structural integrity of the PE import table extracted by pe_imports. + +This validator performs: + +- Top-level decode failure detection and short-circuit for an unrecoverable descriptor array. +- Directory placement is **not** re-checked here — `DATA_DIRECTORY_OUT_OF_RANGE` and `DATA_DIRECTORY_NOT_MAPPED_TO_SECTION` from the RVA-graph backbone (§2.5) own it for both the import directory (index 1) and the IAT directory (index 12), so this validator emits no placement code of its own. Note that a badly placed directory still surfaces here as `IMPORT_TABLE_TRUNCATED` with `table: import_descriptor_read_failed`: the parser cannot read what the backbone has already declared out of range, and the resulting silence is itself a structural fact. The two codes are complementary rather than duplicative — the backbone reports that the declaration is wrong, this validator reports that nothing was recoverable as a consequence. Relocations (§2.13) and debug (§2.14) behave identically. +- Truncation reporting across the descriptor array and per-descriptor thunk arrays, with the tag prefixed by the array actually read. +- Per-descriptor name-source validation: a module identified but carrying no readable table of imported symbol names. +- DLL name string validation: RVA presence, readability, NUL termination, emptiness, printable ASCII compliance, and the filename length bound. +- Per-entry validation of thunks and `IMAGE_IMPORT_BY_NAME` structures, including ordinal validity, hint readability, and name structural correctness. + +Absence of an import directory is not treated as a structural defect — resource-only DLLs and some drivers legitimately import nothing. The bound-import directory (index 11) is a separate structure with its own descriptor and forwarder-reference arrays; descriptor-level bound *state* is interpreted here, but that table is not decoded and is out of scope for this release. + +The import table looks like the delay-load table and is not. Two properties make general-purpose import parsers prone to inconsistent output, and both are cases where a plausible implementation misreports rather than fails loudly: + +`OriginalFirstThunk` may legally be zero. The delay-load INT may not — a zero there is a genuine anomaly — but older linkers (Borland TLINK, some Microsoft toolchains) emit standard imports with only `FirstThunk`, which then holds INT-style thunks on disk that the loader overwrites with resolved addresses at load time. A parser that copies the delay-load treatment flags a large fraction of legitimate binaries; a parser that simply reads `OriginalFirstThunk` unconditionally reports those same binaries as importing nothing. Neither failure is visible without a fixture that exercises the zero case. + +`TimeDateStamp` silently changes what `FirstThunk` contains. Zero means unbound and `FirstThunk` mirrors the name table on disk. `0xFFFFFFFF` means "new-style" bound, with the real timestamps held in the bound-import directory — `FirstThunk` still holds thunks. Any other value means "old-style" bound, and `FirstThunk` holds resolved addresses. The three states are indistinguishable from the thunk bytes alone, so a parser that ignores the field will decode addresses as though they were ordinals-or-name-RVAs and emit fabricated import names. + +The imports parser is implemented as a pure `struct`-level decoder over `pe.get_data`-acquired byte buffers: + +- The 20-byte `IMAGE_IMPORT_DESCRIPTOR` is unpacked via a single `struct.unpack_from` call. No reliance on pefile's `DIRECTORY_ENTRY_IMPORT` interpretation. +- The name source is selected explicitly rather than assumed: `OriginalFirstThunk` when present, otherwise `FirstThunk`, with the choice recorded in `thunk_source` so a consumer can see which array was read. The fallback is treated as normal, not as an anomaly. +- Bound state is derived from `TimeDateStamp` at decode time. The one combination that genuinely defeats name recovery — old-style bound *with* a zero `OriginalFirstThunk` — is reported as a structural fact rather than parsed as garbage. +- The descriptor array walk has both an explicit zero-descriptor terminator check and a hard count limit (4096), with distinct truncation tags for each termination cause. +- Thunk arrays are walked with the same dual-bounded strategy: NULL-terminator detection plus a hard limit (16384 per descriptor). Truncation tags carry the `int` or `iat_fallback` prefix of the array actually read. +- Ordinals are masked to the low 16 bits per the PE spec, matching the loader; the DLL name scan is bounded at 512 bytes and the `IMAGE_IMPORT_BY_NAME` scan at 1024. Name length is not otherwise constrained — mangled C++ symbols legitimately exceed any smaller limit — so printability and length are reported as distinct faults rather than conflated. + +The validator then maps these structural states to a small, well-defined set of reason codes (`IMPORT_DIRECTORY_INVALID_HEADER`, `IMPORT_TABLE_TRUNCATED`, `IMPORT_DESCRIPTOR_INVALID`, `IMPORT_DLL_NAME_INVALID`, `IMPORT_ENTRY_INVALID`), which downstream heuristics and IOC consumers can rely on as a stable contract. Per-descriptor and per-entry malformations are priority-resolved so a descriptor carrying several defects emits one deterministic sub-reason per pathology class, and the per-entry emission is capped at 32 issues per descriptor with the true count always present in the issue details — capped per descriptor rather than per file, so a heavily malformed first module cannot silence the ones after it. The import table is the highest-value forensic surface in the PE format for behavioural triage: the set of resolved DLL and symbol names is what most capability heuristics are built on, so a descriptor whose names cannot be recovered is a fact worth reporting explicitly rather than presenting as an empty import list. + +--- + # **3. Deterministic Heuristics Layer** ### *Heuristics interpret structural truth — they never override it.* @@ -413,6 +448,21 @@ Heuristics include: > This separation is what makes structural findings machine-consumable: the > parent code is the stable contract, the sub-reason is diagnostic detail, and > neither can shadow the other. +> +> **The tag contract is statically verified.** Parsers record faults as +> tombstone tags; validators consume them via priority lists or wholesale +> forwarding. A tag with no consumer is silently dropped — `_first_matching` +> returns `"unknown"` and the caller continues, so the finding never reaches +> output at all. That failure mode is not visible in coverage, in snapshots, +> or on inspection: the affected file simply reports clean. A contract check +> therefore walks every parser's AST, enumerates the tags it can emit +> (including f-string templates and tags returned from helper readers), and +> asserts each has a consumer in its paired validator. It runs in CI across +> every parser/validator pair. Deliberate exemptions — where a parser records +> a tag the validator intentionally ignores in favour of a more authoritative +> source — are declared per-tag rather than per-subsystem, so a second, +> undeclared drop in the same parser still fails. + Heuristics never contradict validators. They only interpret validated truth. diff --git a/examples/generators/c/contract/layer1_core/clean_version_info.c b/examples/generators/c/contract/layer1_core/clean_version_info.c new file mode 100644 index 0000000..9282b7c --- /dev/null +++ b/examples/generators/c/contract/layer1_core/clean_version_info.c @@ -0,0 +1,12 @@ +/* Copyright (c) 2026 MalX Labs and contributors + * SPDX-License-Identifier: MPL-2.0 + * + * Minimal carrier for clean_version_info.rc. The code is irrelevant - the point is + * to produce a real PE with a real .rsrc section, so the RT_VERSION leaf + * is located through the actual resource tree rather than a fake. + */ + +int main(void) +{ + return 0; +} diff --git a/examples/generators/c/contract/layer1_core/clean_version_info.rc b/examples/generators/c/contract/layer1_core/clean_version_info.rc new file mode 100644 index 0000000..6b5ac59 --- /dev/null +++ b/examples/generators/c/contract/layer1_core/clean_version_info.rc @@ -0,0 +1,46 @@ +// Copyright (c) 2026 MalX Labs and contributors +// SPDX-License-Identifier: MPL-2.0 +// +// Valid VS_VERSIONINFO baseline for IOCX end-to-end tests. +// +// This produces a WELL-FORMED resource by construction - a resource +// compiler will not emit a malformed one, and that is precisely what +// makes this fixture useful: it pins the clean path. Malformed cases +// come from vs_versioninfo_builder.py instead. +// +// The string block deliberately contains exactly the eight keys in the +// projection's _DEFAULT_KEYS shortlist, plus one extra ("Comments") to +// exercise the keys_filtered truncation flag at default level. + +#include + +VS_VERSION_INFO VERSIONINFO +FILEVERSION 10,0,0,1 +PRODUCTVERSION 10,0,0,1 +FILEFLAGSMASK VS_FFI_FILEFLAGSMASK +FILEFLAGS 0x0L +FILEOS VOS_NT_WINDOWS32 +FILETYPE VFT_APP +FILESUBTYPE VFT2_UNKNOWN +BEGIN + BLOCK "StringFileInfo" + BEGIN + BLOCK "040904B0" // US English, Unicode + BEGIN + VALUE "CompanyName", "MalX Labs" + VALUE "FileDescription", "IOCX test fixture" + VALUE "FileVersion", "10.0.0.1" + VALUE "InternalName", "fixture" + VALUE "LegalCopyright", "\xA9 MalX Labs. All rights reserved." + VALUE "OriginalFilename", "FIXTURE.EXE" + VALUE "ProductName", "IOCX" + VALUE "ProductVersion", "10.0.0.1" + VALUE "Comments", "Non-shortlist key: expect keys_filtered" + END + END + + BLOCK "VarFileInfo" + BEGIN + VALUE "Translation", 0x409, 1200 + END +END diff --git a/iocx/cli/main.py b/iocx/cli/main.py index c5eb842..11f0580 100644 --- a/iocx/cli/main.py +++ b/iocx/cli/main.py @@ -7,18 +7,64 @@ from ..engine import Engine, EngineConfig from importlib.metadata import version, PackageNotFoundError +_ART = r""" ___ ___ ___ __ __ + |_ _/ _ \ / __|\ \/ / + | | (_) | (__ > < + |___\___/ \___|/_/\_\ +""" + +def _dep_version(name: str) -> str: + """Best-effort dependency version; never raises.""" + try: + from importlib.metadata import version + return version(name) + except Exception: + return "unknown" + + +def _format_version(version: str, *, art: bool = True) -> str: + """ + Build the --version text. + + Dependency versions are included because they are a real variable in + the output: pefile materialises the resource tree the parsers walk, + so two runs disagreeing on a finding may differ only there. + """ + lines = [] + + # ASCII art only when stdout is a terminal - it is noise in CI logs + # and in anything capturing the output. + if art and sys.stdout.isatty(): + lines.append(_ART.rstrip("\n")) + lines.append("") + + py = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}" + + lines.extend([ + f"Deterministic PE static analysis\n", + f"iocx {version}", + f"python {py}", + f"pefile {_dep_version('pefile')}", + "license MPL-2.0", + "", + "MalX Labs - https://github.com/iocx-dev/iocx", + ]) + + return "\n".join(lines) + def get_version(): try: - return version("iocx") + return _format_version(version("iocx")) except PackageNotFoundError: return "0.0.0" def main(): parser = argparse.ArgumentParser( - description="Static IOC extractor for binaries, logs, and text.", + description="An extensible, deterministic static‑analysis engine that extracts high‑signal IOCs from PE binaries and text, built for SOC automation and modern threat‑analysis pipelines.", formatter_class=argparse.RawDescriptionHelpFormatter, + allow_abbrev=False ) # --------------------------- @@ -26,6 +72,7 @@ def main(): # --------------------------- input_group = parser.add_argument_group("Input") output_group = parser.add_argument_group("Output") + pe_analysis_group = parser.add_argument_group("PE Analysis") engine_group = parser.add_argument_group("Engine Options") detector_group = parser.add_argument_group("Detector Options") misc_group = parser.add_argument_group("Misc") @@ -56,15 +103,16 @@ def main(): output_group.add_argument( "-e", "--enrich", action="store_true", - help="Write enrichment data to the JSON output." + help="Write enrichment data to the JSON output. Enrichment is context to extracted IOCs, surfaced via plugins with the enrichment capability." ) - output_group.add_argument( + pe_analysis_group.add_argument( "-a", "--analyse", "--analyze", nargs="?", const="deep", choices=["basic", "deep", "full"], - help="Enable PE analysis (basic, deep, full; default: deep)." + metavar="LEVEL", + help="Enable PE analysis. LEVEL: basic (sections, entropy), deep (+ obfuscation heuristics), full (+ structural validation, full version-info). Default when -a is given without a value: deep." ) # --------------------------- @@ -76,6 +124,14 @@ def main(): help="Disable engine caching." ) + engine_group.add_argument( + "-m", "--min-length", + type=int, + default=4, + metavar="N", + help="Minimum printable string length for the string extractor (default: 4)." + ) + # --------------------------- # Detector Options # --------------------------- @@ -88,21 +144,13 @@ def main(): detector_group.add_argument( "--list-transformers", action="store_true", - help="List available transformer plugins." + help="List available transformer plugins and exit." ) detector_group.add_argument( "--list-enrichers", action="store_true", - help="List available enricher plugins." - ) - - detector_group.add_argument( - "-m", "--min-length", - type=int, - default=4, - metavar="N", - help="Minimum printable string length for the string extractor (default: 4)." + help="List available enricher plugins and exit." ) # --------------------------- @@ -117,7 +165,7 @@ def main(): misc_group.add_argument( "-d", "--dev", action="store_true", - help="Enable local plugins.", + help="Enable local plugins. Local plugins must be placed in the '.iocx/plugins' folder of your home directory.", ) args = parser.parse_args() diff --git a/iocx/engine.py b/iocx/engine.py index ced38ce..2634b85 100644 --- a/iocx/engine.py +++ b/iocx/engine.py @@ -11,10 +11,12 @@ from .parsers.pe_parser import parse_pe, analyse_pe_sections, analyse_data_directories, sanitize_sections, analyse_data_directories_raw from .parsers.string_extractor import extract_strings from .parsers.pe_resources import build_resource_structure -from .parsers.pe_version_info import build_version_info +from .parsers.pe_version_info import build_version_info_structure +from .parsers.version_info_projection import project_version_info from .parsers.pe_load_config import analyse_load_config from .parsers.pe_optional_header import extract_optional_header_metadata from .parsers.pe_exports import build_export_structure +from .parsers.pe_imports import build_import_structure from .parsers.pe_delay_imports import build_delay_import_structure from .parsers.pe_relocations import build_relocation_structure from .parsers.pe_debug import build_debug_structure @@ -138,6 +140,8 @@ def _pipeline_pe(self, path: str) -> Dict[str, Any]: heuristics = [] structural = [] + version_info = build_version_info_structure(pe) + # BASIC: section layout + entropy if analysis_level in ("basic", "deep", "full"): section_analysis = { @@ -149,7 +153,7 @@ def _pipeline_pe(self, path: str) -> Dict[str, Any]: if analysis_level in ("deep", "full"): obf = analyse_obfuscation(section_analysis["sections"], text) - # FULL: future expansion + # FULL if analysis_level == "full": extended = analyse_extended(pe, metadata, text) @@ -170,8 +174,9 @@ def _pipeline_pe(self, path: str) -> Dict[str, Any]: } self._internal_metadata["resources_struct"] = build_resource_structure(pe) - self._internal_metadata["version_info_struct"] = build_version_info(pe) + self._internal_metadata["version_info_struct"] = version_info self._internal_metadata["export_struct"] = build_export_structure(pe) + self._internal_metadata["import_struct"] = build_import_structure(pe) self._internal_metadata["delay_import_struct"] = build_delay_import_structure(pe) self._internal_metadata["data_directories_raw"] = analyse_data_directories_raw(pe) self._internal_metadata["relocation_struct"] = build_relocation_structure(pe) @@ -202,6 +207,8 @@ def _pipeline_pe(self, path: str) -> Dict[str, Any]: analysis["extended"] = extended analysis["heuristics"] = [asdict(h) for h in heuristics] + result["version_info"] = project_version_info(version_info, full=(analysis_level == "full")) + if analysis: result["analysis"] = analysis diff --git a/iocx/parsers/pe_certificates.py b/iocx/parsers/pe_certificates.py index 4443043..d3270a9 100644 --- a/iocx/parsers/pe_certificates.py +++ b/iocx/parsers/pe_certificates.py @@ -216,7 +216,7 @@ def _read_certificates( " Optional[Dict[str, Any]]: @@ -159,7 +165,7 @@ def _read_descriptors( decoded = _decode_descriptor(raw, index) if decoded is None: - errors.append(f"descriptor_unpack_failed_at_{index}") + errors.append("descriptor_unpack_failed") break # Check for zero terminator BEFORE adding to results @@ -253,9 +259,18 @@ def _enrich_descriptor( descriptor["errors"].append(err) else: descriptor["dll_name"] = name - descriptor["dll_name_valid"] = bool(_DLL_NAME_RE.match(name)) - if not descriptor["dll_name_valid"]: + # Three distinct structural faults, reported separately: an empty + # name, a non-printable one, and one exceeding the filename limit + # are different anomalies and a consumer triaging on + # "not_printable" should not be shown a length violation. + if name == "": + descriptor["errors"].append("dll_name_empty") + elif not _PRINTABLE_ASCII_RE.match(name): descriptor["errors"].append("dll_name_not_printable") + elif len(name) > _DLL_NAME_MAX_CHARS: + descriptor["errors"].append("dll_name_too_long") + else: + descriptor["dll_name_valid"] = True # ---- INT (Import Name Table) ---- int_rva = descriptor["int_rva"] @@ -330,9 +345,16 @@ def _decode_import_entry( elif name is None: errors.append("name_read_failed") else: - name_valid = bool(re.match(r"^[\x20-\x7E]{1,512}$", name)) - if not name_valid: + # No length cap: _IMPORT_NAME_MAX_LEN already bounds the read + # (effective max 1021 chars after the WORD hint), and MSVC-mangled + # C++ symbols legitimately exceed any smaller limit. Only + # printability is a structural question at this point. + if name == "": + errors.append("name_empty") + elif not _PRINTABLE_ASCII_RE.match(name): errors.append("name_not_printable") + else: + name_valid = True return { "index": index, diff --git a/iocx/parsers/pe_exports.py b/iocx/parsers/pe_exports.py index b64dc2e..74e2582 100644 --- a/iocx/parsers/pe_exports.py +++ b/iocx/parsers/pe_exports.py @@ -37,6 +37,7 @@ name - resolved name if a name pointer maps to this index, else None name_rva - RVA of the name string, if any + errors - per-entry decode errors Each NamePointerEntry has: index - position in name pointer array @@ -60,8 +61,11 @@ # Forwarder string: ASCII printable, "DllName.SymbolName" or "DllName.#Ordinal" # Conservative bounds: name parts up to 255 chars, total up to 512. +# The symbol branch excludes a leading '#' (0x23) so an over-long ordinal +# such as "Dll.#99999999999" cannot fall through to it and be accepted as an +# ordinary symbol name. Ordinals are 16-bit, hence at most five digits. _FORWARDER_RE = re.compile( - r"^[\x20-\x7E]{1,255}\.(?:#\d{1,10}|[\x20-\x7E]{1,255})$" + r"^[\x20-\x7E]{1,255}\.(?:#\d{1,5}|[\x20-\x22\x24-\x7E][\x20-\x7E]{0,254})$" ) # Name string: PE spec allows ASCII; we accept any printable byte range @@ -334,7 +338,7 @@ def _build_name_pointers( EAT-index -> name, used by _build_functions to enrich function entries. """ entries: List[Dict[str, Any]] = [] - name_by_index: Dict[int, str] = {} + name_by_index: Dict[int, tuple] = {} num_names = header["NumberOfNames"] num_funcs = header["NumberOfFunctions"] @@ -367,7 +371,9 @@ def _build_name_pointers( elif name is not None and name_valid: # Record the resolved name against its EAT index for use by # _build_functions - name_by_index[ordinal_index] = name + if ordinal_index in name_by_index: + entry_errors.append("ordinal_index_duplicate") + name_by_index[ordinal_index] = (name, name_rva) entries.append({ "index": i, @@ -391,7 +397,7 @@ def _build_functions( header: Dict[str, int], dir_start: int, dir_end: int, - name_by_index: Dict[int, str], + name_by_index: Dict[int, tuple], ) -> List[Dict[str, Any]]: """ Build the function entry list from the EAT, joining with name resolution @@ -408,24 +414,25 @@ def _build_functions( is_forwarder = False forwarder: Optional[str] = None forwarder_valid = False - name_rva: Optional[int] = None + entry_errors: List[str] = [] if address_rva is not None and address_rva != 0: # PE spec: an EAT entry RVA that points within the export # directory itself is a forwarder string pointer. if dir_start <= address_rva < dir_end: is_forwarder = True - forwarder, _ = _read_asciiz(pe, address_rva, _FORWARDER_MAX_LEN) + forwarder, fwd_error = _read_asciiz( + pe, address_rva, _FORWARDER_MAX_LEN + ) + if fwd_error is not None: + entry_errors.append(fwd_error) if forwarder is not None: forwarder_valid = bool(_FORWARDER_RE.match(forwarder)) + resolved = name_by_index.get(i) # Look up resolved name from name pointer table cross-reference - name = name_by_index.get(i) - if name is not None: - # The name's RVA is recorded in name_pointers, not duplicated - # here — but we expose name_rva as None to keep the shape - # consistent. - name_rva = None + name = resolved[0] if resolved else None + name_rva = resolved[1] if resolved else None entries.append({ "index": i, @@ -436,6 +443,7 @@ def _build_functions( "forwarder_valid": forwarder_valid, "name": name, "name_rva": name_rva, + "errors": entry_errors, }) return entries diff --git a/iocx/parsers/pe_imports.py b/iocx/parsers/pe_imports.py new file mode 100644 index 0000000..78e1e3d --- /dev/null +++ b/iocx/parsers/pe_imports.py @@ -0,0 +1,514 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 + +""" +Deterministic structural extraction of the PE import table. + +Independent of pefile's DIRECTORY_ENTRY_IMPORT interpretation. +Pefile is used only to: + - Locate the import directory (RVA, size) + - Determine PE32 vs PE32+ via OPTIONAL_HEADER.Magic + - Resolve RVAs to file offsets via pe.get_data + +All structural fields are decoded from the raw 20-byte +IMAGE_IMPORT_DESCRIPTOR structure: + + DWORD OriginalFirstThunk # RVA of the INT. MAY BE ZERO - see below. + DWORD TimeDateStamp # 0 = unbound; -1 = new-style bound + DWORD ForwarderChain # index of first forwarder, -1 = none + DWORD Name # RVA of the ASCIIZ DLL name + DWORD FirstThunk # RVA of the IAT. Always required. + +TWO DIVERGENCES FROM DELAY-LOAD: + +1. `OriginalFirstThunk == 0` is LEGAL for standard imports, unlike the + delay-load INT. Older linkers (Borland TLINK, some MS toolchains) emit + only FirstThunk; on disk that array then holds INT-style thunks which the + loader overwrites with addresses at load time. The parser therefore FALLS + BACK to FirstThunk for name resolution and records which array it used in + `thunk_source`. Flagging this as an anomaly would misreport a large + fraction of legitimate binaries. + +2. `TimeDateStamp` selects how FirstThunk should be read: + 0 - unbound; FirstThunk mirrors the INT on disk + 0xFFFFFFFF - "new-style" bound; real timestamps live in + IMAGE_DIRECTORY_ENTRY_BOUND_IMPORT (index 11) + anything else - "old-style" bound; FirstThunk holds RESOLVED + ADDRESSES on disk, not thunks + When old-style bound AND OriginalFirstThunk is zero there is no readable + name source at all, which is a genuine structural fact rather than a + parse failure - tagged `names_unrecoverable_bound_no_int`. + +The bound-import directory (index 11) is a separate structure and is NOT +parsed here. + +Output contract: + None - no import directory present (not an error) + dict per the documented contract (see ImportStruct in + iocx.schemas.internal_schema). +""" + +from __future__ import annotations + +import re +import struct +from typing import Any, Dict, List, Optional, Tuple + +# IMAGE_DIRECTORY_ENTRY_IMPORT = 1 +_IMPORT_DIRECTORY_INDEX = 1 +_DESCRIPTOR_SIZE = 20 # IMAGE_IMPORT_DESCRIPTOR is 20 bytes + +# OPTIONAL_HEADER.Magic values +_MAGIC_PE32 = 0x10B +_MAGIC_PE32_PLUS = 0x20B + +# TimeDateStamp sentinel for "new-style" bound imports. +_BOUND_NEW_STYLE = 0xFFFFFFFF + +# DLL name length cap to defend against unterminated reads. +_DLL_NAME_MAX_LEN = 512 +# Import-by-name length cap. Mangled C++ symbols legitimately run long, so +# this bounds the read only; printability is checked separately. +_IMPORT_NAME_MAX_LEN = 1024 + +# Hard limit on descriptors to defend against a directory claiming +# arbitrarily many. Real binaries rarely exceed a few hundred. +_MAX_DESCRIPTORS = 4096 + +# Hard limit on imports per descriptor, same rationale. +_MAX_IMPORTS_PER_DESCRIPTOR = 16384 + +# Printable-ASCII structural check, applied to both DLL names and import +# symbol names. Length is NOT constrained here - each caller applies its own +# limit where one is structurally meaningful, so a too-long name is reported +# distinctly from a non-printable one. +_PRINTABLE_ASCII_RE = re.compile(r"^[\x20-\x7E]+$") + +# NTFS filename component limit. An import DLL name is a filename, not a +# path, so 255 is the correct structural bound. +_DLL_NAME_MAX_CHARS = 255 + + +def build_import_structure(pe) -> Optional[Dict[str, Any]]: + """ + Locate and structurally decode the PE import table. + + Returns None if no import directory is present. Otherwise returns a dict + per the module docstring contract. Never raises; decode failures produce + tombstone entries in `errors` and `truncations`. + """ + placement = _locate_import_directory(pe) + if placement is None: + return None + + rva, size = placement + is_64bit = _is_pe32_plus(pe) + thunk_size = 8 if is_64bit else 4 + truncations: List[str] = [] + errors: List[str] = [] + + descriptors = _read_descriptors( + pe, rva, size, thunk_size, truncations, errors, + ) + + return { + "rva": rva, + "size": size, + "is_64bit": is_64bit, + "descriptors": descriptors, + "descriptor_count": len(descriptors), + "truncations": truncations, + "errors": errors, + } + + +# ================================================================= +# Locator +# ================================================================= + +def _locate_import_directory(pe) -> Optional[Tuple[int, int]]: + """Return (rva, size) of the import directory, or None if absent.""" + try: + data_dir = pe.OPTIONAL_HEADER.DATA_DIRECTORY[_IMPORT_DIRECTORY_INDEX] + rva = int(data_dir.VirtualAddress) + size = int(data_dir.Size) + except (AttributeError, IndexError, ValueError, TypeError): + return None + + if rva == 0 or size == 0: + return None + + return (rva, size) + + +def _is_pe32_plus(pe) -> bool: + """Determine PE32+ (64-bit) by reading OPTIONAL_HEADER.Magic.""" + try: + return int(pe.OPTIONAL_HEADER.Magic) == _MAGIC_PE32_PLUS + except (AttributeError, ValueError, TypeError): + # Default to 32-bit if we cannot tell. Conservative - narrower + # thunks, lower risk of over-reading. + return False + + +# ================================================================= +# Descriptor array +# ================================================================= + +def _read_descriptors( + pe, + base_rva: int, + declared_size: int, + thunk_size: int, + truncations: List[str], + errors: List[str], +) -> List[Dict[str, Any]]: + """ + Walk the array of IMAGE_IMPORT_DESCRIPTOR structures. + + The array is terminated by an all-zero descriptor. We also stop at the + declared directory size and enforce a hard count limit. + """ + descriptors: List[Dict[str, Any]] = [] + pos = base_rva + end = base_rva + declared_size + + for index in range(_MAX_DESCRIPTORS): + if pos + _DESCRIPTOR_SIZE > end: + # Reached the declared end without a zero terminator. Only + # meaningful if we decoded at least one descriptor. + if descriptors: + truncations.append("import_descriptor_unterminated") + break + + try: + raw = bytes(pe.get_data(pos, _DESCRIPTOR_SIZE)) + except Exception: + truncations.append("import_descriptor_read_failed") + break + + if len(raw) < _DESCRIPTOR_SIZE: + truncations.append("import_descriptor_truncated") + break + + decoded = _decode_descriptor(raw, index) + if decoded is None: # pragma: no cover - guarded by the length check + errors.append("descriptor_unpack_failed") + break + + # Zero terminator ends the array and is not itself an entry. + if _is_zero_descriptor(decoded): + break + + _enrich_descriptor(pe, decoded, thunk_size, truncations) + descriptors.append(decoded) + pos += _DESCRIPTOR_SIZE + else: + truncations.append("import_descriptor_max_exceeded") + + return descriptors + + +def _decode_descriptor(buf: bytes, index: int) -> Optional[Dict[str, Any]]: + """Unpack a 20-byte IMAGE_IMPORT_DESCRIPTOR.""" + try: + (original_first_thunk, timestamp, forwarder_chain, + name_rva, first_thunk) = struct.unpack_from(" bool: + """An all-zero descriptor signals end of array.""" + return ( + d["original_first_thunk"] == 0 + and d["timestamp"] == 0 + and d["forwarder_chain"] == 0 + and d["name_rva"] == 0 + and d["first_thunk"] == 0 + ) + + +# ================================================================= +# Per-descriptor enrichment +# ================================================================= + +def _enrich_descriptor( + pe, + descriptor: Dict[str, Any], + thunk_size: int, + truncations: List[str], +) -> None: + """Read the DLL name and walk the name-source thunk array.""" + _read_dll_name(pe, descriptor) + + int_rva = descriptor["original_first_thunk"] + iat_rva = descriptor["first_thunk"] + bound_state = descriptor["bound_state"] + + # ---- Select the array to read names from ---- + # + # Normal case: OriginalFirstThunk points at the INT. + # + # Fallback: OriginalFirstThunk == 0 is legal; FirstThunk then holds the + # INT-style thunks on disk. This is NOT an anomaly. + # + # Exception: if the descriptor is old-style bound, FirstThunk holds + # resolved ADDRESSES rather than thunks, so with no INT there is no + # readable name source at all. + if int_rva != 0: + source_rva, source = int_rva, "int" + elif iat_rva != 0: + if bound_state == "bound_old_style": + descriptor["errors"].append("names_unrecoverable_bound_no_int") + return + source_rva, source = iat_rva, "iat_fallback" + else: + # Neither array present - the descriptor names no imports at all. + descriptor["errors"].append("no_thunk_array") + return + + descriptor["thunk_source"] = source + + thunks = _read_thunk_array( + pe, source_rva, thunk_size, source, + descriptor["errors"], truncations, + ) + + high_bit = 1 << (thunk_size * 8 - 1) + for i, value in enumerate(thunks): + descriptor["imports"].append( + _decode_import_entry(pe, i, value, high_bit) + ) + + +def _read_dll_name(pe, descriptor: Dict[str, Any]) -> None: + """Read and structurally check the ASCIIZ DLL name.""" + name_rva = descriptor["name_rva"] + if name_rva == 0: + descriptor["errors"].append("dll_name_rva_zero") + return + + name, err = _read_asciiz(pe, name_rva, _DLL_NAME_MAX_LEN) + if err is not None: + descriptor["errors"].append(err) + return + + descriptor["dll_name"] = name + # Three distinct structural faults, reported separately: an empty name, a + # non-printable one, and one exceeding the filename limit are different + # anomalies and a consumer triaging on "not_printable" should not be + # shown a length violation. + if name == "": + descriptor["errors"].append("dll_name_empty") + elif not _PRINTABLE_ASCII_RE.match(name): + descriptor["errors"].append("dll_name_not_printable") + elif len(name) > _DLL_NAME_MAX_CHARS: + descriptor["errors"].append("dll_name_too_long") + else: + descriptor["dll_name_valid"] = True + + +def _decode_import_entry( + pe, + index: int, + thunk_value: int, + high_bit: int, +) -> Dict[str, Any]: + """ + Build one ImportEntry from a thunk value. + + Thunk semantics: + - High bit set: the low 16 bits are an ordinal. + - High bit clear: the value is an RVA to IMAGE_IMPORT_BY_NAME. + """ + errors: List[str] = [] + is_ordinal = False + ordinal: Optional[int] = None + hint: Optional[int] = None + name: Optional[str] = None + name_rva: Optional[int] = None + name_valid = False + + if thunk_value & high_bit: + is_ordinal = True + # Ordinal is the low 16 bits per PE spec. Bits between 16 and the + # high flag are discarded, matching the loader. + ordinal = thunk_value & 0xFFFF + if ordinal == 0: + errors.append("ordinal_zero") + else: + name_rva = thunk_value + hint, name, read_err = _read_import_by_name(pe, name_rva) + if read_err is not None: + errors.append(read_err) + elif name is None: # pragma: no cover - defensive + errors.append("name_read_failed") + else: + # No length cap: _IMPORT_NAME_MAX_LEN already bounds the read, + # and mangled C++ symbols legitimately exceed any smaller limit. + if name == "": + errors.append("name_empty") + elif not _PRINTABLE_ASCII_RE.match(name): + errors.append("name_not_printable") + else: + name_valid = True + + return { + "index": index, + "thunk_value": thunk_value, + "is_ordinal": is_ordinal, + "ordinal": ordinal, + "hint": hint, + "name": name, + "name_rva": name_rva, + "name_valid": name_valid, + "errors": errors, + } + + +# ================================================================= +# Thunk array reader +# ================================================================= + +def _read_thunk_array( + pe, + rva: int, + thunk_size: int, + tag: str, + descriptor_errors: List[str], + truncations: List[str], +) -> List[int]: + """ + Read a NULL-terminated array of thunks. + + Walks one thunk at a time until a zero terminator is found, the + per-descriptor limit is hit, or the read fails. + """ + thunks: List[int] = [] + pos = rva + fmt = " Tuple[Optional[str], Optional[str]]: + """ + Read a NUL-terminated ASCII string at `rva`. + Returns (string, error_tag); error_tag is None on success. + """ + if rva == 0: + return None, "rva_zero" + + try: + raw = bytes(pe.get_data(rva, max_len)) + except Exception: + return None, "read_failed" + + if not raw: + return None, "empty_read" + + nul_pos = raw.find(b"\x00") + if nul_pos == -1: + return None, "unterminated" + + try: + return raw[:nul_pos].decode("ascii"), None + except UnicodeDecodeError: + return raw[:nul_pos].decode("ascii", errors="replace"), "non_ascii" + + +def _read_import_by_name( + pe, + rva: int, +) -> Tuple[Optional[int], Optional[str], Optional[str]]: + """ + Read IMAGE_IMPORT_BY_NAME: + WORD Hint + BYTE Name[] (NUL-terminated ASCII) + + Returns (hint, name, error_tag). + """ + if rva == 0: + return None, None, "name_rva_zero" + + try: + raw = bytes(pe.get_data(rva, _IMPORT_NAME_MAX_LEN)) + except Exception: + return None, None, "name_read_failed" + + if len(raw) < 3: + return None, None, "name_too_short" + + try: + (hint,) = struct.unpack_from(" Name -> Language, so three levels. Anything +# deeper is malformed; the cap also prevents unbounded recursion, which +# would raise RecursionError out of parse_pe rather than degrade. +_MAX_RESOURCE_DEPTH = 8 + # --------------------------------------------------------------------------- # Low-level helpers # --------------------------------------------------------------------------- @@ -75,7 +87,8 @@ def _safe_file_size(pe) -> int: return size_attr() if callable(size_attr) else size_attr -def _walk_resources(pe, directory, resource_strings, max_allowed=None, visited=None): +def _walk_resources(pe, directory, resource_strings, max_allowed=None, + visited=None, depth=0): if visited is None: visited = set() @@ -84,15 +97,28 @@ def _walk_resources(pe, directory, resource_strings, max_allowed=None, visited=N # 10% of file, capped at 20 MB max_allowed = min(size // 10, 20_000_000) if size else 20_000_000 - # Prevent infinite recursion on malformed resource trees + if depth > _MAX_RESOURCE_DEPTH: + return + + # Identity is sufficient here: pefile has already materialised the tree, + # so what we walk is a finite object graph held alive for the duration + # of the call. The depth cap above covers the case identity does not - + # a legitimately deep chain of distinct directories. dir_id = id(directory) if dir_id in visited: return visited.add(dir_id) for entry in getattr(directory, "entries", []): + # Stop once the string budget is spent. Checked per entry so a tree + # of many small resources is bounded as well as one with a few + # large ones. + if len(resource_strings) >= _MAX_RESOURCE_STRINGS: + return + if hasattr(entry, "directory"): - _walk_resources(pe, entry.directory, resource_strings, max_allowed, visited) + _walk_resources(pe, entry.directory, resource_strings, + max_allowed, visited, depth + 1) elif hasattr(entry, "data"): data_rva = getattr(entry.data.struct, "OffsetToData", 0) size = getattr(entry.data.struct, "Size", 0) @@ -103,10 +129,14 @@ def _walk_resources(pe, directory, resource_strings, max_allowed=None, visited=N try: data = pe.get_data(data_rva, size) except Exception: - # Malformed resources (bad RVA/size) – skip safely + # Malformed resources (bad RVA/size) - skip safely continue - resource_strings.extend(extract_strings_from_bytes(data)) + # Slice the extension too: a single large blob would otherwise + # overshoot the budget in one call, before the loop-top check + # runs again. + budget = _MAX_RESOURCE_STRINGS - len(resource_strings) + resource_strings.extend(extract_strings_from_bytes(data)[:budget]) def _entropy(data: bytes | None) -> float: @@ -225,8 +255,8 @@ def _parse_bound_imports(pe): dll_raw = getattr(entry, "name", None) or getattr(entry, "dll", None) dll = _decode_dll_name(dll_raw) - struct = getattr(entry, "struct", None) - ts = getattr(struct, "TimeDateStamp", 0) if struct else 0 + entry_struct = getattr(entry, "struct", None) + ts = getattr(entry_struct, "TimeDateStamp", 0) if entry_struct else 0 bound_imports.append({"dll": dll, "timestamp": ts}) @@ -315,14 +345,14 @@ def _parse_signatures(pe): return signatures for sec in pe.DIRECTORY_ENTRY_SECURITY: - struct = getattr(sec, "struct", None) - if not struct: + cert_struct = getattr(sec, "struct", None) + if not cert_struct: continue signatures.append( { - "address": getattr(struct, "VirtualAddress", 0), - "size": getattr(struct, "Size", 0), + "address": getattr(cert_struct, "VirtualAddress", 0), + "size": getattr(cert_struct, "Size", 0), } ) @@ -395,21 +425,35 @@ def _parse_header(pe, opt): def _parse_resources(pe): resources: list[dict[str, Any]] = [] resource_strings: list[str] = [] + truncated: list[str] = [] root = getattr(pe, "DIRECTORY_ENTRY_RESOURCE", None) if not root: - return resources, resource_strings + return resources, resource_strings, truncated # Walk the tree and collect resource_strings _walk_resources(pe, root, resource_strings) + if len(resource_strings) >= _MAX_RESOURCE_STRINGS: + truncated.append("resource_strings") - # Extract structured resource entries + # Extract structured resource entries. A pe object without this method + # cannot yield entries at all, which is materially different from a + # binary that has none - record it rather than returning silently if not hasattr(pe, "get_memory_mapped_image"): - return resources, resource_strings + truncated.append("resources_unavailable") + return resources, resource_strings, truncated - mm = pe.get_memory_mapped_image() or b"" + try: + mm = pe.get_memory_mapped_image() or b"" + except Exception: + truncated.append("resources_map_read_failed") + return resources, resource_strings, truncated + entries_capped = False for entry in getattr(pe.DIRECTORY_ENTRY_RESOURCE, "entries", []): + if entries_capped: + break + type_id = getattr(entry, "id", None) type_name = pefile.RESOURCE_TYPE.get(type_id, f"RT_UNKNOWN_{type_id}") @@ -417,6 +461,11 @@ def _parse_resources(pe): continue for res in getattr(entry.directory, "entries", []): + if len(resources) >= _MAX_RESOURCE_ENTRIES: + truncated.append("resources") + entries_capped = True + break + # Capture the resource's named identifier if present res_name = str(res.name) if getattr(res, "name", None) is not None else None lang = getattr(res, "id", None) @@ -473,7 +522,7 @@ def _parse_resources(pe): r["rva"] if r["rva"] is not None else -1, )) - return resources, resource_strings + return resources, resource_strings, truncated def _parse_data_directories(pe): dirs: list[dict[str, Any]] = [] @@ -510,22 +559,33 @@ def _parse_data_directories_raw(pe) -> list[dict[str, int]]: if not opt: return dirs - # Raw file bytes - raw = pe.__data__ + # Raw file bytes. + # `If not raw` also catches an empty buffer, whereas the other parsers + # use `is None`. `Not raw` is arguably better here since a zero-length + # file genuinely has no optional header. + raw = getattr(pe, "__data__", None) + if not raw: + return dirs # File offset of Optional Header opt_offset = opt.get_file_offset() - # For PE32, DataDirectory starts 96 bytes into Optional Header - # (Magic..LoaderFlags = 96 bytes) - DATA_DIR_OFFSET = 96 + # DataDirectory offset within the optional header: 96 for PE32, + # 112 for PE32+. PE32+ widens ImageBase and the four stack/heap + # fields to QWORD (+20) and drops BaseOfData (-4). + magic = getattr(opt, "Magic", 0x10B) + data_dir_offset = 112 if magic == 0x20B else 96 # Each entry is 8 bytes: (DWORD RVA, DWORD Size) - entry_offset = opt_offset + DATA_DIR_OFFSET + entry_offset = opt_offset + data_dir_offset for i in range(16): - rva = struct.unpack_from(" Optional[Dict[str, Any]]: """ @@ -150,7 +153,7 @@ def _read_blocks( try: page_rva, size_of_block = struct.unpack_from("> 12) & 0xF offset = word & 0x0FFF block["entries"].append({ @@ -235,6 +248,11 @@ def _decode_block( "offset": offset, "rva": page_rva + offset, }) + skip_next = (reloc_type == _RELOC_TYPE_HIGHADJ) + + if skip_next: + # Trailing HIGHADJ with no adjustment word - the block ended mid-pair. + block["errors"].append("highadj_missing_adjustment") block["entry_count"] = len(block["entries"]) return block diff --git a/iocx/parsers/pe_resources.py b/iocx/parsers/pe_resources.py index 3dc014b..0f8cf83 100644 --- a/iocx/parsers/pe_resources.py +++ b/iocx/parsers/pe_resources.py @@ -1,115 +1,151 @@ # Copyright (c) 2026 MalX Labs and contributors # SPDX-License-Identifier: MPL-2.0 -from typing import Dict, Any +""" +Structural extraction of the PE resource directory. + +Unlike the other PE parsers in this package, this one consumes pefile's +already-parsed ``DIRECTORY_ENTRY_RESOURCE`` tree rather than decoding raw +bytes. The resource tree is recursive and pefile's traversal is well +exercised; re-implementing it is deferred. The consequence is recorded +honestly: a tree pefile refuses to parse yields ``None`` here, and the +validator then has nothing to report. + +Never raises. A node or entry that cannot be read is recorded as a +tombstone tag in the containing directory's ``errors`` list and skipped, so +one malformed subtree costs that subtree rather than the whole analysis. + +Output contract: + None - no resource directory present (not an error) + dict per ResourcesStruct in iocx.schemas.internal_schema. +""" + +from typing import Any, Dict, List, Optional + import pefile -def build_resource_structure(pe) -> Dict[str, Any]: - """ - Build a structural resource tree suitable for validation. - """ + +def build_resource_structure(pe) -> Optional[Dict[str, Any]]: if not hasattr(pe, "DIRECTORY_ENTRY_RESOURCE"): return None - # Resource directory entry index (IMAGE_DIRECTORY_ENTRY_RESOURCE = 2) - res_dir = pe.OPTIONAL_HEADER.DATA_DIRECTORY[2] - base_rva = res_dir.VirtualAddress + # IMAGE_DIRECTORY_ENTRY_RESOURCE = 2 + try: + res_dir = pe.OPTIONAL_HEADER.DATA_DIRECTORY[2] + base_rva = int(res_dir.VirtualAddress) + except (AttributeError, IndexError, ValueError, TypeError): + return None root_dir = pe.DIRECTORY_ENTRY_RESOURCE + def build_entry(e) -> Dict[str, Any]: + """ + Decode one IMAGE_RESOURCE_DIRECTORY_ENTRY. May raise; the caller + records the failure and skips the entry. + """ + name = str(e.name) if getattr(e, "name", None) is not None else None + entry_id = getattr(e, "id", None) + + if getattr(e, "directory", None) is not None: + return { + "name": name, + "id": entry_id, + "is_directory": True, + "directory": build_directory(e.directory, e.struct), + "data_rva": None, + "data_size": None, + "raw_offset": None, + } + + d = e.data.struct + data_rva = d.OffsetToData + data_size = d.Size + + # Guarded RVA -> offset: a corrupt RVA must not abort the walk. + # -1 is the sentinel the validator treats as out-of-bounds via its + # `data_raw < 0` arm. + try: + raw_offset = pe.get_offset_from_rva(data_rva) + except (pefile.PEFormatError, AttributeError): + raw_offset = -1 + + return { + "name": name, + "id": entry_id, + "is_directory": False, + "directory": None, + "data_rva": data_rva, + "data_size": data_size, + "raw_offset": raw_offset, + } + def build_directory(node, entry_struct=None) -> Dict[str, Any]: """ node: pefile.ResourceDirData - entry_struct: the IMAGE_RESOURCE_DIRECTORY_ENTRY struct that pointed to this directory + entry_struct: the IMAGE_RESOURCE_DIRECTORY_ENTRY that pointed here """ - - # Directory RVA is derived from the entry that referenced it - if entry_struct: - # Mask off high bit (0x80000000) which indicates "is directory" + if entry_struct is not None: + # Mask off the high bit (0x80000000) marking "is directory". offset = entry_struct.OffsetToData & 0x7FFFFFFF rva = base_rva + offset else: - # Root directory: RVA is simply the base RVA rva = base_rva - # Directory size = 16-byte header + 8 bytes per entry - size = 16 + 8 * len(node.entries) - - entries = [] - for e in node.entries: - name = str(e.name) if getattr(e, "name", None) is not None else None - entry_id = getattr(e, "id", None) - - if hasattr(e, "directory") and e.directory is not None: - # Subdirectory - subdir = build_directory(e.directory, e.struct) - entries.append( - { - "name": name, - "id": entry_id, - "is_directory": True, - "directory": subdir, - "data_rva": None, - "data_size": None, - "raw_offset": None, - } - ) - else: - # Data entry - data = e.data - d = data.struct - data_rva = d.OffsetToData - data_size = d.Size - - # Guarded RVA→offset: a corrupt RVA must not crash the - # parser. -1 is the sentinel the validator already treats - # as out-of-bounds via its data_raw < 0 arm. - try: - raw_offset = pe.get_offset_from_rva(data_rva) - except (pefile.PEFormatError, AttributeError): - raw_offset = -1 - - entries.append( - { - "name": name, - "id": entry_id, - "is_directory": False, - "directory": None, - "data_rva": data_rva, - "data_size": data_size, - "raw_offset": raw_offset, - } - ) + errors: List[str] = [] - return { - "rva": rva, - "size": size, - "entries": entries, - } + # The entry list itself may be missing or unreadable on a malformed + # tree. Without this guard the whole parse aborts. + try: + node_entries = list(node.entries) + except Exception: + return {"rva": rva, "size": 16, "entries": [], + "errors": ["directory_entries_unavailable"]} + + # Size is derived from the DECLARED entry count, before any entry is + # skipped, so a partially decodable directory still reports the size + # its header implies: 16-byte header + 8 bytes per entry. + size = 16 + 8 * len(node_entries) + + entries: List[Dict[str, Any]] = [] + for e in node_entries: + try: + entries.append(build_entry(e)) + except Exception: + # One unreadable entry costs that entry, not the directory. + errors.append("entry_decode_failed") + continue + + return {"rva": rva, "size": size, "entries": entries, + "errors": errors} root = build_directory(root_dir) - # Collect string table entries (RT_STRING = 6) - string_tables = [] + # ---- RT_STRING table collection ---- + # A walk failure and a genuine absence of string resources both leave + # this list empty, so the failure is recorded explicitly. + string_tables: List[Dict[str, Any]] = [] + errors: List[str] = [] try: RT_STRING = 6 for type_entry in root_dir.entries: - if getattr(type_entry, "id", None) == RT_STRING and hasattr(type_entry, "directory"): - for name_entry in type_entry.directory.entries: - if hasattr(name_entry, "directory"): - for lang_entry in name_entry.directory.entries: - if hasattr(lang_entry, "data"): - d = lang_entry.data.struct - string_tables.append( - { - "rva": d.OffsetToData, - "size": d.Size, - } - ) - except Exception: # pragma: no cover - pass + if getattr(type_entry, "id", None) != RT_STRING: + continue + if not hasattr(type_entry, "directory"): + continue + for name_entry in type_entry.directory.entries: + if not hasattr(name_entry, "directory"): + continue + for lang_entry in name_entry.directory.entries: + if not hasattr(lang_entry, "data"): + continue + d = lang_entry.data.struct + string_tables.append({"rva": d.OffsetToData, + "size": d.Size}) + except Exception: + errors.append("string_table_walk_failed") return { "root": root, "string_tables": string_tables, + "errors": errors, } diff --git a/iocx/parsers/pe_version_info.py b/iocx/parsers/pe_version_info.py index 7f93219..514a7d6 100644 --- a/iocx/parsers/pe_version_info.py +++ b/iocx/parsers/pe_version_info.py @@ -29,8 +29,17 @@ _VS_FFI_SIGNATURE = 0xFEEF04BD _VS_FFI_STRUCT_VERSION = 0x00010000 +# Hard cap on children walked, matching the bound every other parser +# applies. `unknown_child` does not terminate the walk, so without this the +# errors list scales with the resource leaf size - which is attacker- +# controlled and unbounded when length_consistent is False. +_MAX_CHILDREN = 256 -def build_version_info(pe) -> Optional[Dict[str, Any]]: +# VS_VERSIONINFO blobs are a few KB in practice +_MAX_VERSION_BLOB = 1_000_000 + + +def build_version_info_structure(pe) -> Optional[Dict[str, Any]]: """ Locate and decode the first RT_VERSION leaf in the resource tree. @@ -57,6 +66,15 @@ def build_version_info(pe) -> Optional[Dict[str, Any]]: "errors": ["leaf_struct_unpack"], } + if rva < 0 or size <= 0 or size > _MAX_VERSION_BLOB: + return { + "rva": rva, "size": size, + "decoded": False, "header_ok": False, "length_consistent": False, + "fixed_file_info": None, + "string_file_info": [], "var_file_info": [], + "errors": ["leaf_placement_implausible"], + } + try: raw = bytes(pe.get_data(rva, size)) except Exception: @@ -199,7 +217,13 @@ def _decode_vs_versioninfo(buf: bytes) -> Dict[str, Any]: end = min(w_length, len(buf)) if out["length_consistent"] else len(buf) # ---- Children: StringFileInfo / VarFileInfo ---- + child_count = 0 while pos + 6 <= end: + if child_count >= _MAX_CHILDREN: + out["errors"].append("child_max_exceeded") + break + child_count += 1 + try: c_len = _u16(buf, pos) _c_vlen = _u16(buf, pos + 2) diff --git a/iocx/parsers/version_info_projection.py b/iocx/parsers/version_info_projection.py new file mode 100644 index 0000000..af6eeb6 --- /dev/null +++ b/iocx/parsers/version_info_projection.py @@ -0,0 +1,280 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 + +""" +Public projection of ``version_info_struct``. + +Deliberately a projection rather than a serialisation of the internal +struct: + + * version numbers become dotted quads, not (MS, LS) pairs; + * string values are stripped of control and bidi characters; + * string tables stay separate, so a key present in two language blocks + with different values does not silently collapse to one; + * the raw parser ``errors`` vocabulary stays internal - the public view + carries a count, and the structural reason codes carry the detail; + * every list is bounded, so output size does not scale with input size. + +The default projection emits a CLOSED key set. StringFileInfo keys are +arbitrary strings taken from the file, so allowing them through unfiltered +would let a binary choose the key names in public output, not merely the +values. ``full=True`` opts into the open set. +""" + +import re +from typing import Any, Dict, List, Optional, Tuple + +# Control characters and bidi overrides are stripped from public string +# output. The parser bounds LENGTH and replacement-decodes invalid UTF-16, +# but leaves these in: a bidi override in CompanyName renders text +# backwards in a terminal or dashboard - the filename-spoofing trick, +# arriving through a field nobody inspects. ANSI escapes are the same +# class of problem for terminal consumers. +# +# \t \n \r are stripped too, unlike the usual "printable" allowance: +# these are single-line display fields, and a newline in CompanyName +# breaks line-oriented consumers and enables log injection. +_UNSAFE_CHARS = re.compile( + r"[\x00-\x1f\x7f]" # all C0 controls, plus DEL + r"|[\u202a-\u202e\u2066-\u2069]" # bidi embedding / override +) + +# A defensive second bound on public strings. The parser already caps +# values at 512 and keys at 128, so these only bite if that changes. +_MAX_PUBLIC_STRING = 512 +_MAX_PUBLIC_KEY = 128 + +# Caps on public output. The parser bounds each value's length but not the +# NUMBER of tables, strings or translations: _decode_string_file_info +# advances by t_len >= 6, so a 1 MB blob can yield ~174k tables. Without +# these, output size scales with attacker-controlled input. +_MAX_PUBLIC_TABLES = 16 +_MAX_PUBLIC_STRINGS_PER_TABLE = 64 +_MAX_PUBLIC_TRANSLATIONS = 32 + +# Closed key set for the default projection: the fields that carry triage +# value and appear in essentially every well-formed binary. +_DEFAULT_KEYS = frozenset({ + "CompanyName", + "FileDescription", + "FileVersion", + "InternalName", + "LegalCopyright", + "OriginalFilename", + "ProductName", + "ProductVersion", +}) + + +# ===================================================================== +# Helpers +# ===================================================================== + +def _clean(text: Any, limit: int = _MAX_PUBLIC_STRING) -> Optional[str]: + """ + Strip control and bidi characters from an attacker-controlled string. + + Returns None for anything that is not a str, so a malformed struct + cannot inject a non-string into public output. + """ + if not isinstance(text, str): + return None + return _UNSAFE_CHARS.sub("", text)[:limit] + + +def _dotted_version(pair: Any) -> Optional[str]: + """ + Render a (MS, LS) DWORD pair as major.minor.build.revision. + + Each DWORD holds TWO 16-bit components: + MS = (major << 16) | minor + LS = (build << 16) | revision + + Emitting the raw pair makes every consumer re-derive this, and the + word order is easy to get backwards. + """ + if not isinstance(pair, (tuple, list)) or len(pair) != 2: + return None + ms, ls = pair + if not isinstance(ms, int) or not isinstance(ls, int): + return None + if ms < 0 or ls < 0 or ms > 0xFFFFFFFF or ls > 0xFFFFFFFF: + return None + return f"{ms >> 16}.{ms & 0xFFFF}.{ls >> 16}.{ls & 0xFFFF}" + + +def _error_total(vi: Dict[str, Any]) -> int: + """ + Count every parser error tag, at all levels. + + The top-level list alone is not enough: a blob whose envelope decoded + cleanly but whose string tables carry lang_codepage_key or + string_length would otherwise report decoded=True with a zero count, + reading as healthy when it is not. + """ + total = len(vi.get("errors") or []) + + for sfi in vi.get("string_file_info") or []: + if not isinstance(sfi, dict): + continue + total += len(sfi.get("errors") or []) + for table in sfi.get("tables") or []: + if isinstance(table, dict): + total += len(table.get("errors") or []) + + for vfi in vi.get("var_file_info") or []: + if isinstance(vfi, dict): + total += len(vfi.get("errors") or []) + + return total + + +def _flag(truncated: List[str], tag: str) -> None: + """Record a truncation tag once, preserving first-seen order.""" + if tag not in truncated: + truncated.append(tag) + + +def _project_tables(vi: Dict[str, Any], + full: bool, + truncated: List[str]) -> Tuple[List[Dict[str, Any]], + List[str]]: + """ + Project the string tables, keeping each language block separate. + + A flat merge across tables would let a key present in two language + blocks overwrite itself, with the survivor decided by walk order - + and a disagreement between blocks is itself a repackaging signal. + """ + tables: List[Dict[str, Any]] = [] + languages: List[str] = [] + + for sfi in vi.get("string_file_info") or []: + if not isinstance(sfi, dict): + continue + + for table in sfi.get("tables") or []: + if not isinstance(table, dict): + continue + + if len(tables) >= _MAX_PUBLIC_TABLES: + _flag(truncated, "tables") + return tables, languages + + lang = _clean(table.get("lang_codepage"), _MAX_PUBLIC_KEY) + if lang and lang not in languages: + languages.append(lang) + + raw_strings = table.get("strings") + if not isinstance(raw_strings, dict): + raw_strings = {} + + # Filter and count in ONE pass, so neither the filtered-out + # keys nor an intermediate list scales with input size. A + # separate `candidates` list would be bounded only by the + # parser's per-table string count, which is unbounded. + strings: Dict[str, str] = {} + kept = 0 + for k, v in raw_strings.items(): + key = _clean(k, _MAX_PUBLIC_KEY) + if not key: + continue + if not full and key not in _DEFAULT_KEYS: + _flag(truncated, "keys_filtered") + continue + + # Count only what is actually emitted: filtering before + # counting stops a table padded with junk keys from + # exhausting the cap ahead of the shortlist. + if kept >= _MAX_PUBLIC_STRINGS_PER_TABLE: + _flag(truncated, "strings") + break + + strings[key] = _clean(v) or "" + kept += 1 + + tables.append({"lang_codepage": lang, "strings": strings}) + + return tables, languages + + +def _project_translations(vi: Dict[str, Any], + truncated: List[str]) -> List[str]: + """Render VarFileInfo translation pairs as 8-hex-char lang+codepage.""" + translations: List[str] = [] + + for vfi in vi.get("var_file_info") or []: + if not isinstance(vfi, dict): + continue + + for var in vfi.get("vars") or []: + if not isinstance(var, dict): + continue + + for t in var.get("translations") or []: + if not isinstance(t, dict): + continue + + lang = t.get("lang") + cp = t.get("codepage") + if not isinstance(lang, int) or not isinstance(cp, int): + continue + if not (0 <= lang <= 0xFFFF) or not (0 <= cp <= 0xFFFF): + continue + + rendered = f"{lang:04X}{cp:04X}" + if rendered in translations: + continue + + if len(translations) >= _MAX_PUBLIC_TRANSLATIONS: + _flag(truncated, "translations") + return translations + + translations.append(rendered) + + return translations + + +# ===================================================================== +# Public API +# ===================================================================== + +def project_version_info(vi: Optional[Dict[str, Any]], + *, full: bool = False) -> Optional[Dict[str, Any]]: + """ + Derive the public view of ``version_info_struct``. + + Returns None when no RT_VERSION resource is present, matching the + parser's own contract. Because the parse now runs on every file + regardless of analysis level, None has exactly one meaning - the + binary carries no version-info - and can never mean "this mode did + not look". + + A tombstoned struct (decoded=False, errors populated) is projected + normally rather than discarded: a resource that exists but cannot be + trusted must stay visible, and `decoded` plus + `structural_error_count` are how a consumer tells the two apart. + """ + if vi is None: + return None + + ffi = vi.get("fixed_file_info") + if not isinstance(ffi, dict): + ffi = {} + + truncated: List[str] = [] + tables, languages = _project_tables(vi, full, truncated) + translations = _project_translations(vi, truncated) + + return { + "file_version": _dotted_version(ffi.get("file_version")), + "product_version": _dotted_version(ffi.get("product_version")), + "tables": tables, + "languages": languages, + "translations": translations, + # Structural health, without exposing the tag vocabulary. The + # matching RESOURCE_VERSIONINFO_* reason codes carry the detail. + "decoded": bool(vi.get("decoded")), + "structural_error_count": _error_total(vi), + "truncated": truncated, + } diff --git a/iocx/reason_codes.py b/iocx/reason_codes.py index 5ad4a36..712d686 100644 --- a/iocx/reason_codes.py +++ b/iocx/reason_codes.py @@ -105,6 +105,7 @@ class ReasonCodes: RESOURCE_ENTRY_OUT_OF_BOUNDS = "resource_entry_out_of_bounds" RESOURCE_DIRECTORY_ZERO_LENGTH = "resource_directory_zero_length" RESOURCE_DIRECTORY_LANGUAGE_NOT_ID = "resource_directory_language_not_id" + RESOURCE_DIRECTORY_ENTRY_UNREADABLE = "resource_directory_entry_unreadable" # --- Resource data anomalies --- RESOURCE_DATA_OUT_OF_BOUNDS = "resource_data_out_of_bounds" @@ -119,6 +120,7 @@ class ReasonCodes: # --- Resource string-table anomalies --- RESOURCE_STRING_TABLE_CORRUPT = "resource_string_table_corrupt" + RESOURCE_STRING_TABLE_UNREADABLE = "resource_string_table_unreadable" # --- Load Config Directory structural issues --- LOAD_CONFIG_TOO_SMALL = "load_config_too_small" @@ -203,6 +205,13 @@ class ReasonCodes: EXCEPTION_UNWIND_INFO_INVALID = "exception_unwind_info_invalid" EXCEPTION_UNWIND_CHAIN_INVALID = "exception_unwind_chain_invalid" + # ---- Import table ---- + IMPORT_DIRECTORY_INVALID_HEADER = "import_directory_invalid_header" + IMPORT_TABLE_TRUNCATED = "import_table_truncated" + IMPORT_DESCRIPTOR_INVALID = "import_descriptor_invalid" + IMPORT_DLL_NAME_INVALID = "import_dll_name_invalid" + IMPORT_ENTRY_INVALID = "import_entry_invalid" + # --- Packer heuristics (interpretation layer) --- PACKER_SECTION_NAME = "packer_section_name" PACKER_HIGH_ENTROPY_SECTION = "high_entropy_section" diff --git a/iocx/schemas/internal_schema.py b/iocx/schemas/internal_schema.py index 4a7ffcf..e016ce7 100644 --- a/iocx/schemas/internal_schema.py +++ b/iocx/schemas/internal_schema.py @@ -21,6 +21,7 @@ class ResourceDirectoryNode(TypedDict): rva: int size: int entries: List[ResourceEntry] + errors: List[str] class ResourceStringTable(TypedDict): @@ -31,6 +32,7 @@ class ResourceStringTable(TypedDict): class ResourcesStruct(TypedDict): root: ResourceDirectoryNode string_tables: List[ResourceStringTable] + errors: List[str] # ------------------------- @@ -115,6 +117,7 @@ class ExportFunctionEntry(TypedDict): forwarder_valid: bool name: Optional[str] name_rva: Optional[int] + errors: List[str] class ExportNamePointerEntry(TypedDict): @@ -150,6 +153,71 @@ class ExportStruct(TypedDict, total=False): errors: List[str] +# ------------------------- +# Import table +# ------------------------- +# Produced by parsers.pe_imports.build_import_structure and consumed by +# validators.imports.validate_imports. Absence of the directory is signalled +# by the parser returning None (hence InternalMetadata.import_struct is +# Optional). Placement is NOT represented here beyond (rva, size): the +# RVA-graph backbone owns it for both index 1 and index 12. + +class ImportEntry(TypedDict, total=False): + index: int + thunk_value: int # raw thunk as read from the name-source array + is_ordinal: bool # high bit set (0x80000000 / 0x8000000000000000) + ordinal: Optional[int] # low 16 bits when is_ordinal; None otherwise + hint: Optional[int] # IMAGE_IMPORT_BY_NAME hint; None for ordinals + name: Optional[str] # None for ordinals or on decode failure + name_rva: Optional[int] # RVA of IMAGE_IMPORT_BY_NAME; None for ordinals + name_valid: bool + errors: List[str] # ordinal_zero | name_rva_zero | name_read_failed | + # name_too_short | hint_unpack_failed | + # name_unterminated | name_non_ascii | + # name_empty | name_not_printable + + +class ImportDescriptor(TypedDict, total=False): + index: int + original_first_thunk: int # RVA of the INT. MAY LEGALLY BE ZERO - see below. + timestamp: int # TimeDateStamp; selects bound_state + forwarder_chain: int + name_rva: int # RVA of the ASCIIZ DLL name + first_thunk: int # RVA of the IAT + bound_state: str # derived from timestamp: + # "unbound" (0) + # "bound_new_style" (0xFFFFFFFF) + # "bound_old_style" (any other value) + dll_name: Optional[str] # may be "" when the string terminated immediately + dll_name_valid: bool + thunk_source: Optional[str] # which array the imports were read from: + # "int" - OriginalFirstThunk (normal) + # "iat_fallback" - FirstThunk, because + # OriginalFirstThunk was zero. + # LEGAL, not an anomaly. + # None - no readable source; see errors + imports: List[ImportEntry] + errors: List[str] # dll_name_rva_zero | rva_zero | read_failed | + # empty_read | unterminated | non_ascii | + # dll_name_empty | dll_name_not_printable | + # dll_name_too_long | + # names_unrecoverable_bound_no_int | no_thunk_array + + +class ImportStruct(TypedDict, total=False): + rva: int + size: int + is_64bit: bool # PE32+ selects 8-byte thunks + descriptors: List[ImportDescriptor] + descriptor_count: int # derived: len(descriptors) + truncations: List[str] # import_descriptor_{truncated,read_failed, + # unterminated,max_exceeded} | + # {int,iat_fallback}_{truncated,read_failed, + # unpack_failed,max_exceeded} + errors: List[str] # top-level; presence short-circuits to + # IMPORT_DIRECTORY_INVALID_HEADER + + # ------------------------- # Delay-load import # ------------------------- @@ -361,6 +429,7 @@ class InternalMetadata(TypedDict, total=False): version_info_struct: Optional[VersionInfoStruct] data_directories_raw: List[DataDirectoryRaw] export_struct: Optional[ExportStruct] + import_struct: Optional[ImportStruct] delay_import_struct: Optional[DelayImportStruct] exception_struct: Optional[ExceptionStruct] relocation_struct: Optional[RelocationStruct] diff --git a/iocx/validators/__init__.py b/iocx/validators/__init__.py index 75a2e18..d858bd4 100644 --- a/iocx/validators/__init__.py +++ b/iocx/validators/__init__.py @@ -16,6 +16,7 @@ from .resources import validate_resources from .version_info import validate_version_info from .exports import validate_exports +from .imports import validate_imports from .delay_imports import validate_delay_imports from .entropy import validate_entropy @@ -54,6 +55,12 @@ "version_info": validate_version_info, # Exports "exports": validate_exports, + # Imports (dir 1): descriptor array, DLL names, INT/IAT thunk decode. + # Placement owned by rva_graph for both the import directory and the IAT + # (dir 12). Grouped with the other name-table validators rather than the + # ascending deep-parse cluster, since it shares their fixture shape and + # sub-reason vocabulary. + "imports": validate_imports, # Delay imports "delay_imports": validate_delay_imports, # Entropy metrics (high entropy sections, overlays, uniform patterns) diff --git a/iocx/validators/delay_imports.py b/iocx/validators/delay_imports.py index 0a7f02e..a3090e3 100644 --- a/iocx/validators/delay_imports.py +++ b/iocx/validators/delay_imports.py @@ -34,14 +34,25 @@ from .decorators import depends_on -# Priority-resolved sub-reasons for per-entry name/RVA pathologies. +# Priority-resolved sub-reasons for DLL-name pathologies. # First-matching wins for deterministic emission. +# Ordered so the earlier, more fundamental failures win: the RVA itself, then +# read faults from _read_asciiz, then content faults from the three-way check. +# The _read_asciiz tags and the content tags are mutually exclusive in +# practice (a non-None err skips the content check entirely), so their +# relative order is defensive rather than load-bearing. _DLL_NAME_ERROR_PRIORITY = [ + # RVA-level "dll_name_rva_zero", + # _read_asciiz failures "read_failed", + "empty_read", "unterminated", - "dll_name_not_printable", "non_ascii", + # content checks (three-way split, mutually exclusive) + "dll_name_empty", + "dll_name_not_printable", + "dll_name_too_long", ] _INT_RVA_ERROR_PRIORITY = [ @@ -60,15 +71,24 @@ "iat_unpack_failed", ] +# Priority-resolved sub-reasons for per-import-entry pathologies. +# First-matching wins for deterministic emission. +# Ordered by decode stage: INT thunk faults, then ordinal faults, then +# IMAGE_IMPORT_BY_NAME read faults, then name content faults. _ENTRY_ERROR_PRIORITY = [ + # INT thunk "int_entry_missing", "int_entry_zero", + # ordinal path "ordinal_zero", + # IMAGE_IMPORT_BY_NAME read faults "name_read_failed", "name_too_short", "hint_unpack_failed", "name_unterminated", "name_non_ascii", + # name content checks (mutually exclusive) + "name_empty", "name_not_printable", ] diff --git a/iocx/validators/exception_table.py b/iocx/validators/exception_table.py index deac360..f3d3550 100644 --- a/iocx/validators/exception_table.py +++ b/iocx/validators/exception_table.py @@ -127,8 +127,6 @@ # Priority-resolved sub-reasons for per-entry table pathologies. # First-matching wins for deterministic emission. _ENTRY_ERROR_PRIORITY = [ - "entry_truncated", - "entry_read_failed", "entry_unpack_failed", "begin_rva_zero", "end_rva_zero", diff --git a/iocx/validators/exports.py b/iocx/validators/exports.py index 8094957..23f5868 100644 --- a/iocx/validators/exports.py +++ b/iocx/validators/exports.py @@ -33,10 +33,15 @@ # Deterministic priority orders for mapping parser error tags to a single # reason. The validator emits at most one issue per malformed entry per # pathology class; the first tag in priority order wins. +# +# The middle three tags come from _read_asciiz rather than the name-pointer +# walk itself, so they are listed here otherwise they are silently dropped. _NAME_RVA_ERROR_PRIORITY = [ "name_rva_missing", "name_rva_zero", + "rva_zero", # _read_asciiz's own zero-RVA guard "read_failed", + "empty_read", # get_data returned zero bytes "unterminated", ] @@ -241,6 +246,18 @@ def _validate_name_pointers(exp: Dict[str, Any], "num_functions": num_funcs, "sub_reason": "out_of_range"}, )) + elif "ordinal_index_duplicate" in entry_errors: + # Two name pointers resolve to the same EAT index. The function + # view keeps only the last, so an export name becomes unreachable + # through the resolved-function list while remaining present in + # the name pointer table. + issues.append(StructuralIssue( + issue=ReasonCodes.EXPORT_NAME_ORDINAL_INDEX_INVALID, + details={"index": index, + "ordinal_index": entry.get("ordinal_index"), + "name": entry.get("name"), + "sub_reason": "duplicate"}, + )) def _validate_name_pointer_ordering(exp: Dict[str, Any], diff --git a/iocx/validators/imports.py b/iocx/validators/imports.py new file mode 100644 index 0000000..50a3934 --- /dev/null +++ b/iocx/validators/imports.py @@ -0,0 +1,225 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 + +""" +Validate the import-table structure produced by parser pe_imports. + +Absence of an import directory is NOT a structural defect - a few legitimate +binaries (resource-only DLLs, some drivers) import nothing. We only emit +codes when the directory is present and structurally malformed. + +PLACEMENT IS NOT CHECKED HERE. The import directory (index 1) and the IAT +directory (index 12) are both plain RVAs, so the RVA-graph backbone (2.5) +already owns their placement, mapping, bounds and mutual-overlap truth. This +validator deliberately defers, in the same way relocations (2.13) and debug +(2.14) do, rather than asserting locally and double-counting. + +The bound-import directory (index 11) is a separate structure that the +parser does not decode; descriptor-level bound STATE is interpreted here, +but the BOUND_IMPORT table itself is out of scope. + +Reason codes emitted: + IMPORT_DIRECTORY_INVALID_HEADER + IMPORT_TABLE_TRUNCATED + IMPORT_DESCRIPTOR_INVALID + IMPORT_DLL_NAME_INVALID + IMPORT_ENTRY_INVALID +""" + +from typing import Any, Dict, List + +from iocx.reason_codes import ReasonCodes +from iocx.validators.schema import StructuralIssue +from iocx.schemas.internal_schema import InternalMetadata +from .decorators import depends_on + + +# Priority-resolved sub-reasons. First match wins, so the ordering encodes +# which fault is the more fundamental when an entry carries several. +# +# Every tag the parser can place in the corresponding list appears here. +# A tag with no entry is silently dropped by _first_matching, so these lists +# are the parser->validator contract and must be kept exhaustive. + +# descriptor["errors"], DLL-name class. +# Ordered: the RVA itself, then read faults from _read_asciiz, then content +# faults from the three-way split. The read tags and the content tags are +# mutually exclusive in practice (a non-None err skips the content check), so +# their relative order is defensive rather than load-bearing. +_DLL_NAME_ERROR_PRIORITY = [ + "dll_name_rva_zero", + "rva_zero", + "read_failed", + "empty_read", + "unterminated", + "non_ascii", + "dll_name_empty", + "dll_name_not_printable", + "dll_name_too_long", +] + +# descriptor["errors"], thunk-source class. Mutually exclusive by +# construction: the parser returns immediately after appending either. +_THUNK_SOURCE_ERROR_PRIORITY = [ + "names_unrecoverable_bound_no_int", + "no_thunk_array", +] + +# entry["errors"]. +# Ordered by decode stage: the ordinal path, then IMAGE_IMPORT_BY_NAME read +# faults, then name content faults. +_ENTRY_ERROR_PRIORITY = [ + "ordinal_zero", + "name_rva_zero", + "name_read_failed", + "name_too_short", + "hint_unpack_failed", + "name_unterminated", + "name_non_ascii", + "name_empty", + "name_not_printable", +] + +# Cap on per-entry issues raised for a single descriptor, so a hostile +# import table cannot flood the stream. The true count is always in details. +_MAX_ENTRY_ISSUES_PER_DESCRIPTOR = 32 + + +@depends_on("internal") +def validate_imports(internal: InternalMetadata) -> List[StructuralIssue]: + issues: List[StructuralIssue] = [] + + imp = internal.get("import_struct") + if imp is None: + return issues # no import directory - not a defect + + # Top-level decode failure short-circuits: without a descriptor array + if imp.get("errors"): + issues.append(StructuralIssue( + issue=ReasonCodes.IMPORT_DIRECTORY_INVALID_HEADER, + details={"sub_reason": "top_level_decode", + "errors": list(imp["errors"])}, + )) + return issues + + _validate_truncations(imp, issues) + _validate_descriptors(imp, issues) + + return issues + + +# ================================================================= +# Truncations +# ================================================================= + +def _validate_truncations(imp: Dict[str, Any], + issues: List[StructuralIssue]) -> None: + """ + One issue per parser truncation tag, so the consumer sees one issue per + truncated table rather than a single bundled report. + + Note the thunk-array tags are prefixed by the array actually read - `int` + when OriginalFirstThunk was present, `iat_fallback` when the parser fell + back to FirstThunk. That distinction is preserved verbatim in `table` so + a consumer can tell which array was short. + """ + for tag in imp.get("truncations", []) or []: + issues.append(StructuralIssue( + issue=ReasonCodes.IMPORT_TABLE_TRUNCATED, + details={"table": tag}, + )) + + +# ================================================================= +# Descriptors +# ================================================================= + +def _validate_descriptors(imp: Dict[str, Any], + issues: List[StructuralIssue]) -> None: + for descriptor in imp.get("descriptors", []) or []: + index = descriptor.get("index") + errors = descriptor.get("errors", []) or [] + + # ---- DLL name ---- + reason = _first_matching(errors, _DLL_NAME_ERROR_PRIORITY) + if reason != "unknown": + issues.append(StructuralIssue( + issue=ReasonCodes.IMPORT_DLL_NAME_INVALID, + details={"index": index, + "dll_name_rva": descriptor.get("name_rva"), + "dll_name": descriptor.get("dll_name"), + "sub_reason": reason}, + )) + + # ---- Thunk source ---- + # A descriptor with no readable name source names no imports at all. + # Distinct from the DLL-name class: the module is identified, its + # imported symbols are not. + reason = _first_matching(errors, _THUNK_SOURCE_ERROR_PRIORITY) + if reason != "unknown": + issues.append(StructuralIssue( + issue=ReasonCodes.IMPORT_DESCRIPTOR_INVALID, + details={"index": index, + "dll_name": descriptor.get("dll_name"), + "bound_state": descriptor.get("bound_state"), + "original_first_thunk": + descriptor.get("original_first_thunk"), + "first_thunk": descriptor.get("first_thunk"), + "sub_reason": reason}, + )) + + _validate_entries(descriptor, issues) + + +def _validate_entries(descriptor: Dict[str, Any], + issues: List[StructuralIssue]) -> None: + """ + Emit per-import-entry issues, at most one per entry, priority-resolved. + + Emission is capped so a hostile table cannot flood the stream; + `invalid_entry_count` always carries the true total. + """ + descriptor_index = descriptor.get("index") + dll_name = descriptor.get("dll_name") + + invalid: List[Dict[str, Any]] = [] + for entry in descriptor.get("imports", []) or []: + entry_errors = entry.get("errors", []) or [] + if not entry_errors: + continue + reason = _first_matching(entry_errors, _ENTRY_ERROR_PRIORITY) + if reason == "unknown": + continue + invalid.append({"entry": entry, "sub_reason": reason}) + + if not invalid: + return + + for item in invalid[:_MAX_ENTRY_ISSUES_PER_DESCRIPTOR]: + entry = item["entry"] + issues.append(StructuralIssue( + issue=ReasonCodes.IMPORT_ENTRY_INVALID, + details={ + "descriptor_index": descriptor_index, + "dll_name": dll_name, + "entry_index": entry.get("index"), + "is_ordinal": entry.get("is_ordinal"), + "ordinal": entry.get("ordinal"), + "name": entry.get("name"), + "name_rva": entry.get("name_rva"), + "sub_reason": item["sub_reason"], + "invalid_entry_count": len(invalid), + }, + )) + + +# ================================================================= +# Helpers +# ================================================================= + +def _first_matching(errors: List[str], candidates: List[str]) -> str: + """Return the first tag from `candidates` present in `errors`.""" + for c in candidates: + if c in errors: + return c + return "unknown" diff --git a/iocx/validators/relocations.py b/iocx/validators/relocations.py index dfd21fa..7cd0dca 100644 --- a/iocx/validators/relocations.py +++ b/iocx/validators/relocations.py @@ -41,6 +41,7 @@ "size_of_block_too_small", "size_of_block_not_word_aligned", "entry_count_exceeds_max", + "highadj_missing_adjustment", ] # Cap on how many invalid-entry issues a single block may raise, so a diff --git a/iocx/validators/resources.py b/iocx/validators/resources.py index d66c0bf..88d56ec 100644 --- a/iocx/validators/resources.py +++ b/iocx/validators/resources.py @@ -1,7 +1,26 @@ # Copyright (c) 2026 MalX Labs and contributors # SPDX-License-Identifier: MPL-2.0 -from typing import Dict, Any, List, Set +""" +Validate the resource directory tree produced by parser pe_resources. + +Absence of a resource directory is NOT a structural defect. Neither is the +absence of a .rsrc section: some producers place resources elsewhere, and +this validator's bounds checks are all expressed against .rsrc, so without +it there is nothing to compare against and we stay silent rather than +guess. + +Parser tombstone tags consumed here: + entry_decode_failed - one entry in a directory was unreadable + directory_entries_unavailable - a directory's entry list was unreadable + string_table_walk_failed - the RT_STRING traversal raised + +All three describe recoverability rather than placement, so they map to +their own codes and never displace the bounds findings below. +""" + +from typing import Any, Dict, List, Set + from iocx.reason_codes import ReasonCodes from iocx.validators.schema import StructuralIssue from iocx.schemas.internal_schema import InternalMetadata @@ -9,13 +28,23 @@ from .decorators import depends_on +# Per-directory parser tags, priority-resolved so a directory carrying both +# reports the more fundamental one. An unreadable entry LIST subsumes any +# per-entry failure, because no entry was reached at all. +_DIRECTORY_ERROR_PRIORITY = [ + "directory_entries_unavailable", + "entry_decode_failed", +] + + @depends_on("internal", "analysis") -def validate_resources(metadata: InternalMetadata, analysis: AnalysisDict) -> List[StructuralIssue]: +def validate_resources(metadata: InternalMetadata, + analysis: AnalysisDict) -> List[StructuralIssue]: issues: List[StructuralIssue] = [] resources = metadata.get("resources_struct") if not resources: - return issues # No resource directory → no issues + return issues # No resource directory -> no issues sections = analysis["sections"] file_size = analysis["file_size"] @@ -24,14 +53,13 @@ def validate_resources(metadata: InternalMetadata, analysis: AnalysisDict) -> Li # --------------------------------------------------------- # Locate .rsrc section # --------------------------------------------------------- - rsrc_section = next((sec for sec in sections if sec["name"].lower() == ".rsrc"), None) + rsrc_section = next( + (sec for sec in sections if sec["name"].lower() == ".rsrc"), None) if rsrc_section is None: - return issues # No resource section → nothing to validate + return issues # No resource section -> nothing to validate rsrc_va = rsrc_section["virtual_address"] rsrc_vs = rsrc_section["virtual_size"] - rsrc_raw = rsrc_section["raw_address"] # reserved - rsrc_raw_size = rsrc_section["raw_size"] # reserved def rva_in_rsrc(rva: int, size: int = 0) -> bool: return rsrc_va <= rva and (rva + size) <= (rsrc_va + rsrc_vs) @@ -42,7 +70,8 @@ def va_overlaps_section(start: int, size: int, sec: Dict[str, Any]) -> bool: sec_end = sec_start + sec["virtual_size"] return max(start, sec_start) < min(end, sec_end) - def raw_overlaps_section(raw_start: int, size: int, sec: Dict[str, Any]) -> bool: + def raw_overlaps_section(raw_start: int, size: int, + sec: Dict[str, Any]) -> bool: end = raw_start + size sec_start = sec["raw_address"] sec_end = sec_start + sec["raw_size"] @@ -62,13 +91,13 @@ def validate_directory(dir_node: Dict[str, Any], depth: int = 0) -> None: # Two distinct malformations reach here: # * the ROOT directory lies outside .rsrc entirely, or # * a CHILD starts inside .rsrc but its extent overflows the end - - # the caller's entry check tests target_rva WITHOUT a size, so such - # a child passes there and is caught here. + # the caller's entry check tests target_rva WITHOUT a size, so + # such a child passes there and is caught here. # # A child lying wholly outside is already reported by the caller as - # RESOURCE_ENTRY_OUT_OF_BOUNDS, and its `continue` prevents descent, so - # the two codes never double-count. `depth` distinguishes the cases: - # depth 0 is the root, deeper values are the overflow case. + # RESOURCE_ENTRY_OUT_OF_BOUNDS, and its `continue` prevents descent, + # so the two codes never double-count. `depth` distinguishes the + # cases: depth 0 is the root, deeper values are the overflow case. if not rva_in_rsrc(rva, size): issues.append(StructuralIssue( issue=ReasonCodes.RESOURCE_DIRECTORY_OUT_OF_BOUNDS, @@ -79,9 +108,9 @@ def validate_directory(dir_node: Dict[str, Any], depth: int = 0) -> None: entries = dir_node["entries"] - # Reserved: only reachable if size is sourced from the on-disk - # IMAGE_RESOURCE_DIRECTORY header rather than derived from len(entries) - # Zero-length directory + # Reserved: unreachable while the parser derives size from the entry + # count (minimum 16). Only becomes live if size is ever sourced from + # the on-disk IMAGE_RESOURCE_DIRECTORY header instead. if size == 0: issues.append(StructuralIssue( issue=ReasonCodes.RESOURCE_DIRECTORY_ZERO_LENGTH, @@ -98,8 +127,26 @@ def validate_directory(dir_node: Dict[str, Any], depth: int = 0) -> None: return visited_dirs.add(rva) + # ---- Parser-level decode failures for THIS directory ---- + # Reported after the placement and loop checks, so a directory we + # decline to process does not also report its contents. The count is + # carried because one tag may stand for several skipped entries, and + # the gap between `size` and len(entries) is only interpretable with + # it. + dir_errors = dir_node.get("errors") or [] + reason = _first_matching(dir_errors, _DIRECTORY_ERROR_PRIORITY) + if reason != "unknown": + issues.append(StructuralIssue( + issue=ReasonCodes.RESOURCE_DIRECTORY_ENTRY_UNREADABLE, + details={"rva": rva, "depth": depth, "sub_reason": reason, + "failed_entry_count": dir_errors.count( + "entry_decode_failed"), + "declared_size": size, + "decoded_entries": len(entries)}, + )) + # Language layer (depth 2) must use integer LCIDs: - # Per PE spec, the Type → Name → Language tree's deepest directory + # Per PE spec, the Type -> Name -> Language tree's deepest directory # layer is keyed by language ID. Named entries here are malformed. if depth == 2: for e in entries: @@ -118,16 +165,17 @@ def validate_directory(dir_node: Dict[str, Any], depth: int = 0) -> None: if not rva_in_rsrc(target_rva): issues.append(StructuralIssue( issue=ReasonCodes.RESOURCE_ENTRY_OUT_OF_BOUNDS, - details={"directory_rva": rva, "target_rva": target_rva}, + details={"directory_rva": rva, + "target_rva": target_rva}, )) continue - validate_directory(target, depth + 1) # <-- depth bumped + validate_directory(target, depth + 1) continue # Data entries should only appear at depth 2 (Language layer): # A data leaf at depth 0 or 1 means the tree shape violates the - # Type → Name → Language hierarchy. + # Type -> Name -> Language hierarchy. if depth != 2: issues.append(StructuralIssue( issue=ReasonCodes.RESOURCE_DATA_AT_INVALID_DEPTH, @@ -158,12 +206,13 @@ def validate_directory(dir_node: Dict[str, Any], depth: int = 0) -> None: )) continue - # Raw bounds (data_raw == -1 sentinel from a guarded RVA→offset + # Raw bounds (data_raw == -1 sentinel from a guarded RVA->offset # lookup also lands here, preserving the existing reason code). if data_raw < 0 or data_raw + data_size > file_size: issues.append(StructuralIssue( issue=ReasonCodes.RESOURCE_DATA_OUT_OF_BOUNDS, - details={"data_raw": data_raw, "data_size": data_size, "file_size": file_size}, + details={"data_raw": data_raw, "data_size": data_size, + "file_size": file_size}, )) continue @@ -171,30 +220,33 @@ def validate_directory(dir_node: Dict[str, Any], depth: int = 0) -> None: if data_raw <= overlay_offset < data_raw + data_size: issues.append(StructuralIssue( issue=ReasonCodes.RESOURCE_DATA_OVERLAPS_OTHER_DATA, - details={"data_raw": data_raw, "data_size": data_size, "overlay_offset": overlay_offset}, + details={"data_raw": data_raw, "data_size": data_size, + "overlay_offset": overlay_offset}, )) - # Raw overlap with other sections + # Raw overlap with other sections. Stops at the first + # intersecting section: a blob crossing several reports once. for sec in sections: if sec is rsrc_section: continue - if raw_overlaps_section(data_raw, data_size, sec): issues.append(StructuralIssue( issue=ReasonCodes.RESOURCE_DATA_OVERLAPS_OTHER_DATA, - details={"data_raw": data_raw, "data_size": data_size, "section": sec["name"]}, + details={"data_raw": data_raw, "data_size": data_size, + "section": sec["name"]}, )) break - # VA overlap with other sections + # VA overlap with other sections. Independent of the raw check + # above, so a blob may report once per check. for sec in sections: if sec is rsrc_section: continue - if va_overlaps_section(data_rva, data_size, sec): issues.append(StructuralIssue( issue=ReasonCodes.RESOURCE_DATA_OVERLAPS_OTHER_DATA, - details={"data_rva": data_rva, "data_size": data_size, "section": sec["name"]}, + details={"data_rva": data_rva, "data_size": data_size, + "section": sec["name"]}, )) break @@ -206,6 +258,22 @@ def validate_directory(dir_node: Dict[str, Any], depth: int = 0) -> None: # --------------------------------------------------------- # String table validation # --------------------------------------------------------- + # A walk failure and a genuine absence of RT_STRING resources both leave + # string_tables empty, so the parser records the failure explicitly. + # Without this the two are indistinguishable and a malformed subtree is + # reported as clean. Deliberately does NOT return: a partial walk still + # has collected tables worth bounds-checking. + if "string_table_walk_failed" in (resources.get("errors") or []): + issues.append(StructuralIssue( + issue=ReasonCodes.RESOURCE_STRING_TABLE_UNREADABLE, + details={"sub_reason": "walk_failed", + "collected_tables": len( + resources.get("string_tables") or [])}, + )) + + # One issue per file rather than per table: a corrupt string-table + # region is a property of the resource section, and reporting every + # entry would flood on a systematically broken tree. for st in resources.get("string_tables", []): rva = st["rva"] size = st["size"] @@ -217,3 +285,11 @@ def validate_directory(dir_node: Dict[str, Any], depth: int = 0) -> None: break return issues + + +def _first_matching(errors: List[str], candidates: List[str]) -> str: + """Return the first tag from `candidates` present in `errors`.""" + for c in candidates: + if c in errors: + return c + return "unknown" diff --git a/iocx/validators/signature.py b/iocx/validators/signature.py index ddf65d4..501f334 100644 --- a/iocx/validators/signature.py +++ b/iocx/validators/signature.py @@ -44,13 +44,6 @@ from iocx.schemas.analysis import AnalysisDict from .decorators import depends_on -# Parser per-certificate error tags that indicate a genuine structural decode -# failure (as opposed to a bad field VALUE, which the SIGNATURE_INVALID_* -# checks below already own). Kept deliberately narrow to avoid double-counting. -_STRUCTURAL_CERT_ERROR_TAGS = { - "length_too_small", # also covered by SIGNATURE_INVALID_LENGTH; see note -} - @depends_on("internal", "metadata", "analysis") def validate_signature(internal: InternalMetadata, diff --git a/iocx/validators/version_info.py b/iocx/validators/version_info.py index 03481dd..7afbe4e 100644 --- a/iocx/validators/version_info.py +++ b/iocx/validators/version_info.py @@ -8,9 +8,30 @@ custom-action DLLs and many cross-compiled binaries legitimately omit it. We only emit structural codes when an RT_VERSION resource is present and malformed. + +Parser tombstone tags and where each is consumed +------------------------------------------------ +The parser records faults in several lists at different levels, and the +level determines which branch here reads them: + + decoded is False -> the whole top-level `errors` list is forwarded by the + "undecoded" branch: leaf_struct_unpack, read_failed, too_short, + header_unpack. + + decoded is True -> the "undecoded" branch is skipped, so tags appended to + the top-level list AFTER that point need their own consumers: + * fixed_file_info_unpack / fixed_file_info_truncated, read by the + FFI parse_failed branch via its startswith filter; + * child_header_unpack / child_length_invalid / unknown_child, read + by the child-dispatch branch below. Without it a blob carrying an + unrecognised child, or a child whose wLength runs past the + envelope, reports entirely clean. + + per-item lists -> string_file_info[].errors, its tables[].errors, and + var_file_info[].errors are each forwarded wholesale by their loops. """ -from typing import List +from typing import Any, Dict, List from iocx.reason_codes import ReasonCodes from iocx.validators.schema import StructuralIssue @@ -19,8 +40,22 @@ from .decorators import depends_on +# Child-dispatch faults, priority-resolved. The first two each `break` the +# parser's walk, so at most one of them occurs and never both; the third +# does not break and may repeat. A walk-terminating fault is the more +# fundamental fact - it means the remaining children were never examined - +# so both precede unknown_child. +_CHILD_ERROR_PRIORITY = [ + "child_max_exceeded", + "child_header_unpack", + "child_length_invalid", + "unknown_child", +] + + @depends_on("internal", "analysis") -def validate_version_info(metadata: InternalMetadata, analysis: AnalysisDict) -> List[StructuralIssue]: +def validate_version_info(metadata: InternalMetadata, + analysis: AnalysisDict) -> List[StructuralIssue]: issues: List[StructuralIssue] = [] vi = metadata.get("version_info_struct") @@ -64,16 +99,32 @@ def validate_version_info(metadata: InternalMetadata, analysis: AnalysisDict) -> details={"sub_reason": "length_inconsistent"}, )) + # ---- Child dispatch ---- + # These tags are appended to the top-level errors list AFTER `decoded` + # is set, so the undecoded branch above has already been skipped and no + # other branch reads them. Priority-resolved to one issue per blob; the + # full matching set is carried in details, since unknown_child can + # repeat and its count is the signal. + vi_errors = vi.get("errors") or [] + child_reason = _first_matching(vi_errors, _CHILD_ERROR_PRIORITY) + if child_reason != "unknown": + issues.append(StructuralIssue( + issue=ReasonCodes.RESOURCE_VERSIONINFO_INVALID_HEADER, + details={"sub_reason": child_reason, + "errors": [e for e in vi_errors + if e in _CHILD_ERROR_PRIORITY]}, + )) + # ---- VS_FIXEDFILEINFO ---- ffi = vi.get("fixed_file_info") if ffi is None: # Only flag if there were parse errors; some binaries legitimately # omit VS_FIXEDFILEINFO with wValueLength == 0. - if any(e.startswith("fixed_file_info") for e in vi.get("errors", [])): + if any(e.startswith("fixed_file_info") for e in vi_errors): issues.append(StructuralIssue( issue=ReasonCodes.RESOURCE_VERSIONINFO_INVALID_FIXEDINFO, details={"sub_reason": "parse_failed", - "errors": [e for e in vi["errors"] + "errors": [e for e in vi_errors if e.startswith("fixed_file_info")]}, )) else: @@ -117,3 +168,11 @@ def validate_version_info(metadata: InternalMetadata, analysis: AnalysisDict) -> )) return issues + + +def _first_matching(errors: List[str], candidates: List[str]) -> str: + """Return the first tag from `candidates` present in `errors`.""" + for c in candidates: + if c in errors: + return c + return "unknown" diff --git a/pyproject.toml b/pyproject.toml index a489d82..db43e41 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "iocx" -version = "0.7.6.1" +version = "0.7.6.2" description = "A deterministic, high‑performance static‑analysis engine that extracts high‑signal IOCs from PE binaries, text, and logs — built for SOC automation and modern threat‑analysis pipelines." authors = [ { name = "MalX Labs" } diff --git a/tests/contract/fixtures/layer1_core/clean_version_info.core.exe b/tests/contract/fixtures/layer1_core/clean_version_info.core.exe new file mode 100644 index 0000000..6a85570 Binary files /dev/null and b/tests/contract/fixtures/layer1_core/clean_version_info.core.exe differ diff --git a/tests/contract/fixtures/layer1_core/clean_version_info.full.exe b/tests/contract/fixtures/layer1_core/clean_version_info.full.exe new file mode 100644 index 0000000..6a85570 Binary files /dev/null and b/tests/contract/fixtures/layer1_core/clean_version_info.full.exe differ diff --git a/tests/contract/snapshots/basic.json b/tests/contract/snapshots/basic.json index 7850153..8e7c078 100644 --- a/tests/contract/snapshots/basic.json +++ b/tests/contract/snapshots/basic.json @@ -48,6 +48,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [] } diff --git a/tests/contract/snapshots/core.json b/tests/contract/snapshots/core.json index 287c5fc..1b0d3f7 100644 --- a/tests/contract/snapshots/core.json +++ b/tests/contract/snapshots/core.json @@ -47,5 +47,6 @@ "rich_header": null, "signatures": [], "has_signature": false - } + }, + "version_info": null } diff --git a/tests/contract/snapshots/deep.json b/tests/contract/snapshots/deep.json index 538e0ca..00b1f4b 100644 --- a/tests/contract/snapshots/deep.json +++ b/tests/contract/snapshots/deep.json @@ -48,6 +48,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [], "obfuscation": [] diff --git a/tests/contract/snapshots/enrich.json b/tests/contract/snapshots/enrich.json index b83b22b..d2c265e 100644 --- a/tests/contract/snapshots/enrich.json +++ b/tests/contract/snapshots/enrich.json @@ -48,5 +48,6 @@ "signatures": [], "has_signature": false }, + "version_info": null, "enrichment": {} } diff --git a/tests/contract/snapshots/full.json b/tests/contract/snapshots/full.json index bf5cb34..84e080e 100644 --- a/tests/contract/snapshots/full.json +++ b/tests/contract/snapshots/full.json @@ -48,6 +48,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [], "obfuscation": [], diff --git a/tests/contract/snapshots/layer1_core/clean_iocx_demo.core.json b/tests/contract/snapshots/layer1_core/clean_iocx_demo.core.json index b626dab..723df78 100644 --- a/tests/contract/snapshots/layer1_core/clean_iocx_demo.core.json +++ b/tests/contract/snapshots/layer1_core/clean_iocx_demo.core.json @@ -319,5 +319,6 @@ "rich_header": null, "signatures": [], "has_signature": false - } + }, + "version_info": null } diff --git a/tests/contract/snapshots/layer1_core/clean_version_info.core.json b/tests/contract/snapshots/layer1_core/clean_version_info.core.json new file mode 100644 index 0000000..b9db810 --- /dev/null +++ b/tests/contract/snapshots/layer1_core/clean_version_info.core.json @@ -0,0 +1,548 @@ +{ + "file": "tests/contract/fixtures/layer1_core/clean_version_info.core.exe", + "type": "PE", + "iocs": { + "urls": [], + "domains": [], + "ips": [ + "10.0.0.1" + ], + "hashes": [ + "0123456789" + ], + "emails": [], + "filepaths": [], + "base64": [], + "crypto.btc": [], + "crypto.eth": [] + }, + "metadata": { + "file_type": "PE", + "imports": [ + "KERNEL32.dll" + ], + "sections": [ + ".text", + ".rdata", + ".data", + ".pdata", + ".fptable", + ".rsrc", + ".reloc" + ], + "resources": [ + { + "type": "RT_VERSION", + "name": null, + "language": 1, + "language_name": "ar", + "codepage": null, + "size": 820, + "entropy": 3.4349, + "rva": 122976, + "raw_offset": 106080, + "errors": null + } + ], + "resource_strings": [ + "VS_VERSION_INFO", + "StringFileInfo", + "040904B0", + "CompanyName", + "MalX Labs", + "FileDescription", + "IOCX test fixture", + "FileVersion", + "10.0.0.1", + "InternalName", + "fixture", + "LegalCopyright", + " MalX Labs. All rights reserved.", + "OriginalFilename", + "FIXTURE.EXE", + "ProductName", + "IOCX", + "ProductVersion", + "Comments", + "Non-shortlist key: expect keys_filtered", + "VarFileInfo", + "Translation" + ], + "import_details": [ + { + "dll": "KERNEL32.dll", + "function": "QueryPerformanceCounter", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetCurrentProcessId", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetCurrentThreadId", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetSystemTimeAsFileTime", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "InitializeSListHead", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "RtlCaptureContext", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "RtlLookupFunctionEntry", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "RtlVirtualUnwind", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "IsDebuggerPresent", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "UnhandledExceptionFilter", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "SetUnhandledExceptionFilter", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetStartupInfoW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "IsProcessorFeaturePresent", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetModuleHandleW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "WriteConsoleW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "RtlUnwindEx", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetLastError", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "SetLastError", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "EnterCriticalSection", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "LeaveCriticalSection", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "DeleteCriticalSection", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "InitializeCriticalSectionAndSpinCount", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "TlsAlloc", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "TlsGetValue", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "TlsSetValue", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "TlsFree", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FreeLibrary", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetProcAddress", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "LoadLibraryExW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "EncodePointer", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "RaiseException", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "RtlPcToFileHeader", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetStdHandle", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "WriteFile", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetModuleFileNameW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetCurrentProcess", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "ExitProcess", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "TerminateProcess", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetModuleHandleExW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetCommandLineA", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetCommandLineW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "HeapAlloc", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "HeapFree", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FindClose", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FindFirstFileExW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FindNextFileW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "IsValidCodePage", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetACP", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetOEMCP", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetCPInfo", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "MultiByteToWideChar", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "WideCharToMultiByte", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetEnvironmentStringsW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FreeEnvironmentStringsW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "SetEnvironmentVariableW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "SetStdHandle", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetFileType", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetStringTypeW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FlsAlloc", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FlsGetValue", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FlsSetValue", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FlsFree", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "InitializeCriticalSectionEx", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "VirtualProtect", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "CompareStringW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "LCMapStringW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetProcessHeap", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "HeapSize", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "HeapReAlloc", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FlushFileBuffers", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetConsoleOutputCP", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetConsoleMode", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "SetFilePointerEx", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "CreateFileW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "CloseHandle", + "ordinal": null + } + ], + "delayed_imports": [], + "bound_imports": [], + "exports": [], + "tls": null, + "header": { + "entry_point": 4708, + "image_base": 5368709120, + "subsystem": 3, + "subsystem_name": "WINDOWS_CUI", + "timestamp": 1788865310, + "machine": 34404, + "machine_name": "AMD64", + "characteristics": 34 + }, + "optional_header": { + "section_alignment": 4096, + "file_alignment": 512, + "size_of_image": 131072, + "size_of_headers": 1024, + "linker_version": "14.44", + "os_version": "6.0", + "subsystem_version": "6.0", + "dll_characteristics": 33120, + "dll_characteristics_flags": [ + "HIGH_ENTROPY_VA", + "DYNAMIC_BASE", + "NX_COMPAT", + "TERMINAL_SERVER_AWARE" + ], + "dll_characteristics_unknown_bits": null, + "win32_version_value": 0, + "loader_flags": 0, + "stack_reserve_size": 1048576, + "stack_commit_size": 4096, + "heap_reserve_size": 1048576, + "heap_commit_size": 4096 + }, + "rich_header": { + "key": "c3485428", + "raw_data": "87293a7bc3485428c3485428c3485428bac951294e485428bac95029cf485428bac95729cb485428bac95529c0485428c34855289f48542844c15729ca48542844c15029d348542844c15129eb48542858c15029c248542858c1ab28c248542858c15629c2485428", + "clear_data": "44616e53000000000000000000000000798105018d000000798104010c00000079810301080000007981010103000000000001005c0000008789030109000000878904011000000087890501280000009b890401010000009b89ff00010000009b89020101000000", + "checksum": 676612291, + "values": [ + 17138041, + 141, + 17072505, + 12, + 17006969, + 8, + 16875897, + 3, + 65536, + 92, + 17009031, + 9, + 17074567, + 16, + 17140103, + 40, + 17074587, + 1, + 16746907, + 1, + 16943515, + 1 + ] + }, + "signatures": [], + "has_signature": false + }, + "version_info": { + "file_version": "10.0.0.1", + "product_version": "10.0.0.1", + "tables": [ + { + "lang_codepage": "040904B0", + "strings": { + "CompanyName": "MalX Labs", + "FileDescription": "IOCX test fixture", + "FileVersion": "10.0.0.1", + "InternalName": "fixture", + "LegalCopyright": "\u00a9 MalX Labs. All rights reserved.", + "OriginalFilename": "FIXTURE.EXE", + "ProductName": "IOCX", + "ProductVersion": "10.0.0.1" + } + } + ], + "languages": [ + "040904B0" + ], + "translations": [ + "040904B0" + ], + "decoded": true, + "structural_error_count": 0, + "truncated": [ + "keys_filtered" + ] + } +} diff --git a/tests/contract/snapshots/layer1_core/clean_version_info.full.json b/tests/contract/snapshots/layer1_core/clean_version_info.full.json new file mode 100644 index 0000000..0cbc0f4 --- /dev/null +++ b/tests/contract/snapshots/layer1_core/clean_version_info.full.json @@ -0,0 +1,860 @@ +{ + "file": "tests/contract/fixtures/layer1_core/clean_version_info.full.exe", + "type": "PE", + "iocs": { + "urls": [], + "domains": [], + "ips": [ + "10.0.0.1" + ], + "hashes": [ + "0123456789" + ], + "emails": [], + "filepaths": [], + "base64": [], + "crypto.btc": [], + "crypto.eth": [] + }, + "metadata": { + "file_type": "PE", + "imports": [ + "KERNEL32.dll" + ], + "sections": [ + ".text", + ".rdata", + ".data", + ".pdata", + ".fptable", + ".rsrc", + ".reloc" + ], + "resources": [ + { + "type": "RT_VERSION", + "name": null, + "language": 1, + "language_name": "ar", + "codepage": null, + "size": 820, + "entropy": 3.4349, + "rva": 122976, + "raw_offset": 106080, + "errors": null + } + ], + "resource_strings": [ + "VS_VERSION_INFO", + "StringFileInfo", + "040904B0", + "CompanyName", + "MalX Labs", + "FileDescription", + "IOCX test fixture", + "FileVersion", + "10.0.0.1", + "InternalName", + "fixture", + "LegalCopyright", + " MalX Labs. All rights reserved.", + "OriginalFilename", + "FIXTURE.EXE", + "ProductName", + "IOCX", + "ProductVersion", + "Comments", + "Non-shortlist key: expect keys_filtered", + "VarFileInfo", + "Translation" + ], + "import_details": [ + { + "dll": "KERNEL32.dll", + "function": "QueryPerformanceCounter", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetCurrentProcessId", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetCurrentThreadId", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetSystemTimeAsFileTime", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "InitializeSListHead", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "RtlCaptureContext", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "RtlLookupFunctionEntry", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "RtlVirtualUnwind", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "IsDebuggerPresent", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "UnhandledExceptionFilter", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "SetUnhandledExceptionFilter", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetStartupInfoW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "IsProcessorFeaturePresent", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetModuleHandleW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "WriteConsoleW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "RtlUnwindEx", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetLastError", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "SetLastError", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "EnterCriticalSection", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "LeaveCriticalSection", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "DeleteCriticalSection", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "InitializeCriticalSectionAndSpinCount", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "TlsAlloc", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "TlsGetValue", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "TlsSetValue", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "TlsFree", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FreeLibrary", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetProcAddress", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "LoadLibraryExW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "EncodePointer", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "RaiseException", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "RtlPcToFileHeader", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetStdHandle", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "WriteFile", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetModuleFileNameW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetCurrentProcess", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "ExitProcess", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "TerminateProcess", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetModuleHandleExW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetCommandLineA", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetCommandLineW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "HeapAlloc", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "HeapFree", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FindClose", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FindFirstFileExW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FindNextFileW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "IsValidCodePage", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetACP", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetOEMCP", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetCPInfo", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "MultiByteToWideChar", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "WideCharToMultiByte", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetEnvironmentStringsW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FreeEnvironmentStringsW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "SetEnvironmentVariableW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "SetStdHandle", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetFileType", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetStringTypeW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FlsAlloc", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FlsGetValue", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FlsSetValue", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FlsFree", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "InitializeCriticalSectionEx", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "VirtualProtect", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "CompareStringW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "LCMapStringW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetProcessHeap", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "HeapSize", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "HeapReAlloc", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "FlushFileBuffers", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetConsoleOutputCP", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "GetConsoleMode", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "SetFilePointerEx", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "CreateFileW", + "ordinal": null + }, + { + "dll": "KERNEL32.dll", + "function": "CloseHandle", + "ordinal": null + } + ], + "delayed_imports": [], + "bound_imports": [], + "exports": [], + "tls": null, + "header": { + "entry_point": 4708, + "image_base": 5368709120, + "subsystem": 3, + "subsystem_name": "WINDOWS_CUI", + "timestamp": 1788865310, + "machine": 34404, + "machine_name": "AMD64", + "characteristics": 34 + }, + "optional_header": { + "section_alignment": 4096, + "file_alignment": 512, + "size_of_image": 131072, + "size_of_headers": 1024, + "linker_version": "14.44", + "os_version": "6.0", + "subsystem_version": "6.0", + "dll_characteristics": 33120, + "dll_characteristics_flags": [ + "HIGH_ENTROPY_VA", + "DYNAMIC_BASE", + "NX_COMPAT", + "TERMINAL_SERVER_AWARE" + ], + "dll_characteristics_unknown_bits": null, + "win32_version_value": 0, + "loader_flags": 0, + "stack_reserve_size": 1048576, + "stack_commit_size": 4096, + "heap_reserve_size": 1048576, + "heap_commit_size": 4096 + }, + "rich_header": { + "key": "c3485428", + "raw_data": "87293a7bc3485428c3485428c3485428bac951294e485428bac95029cf485428bac95729cb485428bac95529c0485428c34855289f48542844c15729ca48542844c15029d348542844c15129eb48542858c15029c248542858c1ab28c248542858c15629c2485428", + "clear_data": "44616e53000000000000000000000000798105018d000000798104010c00000079810301080000007981010103000000000001005c0000008789030109000000878904011000000087890501280000009b890401010000009b89ff00010000009b89020101000000", + "checksum": 676612291, + "values": [ + 17138041, + 141, + 17072505, + 12, + 17006969, + 8, + 16875897, + 3, + 65536, + 92, + 17009031, + 9, + 17074567, + 16, + 17140103, + 40, + 17074587, + 1, + 16746907, + 1, + 16943515, + 1 + ] + }, + "signatures": [], + "has_signature": false + }, + "version_info": { + "file_version": "10.0.0.1", + "product_version": "10.0.0.1", + "tables": [ + { + "lang_codepage": "040904B0", + "strings": { + "CompanyName": "MalX Labs", + "FileDescription": "IOCX test fixture", + "FileVersion": "10.0.0.1", + "InternalName": "fixture", + "LegalCopyright": "\u00a9 MalX Labs. All rights reserved.", + "OriginalFilename": "FIXTURE.EXE", + "ProductName": "IOCX", + "ProductVersion": "10.0.0.1", + "Comments": "Non-shortlist key: expect keys_filtered" + } + } + ], + "languages": [ + "040904B0" + ], + "translations": [ + "040904B0" + ], + "decoded": true, + "structural_error_count": 0, + "truncated": [] + }, + "analysis": { + "sections": [ + { + "name": ".text", + "raw_size": 57856, + "virtual_size": 57408, + "characteristics": 1610612768, + "entropy": 6.414454035980093 + }, + { + "name": ".rdata", + "raw_size": 39424, + "virtual_size": 39090, + "characteristics": 1073741888, + "entropy": 4.701216004495569 + }, + { + "name": ".data", + "raw_size": 3072, + "virtual_size": 7032, + "characteristics": 3221225536, + "entropy": 1.8915089779871357 + }, + { + "name": ".pdata", + "raw_size": 4096, + "virtual_size": 3972, + "characteristics": 1073741888, + "entropy": 4.735059804196878 + }, + { + "name": ".fptable", + "raw_size": 512, + "virtual_size": 256, + "characteristics": 3221225536, + "entropy": 0.0 + }, + { + "name": ".rsrc", + "raw_size": 1024, + "virtual_size": 920, + "characteristics": 1073741888, + "entropy": 3.0277249103536312 + }, + { + "name": ".reloc", + "raw_size": 2048, + "virtual_size": 1636, + "characteristics": 1107296320, + "entropy": 4.883604262379489 + } + ], + "obfuscation": [], + "extended": [ + { + "value": "summary", + "start": 0, + "end": 0, + "category": "pe_metadata", + "metadata": { + "dll_count": 1, + "import_count": 75, + "delayed_import_count": 0, + "bound_import_count": 0, + "export_count": 0, + "resource_count": 1, + "has_tls": false, + "has_signature": false + } + }, + { + "value": "imports", + "start": 0, + "end": 0, + "category": "pe_metadata", + "metadata": { + "dll": "KERNEL32.dll", + "functions": [ + "CloseHandle", + "CompareStringW", + "CreateFileW", + "DeleteCriticalSection", + "EncodePointer", + "EnterCriticalSection", + "ExitProcess", + "FindClose", + "FindFirstFileExW", + "FindNextFileW", + "FlsAlloc", + "FlsFree", + "FlsGetValue", + "FlsSetValue", + "FlushFileBuffers", + "FreeEnvironmentStringsW", + "FreeLibrary", + "GetACP", + "GetCommandLineA", + "GetCommandLineW", + "GetConsoleMode", + "GetConsoleOutputCP", + "GetCPInfo", + "GetCurrentProcess", + "GetCurrentProcessId", + "GetCurrentThreadId", + "GetEnvironmentStringsW", + "GetFileType", + "GetLastError", + "GetModuleFileNameW", + "GetModuleHandleExW", + "GetModuleHandleW", + "GetOEMCP", + "GetProcAddress", + "GetProcessHeap", + "GetStartupInfoW", + "GetStdHandle", + "GetStringTypeW", + "GetSystemTimeAsFileTime", + "HeapAlloc", + "HeapFree", + "HeapReAlloc", + "HeapSize", + "InitializeCriticalSectionAndSpinCount", + "InitializeCriticalSectionEx", + "InitializeSListHead", + "IsDebuggerPresent", + "IsProcessorFeaturePresent", + "IsValidCodePage", + "LCMapStringW", + "LeaveCriticalSection", + "LoadLibraryExW", + "MultiByteToWideChar", + "QueryPerformanceCounter", + "RaiseException", + "RtlCaptureContext", + "RtlLookupFunctionEntry", + "RtlPcToFileHeader", + "RtlUnwindEx", + "RtlVirtualUnwind", + "SetEnvironmentVariableW", + "SetFilePointerEx", + "SetLastError", + "SetStdHandle", + "SetUnhandledExceptionFilter", + "TerminateProcess", + "TlsAlloc", + "TlsFree", + "TlsGetValue", + "TlsSetValue", + "UnhandledExceptionFilter", + "VirtualProtect", + "WideCharToMultiByte", + "WriteConsoleW", + "WriteFile" + ] + } + }, + { + "value": "exports", + "start": 0, + "end": 0, + "category": "pe_metadata", + "metadata": { + "count": 0, + "names": [], + "forwarded": [] + } + }, + { + "value": "header", + "start": 0, + "end": 0, + "category": "pe_metadata", + "metadata": { + "entry_point": 4708, + "image_base": 5368709120, + "subsystem": 3, + "subsystem_name": "WINDOWS_CUI", + "timestamp": 1788865310, + "machine": 34404, + "machine_name": "AMD64", + "characteristics": 34 + } + }, + { + "value": "optional_header", + "start": 0, + "end": 0, + "category": "pe_metadata", + "metadata": { + "section_alignment": 4096, + "file_alignment": 512, + "size_of_image": 131072, + "size_of_headers": 1024, + "linker_version": "14.44", + "os_version": "6.0", + "subsystem_version": "6.0", + "dll_characteristics": 33120, + "dll_characteristics_flags": [ + "HIGH_ENTROPY_VA", + "DYNAMIC_BASE", + "NX_COMPAT", + "TERMINAL_SERVER_AWARE" + ], + "dll_characteristics_unknown_bits": null, + "win32_version_value": 0, + "loader_flags": 0, + "stack_reserve_size": 1048576, + "stack_commit_size": 4096, + "heap_reserve_size": 1048576, + "heap_commit_size": 4096 + } + }, + { + "value": "rich_header", + "start": 0, + "end": 0, + "category": "pe_metadata", + "metadata": { + "key": "c3485428", + "raw_data": "87293a7bc3485428c3485428c3485428bac951294e485428bac95029cf485428bac95729cb485428bac95529c0485428c34855289f48542844c15729ca48542844c15029d348542844c15129eb48542858c15029c248542858c1ab28c248542858c15629c2485428", + "clear_data": "44616e53000000000000000000000000798105018d000000798104010c00000079810301080000007981010103000000000001005c0000008789030109000000878904011000000087890501280000009b890401010000009b89ff00010000009b89020101000000", + "checksum": 676612291, + "values": [ + 17138041, + 141, + 17072505, + 12, + 17006969, + 8, + 16875897, + 3, + 65536, + 92, + 17009031, + 9, + 17074567, + 16, + 17140103, + 40, + 17074587, + 1, + 16746907, + 1, + 16943515, + 1 + ] + } + }, + { + "value": "resources", + "start": 0, + "end": 0, + "category": "pe_metadata", + "metadata": { + "count": 1, + "types": [ + "RT_VERSION" + ], + "entropy_min": 3.4349, + "entropy_max": 3.4349, + "entropy_avg": 3.4349 + } + } + ], + "heuristics": [ + { + "value": "anti_debug_heuristic", + "start": 0, + "end": 0, + "category": "pe_heuristic", + "metadata": { + "dll": "kernel32.dll", + "function": "QueryPerformanceCounter", + "reason": "timing_api_import" + } + }, + { + "value": "anti_debug_heuristic", + "start": 0, + "end": 0, + "category": "pe_heuristic", + "metadata": { + "dll": "kernel32.dll", + "function": "IsDebuggerPresent", + "reason": "anti_debug_api_import" + } + }, + { + "value": "pe_structure_anomaly", + "start": 0, + "end": 0, + "category": "pe_heuristic", + "metadata": { + "check": 5368775264, + "dispatch": 5368775280, + "table": 0, + "count": 0, + "reason": "load_config_guard_cf_inconsistent" + } + }, + { + "value": "pe_structure_anomaly", + "start": 0, + "end": 0, + "category": "pe_heuristic", + "metadata": { + "cookie_rva": 5368815680, + "sub_reason": "unmapped", + "reason": "load_config_cookie_invalid" + } + } + ] + } +} diff --git a/tests/contract/snapshots/layer2_edge/load_config_clang.full.json b/tests/contract/snapshots/layer2_edge/load_config_clang.full.json index b0ca6b8..b07de2c 100644 --- a/tests/contract/snapshots/layer2_edge/load_config_clang.full.json +++ b/tests/contract/snapshots/layer2_edge/load_config_clang.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { diff --git a/tests/contract/snapshots/layer2_edge/load_config_cookie_valid.full.json b/tests/contract/snapshots/layer2_edge/load_config_cookie_valid.full.json index 725727e..c55bab9 100644 --- a/tests/contract/snapshots/layer2_edge/load_config_cookie_valid.full.json +++ b/tests/contract/snapshots/layer2_edge/load_config_cookie_valid.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { diff --git a/tests/contract/snapshots/layer2_edge/load_config_full_msvc.full.json b/tests/contract/snapshots/layer2_edge/load_config_full_msvc.full.json index 5bbd33c..7bf70b0 100644 --- a/tests/contract/snapshots/layer2_edge/load_config_full_msvc.full.json +++ b/tests/contract/snapshots/layer2_edge/load_config_full_msvc.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { diff --git a/tests/contract/snapshots/layer2_edge/load_config_large_padded.full.json b/tests/contract/snapshots/layer2_edge/load_config_large_padded.full.json index 3b25e88..f226c89 100644 --- a/tests/contract/snapshots/layer2_edge/load_config_large_padded.full.json +++ b/tests/contract/snapshots/layer2_edge/load_config_large_padded.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { diff --git a/tests/contract/snapshots/layer2_edge/load_config_minimal_mingw.full.json b/tests/contract/snapshots/layer2_edge/load_config_minimal_mingw.full.json index e5b2cda..2344ee5 100644 --- a/tests/contract/snapshots/layer2_edge/load_config_minimal_mingw.full.json +++ b/tests/contract/snapshots/layer2_edge/load_config_minimal_mingw.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,10 +153,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_too_small", "rva": 12288, "size": 16, - "min_size": 112 + "min_size": 112, + "reason": "load_config_too_small" } } ] diff --git a/tests/contract/snapshots/layer2_edge/load_config_seh_table.full.json b/tests/contract/snapshots/layer2_edge/load_config_seh_table.full.json index adbc2eb..cb8cc02 100644 --- a/tests/contract/snapshots/layer2_edge/load_config_seh_table.full.json +++ b/tests/contract/snapshots/layer2_edge/load_config_seh_table.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,10 +153,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_too_small", "rva": 12288, "size": 44, - "min_size": 112 + "min_size": 112, + "reason": "load_config_too_small" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/broken_rva_addresses.full.json b/tests/contract/snapshots/layer3_adversarial/broken_rva_addresses.full.json index fab3a1c..e4a9c9f 100644 --- a/tests/contract/snapshots/layer3_adversarial/broken_rva_addresses.full.json +++ b/tests/contract/snapshots/layer3_adversarial/broken_rva_addresses.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,10 +153,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_overlay", "entry_point": 4096, "entry_point_file_offset": 512, - "overlay_offset": 392 + "overlay_offset": 392, + "reason": "entrypoint_in_overlay" } }, { @@ -164,10 +165,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_overlaps_headers", "section": ".zero", "raw_address": 0, - "size_of_headers": 512 + "size_of_headers": 512, + "reason": "section_overlaps_headers" } }, { @@ -176,8 +177,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_zero_length", - "section": ".zero" + "section": ".zero", + "reason": "section_zero_length" } }, { @@ -186,11 +187,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_out_of_order_raw", "raw_addresses": [ 512, 0 - ] + ], + "reason": "section_out_of_order_raw" } }, { @@ -199,11 +200,21 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_out_of_range", "directory": "IMAGE_DIRECTORY_ENTRY_IMPORT", "rva": 36864, "size": 512, - "size_of_image": 16384 + "size_of_image": 16384, + "reason": "data_directory_out_of_range" + } + }, + { + "value": "pe_structure_anomaly", + "start": 0, + "end": 0, + "category": "pe_heuristic", + "metadata": { + "table": "import_descriptor_read_failed", + "reason": "import_table_truncated" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/corrupted_data_directories.full.json b/tests/contract/snapshots/layer3_adversarial/corrupted_data_directories.full.json index fd5e9ac..6fcfef5 100644 --- a/tests/contract/snapshots/layer3_adversarial/corrupted_data_directories.full.json +++ b/tests/contract/snapshots/layer3_adversarial/corrupted_data_directories.full.json @@ -57,6 +57,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -144,10 +145,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_overlay", "entry_point": 4096, "entry_point_file_offset": 512, - "overlay_offset": 392 + "overlay_offset": 392, + "reason": "entrypoint_in_overlay" } }, { @@ -156,11 +157,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_out_of_range", "directory": "IMAGE_DIRECTORY_ENTRY_RESOURCE", "rva": 8192, "size": 12288, - "size_of_image": 12288 + "size_of_image": 12288, + "reason": "data_directory_out_of_range" } }, { @@ -169,11 +170,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_out_of_range", "directory": "IMAGE_DIRECTORY_ENTRY_EXCEPTION", "rva": 12032, "size": 8192, - "size_of_image": 12288 + "size_of_image": 12288, + "reason": "data_directory_out_of_range" } }, { @@ -182,9 +183,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_overlap", "directory_a": "IMAGE_DIRECTORY_ENTRY_RESOURCE", - "directory_b": "IMAGE_DIRECTORY_ENTRY_EXCEPTION" + "directory_b": "IMAGE_DIRECTORY_ENTRY_EXCEPTION", + "reason": "data_directory_overlap" } }, { @@ -193,11 +194,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "certificate_table_malformed", + "sub_reason": "top_level_decode", "errors": [ "certificate_offset_past_eof" ], - "sub_reason": "top_level_decode" + "reason": "certificate_table_malformed" } }, { @@ -206,10 +207,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "exception_directory_out_of_bounds", "rva": 12032, "size": 8192, - "size_of_image": 12288 + "size_of_image": 12288, + "reason": "exception_directory_out_of_bounds" } }, { @@ -218,10 +219,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "exception_directory_size_not_multiple", "size": 8192, "entry_size": 12, - "remainder": 8 + "remainder": 8, + "reason": "exception_directory_size_not_multiple" } }, { @@ -230,8 +231,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "exception_table_truncated", - "table": "exception_table_ragged_tail" + "table": "exception_table_ragged_tail", + "reason": "exception_table_truncated" } }, { @@ -240,8 +241,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "exception_table_truncated", - "table": "exception_entry_read_failed" + "table": "exception_entry_read_failed", + "reason": "exception_table_truncated" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/crypto_entropy_payload.full.json b/tests/contract/snapshots/layer3_adversarial/crypto_entropy_payload.full.json index c3062e6..2f65731 100644 --- a/tests/contract/snapshots/layer3_adversarial/crypto_entropy_payload.full.json +++ b/tests/contract/snapshots/layer3_adversarial/crypto_entropy_payload.full.json @@ -331,6 +331,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -634,9 +635,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "timing_api_import", "dll": "kernel32.dll", - "function": "QueryPerformanceCounter" + "function": "QueryPerformanceCounter", + "reason": "timing_api_import" } }, { @@ -645,9 +646,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "IsDebuggerPresent" + "function": "IsDebuggerPresent", + "reason": "anti_debug_api_import" } }, { @@ -656,11 +657,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_guard_cf_inconsistent", "check": 5368717704, "dispatch": 5368717720, "table": 0, - "count": 0 + "count": 0, + "reason": "load_config_guard_cf_inconsistent" } }, { @@ -669,9 +670,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_invalid", "cookie_rva": 5368721472, - "sub_reason": "unmapped" + "sub_reason": "unmapped", + "reason": "load_config_cookie_invalid" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/directory_raw_mismatch.full.json b/tests/contract/snapshots/layer3_adversarial/directory_raw_mismatch.full.json index 2e953f3..9f57a08 100644 --- a/tests/contract/snapshots/layer3_adversarial/directory_raw_mismatch.full.json +++ b/tests/contract/snapshots/layer3_adversarial/directory_raw_mismatch.full.json @@ -57,6 +57,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -144,10 +145,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_overlay", "entry_point": 4096, "entry_point_file_offset": 512, - "overlay_offset": 392 + "overlay_offset": 392, + "reason": "entrypoint_in_overlay" } }, { @@ -156,13 +157,13 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_raw_mismatch", "directory": "IMAGE_DIRECTORY_ENTRY_EXCEPTION", "rva": 6144, "raw_offset": 2560, "section": ".text", "section_raw_start": 512, - "section_raw_end": 1024 + "section_raw_end": 1024, + "reason": "data_directory_raw_mismatch" } }, { @@ -171,10 +172,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_in_overlay", "directory": "IMAGE_DIRECTORY_ENTRY_EXCEPTION", "rva": 6144, - "raw_offset": 2560 + "raw_offset": 2560, + "reason": "data_directory_in_overlay" } }, { @@ -183,10 +184,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "exception_directory_size_not_multiple", "size": 256, "entry_size": 12, - "remainder": 4 + "remainder": 4, + "reason": "exception_directory_size_not_multiple" } }, { @@ -195,8 +196,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "exception_table_truncated", - "table": "exception_table_ragged_tail" + "table": "exception_table_ragged_tail", + "reason": "exception_table_truncated" } }, { @@ -205,8 +206,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "exception_table_truncated", - "table": "exception_entry_truncated" + "table": "exception_entry_truncated", + "reason": "exception_table_truncated" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/directory_zero_size_nonzero_rva.full.json b/tests/contract/snapshots/layer3_adversarial/directory_zero_size_nonzero_rva.full.json index 745952a..a9a35a4 100644 --- a/tests/contract/snapshots/layer3_adversarial/directory_zero_size_nonzero_rva.full.json +++ b/tests/contract/snapshots/layer3_adversarial/directory_zero_size_nonzero_rva.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,10 +153,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_overlay", "entry_point": 4096, "entry_point_file_offset": 512, - "overlay_offset": 512 + "overlay_offset": 512, + "reason": "entrypoint_in_overlay" } }, { @@ -164,10 +165,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_overlaps_headers", "section": ".zero", "raw_address": 0, - "size_of_headers": 512 + "size_of_headers": 512, + "reason": "section_overlaps_headers" } }, { @@ -176,8 +177,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_zero_length", - "section": ".zero" + "section": ".zero", + "reason": "section_zero_length" } }, { @@ -186,11 +187,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_out_of_order_raw", "raw_addresses": [ 512, 0 - ] + ], + "reason": "section_out_of_order_raw" } }, { @@ -199,11 +200,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_out_of_range", "directory": "IMAGE_DIRECTORY_ENTRY_IMPORT", "rva": 36864, "size": 512, - "size_of_image": 16384 + "size_of_image": 16384, + "reason": "data_directory_out_of_range" } }, { @@ -212,10 +213,20 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_zero_size_nonzero_rva", "directory": "IMAGE_DIRECTORY_ENTRY_LOAD_CONFIG", "rva": 4096, - "size": 0 + "size": 0, + "reason": "data_directory_zero_size_nonzero_rva" + } + }, + { + "value": "pe_structure_anomaly", + "start": 0, + "end": 0, + "category": "pe_heuristic", + "metadata": { + "table": "import_descriptor_read_failed", + "reason": "import_table_truncated" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/filepaths_strings_adversarial.full.json b/tests/contract/snapshots/layer3_adversarial/filepaths_strings_adversarial.full.json index 213c0ca..b67a383 100644 --- a/tests/contract/snapshots/layer3_adversarial/filepaths_strings_adversarial.full.json +++ b/tests/contract/snapshots/layer3_adversarial/filepaths_strings_adversarial.full.json @@ -27,7 +27,7 @@ "~user/docs/readme.md", "%APPDATA%\\MyApp\\config.json", "$HOME/.config/tool/settings.ini", - "C:\\Users\\Pub", + "C:\\Users\\Pub", "/usr/loc", "C:\\Temp\\my", "/var/log/my", diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_000_entrypoint_zero.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_000_entrypoint_zero.full.json index 3d17ebf..58660ac 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_000_entrypoint_zero.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_000_entrypoint_zero.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_001_entrypoint_negative.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_001_entrypoint_negative.full.json index 7e6447e..c30192b 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_001_entrypoint_negative.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_001_entrypoint_negative.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,10 +161,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 4294967295, "size_of_image": 16384, - "position": "beyond_size_of_image" + "position": "beyond_size_of_image", + "reason": "entrypoint_out_of_bounds" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_002_entrypoint_in_headers.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_002_entrypoint_in_headers.full.json index a5a4d8a..4796c60 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_002_entrypoint_in_headers.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_002_entrypoint_in_headers.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,9 +161,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 512, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -171,10 +172,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 512, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_003_entrypoint_gap_between_sections.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_003_entrypoint_gap_between_sections.full.json index e92bf5a..223ccc4 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_003_entrypoint_gap_between_sections.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_003_entrypoint_gap_between_sections.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,10 +161,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 7936, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_004_entrypoint_non_exec_section.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_004_entrypoint_non_exec_section.full.json index 579fed2..198073e 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_004_entrypoint_non_exec_section.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_004_entrypoint_non_exec_section.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,10 +161,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_section_not_executable", "entry_point": 8208, "section": ".rdata", - "characteristics": 1073741888 + "characteristics": 1073741888, + "reason": "entrypoint_section_not_executable" } }, { @@ -172,10 +173,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_non_code_section", "entry_point": 8208, "section": ".rdata", - "characteristics": 1073741888 + "characteristics": 1073741888, + "reason": "entrypoint_in_non_code_section" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_005_entrypoint_rsrc.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_005_entrypoint_rsrc.full.json index 8071d3a..602bbf4 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_005_entrypoint_rsrc.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_005_entrypoint_rsrc.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,10 +161,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_section_not_executable", "entry_point": 12320, "section": ".rsrc", - "characteristics": 1073741888 + "characteristics": 1073741888, + "reason": "entrypoint_section_not_executable" } }, { @@ -172,10 +173,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_non_code_section", "entry_point": 12320, "section": ".rsrc", - "characteristics": 1073741888 + "characteristics": 1073741888, + "reason": "entrypoint_in_non_code_section" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_006_entrypoint_discardable.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_006_entrypoint_discardable.full.json index 8e20fe3..9955533 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_006_entrypoint_discardable.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_006_entrypoint_discardable.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,10 +161,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_discardable_section", "entry_point": 4112, "section": ".text", - "characteristics": 1644167200 + "characteristics": 1644167200, + "reason": "entrypoint_in_discardable_section" } }, { @@ -172,9 +173,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_discardable_code", "section": ".text", - "characteristics": 1644167200 + "characteristics": 1644167200, + "reason": "section_discardable_code" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_007_entrypoint_zero_length_section.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_007_entrypoint_zero_length_section.full.json index 46463ec..385396d 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_007_entrypoint_zero_length_section.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_007_entrypoint_zero_length_section.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,10 +161,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_truncated_region", "entry_point": 4096, "section": ".text", - "sub_reason": "zero_length_section" + "sub_reason": "zero_length_section", + "reason": "entrypoint_in_truncated_region" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_008_entrypoint_beyond_virtual_size.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_008_entrypoint_beyond_virtual_size.full.json index ab38a91..96f52fe 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_008_entrypoint_beyond_virtual_size.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_008_entrypoint_beyond_virtual_size.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,10 +161,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 6144, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_009_entrypoint_in_overlay.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_009_entrypoint_in_overlay.full.json index fdd57b6..b18d3a4 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_009_entrypoint_in_overlay.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_009_entrypoint_in_overlay.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,10 +161,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 20480, "size_of_image": 16384, - "position": "beyond_size_of_image" + "position": "beyond_size_of_image", + "reason": "entrypoint_out_of_bounds" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_010_sections_rwx.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_010_sections_rwx.full.json index f79d137..4f5c5f6 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_010_sections_rwx.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_010_sections_rwx.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -171,9 +172,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "rwx_section", "section": ".text", - "characteristics": 3758096416 + "characteristics": 3758096416, + "reason": "rwx_section" } }, { @@ -182,8 +183,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -192,9 +193,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -203,10 +204,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -215,9 +216,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_rwx", "section": ".text", - "characteristics": 3758096416 + "characteristics": 3758096416, + "reason": "section_rwx" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_011_sections_code_not_exec.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_011_sections_code_not_exec.full.json index 668cf6a..011e609 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_011_sections_code_not_exec.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_011_sections_code_not_exec.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_non_executable_code_like", "section": ".text", - "characteristics": 1073741856 + "characteristics": 1073741856, + "reason": "section_non_executable_code_like" } }, { @@ -204,9 +205,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_codelike_name_not_executable", "section": ".text", - "characteristics": 1073741856 + "characteristics": 1073741856, + "reason": "section_codelike_name_not_executable" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_012_sections_codelike_not_exec.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_012_sections_codelike_not_exec.full.json index 1dd4f49..86fd754 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_012_sections_codelike_not_exec.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_012_sections_codelike_not_exec.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_non_executable_code_like", "section": ".text", - "characteristics": 1073741856 + "characteristics": 1073741856, + "reason": "section_non_executable_code_like" } }, { @@ -204,9 +205,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_codelike_name_not_executable", "section": ".text", - "characteristics": 1073741856 + "characteristics": 1073741856, + "reason": "section_codelike_name_not_executable" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_013_sections_non_ascii_name.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_013_sections_non_ascii_name.full.json index fb8cb1b..716eea5 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_013_sections_non_ascii_name.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_013_sections_non_ascii_name.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,8 +194,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_name_empty_or_padding", - "section": "" + "section": "", + "reason": "section_name_empty_or_padding" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_014_sections_empty_name.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_014_sections_empty_name.full.json index 08fbf90..3ecba3c 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_014_sections_empty_name.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_014_sections_empty_name.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,8 +194,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_name_empty_or_padding", - "section": "" + "section": "", + "reason": "section_name_empty_or_padding" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_015_sections_impossible_flags.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_015_sections_impossible_flags.full.json index 77355f3..83e69cc 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_015_sections_impossible_flags.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_015_sections_impossible_flags.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -171,9 +172,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "rwx_section", "section": ".text", - "characteristics": 3791650848 + "characteristics": 3791650848, + "reason": "rwx_section" } }, { @@ -182,8 +183,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -192,9 +193,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -203,10 +204,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -215,9 +216,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_rwx", "section": ".text", - "characteristics": 3791650848 + "characteristics": 3791650848, + "reason": "section_rwx" } }, { @@ -226,9 +227,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_impossible_flags", "section": ".text", - "characteristics": 3791650848 + "characteristics": 3791650848, + "reason": "section_impossible_flags" } }, { @@ -237,9 +238,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_discardable_code", "section": ".text", - "characteristics": 3791650848 + "characteristics": 3791650848, + "reason": "section_discardable_code" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_016_sections_raw_misaligned.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_016_sections_raw_misaligned.full.json index c98efee..294282d 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_016_sections_raw_misaligned.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_016_sections_raw_misaligned.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,11 +194,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_misaligned", "section": ".text", "raw_address": 1040, "raw_size": 512, - "file_alignment": 512 + "file_alignment": 512, + "reason": "section_raw_misaligned" } }, { @@ -206,9 +207,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_overlap", "section_a": ".text", - "section_b": ".rdata" + "section_b": ".rdata", + "reason": "section_raw_overlap" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_017_sections_overlap_headers.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_017_sections_overlap_headers.full.json index d959f00..2d3c4e9 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_017_sections_overlap_headers.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_017_sections_overlap_headers.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,10 +194,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_overlaps_headers", "section": ".text", "raw_address": 512, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "section_overlaps_headers" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_018_sections_zero_length.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_018_sections_zero_length.full.json index 2b655bc..168d784 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_018_sections_zero_length.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_018_sections_zero_length.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,8 +194,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_zero_length", - "section": ".text" + "section": ".text", + "reason": "section_zero_length" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_019_sections_raw_overlap.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_019_sections_raw_overlap.full.json index 7b5c66d..881823a 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_019_sections_raw_overlap.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_019_sections_raw_overlap.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,11 +194,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_misaligned", "section": ".rdata", "raw_address": 1280, "raw_size": 512, - "file_alignment": 512 + "file_alignment": 512, + "reason": "section_raw_misaligned" } }, { @@ -206,9 +207,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_overlap", "section_a": ".text", - "section_b": ".rdata" + "section_b": ".rdata", + "reason": "section_raw_overlap" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_020_sections_virtual_overlap.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_020_sections_virtual_overlap.full.json index 209fd57..481e973 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_020_sections_virtual_overlap.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_020_sections_virtual_overlap.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -179,8 +180,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -189,9 +190,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -200,10 +201,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -212,9 +213,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_overlap", "section_a": ".text", - "section_b": ".rdata" + "section_b": ".rdata", + "reason": "section_overlap" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_021_sections_out_of_order_raw.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_021_sections_out_of_order_raw.full.json index 690da5b..66b1b69 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_021_sections_out_of_order_raw.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_021_sections_out_of_order_raw.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,12 +194,12 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_out_of_order_raw", "raw_addresses": [ 1536, 1024, 2048 - ] + ], + "reason": "section_out_of_order_raw" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_022_sections_out_of_order_virtual.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_022_sections_out_of_order_virtual.full.json index c93ec7c..9b38607 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_022_sections_out_of_order_virtual.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_022_sections_out_of_order_virtual.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,12 +194,12 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_out_of_order_raw", "raw_addresses": [ 1536, 1024, 2048 - ] + ], + "reason": "section_out_of_order_raw" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_023_sections_negative_fields.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_023_sections_negative_fields.full.json index 4d94344..1d020b8 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_023_sections_negative_fields.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_023_sections_negative_fields.full.json @@ -55,6 +55,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [], "obfuscation": [], @@ -134,8 +135,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -144,9 +145,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_024_opt_size_of_image_too_small.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_024_opt_size_of_image_too_small.full.json index 6eea2ca..8ae2010 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_024_opt_size_of_image_too_small.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_024_opt_size_of_image_too_small.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 8192, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_inconsistent_size", "size_of_image": 8192, - "max_section_end": 16384 + "max_section_end": 16384, + "reason": "optional_header_inconsistent_size" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_025_opt_size_of_headers_misaligned.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_025_opt_size_of_headers_misaligned.full.json index bd48465..cb0a195 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_025_opt_size_of_headers_misaligned.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_025_opt_size_of_headers_misaligned.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_026_opt_size_of_headers_too_small.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_026_opt_size_of_headers_too_small.full.json index 595f1fb..b3b0c1c 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_026_opt_size_of_headers_too_small.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_026_opt_size_of_headers_too_small.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 512 + "size_of_headers": 512, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_027_opt_section_alignment_invalid.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_027_opt_section_alignment_invalid.full.json index 8f92fba..a52f8b5 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_027_opt_section_alignment_invalid.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_027_opt_section_alignment_invalid.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_section_alignment", "section_alignment": 384, - "file_alignment": 512 + "file_alignment": 512, + "reason": "optional_header_invalid_section_alignment" } }, { @@ -204,9 +205,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_section_alignment", "section_alignment": 384, - "sub_reason": "not_power_of_two" + "sub_reason": "not_power_of_two", + "reason": "optional_header_invalid_section_alignment" } }, { @@ -215,9 +216,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_size_of_image_misaligned", "size_of_image": 16384, - "section_alignment": 384 + "section_alignment": 384, + "reason": "optional_header_size_of_image_misaligned" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_028_opt_file_alignment_invalid.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_028_opt_file_alignment_invalid.full.json index 89f2daa..280b7ec 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_028_opt_file_alignment_invalid.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_028_opt_file_alignment_invalid.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1536 + "size_of_headers": 1536, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,11 +194,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_misaligned", "section": ".text", "raw_address": 1024, "raw_size": 512, - "file_alignment": 768 + "file_alignment": 768, + "reason": "section_raw_misaligned" } }, { @@ -206,10 +207,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_overlaps_headers", "section": ".text", "raw_address": 1024, - "size_of_headers": 1536 + "size_of_headers": 1536, + "reason": "section_overlaps_headers" } }, { @@ -218,11 +219,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_misaligned", "section": ".rsrc", "raw_address": 2048, "raw_size": 512, - "file_alignment": 768 + "file_alignment": 768, + "reason": "section_raw_misaligned" } }, { @@ -231,9 +232,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_file_alignment", "file_alignment": 768, - "sub_reason": "not_power_of_two" + "sub_reason": "not_power_of_two", + "reason": "optional_header_invalid_file_alignment" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_029_opt_size_fields_too_small.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_029_opt_size_fields_too_small.full.json index 22b3fc8..b4e8319 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_029_opt_size_fields_too_small.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_029_opt_size_fields_too_small.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 4096, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_inconsistent_size", "size_of_image": 4096, - "max_section_end": 16384 + "max_section_end": 16384, + "reason": "optional_header_inconsistent_size" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_030_opt_image_base_misaligned.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_030_opt_image_base_misaligned.full.json index fc3cdb0..2a87049 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_030_opt_image_base_misaligned.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_030_opt_image_base_misaligned.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_031_opt_num_dirs_invalid.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_031_opt_num_dirs_invalid.full.json index ba2eccd..fd8f7cc 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_031_opt_num_dirs_invalid.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_031_opt_num_dirs_invalid.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,8 +194,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", - "number_of_rva_and_sizes": 20 + "number_of_rva_and_sizes": 20, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_032_opt_num_dirs_too_small.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_032_opt_num_dirs_too_small.full.json index 068e3a3..792f51b 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_032_opt_num_dirs_too_small.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_032_opt_num_dirs_too_small.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 1, - "actual_directories": 2 + "actual_directories": 2, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_033_opt_size_of_image_misaligned.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_033_opt_size_of_image_misaligned.full.json index 6240b9e..bcd383c 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_033_opt_size_of_image_misaligned.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_033_opt_size_of_image_misaligned.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 6144, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_inconsistent_size", "size_of_image": 6144, - "max_section_end": 16384 + "max_section_end": 16384, + "reason": "optional_header_inconsistent_size" } }, { @@ -204,9 +205,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_size_of_image_misaligned", "size_of_image": 6144, - "section_alignment": 4096 + "section_alignment": 4096, + "reason": "optional_header_size_of_image_misaligned" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_034_ddir_negative_rva.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_034_ddir_negative_rva.full.json index ecf8645..d6cdc70 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_034_ddir_negative_rva.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_034_ddir_negative_rva.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 0, - "actual_directories": 1 + "actual_directories": 1, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_035_ddir_negative_size.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_035_ddir_negative_size.full.json index 0dafe30..3573636 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_035_ddir_negative_size.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_035_ddir_negative_size.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 0, - "actual_directories": 1 + "actual_directories": 1, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_036_ddir_zero_zero.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_036_ddir_zero_zero.full.json index 959304c..89c681e 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_036_ddir_zero_zero.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_036_ddir_zero_zero.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_037_ddir_zero_rva_nonzero_size.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_037_ddir_zero_rva_nonzero_size.full.json index 02e8f93..67b5db0 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_037_ddir_zero_rva_nonzero_size.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_037_ddir_zero_rva_nonzero_size.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 0, - "actual_directories": 1 + "actual_directories": 1, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_038_ddir_zero_size_nonzero_rva.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_038_ddir_zero_size_nonzero_rva.full.json index 4f2fc11..66b8040 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_038_ddir_zero_size_nonzero_rva.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_038_ddir_zero_size_nonzero_rva.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 0, - "actual_directories": 1 + "actual_directories": 1, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_039_ddir_in_headers.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_039_ddir_in_headers.full.json index d2202fe..f6158b1 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_039_ddir_in_headers.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_039_ddir_in_headers.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 0, - "actual_directories": 1 + "actual_directories": 1, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_040_ddir_out_of_range.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_040_ddir_out_of_range.full.json index 3a3086e..cc94c1c 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_040_ddir_out_of_range.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_040_ddir_out_of_range.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 0, - "actual_directories": 1 + "actual_directories": 1, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_041_ddir_raw_mismatch.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_041_ddir_raw_mismatch.full.json index a0d769a..17a7bed 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_041_ddir_raw_mismatch.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_041_ddir_raw_mismatch.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 0, - "actual_directories": 1 + "actual_directories": 1, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_042_ddir_in_overlay.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_042_ddir_in_overlay.full.json index a0b304b..d5034fe 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_042_ddir_in_overlay.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_042_ddir_in_overlay.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 0, - "actual_directories": 1 + "actual_directories": 1, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_043_ddir_not_mapped.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_043_ddir_not_mapped.full.json index 399ed1c..9bc242a 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_043_ddir_not_mapped.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_043_ddir_not_mapped.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 0, - "actual_directories": 1 + "actual_directories": 1, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_044_ddir_spans_sections.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_044_ddir_spans_sections.full.json index 9281212..8ff4cf6 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_044_ddir_spans_sections.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_044_ddir_spans_sections.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 0, - "actual_directories": 1 + "actual_directories": 1, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/fixture_045_ddir_overlap.full.json b/tests/contract/snapshots/layer3_adversarial/fixture_045_ddir_overlap.full.json index dce5414..132682e 100644 --- a/tests/contract/snapshots/layer3_adversarial/fixture_045_ddir_overlap.full.json +++ b/tests/contract/snapshots/layer3_adversarial/fixture_045_ddir_overlap.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -160,8 +161,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_zero_or_negative", - "entry_point": 0 + "entry_point": 0, + "reason": "entrypoint_zero_or_negative" } }, { @@ -170,9 +171,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_headers", "entry_point": 0, - "size_of_headers": 1024 + "size_of_headers": 1024, + "reason": "entrypoint_in_headers" } }, { @@ -181,10 +182,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 0, "size_of_image": 16384, - "position": "within_size_of_image_but_no_section" + "position": "within_size_of_image_but_no_section", + "reason": "entrypoint_out_of_bounds" } }, { @@ -193,9 +194,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 0, - "actual_directories": 2 + "actual_directories": 2, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/franken_malformed_pe.full.json b/tests/contract/snapshots/layer3_adversarial/franken_malformed_pe.full.json index 7030fea..9a92167 100644 --- a/tests/contract/snapshots/layer3_adversarial/franken_malformed_pe.full.json +++ b/tests/contract/snapshots/layer3_adversarial/franken_malformed_pe.full.json @@ -60,6 +60,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -187,10 +188,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 12288, "size_of_image": 8192, - "position": "beyond_size_of_image" + "position": "beyond_size_of_image", + "reason": "entrypoint_out_of_bounds" } }, { @@ -199,11 +200,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_misaligned", "section": ".rdata", "raw_address": 768, "raw_size": 1536, - "file_alignment": 512 + "file_alignment": 512, + "reason": "section_raw_misaligned" } }, { @@ -212,11 +213,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_misaligned", "section": ".data", "raw_address": 2384, "raw_size": 768, - "file_alignment": 512 + "file_alignment": 512, + "reason": "section_raw_misaligned" } }, { @@ -225,9 +226,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_overlap", "section_a": ".text", - "section_b": ".rdata" + "section_b": ".rdata", + "reason": "section_raw_overlap" } }, { @@ -236,9 +237,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_overlap", "section_a": ".data", - "section_b": ".rsrc" + "section_b": ".rsrc", + "reason": "section_raw_overlap" } }, { @@ -247,9 +248,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_overlap", "section_a": ".text", - "section_b": ".rdata" + "section_b": ".rdata", + "reason": "section_overlap" } }, { @@ -258,9 +259,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_inconsistent_size", "size_of_image": 8192, - "max_section_end": 11776 + "max_section_end": 11776, + "reason": "optional_header_inconsistent_size" } }, { @@ -269,11 +270,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_out_of_range", "directory": "IMAGE_DIRECTORY_ENTRY_IMPORT", "rva": 20480, "size": 512, - "size_of_image": 8192 + "size_of_image": 8192, + "reason": "data_directory_out_of_range" } }, { @@ -282,10 +283,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_zero_rva_nonzero_size", "directory": "IMAGE_DIRECTORY_ENTRY_RESOURCE", "rva": 0, - "size": 256 + "size": 256, + "reason": "data_directory_zero_rva_nonzero_size" } }, { @@ -294,10 +295,20 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "exception_directory_size_not_multiple", "size": 512, "entry_size": 12, - "remainder": 8 + "remainder": 8, + "reason": "exception_directory_size_not_multiple" + } + }, + { + "value": "pe_structure_anomaly", + "start": 0, + "end": 0, + "category": "pe_heuristic", + "metadata": { + "table": "exception_table_ragged_tail", + "reason": "exception_table_truncated" } }, { @@ -306,8 +317,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "exception_table_truncated", - "table": "exception_table_ragged_tail" + "table": "exception_entry_read_failed", + "reason": "exception_table_truncated" } }, { @@ -316,8 +327,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "exception_table_truncated", - "table": "exception_entry_read_failed" + "table": "import_descriptor_read_failed", + "reason": "import_table_truncated" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/franken_malformed_pe.pe32.full.json b/tests/contract/snapshots/layer3_adversarial/franken_malformed_pe.pe32.full.json index 8a29915..95a821c 100644 --- a/tests/contract/snapshots/layer3_adversarial/franken_malformed_pe.pe32.full.json +++ b/tests/contract/snapshots/layer3_adversarial/franken_malformed_pe.pe32.full.json @@ -60,6 +60,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -187,10 +188,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 12288, "size_of_image": 8192, - "position": "beyond_size_of_image" + "position": "beyond_size_of_image", + "reason": "entrypoint_out_of_bounds" } }, { @@ -199,11 +200,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_misaligned", "section": ".rdata", "raw_address": 768, "raw_size": 1536, - "file_alignment": 512 + "file_alignment": 512, + "reason": "section_raw_misaligned" } }, { @@ -212,11 +213,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_misaligned", "section": ".data", "raw_address": 2384, "raw_size": 768, - "file_alignment": 512 + "file_alignment": 512, + "reason": "section_raw_misaligned" } }, { @@ -225,9 +226,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_overlap", "section_a": ".text", - "section_b": ".rdata" + "section_b": ".rdata", + "reason": "section_raw_overlap" } }, { @@ -236,9 +237,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_overlap", "section_a": ".data", - "section_b": ".rsrc" + "section_b": ".rsrc", + "reason": "section_raw_overlap" } }, { @@ -247,9 +248,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_overlap", "section_a": ".text", - "section_b": ".rdata" + "section_b": ".rdata", + "reason": "section_overlap" } }, { @@ -258,9 +259,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_inconsistent_size", "size_of_image": 8192, - "max_section_end": 11776 + "max_section_end": 11776, + "reason": "optional_header_inconsistent_size" } }, { @@ -269,11 +270,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_out_of_range", "directory": "IMAGE_DIRECTORY_ENTRY_IMPORT", "rva": 20480, "size": 512, - "size_of_image": 8192 + "size_of_image": 8192, + "reason": "data_directory_out_of_range" } }, { @@ -282,10 +283,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_zero_rva_nonzero_size", "directory": "IMAGE_DIRECTORY_ENTRY_RESOURCE", "rva": 0, - "size": 256 + "size": 256, + "reason": "data_directory_zero_rva_nonzero_size" } }, { @@ -294,10 +295,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "exception_directory_size_not_multiple", "size": 512, "entry_size": 12, - "remainder": 8 + "remainder": 8, + "reason": "exception_directory_size_not_multiple" } }, { @@ -306,9 +307,19 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "exception_unsupported_machine", "arch": "unsupported", - "machine": 332 + "machine": 332, + "reason": "exception_unsupported_machine" + } + }, + { + "value": "pe_structure_anomaly", + "start": 0, + "end": 0, + "category": "pe_heuristic", + "metadata": { + "table": "import_descriptor_read_failed", + "reason": "import_table_truncated" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/franken_url_domain_ip.full.json b/tests/contract/snapshots/layer3_adversarial/franken_url_domain_ip.full.json index 2cb4449..31bb10d 100644 --- a/tests/contract/snapshots/layer3_adversarial/franken_url_domain_ip.full.json +++ b/tests/contract/snapshots/layer3_adversarial/franken_url_domain_ip.full.json @@ -367,6 +367,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -671,9 +672,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "OutputDebugStringA" + "function": "OutputDebugStringA", + "reason": "anti_debug_api_import" } }, { @@ -682,9 +683,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "IsDebuggerPresent" + "function": "IsDebuggerPresent", + "reason": "anti_debug_api_import" } }, { @@ -693,9 +694,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "timing_api_import", "dll": "kernel32.dll", - "function": "QueryPerformanceCounter" + "function": "QueryPerformanceCounter", + "reason": "timing_api_import" } }, { @@ -704,11 +705,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_guard_cf_inconsistent", "check": 5368717712, "dispatch": 5368717728, "table": 0, - "count": 0 + "count": 0, + "reason": "load_config_guard_cf_inconsistent" } }, { @@ -717,9 +718,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_invalid", "cookie_rva": 5368725760, - "sub_reason": "unmapped" + "sub_reason": "unmapped", + "reason": "load_config_cookie_invalid" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/heuristic_rich.full.json b/tests/contract/snapshots/layer3_adversarial/heuristic_rich.full.json index b5f0997..1201d02 100644 --- a/tests/contract/snapshots/layer3_adversarial/heuristic_rich.full.json +++ b/tests/contract/snapshots/layer3_adversarial/heuristic_rich.full.json @@ -359,6 +359,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -691,8 +692,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "packer_section_name", - "section": "UPX0" + "section": "UPX0", + "reason": "packer_section_name" } }, { @@ -701,9 +702,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "CheckRemoteDebuggerPresent" + "function": "CheckRemoteDebuggerPresent", + "reason": "anti_debug_api_import" } }, { @@ -712,9 +713,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "timing_api_import", "dll": "kernel32.dll", - "function": "GetTickCount" + "function": "GetTickCount", + "reason": "timing_api_import" } }, { @@ -723,9 +724,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "timing_api_import", "dll": "kernel32.dll", - "function": "GetTickCount64" + "function": "GetTickCount64", + "reason": "timing_api_import" } }, { @@ -734,9 +735,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "IsDebuggerPresent" + "function": "IsDebuggerPresent", + "reason": "anti_debug_api_import" } }, { @@ -745,9 +746,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "OutputDebugStringA" + "function": "OutputDebugStringA", + "reason": "anti_debug_api_import" } }, { @@ -756,9 +757,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "timing_api_import", "dll": "kernel32.dll", - "function": "QueryPerformanceCounter" + "function": "QueryPerformanceCounter", + "reason": "timing_api_import" } }, { @@ -767,10 +768,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_overlaps_headers", "section": ".bss", "raw_address": 0, - "size_of_headers": 1536 + "size_of_headers": 1536, + "reason": "section_overlaps_headers" } }, { @@ -779,7 +780,6 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_out_of_order_raw", "raw_addresses": [ 1536, 8192, @@ -802,7 +802,8 @@ 89600, 95744, 100864 - ] + ], + "reason": "section_out_of_order_raw" } }, { @@ -811,9 +812,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_overlap", "directory_a": "IMAGE_DIRECTORY_ENTRY_IMPORT", - "directory_b": "IMAGE_DIRECTORY_ENTRY_IAT" + "directory_b": "IMAGE_DIRECTORY_ENTRY_IAT", + "reason": "data_directory_overlap" } }, { @@ -822,10 +823,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "callback_outside_tls_range", "callbacks": 5368754232, "start_address": 5368758272, - "end_address": 5368758280 + "end_address": 5368758280, + "reason": "callback_outside_tls_range" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/invalid_optional_header.full.json b/tests/contract/snapshots/layer3_adversarial/invalid_optional_header.full.json index ac20c08..bf55178 100644 --- a/tests/contract/snapshots/layer3_adversarial/invalid_optional_header.full.json +++ b/tests/contract/snapshots/layer3_adversarial/invalid_optional_header.full.json @@ -55,6 +55,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [], "obfuscation": [], @@ -134,9 +135,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_size_of_headers", "size_of_headers": 2048, - "file_alignment": 16384 + "file_alignment": 16384, + "reason": "optional_header_invalid_size_of_headers" } }, { @@ -145,9 +146,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_section_alignment", "section_alignment": 4096, - "file_alignment": 16384 + "file_alignment": 16384, + "reason": "optional_header_invalid_section_alignment" } }, { @@ -156,9 +157,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 1, - "actual_directories": 4 + "actual_directories": 3, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } }, { @@ -167,9 +168,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_size_of_image_misaligned", "size_of_image": 512, - "section_alignment": 4096 + "section_alignment": 4096, + "reason": "optional_header_size_of_image_misaligned" } }, { @@ -178,11 +179,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_out_of_range", "directory": "IMAGE_DIRECTORY_ENTRY_EXPORT", "rva": 4096, "size": 512, - "size_of_image": 512 + "size_of_image": 512, + "reason": "data_directory_out_of_range" } }, { @@ -191,11 +192,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "export_directory_invalid_header", "sub_reason": "top_level_decode", "errors": [ "header_read_failed" - ] + ], + "reason": "export_directory_invalid_header" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/invalid_optional_header.pe32.full.json b/tests/contract/snapshots/layer3_adversarial/invalid_optional_header.pe32.full.json index 5dd69d1..ffa5427 100644 --- a/tests/contract/snapshots/layer3_adversarial/invalid_optional_header.pe32.full.json +++ b/tests/contract/snapshots/layer3_adversarial/invalid_optional_header.pe32.full.json @@ -57,6 +57,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -144,10 +145,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_out_of_bounds", "entry_point": 2415919104, "size_of_image": 512, - "position": "beyond_size_of_image" + "position": "beyond_size_of_image", + "reason": "entrypoint_out_of_bounds" } }, { @@ -156,11 +157,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_misaligned", "section": ".text", "raw_address": 512, "raw_size": 512, - "file_alignment": 16384 + "file_alignment": 16384, + "reason": "section_raw_misaligned" } }, { @@ -169,10 +170,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_overlaps_headers", "section": ".text", "raw_address": 512, - "size_of_headers": 2048 + "size_of_headers": 2048, + "reason": "section_overlaps_headers" } }, { @@ -181,9 +182,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_inconsistent_size", "size_of_image": 512, - "max_section_end": 8192 + "max_section_end": 8192, + "reason": "optional_header_inconsistent_size" } }, { @@ -192,9 +193,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_size_of_headers", "size_of_headers": 2048, - "file_alignment": 16384 + "file_alignment": 16384, + "reason": "optional_header_invalid_size_of_headers" } }, { @@ -203,9 +204,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_section_alignment", "section_alignment": 4096, - "file_alignment": 16384 + "file_alignment": 16384, + "reason": "optional_header_invalid_section_alignment" } }, { @@ -214,9 +215,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_invalid_number_of_rva_and_sizes", "number_of_rva_and_sizes": 1, - "actual_directories": 3 + "actual_directories": 3, + "reason": "optional_header_invalid_number_of_rva_and_sizes" } }, { @@ -225,9 +226,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_size_of_image_misaligned", "size_of_image": 512, - "section_alignment": 4096 + "section_alignment": 4096, + "reason": "optional_header_size_of_image_misaligned" } }, { @@ -236,11 +237,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_out_of_range", "directory": "IMAGE_DIRECTORY_ENTRY_EXPORT", "rva": 4096, "size": 512, - "size_of_image": 512 + "size_of_image": 512, + "reason": "data_directory_out_of_range" } }, { @@ -249,10 +250,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "export_directory_out_of_bounds", "rva": 4096, "size": 512, - "size_of_image": 512 + "size_of_image": 512, + "reason": "export_directory_out_of_bounds" } }, { @@ -261,8 +262,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "export_table_truncated", - "table": "export_directory_header" + "table": "export_directory_header", + "reason": "export_table_truncated" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/invalid_section_alignment.full.json b/tests/contract/snapshots/layer3_adversarial/invalid_section_alignment.full.json index d2997a3..e4ce97c 100644 --- a/tests/contract/snapshots/layer3_adversarial/invalid_section_alignment.full.json +++ b/tests/contract/snapshots/layer3_adversarial/invalid_section_alignment.full.json @@ -57,6 +57,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -144,11 +145,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_misaligned", "section": ".text", "raw_address": 291, "raw_size": 4096, - "file_alignment": 512 + "file_alignment": 512, + "reason": "section_raw_misaligned" } }, { @@ -157,10 +158,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_overlaps_headers", "section": ".text", "raw_address": 291, - "size_of_headers": 512 + "size_of_headers": 512, + "reason": "section_overlaps_headers" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/load_config_cookie_too_small.full.json b/tests/contract/snapshots/layer3_adversarial/load_config_cookie_too_small.full.json index 9597a2c..ab7786e 100644 --- a/tests/contract/snapshots/layer3_adversarial/load_config_cookie_too_small.full.json +++ b/tests/contract/snapshots/layer3_adversarial/load_config_cookie_too_small.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,10 +153,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_too_small", "rva": 12288, "size": 12, - "min_size": 112 + "min_size": 112, + "reason": "load_config_too_small" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_cookie_in_overlay.full.json b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_cookie_in_overlay.full.json index b300fe8..435e5e2 100644 --- a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_cookie_in_overlay.full.json +++ b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_cookie_in_overlay.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,11 +153,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_invalid", "cookie_rva": 12800, "section": ".rdata", "characteristics": 1073741888, - "sub_reason": "non_writable_section" + "sub_reason": "non_writable_section", + "reason": "load_config_cookie_invalid" } }, { @@ -165,10 +166,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_in_overlay", "cookie_rva": 12800, "cookie_raw": 2048, - "overlay_offset": 2048 + "overlay_offset": 2048, + "reason": "load_config_cookie_in_overlay" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_cookie_invalid.full.json b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_cookie_invalid.full.json index 824a04d..8a8d561 100644 --- a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_cookie_invalid.full.json +++ b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_cookie_invalid.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,9 +153,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_invalid", "cookie_rva": 2415919104, - "sub_reason": "unmapped" + "sub_reason": "unmapped", + "reason": "load_config_cookie_invalid" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_guard_cf_inconsistent.full.json b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_guard_cf_inconsistent.full.json index 08bac37..a2a31f4 100644 --- a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_guard_cf_inconsistent.full.json +++ b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_guard_cf_inconsistent.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,11 +153,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_guard_cf_inconsistent", "check": 13824, "dispatch": 0, "table": 14080, - "count": 0 + "count": 0, + "reason": "load_config_guard_cf_inconsistent" } }, { @@ -165,11 +166,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_invalid", "cookie_rva": 13568, "section": ".rdata", "characteristics": 1073741888, - "sub_reason": "non_writable_section" + "sub_reason": "non_writable_section", + "reason": "load_config_cookie_invalid" } }, { @@ -178,10 +179,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_in_overlay", "cookie_rva": 13568, "cookie_raw": 2816, - "overlay_offset": 1684 + "overlay_offset": 1684, + "reason": "load_config_cookie_in_overlay" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_seh_invalid.full.json b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_seh_invalid.full.json index 7ba654a..f491e14 100644 --- a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_seh_invalid.full.json +++ b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_seh_invalid.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,11 +153,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_invalid", "cookie_rva": 13568, "section": ".rdata", "characteristics": 1073741888, - "sub_reason": "non_writable_section" + "sub_reason": "non_writable_section", + "reason": "load_config_cookie_invalid" } }, { @@ -165,10 +166,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_in_overlay", "cookie_rva": 13568, "cookie_raw": 2816, - "overlay_offset": 1684 + "overlay_offset": 1684, + "reason": "load_config_cookie_in_overlay" } }, { @@ -177,10 +178,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_seh_invalid", "seh_table_rva": 0, "seh_count": 4, - "sub_reason": "missing_table_rva" + "sub_reason": "missing_table_rva", + "reason": "load_config_seh_invalid" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_size_exceeds_section.full.json b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_size_exceeds_section.full.json index 5e61dd2..c13418b 100644 --- a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_size_exceeds_section.full.json +++ b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_size_exceeds_section.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,11 +153,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_out_of_range", "directory": "IMAGE_DIRECTORY_ENTRY_LOAD_CONFIG", "rva": 12288, "size": 8192, - "size_of_image": 16384 + "size_of_image": 16384, + "reason": "data_directory_out_of_range" } }, { @@ -165,9 +166,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_invalid", "cookie_rva": 13568, - "sub_reason": "unmapped" + "sub_reason": "unmapped", + "reason": "load_config_cookie_invalid" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_size_too_small.full.json b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_size_too_small.full.json index c9e55e4..ceb4454 100644 --- a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_size_too_small.full.json +++ b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_size_too_small.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,10 +153,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_too_small", "rva": 12288, "size": 32, - "min_size": 112 + "min_size": 112, + "reason": "load_config_too_small" } }, { @@ -164,11 +165,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_invalid", "cookie_rva": 13568, "section": ".rdata", "characteristics": 1073741888, - "sub_reason": "non_writable_section" + "sub_reason": "non_writable_section", + "reason": "load_config_cookie_invalid" } }, { @@ -177,10 +178,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_in_overlay", "cookie_rva": 13568, "cookie_raw": 2816, - "overlay_offset": 1664 + "overlay_offset": 1664, + "reason": "load_config_cookie_in_overlay" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_truncated.full.json b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_truncated.full.json index 0b1d2ca..f6065b4 100644 --- a/tests/contract/snapshots/layer3_adversarial/load_config_malformed_truncated.full.json +++ b/tests/contract/snapshots/layer3_adversarial/load_config_malformed_truncated.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,10 +153,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_truncated", "rva": 12288, "declared_size": 112, - "parsed_size": 64 + "parsed_size": 64, + "reason": "load_config_truncated" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/load_config_rva_negative.full.json b/tests/contract/snapshots/layer3_adversarial/load_config_rva_negative.full.json index fb776cf..f77bcea 100644 --- a/tests/contract/snapshots/layer3_adversarial/load_config_rva_negative.full.json +++ b/tests/contract/snapshots/layer3_adversarial/load_config_rva_negative.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,11 +153,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_out_of_range", "directory": "IMAGE_DIRECTORY_ENTRY_LOAD_CONFIG", "rva": 4294967295, "size": 148, - "size_of_image": 16384 + "size_of_image": 16384, + "reason": "data_directory_out_of_range" } }, { @@ -165,10 +166,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_truncated", "rva": 4294967295, "declared_size": 148, - "parsed_size": 0 + "parsed_size": 0, + "reason": "load_config_truncated" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/load_config_rva_zero.full.json b/tests/contract/snapshots/layer3_adversarial/load_config_rva_zero.full.json index b2b7cd3..52a14d7 100644 --- a/tests/contract/snapshots/layer3_adversarial/load_config_rva_zero.full.json +++ b/tests/contract/snapshots/layer3_adversarial/load_config_rva_zero.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,10 +153,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_zero_rva_nonzero_size", "directory": "IMAGE_DIRECTORY_ENTRY_LOAD_CONFIG", "rva": 0, - "size": 148 + "size": 148, + "reason": "data_directory_zero_rva_nonzero_size" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/load_config_zero_size_but_fields_present.full.json b/tests/contract/snapshots/layer3_adversarial/load_config_zero_size_but_fields_present.full.json index 9b46425..ebf23b7 100644 --- a/tests/contract/snapshots/layer3_adversarial/load_config_zero_size_but_fields_present.full.json +++ b/tests/contract/snapshots/layer3_adversarial/load_config_zero_size_but_fields_present.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,10 +153,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_zero_size_nonzero_rva", "directory": "IMAGE_DIRECTORY_ENTRY_LOAD_CONFIG", "rva": 12288, - "size": 0 + "size": 0, + "reason": "data_directory_zero_size_nonzero_rva" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/load_config_zero_size_invalid_rva.full.json b/tests/contract/snapshots/layer3_adversarial/load_config_zero_size_invalid_rva.full.json index 09f061e..93d45c0 100644 --- a/tests/contract/snapshots/layer3_adversarial/load_config_zero_size_invalid_rva.full.json +++ b/tests/contract/snapshots/layer3_adversarial/load_config_zero_size_invalid_rva.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,10 +153,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_zero_size_nonzero_rva", "directory": "IMAGE_DIRECTORY_ENTRY_LOAD_CONFIG", "rva": 2415919104, - "size": 0 + "size": 0, + "reason": "data_directory_zero_size_nonzero_rva" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/load_config_zero_size_valid_rva.full.json b/tests/contract/snapshots/layer3_adversarial/load_config_zero_size_valid_rva.full.json index f1ebb87..54ba9c1 100644 --- a/tests/contract/snapshots/layer3_adversarial/load_config_zero_size_valid_rva.full.json +++ b/tests/contract/snapshots/layer3_adversarial/load_config_zero_size_valid_rva.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -152,10 +153,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_zero_size_nonzero_rva", "directory": "IMAGE_DIRECTORY_ENTRY_LOAD_CONFIG", "rva": 12288, - "size": 0 + "size": 0, + "reason": "data_directory_zero_size_nonzero_rva" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/malformed_domain.full.json b/tests/contract/snapshots/layer3_adversarial/malformed_domain.full.json index a8d430a..a9022fa 100644 --- a/tests/contract/snapshots/layer3_adversarial/malformed_domain.full.json +++ b/tests/contract/snapshots/layer3_adversarial/malformed_domain.full.json @@ -345,6 +345,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -649,9 +650,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "OutputDebugStringA" + "function": "OutputDebugStringA", + "reason": "anti_debug_api_import" } }, { @@ -660,9 +661,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "IsDebuggerPresent" + "function": "IsDebuggerPresent", + "reason": "anti_debug_api_import" } }, { @@ -671,9 +672,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "timing_api_import", "dll": "kernel32.dll", - "function": "QueryPerformanceCounter" + "function": "QueryPerformanceCounter", + "reason": "timing_api_import" } }, { @@ -682,11 +683,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_guard_cf_inconsistent", "check": 5368717712, "dispatch": 5368717728, "table": 0, - "count": 0 + "count": 0, + "reason": "load_config_guard_cf_inconsistent" } }, { @@ -695,9 +696,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_invalid", "cookie_rva": 5368721536, - "sub_reason": "unmapped" + "sub_reason": "unmapped", + "reason": "load_config_cookie_invalid" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/malformed_import_table.full.json b/tests/contract/snapshots/layer3_adversarial/malformed_import_table.full.json index b00f5e2..979dddf 100644 --- a/tests/contract/snapshots/layer3_adversarial/malformed_import_table.full.json +++ b/tests/contract/snapshots/layer3_adversarial/malformed_import_table.full.json @@ -57,6 +57,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -144,10 +145,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_overlay", "entry_point": 4096, "entry_point_file_offset": 512, - "overlay_offset": 392 + "overlay_offset": 392, + "reason": "entrypoint_in_overlay" } }, { @@ -156,11 +157,21 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "data_directory_out_of_range", "directory": "IMAGE_DIRECTORY_ENTRY_IMPORT", "rva": 3735928559, "size": 512, - "size_of_image": 12288 + "size_of_image": 12288, + "reason": "data_directory_out_of_range" + } + }, + { + "value": "pe_structure_anomaly", + "start": 0, + "end": 0, + "category": "pe_heuristic", + "metadata": { + "table": "import_descriptor_read_failed", + "reason": "import_table_truncated" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/malformed_ip.full.json b/tests/contract/snapshots/layer3_adversarial/malformed_ip.full.json index 933aa6f..f180932 100644 --- a/tests/contract/snapshots/layer3_adversarial/malformed_ip.full.json +++ b/tests/contract/snapshots/layer3_adversarial/malformed_ip.full.json @@ -351,6 +351,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -655,9 +656,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "OutputDebugStringA" + "function": "OutputDebugStringA", + "reason": "anti_debug_api_import" } }, { @@ -666,9 +667,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "IsDebuggerPresent" + "function": "IsDebuggerPresent", + "reason": "anti_debug_api_import" } }, { @@ -677,9 +678,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "timing_api_import", "dll": "kernel32.dll", - "function": "QueryPerformanceCounter" + "function": "QueryPerformanceCounter", + "reason": "timing_api_import" } }, { @@ -688,11 +689,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_guard_cf_inconsistent", "check": 5368717712, "dispatch": 5368717728, "table": 0, - "count": 0 + "count": 0, + "reason": "load_config_guard_cf_inconsistent" } }, { @@ -701,9 +702,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_invalid", "cookie_rva": 5368721600, - "sub_reason": "unmapped" + "sub_reason": "unmapped", + "reason": "load_config_cookie_invalid" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/malformed_url.full.json b/tests/contract/snapshots/layer3_adversarial/malformed_url.full.json index bcaf223..86b6ed8 100644 --- a/tests/contract/snapshots/layer3_adversarial/malformed_url.full.json +++ b/tests/contract/snapshots/layer3_adversarial/malformed_url.full.json @@ -349,6 +349,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -653,9 +654,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "OutputDebugStringA" + "function": "OutputDebugStringA", + "reason": "anti_debug_api_import" } }, { @@ -664,9 +665,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "IsDebuggerPresent" + "function": "IsDebuggerPresent", + "reason": "anti_debug_api_import" } }, { @@ -675,9 +676,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "timing_api_import", "dll": "kernel32.dll", - "function": "QueryPerformanceCounter" + "function": "QueryPerformanceCounter", + "reason": "timing_api_import" } }, { @@ -686,11 +687,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_guard_cf_inconsistent", "check": 5368717712, "dispatch": 5368717728, "table": 0, - "count": 0 + "count": 0, + "reason": "load_config_guard_cf_inconsistent" } }, { @@ -699,9 +700,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_invalid", "cookie_rva": 5368721536, - "sub_reason": "unmapped" + "sub_reason": "unmapped", + "reason": "load_config_cookie_invalid" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/overlapping_sections.full.json b/tests/contract/snapshots/layer3_adversarial/overlapping_sections.full.json index fea2f64..13700d7 100644 --- a/tests/contract/snapshots/layer3_adversarial/overlapping_sections.full.json +++ b/tests/contract/snapshots/layer3_adversarial/overlapping_sections.full.json @@ -58,6 +58,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -171,10 +172,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_overlay", "entry_point": 4096, "entry_point_file_offset": 512, - "overlay_offset": 392 + "overlay_offset": 392, + "reason": "entrypoint_in_overlay" } }, { @@ -183,9 +184,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_raw_overlap", "section_a": ".text", - "section_b": ".data" + "section_b": ".data", + "reason": "section_raw_overlap" } }, { @@ -194,9 +195,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "section_overlap", "section_a": ".text", - "section_b": ".data" + "section_b": ".data", + "reason": "section_overlap" } }, { @@ -205,9 +206,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_inconsistent_size", "size_of_image": 12288, - "max_section_end": 14336 + "max_section_end": 14336, + "reason": "optional_header_inconsistent_size" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/packed_lookalike.full.json b/tests/contract/snapshots/layer3_adversarial/packed_lookalike.full.json index 6d07915..1ac87d1 100644 --- a/tests/contract/snapshots/layer3_adversarial/packed_lookalike.full.json +++ b/tests/contract/snapshots/layer3_adversarial/packed_lookalike.full.json @@ -61,6 +61,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -192,10 +193,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "high_entropy_section", "section": ".text", "entropy": 7.980294617270556, - "raw_size": 8192 + "raw_size": 8192, + "reason": "high_entropy_section" } }, { @@ -204,8 +205,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "packer_section_name", - "section": ".upx0" + "section": ".upx0", + "reason": "packer_section_name" } }, { @@ -214,8 +215,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "packer_section_name", - "section": ".upx1" + "section": ".upx1", + "reason": "packer_section_name" } }, { @@ -224,9 +225,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "optional_header_inconsistent_size", "size_of_image": 16384, - "max_section_end": 20480 + "max_section_end": 20480, + "reason": "optional_header_inconsistent_size" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/string_obfuscation_tricks.full.json b/tests/contract/snapshots/layer3_adversarial/string_obfuscation_tricks.full.json index 21978d8..a67a97e 100644 --- a/tests/contract/snapshots/layer3_adversarial/string_obfuscation_tricks.full.json +++ b/tests/contract/snapshots/layer3_adversarial/string_obfuscation_tricks.full.json @@ -342,6 +342,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -646,9 +647,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "OutputDebugStringA" + "function": "OutputDebugStringA", + "reason": "anti_debug_api_import" } }, { @@ -657,9 +658,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "anti_debug_api_import", "dll": "kernel32.dll", - "function": "IsDebuggerPresent" + "function": "IsDebuggerPresent", + "reason": "anti_debug_api_import" } }, { @@ -668,9 +669,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "timing_api_import", "dll": "kernel32.dll", - "function": "QueryPerformanceCounter" + "function": "QueryPerformanceCounter", + "reason": "timing_api_import" } }, { @@ -679,11 +680,11 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_guard_cf_inconsistent", "check": 5368717712, "dispatch": 5368717728, "table": 0, - "count": 0 + "count": 0, + "reason": "load_config_guard_cf_inconsistent" } }, { @@ -692,9 +693,9 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "load_config_cookie_invalid", "cookie_rva": 5368721472, - "sub_reason": "unmapped" + "sub_reason": "unmapped", + "reason": "load_config_cookie_invalid" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/truncated_rich_header.full.json b/tests/contract/snapshots/layer3_adversarial/truncated_rich_header.full.json index e7d3f86..826ab80 100644 --- a/tests/contract/snapshots/layer3_adversarial/truncated_rich_header.full.json +++ b/tests/contract/snapshots/layer3_adversarial/truncated_rich_header.full.json @@ -57,6 +57,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -144,10 +145,10 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "entrypoint_in_overlay", "entry_point": 4096, "entry_point_file_offset": 512, - "overlay_offset": 392 + "overlay_offset": 392, + "reason": "entrypoint_in_overlay" } } ] diff --git a/tests/contract/snapshots/layer3_adversarial/upx_name_only.full.json b/tests/contract/snapshots/layer3_adversarial/upx_name_only.full.json index 821c000..7a400f7 100644 --- a/tests/contract/snapshots/layer3_adversarial/upx_name_only.full.json +++ b/tests/contract/snapshots/layer3_adversarial/upx_name_only.full.json @@ -59,6 +59,7 @@ "signatures": [], "has_signature": false }, + "version_info": null, "analysis": { "sections": [ { @@ -179,8 +180,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "packer_section_name", - "section": ".upx0" + "section": ".upx0", + "reason": "packer_section_name" } }, { @@ -189,8 +190,8 @@ "end": 0, "category": "pe_heuristic", "metadata": { - "reason": "packer_section_name", - "section": ".upx1" + "section": ".upx1", + "reason": "packer_section_name" } } ] diff --git a/tests/contract/tag_contract.py b/tests/contract/tag_contract.py new file mode 100644 index 0000000..07a0342 --- /dev/null +++ b/tests/contract/tag_contract.py @@ -0,0 +1,308 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 +"""Static verification of the parser -> validator tag contract.""" + +from __future__ import annotations + +import ast +from dataclasses import dataclass, field +from typing import Dict, List, Optional, Set, Tuple + +_ERROR_SINKS = {"errors", "entry_errors", "descriptor_errors", "block_errors"} +_TRUNCATION_SINKS = {"truncations"} +_ALL_SINKS = _ERROR_SINKS | _TRUNCATION_SINKS +_DEFAULT_TEMPLATE_VARS: Dict[str, List[str]] = {} + + +@dataclass +class ParserTags: + top_errors: Set[str] = field(default_factory=set) + top_truncations: Set[str] = field(default_factory=set) + item_errors: Set[str] = field(default_factory=set) + unexpanded: Set[str] = field(default_factory=set) + + @property + def errors(self) -> Set[str]: + return self.top_errors | self.item_errors + + @property + def truncations(self) -> Set[str]: + return self.top_truncations + + +@dataclass +class ValidatorConsumption: + matched: Set[str] = field(default_factory=set) + iterated_sinks: Set[str] = field(default_factory=set) + forwarded_sinks: Set[str] = field(default_factory=set) + item_forwarded_sinks: Set[str] = field(default_factory=set) + + @property + def wholesale_sinks(self) -> Set[str]: + return (self.iterated_sinks | self.forwarded_sinks | self.item_forwarded_sinks) + + +@dataclass +class ContractResult: + parser: str + validator: str + emitted_errors: Set[str] + emitted_truncations: Set[str] + matched: Set[str] + iterated_sinks: Set[str] + forwarded_sinks: Set[str] + dropped: Set[str] + phantom: Set[str] + unexpanded: Set[str] + + @property + def ok(self) -> bool: + return not (self.dropped or self.unexpanded) + + def report(self) -> str: + lines = [f"{self.parser} -> {self.validator}"] + lines.append(f" emitted (errors) : {len(self.emitted_errors)}") + lines.append(f" emitted (truncations) : {len(self.emitted_truncations)}") + lines.append(f" iterated wholesale : {sorted(self.iterated_sinks) or 'none'}") + lines.append(f" forwarded wholesale : {sorted(self.forwarded_sinks) or 'none'}") + if self.dropped: + lines.append(f" DROPPED : {sorted(self.dropped)}") + if self.unexpanded: + lines.append(f" UNEXPANDED TEMPLATES : {sorted(self.unexpanded)}") + if self.phantom: + lines.append(f" phantom (unreachable) : {sorted(self.phantom)}") + if self.ok and not self.phantom: + lines.append(" OK - every emittable tag has a consumer") + return "\n".join(lines) + + +def _is_entry_point(fn: ast.AST) -> bool: + return (isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef)) + and fn.name.startswith("build_") + and fn.name.endswith("_structure")) + + +def _is_tag_collection(name: str) -> bool: + upper = name.upper() + return ("ERROR" in upper or upper.endswith("_TAGS") + or upper.endswith("_PRIORITY")) + + +def _sink_ref(node: ast.AST) -> Optional[Tuple[str, bool]]: + if not (isinstance(node, ast.Call) + and isinstance(node.func, ast.Attribute) + and node.func.attr == "append"): + return None + target = node.func.value + if isinstance(target, ast.Name) and target.id in _ALL_SINKS: + return (target.id, True) + if isinstance(target, ast.Subscript) and isinstance(target.slice, ast.Constant): + if target.slice.value in _ALL_SINKS: + return (target.slice.value, False) + return None + + +def _function_scopes(tree: ast.AST) -> Dict[ast.AST, Set[str]]: + return {fn: {a.arg for a in fn.args.args} + for fn in ast.walk(tree) + if isinstance(fn, (ast.FunctionDef, ast.AsyncFunctionDef))} + + +def _expand_fstring(node, template_vars): + var = None + prefix = suffix = "" + seen_var = False + has_placeholder = False + for part in node.values: + if isinstance(part, ast.FormattedValue): + has_placeholder = True + if isinstance(part.value, ast.Name): + var = part.value.id + seen_var = True + else: + return [], ast.unparse(node) + elif isinstance(part, ast.Constant): + if seen_var: + suffix += str(part.value) + else: + prefix += str(part.value) + if not has_placeholder: + return [prefix], None + if var is None or var not in template_vars: + return [], ast.unparse(node) + return [f"{prefix}{v}{suffix}" for v in template_vars[var]], None + + +def extract_parser_tags(source, template_vars=None) -> ParserTags: + template_vars = {**_DEFAULT_TEMPLATE_VARS, **(template_vars or {})} + tree = ast.parse(source) + out = ParserTags() + scopes = _function_scopes(tree) + + def _bucket_for(sink, is_top): + if sink in _TRUNCATION_SINKS: + return out.top_truncations + return out.top_errors if is_top else out.item_errors + + def _add(bucket, arg): + if isinstance(arg, ast.Constant) and isinstance(arg.value, str): + bucket.add(arg.value) + elif isinstance(arg, ast.JoinedStr): + expansions, unexpanded = _expand_fstring(arg, template_vars) + if unexpanded is not None: + out.unexpanded.add(unexpanded) + else: + bucket.update(expansions) + + for fn, params in scopes.items(): + entry_point = _is_entry_point(fn) + for node in ast.walk(fn): + ref = _sink_ref(node) + if ref is not None and node.args: + sink, is_bare = ref + # Only the CANONICAL sink names can hold the struct's own + # lists. A parameter called descriptor_errors / entry_errors + # / block_errors is a per-item list passed down by its + # owner, so it stays checkable however it is reached. + canonical = sink in ("errors", "truncations") + is_top = (is_bare and canonical + and (sink in params or entry_point)) + _add(_bucket_for(sink, is_top), node.args[0]) + if isinstance(node, ast.Call): + for kw in node.keywords: + if kw.arg in _ALL_SINKS and isinstance(kw.value, ast.List): + bucket = _bucket_for(kw.arg, is_top=entry_point) + for el in kw.value.elts: + _add(bucket, el) + + for node in ast.walk(tree): + if isinstance(node, ast.Return) and isinstance(node.value, ast.Tuple): + last = node.value.elts[-1] + if isinstance(last, ast.Constant) and isinstance(last.value, str): + out.item_errors.add(last.value) + if isinstance(node, ast.Dict): + for k, v in zip(node.keys, node.values): + if (isinstance(k, ast.Constant) and k.value in _ALL_SINKS + and isinstance(v, ast.List)): + bucket = (out.top_truncations + if k.value in _TRUNCATION_SINKS + else out.item_errors) + for el in v.elts: + if isinstance(el, ast.Constant) and isinstance(el.value, str): + bucket.add(el.value) + return out + + +def _iterated_sink(node): + for sub in ast.walk(node.iter): + if isinstance(sub, ast.Constant) and sub.value in _ALL_SINKS: + return sub.value + return None + + +def _loop_var_reaches_emission(node): + if not isinstance(node.target, ast.Name): + return False + var = node.target.id + for sub in ast.walk(node): + if sub is node.target: + continue + if isinstance(sub, ast.Name) and sub.id == var and isinstance(sub.ctx, ast.Load): + return True + return False + + +def _forwarded_sinks_in_dict(node): + found = set() + for k, v in zip(node.keys, node.values): + if not (isinstance(k, ast.Constant) and isinstance(k.value, str)): + continue + if isinstance(v, ast.List): + continue + for sub in ast.walk(v): + if isinstance(sub, ast.Constant) and sub.value in _ALL_SINKS: + found.add(sub.value) + return found + + +def _item_forwarded_sinks(tree: ast.AST) -> Set[str]: + """ + Sinks forwarded wholesale from a LOOP VARIABLE, e.g. + + for sfi in vi.get("string_file_info", []): + emit(details={"errors": sfi["errors"]}) + + That copies a PER-ITEM list in full, so its tags cannot drop - distinct + from `details={"errors": list(imp["errors"])}`, which forwards the + struct's own top-level list. Only the loop-variable reference tells the + two apart. + """ + found: Set[str] = set() + for node in ast.walk(tree): + if not (isinstance(node, ast.For) and isinstance(node.target, ast.Name)): + continue + loop_var = node.target.id + for sub in ast.walk(node): + if not isinstance(sub, ast.Dict): + continue + for k, v in zip(sub.keys, sub.values): + if not (isinstance(k, ast.Constant) and k.value in _ALL_SINKS): + continue + if isinstance(v, ast.List): + continue # literal payload, not a forward + if any(isinstance(x, ast.Name) and x.id == loop_var + for x in ast.walk(v)): + found.add(k.value) + return found + + +def extract_validator_consumption(source) -> ValidatorConsumption: + tree = ast.parse(source) + out = ValidatorConsumption() + for node in ast.walk(tree): + if isinstance(node, ast.Assign) and isinstance( + node.value, (ast.List, ast.Set, ast.Tuple)): + names = [t.id for t in node.targets if isinstance(t, ast.Name)] + if any(_is_tag_collection(n) for n in names): + for el in node.value.elts: + if isinstance(el, ast.Constant) and isinstance(el.value, str): + out.matched.add(el.value) + if (isinstance(node, ast.Compare) and len(node.ops) == 1 + and isinstance(node.ops[0], ast.In) + and isinstance(node.left, ast.Constant) + and isinstance(node.left.value, str)): + out.matched.add(node.left.value) + if isinstance(node, ast.For): + sink = _iterated_sink(node) + if sink is not None and _loop_var_reaches_emission(node): + out.iterated_sinks.add(sink) + if isinstance(node, ast.Dict): + out.forwarded_sinks |= _forwarded_sinks_in_dict(node) + + out.item_forwarded_sinks = _item_forwarded_sinks(tree) + return out + + +def check_contract(parser_source, validator_source, parser_name="parser", + validator_name="validator", template_vars=None) -> ContractResult: + tags = extract_parser_tags(parser_source, template_vars) + consumption = extract_validator_consumption(validator_source) + unchecked = set() + if _TRUNCATION_SINKS & consumption.wholesale_sinks: + unchecked |= tags.top_truncations + if _ERROR_SINKS & consumption.forwarded_sinks: + unchecked |= tags.top_errors + if _ERROR_SINKS & consumption.iterated_sinks: + unchecked |= tags.errors + if _ERROR_SINKS & consumption.item_forwarded_sinks: + unchecked |= tags.item_errors + checkable = (tags.errors | tags.truncations) - unchecked + return ContractResult( + parser=parser_name, validator=validator_name, + emitted_errors=tags.errors, emitted_truncations=tags.truncations, + matched=consumption.matched, + iterated_sinks=consumption.iterated_sinks, + forwarded_sinks=consumption.forwarded_sinks, + dropped=checkable - consumption.matched, + phantom=consumption.matched - tags.errors - tags.truncations, + unexpanded=tags.unexpanded) diff --git a/tests/contract/test_tag_contract.py b/tests/contract/test_tag_contract.py new file mode 100644 index 0000000..ae81d1e --- /dev/null +++ b/tests/contract/test_tag_contract.py @@ -0,0 +1,311 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 + +""" +CI guard: every tombstone tag a parser can emit must have a validator that +consumes it. + +Add a (parser, validator, template_vars) row to _PAIRS for each subsystem. +The `template_vars` mapping supplies the call-site values for f-string tag +templates - a parser that builds f"{tag}_truncated" needs its `tag` values +listed, or the check fails loudly rather than silently skipping them. + +Two registries record deliberate gaps, and they cover different things: + + _PARSERLESS_VALIDATORS validators with no struct-level decoder at all. + They cannot appear in _PAIRS - there is no parser + module to pass - so the contract is inapplicable + rather than exempted. + + _TAGLESS_PARSERS parsers that ARE in _PAIRS but emit no tags, which + makes the drop check vacuously true. Their drop / + phantom / template checks still run; only the + zero-tags assertion is skipped. +""" + +from __future__ import annotations + +import inspect +import pkgutil +from typing import Dict, List, Set + +import pytest + +from tag_contract import check_contract + +from iocx.parsers import (pe_imports, pe_relocations, pe_tls, pe_debug, + pe_exports, pe_delay_imports, pe_certificates, pe_exception, + pe_resources, pe_version_info, pe_load_config, pe_optional_header) +from iocx.validators import (imports, relocations, tls, debug, + exports, delay_imports, signature, exception_table, + resources, version_info, load_config_directory, optional_header) + +_PAIRS = [ + # (label, parser module, validator module, template_vars) + # + # template_vars supplies the call-site values for f-string tag templates. + # pe_imports builds thunk-array tags as f"{tag}_truncated" etc., where + # `tag` is _read_thunk_array's parameter. It is passed exactly two + # values, from _enrich_descriptor's name-source selection: + # "int" - OriginalFirstThunk present (normal) + # "iat_fallback" - OriginalFirstThunk zero, fell back to FirstThunk + # Both must be listed, or those eight tags are silently unchecked. + ("pe_imports", pe_imports, imports, {"tag": ["int", "iat_fallback"]}), + ("pe_relocations", pe_relocations, relocations, {}), + ("pe_tls", pe_tls, tls, {}), + ("pe_debug", pe_debug, debug, {}), + ("pe_exports", pe_exports, exports, {"tag": ["eat", "enpt", "eot"]}), + ("pe_delay_imports", pe_delay_imports, delay_imports, {"tag": ["int", "iat"]}), + ("pe_certificates", pe_certificates, signature, {}), + ("pe_exception", pe_exception, exception_table, {}), + ("pe_resources", pe_resources, resources, {}), + ("pe_version_info", pe_version_info, version_info, {}), + # The two below are registered pairs like any other - their drop, + # phantom and template checks all run. They appear in _TAGLESS_PARSERS + # only to skip the zero-tags assertion, which would otherwise fail + # because neither parser records tombstone tags. See that registry for + # the reasoning in each case. + ("pe_load_config", pe_load_config, load_config_directory,{}), + ("pe_optional_header", pe_optional_header, optional_header, {}), +] + +# Tags a parser emits that no validator consumes, deliberately. +# Scoped per-tag so a SECOND drop in the same subsystem still fails. +_KNOWN_DELIBERATE_DROPS = { + # pe_tls records this, but validate_tls derives TLS_INVALID_RANGE from + # the start/end VA fields directly - more authoritative than a derived + # tag. Redundant signalling, not a lost finding. + "pe_tls": {"tls_raw_data_end_before_start"}, + # validate_signature checks cert["revision"] and cert["cert_type"] + # against the known-value sets directly rather than consuming these + # tags. The facts reach output as SIGNATURE_INVALID_REVISION / + # SIGNATURE_INVALID_TYPE. Redundant signalling, not a lost finding. + "pe_certificates": {"unknown_revision", "unknown_cert_type", "length_too_small"}, +} + +# Validators with no dedicated parser. Each reads the metadata and analysis +# layers directly rather than decoding structure from bytes, so there are no +# tombstone tags to drop. +_PARSERLESS_VALIDATORS: Dict[str, str] = { + "entrypoint": ( + "Maps the entry point against sections and the overlay. Reads " + "PublicMetadata and AnalysisDict; decodes no structure." + ), + "sections": ( + "Section flags, names, alignment and overlap. Reads PublicMetadata " + "and AnalysisDict; decodes no structure." + ), + "rva_graph": ( + "The directory placement backbone that relocations, debug, imports " + "and the security directory all defer to. Reads PublicMetadata and " + "AnalysisDict; decodes no structure." + ), + "entropy": ( + "Computes entropy metrics over regions supplied by AnalysisDict. " + "Decodes no structure." + ), +} + + +# Parsers that legitimately emit no tombstone tags. A tagless parser passes +# the drop check vacuously, so it must be declared rather than inferred: +# otherwise a tag-extraction failure looks identical to a parser that has +# nothing to extract. +_TAGLESS_PARSERS: Set[str] = { + # Reads two named fields from pefile's parsed OPTIONAL_HEADER with None + # fallbacks. No byte-level decode, so no decode failure to tombstone. + "pe_optional_header", + + # Returns a fields dict with no error list. NOTE this is a design gap + # rather than a clean absence: four distinct conditions - no directory, + # a malformed entry, rva == 0, and an unmapped rva - all collapse to + # parsed_size == 0, so the validator cannot tell "absent" from + # "declared but unreadable". Tracked separately; listed here so the + # vacuous drop check is explicit rather than silent. + "pe_load_config", +} + + +# Modules under iocx/validators that are not validators. +_NON_VALIDATOR_MODULES = {"schema", "decorators"} + + +def _pair_ids() -> List[str]: + """ + Parametrise labels for _PAIRS. + + Copes with pytest.param rows: ParameterSet is a NamedTuple whose field 0 + is `values` - the whole argument tuple - not the label. A bare + `p[0]` therefore yields the tuple and pytest rejects it at collection. + Used by every parametrised test here so the two shapes cannot diverge. + """ + return [(p.values[0] if hasattr(p, "values") else p[0]) for p in _PAIRS] + + +def _discover_validator_modules() -> Set[str]: + """Every validator module name in the package.""" + import iocx.validators + return { + name + for _, name, _ in pkgutil.iter_modules(iocx.validators.__path__) + if not name.startswith("_") and name not in _NON_VALIDATOR_MODULES + } + + +def _paired_validator_names() -> Set[str]: + """Validator module names covered by _PAIRS.""" + return { + (p.values[2] if hasattr(p, "values") else p[2]).__name__.rsplit(".", 1)[-1] + for p in _PAIRS + } + + +@pytest.mark.contract +class TestContractCoverage: + + def test_every_validator_is_paired_or_exempt(self): + """ + The registration guard. Without it, a new parser/validator pair that + nobody adds to _PAIRS is silently unchecked - and every finding this + contract check has produced came from a pair that WAS registered. + """ + found = _discover_validator_modules() + unregistered = found - _paired_validator_names() - set(_PARSERLESS_VALIDATORS) + assert not unregistered, ( + f"validator modules in neither _PAIRS nor _PARSERLESS_VALIDATORS: " + f"{sorted(unregistered)}. Add the pair to _PAIRS, or record why " + f"the validator has no parser." + ) + + def test_no_stale_parserless_exemptions(self): + """An exemption for a validator that no longer exists is misleading.""" + found = _discover_validator_modules() + stale = set(_PARSERLESS_VALIDATORS) - found + assert not stale, ( + f"_PARSERLESS_VALIDATORS names modules that do not exist: " + f"{sorted(stale)}" + ) + + def test_no_stale_pairs(self): + found = _discover_validator_modules() + stale = _paired_validator_names() - found + assert not stale, ( + f"_PAIRS names validator modules that do not exist: {sorted(stale)}" + ) + + def test_parserless_exemptions_carry_a_reason(self): + """ + The reason is the point: it records WHY there is no parser, so a + later reader can tell 'no decoder by design' from 'not built yet'. + """ + for name, reason in _PARSERLESS_VALIDATORS.items(): + assert reason and len(reason) > 20, ( + f"{name} needs a substantive reason, not {reason!r}" + ) + + def test_a_validator_is_not_both_paired_and_exempt(self): + overlap = _paired_validator_names() & set(_PARSERLESS_VALIDATORS) + assert not overlap, ( + f"validators both paired and exempted: {sorted(overlap)}" + ) + + +@pytest.mark.contract +@pytest.mark.parametrize("name,parser_mod,validator_mod,templates", + _PAIRS, ids=_pair_ids()) +def test_no_parser_emits_zero_tags(name, parser_mod, validator_mod, templates): + """ + A parser with no tags passes the drop check vacuously - emitted is + empty, so `emitted - matched` is empty regardless of what the validator + consumes. That is indistinguishable from a tag-extraction failure in the + checker itself, which has happened nine times during development. + + A genuinely tagless parser must opt in via _TAGLESS_PARSERS. + """ + if name in _TAGLESS_PARSERS: + pytest.skip(f"{name} records no tombstone tags by design") + + result = check_contract( + inspect.getsource(parser_mod), + inspect.getsource(validator_mod), + parser_name=name, + validator_name=validator_mod.__name__, + template_vars=templates, + ) + assert result.emitted_errors or result.emitted_truncations, ( + f"{name} emitted no tags at all, so its drop check proves nothing. " + f"Either the parser is genuinely tagless - add it to " + f"_TAGLESS_PARSERS - or tag extraction is broken for this shape." + ) + + +@pytest.mark.contract +@pytest.mark.parametrize("name,parser_mod,validator_mod,templates", + _PAIRS, ids=_pair_ids()) +def test_no_tag_is_silently_dropped(name, parser_mod, validator_mod, templates): + """ + A tag with no consumer vanishes: _first_matching returns "unknown" and + the caller continues, so the structural finding never reaches output. + """ + result = check_contract( + inspect.getsource(parser_mod), + inspect.getsource(validator_mod), + parser_name=name, + validator_name=validator_mod.__name__, + template_vars=templates, + ) + unexpected = result.dropped - _KNOWN_DELIBERATE_DROPS.get(name, set()) + assert not unexpected, ( + f"\n{result.report()}\n\n" + f"These tags are emitted by {name} but consumed by nothing. Add them " + f"to the relevant priority list, or remove them from the parser." + ) + + +@pytest.mark.contract +@pytest.mark.parametrize("name,parser_mod,validator_mod,templates", + _PAIRS, ids=_pair_ids()) +def test_no_unexpandable_tag_template(name, parser_mod, validator_mod, templates): + """ + An f-string tag whose variable has no declared expansion cannot be + checked. Fail rather than skip - a silent skip is how the original bugs + stayed hidden. + """ + result = check_contract( + inspect.getsource(parser_mod), + inspect.getsource(validator_mod), + parser_name=name, + validator_name=validator_mod.__name__, + template_vars=templates, + ) + assert not result.unexpanded, ( + f"\n{result.report()}\n\n" + f"Add the call-site values for these templates to _PAIRS." + ) + + +@pytest.mark.contract +@pytest.mark.parametrize("name,parser_mod,validator_mod,templates", + _PAIRS, ids=_pair_ids()) +def test_no_phantom_tags(name, parser_mod, validator_mod, templates): + """ + A tag named in a priority list that no parser can produce is inert but + misleading: it implies a routing that does not exist, and would produce + a wrong sub_reason if the name were ever reused at another level. + + If you keep deliberate defensive entries - a tag listed against a guard + that cannot currently fire - add a _KNOWN_PHANTOMS mapping scoped + per-tag, mirroring _KNOWN_DELIBERATE_DROPS. Do not xfail the whole + pair: that would also silence a genuine phantom. + """ + result = check_contract( + inspect.getsource(parser_mod), + inspect.getsource(validator_mod), + parser_name=name, + validator_name=validator_mod.__name__, + template_vars=templates, + ) + assert not result.phantom, ( + f"\n{result.report()}\n\n" + f"These tags appear in a priority list but cannot be emitted." + ) diff --git a/tests/contract/test_tag_self.py b/tests/contract/test_tag_self.py new file mode 100644 index 0000000..7fe8ef5 --- /dev/null +++ b/tests/contract/test_tag_self.py @@ -0,0 +1,519 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 + +""" +Self-tests for the tag-contract checker. + +The checker is static analysis over parser/validator source, so every code +shape it must recognise is a separate case - and every shape it misread in +the past is a regression guard. Eight defects were found by feeding it real +parsers one at a time; each cost a review cycle to diagnose. These fixtures +turn that into a unit-test failure. + +The fixtures are DELIBERATELY MINIMAL rather than excerpts of real parsers: +each isolates one syntactic shape, so a failure names the shape directly. + +Naming conventions the checker depends on, and which it CANNOT verify: + * a parser's module entry point is `build__structure` + * a validator's tag list is named `*_ERROR_*`, `*_TAGS` or `*_PRIORITY` + * a per-item error list passed as a parameter is NOT named plain `errors` +Violating any of these produces a wrong verdict, not an exception, so +TestNamingConventionDependencies documents them explicitly. +""" + +from __future__ import annotations + +import pytest + +from tag_contract import ( + check_contract, + extract_parser_tags, + extract_validator_consumption, + _is_entry_point, + _is_tag_collection, +) + + +# ================================================================= +# Shape 1 - direct appends +# ================================================================= + +@pytest.mark.contract +class TestDirectAppends: + + def test_bare_name_parameter_is_top_level(self): + src = ''' +def _walk(pe, truncations, errors): + errors.append("top_tag") + truncations.append("trunc_tag") +''' + tags = extract_parser_tags(src) + assert tags.top_errors == {"top_tag"} + assert tags.top_truncations == {"trunc_tag"} + assert tags.item_errors == set() + + def test_bare_name_local_outside_entry_point_is_per_item(self): + src = ''' +def _decode_entry(pe, value): + errors = [] + errors.append("item_tag") + return {"errors": errors} +''' + tags = extract_parser_tags(src) + assert tags.item_errors == {"item_tag"} + assert tags.top_errors == set() + + def test_bare_name_local_inside_entry_point_is_top_level(self): + """ + pe_certificates creates its own lists as locals in the entry point + and returns them as the struct's top-level errors. Same syntax as + the per-item case above, opposite level. + """ + src = ''' +def build_certificate_structure(pe): + errors = [] + errors.append("raw_file_unavailable") + return {"errors": errors} +''' + tags = extract_parser_tags(src) + assert tags.top_errors == {"raw_file_unavailable"} + assert tags.item_errors == set() + + def test_subscript_append_is_per_item(self): + src = ''' +def _decode(pe, descriptor): + descriptor["errors"].append("dll_name_empty") +''' + tags = extract_parser_tags(src) + assert tags.item_errors == {"dll_name_empty"} + + def test_differently_named_parameter_is_per_item(self): + """ + `descriptor_errors` is a parameter but not the canonical sink name, + so it holds a per-descriptor list rather than the struct's own. + """ + src = ''' +def _read_thunks(pe, rva, descriptor_errors, truncations): + descriptor_errors.append("int_rva_zero") +''' + tags = extract_parser_tags(src) + assert tags.item_errors == {"int_rva_zero"} + assert tags.top_errors == set() + + +# ================================================================= +# Shape 2 - f-string templates +# ================================================================= + +@pytest.mark.contract +class TestFStringTemplates: + + def test_template_expands_against_declared_values(self): + src = ''' +def _read(pe, rva, tag, truncations): + truncations.append(f"{tag}_truncated") +''' + tags = extract_parser_tags(src, {"tag": ["int", "iat"]}) + assert tags.top_truncations == {"int_truncated", "iat_truncated"} + assert tags.unexpanded == set() + + def test_template_without_values_is_reported_unexpanded(self): + """Fail loudly rather than skip - a silent skip is how the original + drops stayed hidden.""" + src = ''' +def _read(pe, rva, tag, truncations): + truncations.append(f"{tag}_truncated") +''' + tags = extract_parser_tags(src, {}) + assert tags.top_truncations == set() + assert tags.unexpanded == {"f'{tag}_truncated'"} + + def test_fstring_with_no_placeholder_is_a_literal(self): + """ + `f"literal"` stays a JoinedStr in the AST. An earlier version + reported it as an unexpandable template. + """ + src = ''' +def _walk(pe, errors): + errors.append(f"block_header_unpack_failed") +''' + tags = extract_parser_tags(src) + assert tags.top_errors == {"block_header_unpack_failed"} + assert tags.unexpanded == set() + + def test_non_name_placeholder_is_unexpandable(self): + src = ''' +def _walk(pe, errors, index): + errors.append(f"failed_at_{index + 1}") +''' + tags = extract_parser_tags(src) + assert tags.unexpanded + assert tags.top_errors == set() + + +# ================================================================= +# Shape 3 - helper tuple returns +# ================================================================= + +@pytest.mark.contract +class TestHelperTupleReturns: + + def test_tag_in_last_tuple_position(self): + src = ''' +def _read_asciiz(pe, rva, max_len): + if rva == 0: + return None, "rva_zero" + return "name", None +''' + assert extract_parser_tags(src).item_errors == {"rva_zero"} + + def test_tag_on_a_separate_line_from_return(self): + """ + The UnicodeDecodeError shape: the literal sits on a different line + from the `return`, which defeated regex extraction. + """ + src = ''' +def _read_asciiz(pe, rva, max_len): + try: + return raw.decode("ascii"), None + except UnicodeDecodeError: + s = raw.decode("ascii", errors="replace") + return s, "non_ascii" +''' + assert "non_ascii" in extract_parser_tags(src).item_errors + + def test_three_element_tuple_return(self): + src = ''' +def _read_import_by_name(pe, rva): + return None, None, "name_too_short" +''' + assert extract_parser_tags(src).item_errors == {"name_too_short"} + + +# ================================================================= +# Shape 4 - literal lists in returned dicts +# ================================================================= + +@pytest.mark.contract +class TestDictLiteralLists: + + def test_literal_error_list_in_returned_dict(self): + src = ''' +def _decode_entry(buf, index): + return {"index": index, "errors": ["entry_unpack_failed"]} +''' + assert extract_parser_tags(src).item_errors == {"entry_unpack_failed"} + + def test_non_list_dict_values_do_not_raise(self): + """ + Regression: an over-broad `for el in v.elts` ran for every dict + value, raising AttributeError on Name and Constant nodes. + """ + src = ''' +def _decode(buf, index): + return {"index": index, "errors": errors, "name": None, + "count": 0, "nested": {"a": 1}} +''' + tags = extract_parser_tags(src) # must not raise + assert tags.item_errors == set() + + def test_dict_with_mixed_literal_and_variable_sinks(self): + src = ''' +def _decode(buf, index): + return {"errors": ["tag_a"], "truncations": truncations} +''' + tags = extract_parser_tags(src) + assert tags.item_errors == {"tag_a"} + + +# ================================================================= +# Shape 5 - kwarg list literals +# ================================================================= + +@pytest.mark.contract +class TestKwargListLiterals: + + def test_kwarg_in_entry_point_is_top_level(self): + """ + pe_exports: _empty_result builds the TOP-LEVEL struct, so a kwarg + passed to it from the entry point fills the struct's own errors. + """ + src = ''' +def build_export_structure(pe): + return _empty_result(rva, size, errors=["header_read_failed"]) + +def _empty_result(rva, size, *, errors=None, truncations=None): + return {"errors": errors or [], "truncations": truncations or []} +''' + tags = extract_parser_tags(src) + assert tags.top_errors == {"header_read_failed"} + assert tags.item_errors == set() + + def test_kwarg_outside_entry_point_is_per_item(self): + """ + pe_exception: _unwind_result builds a PER-ENTRY record. Identical + syntax to the case above, opposite level. + """ + src = ''' +def build_exception_structure(pe): + return {"functions": functions} + +def _decode_unwind_info(pe, rva): + return _unwind_result(errors=["unwind_read_failed"]) +''' + tags = extract_parser_tags(src) + assert tags.item_errors == {"unwind_read_failed"} + assert tags.top_errors == set() + + def test_kwarg_truncations_always_top_level(self): + src = ''' +def _walk(pe): + return _result(truncations=["table_truncated"]) +''' + assert extract_parser_tags(src).top_truncations == {"table_truncated"} + + +# ================================================================= +# Validator consumption +# ================================================================= + +@pytest.mark.contract +class TestValidatorConsumption: + + def test_priority_list_recognised_by_name(self): + src = '_ENTRY_ERROR_PRIORITY = ["a", "b"]' + assert extract_validator_consumption(src).matched == {"a", "b"} + + @pytest.mark.parametrize("name", [ + "_DLL_NAME_ERROR_PRIORITY", "_HEADER_DECODE_ERROR_TAGS", + "_UNWIND_ERROR_PRIORITY", "_STRUCTURAL_CERT_ERROR_TAGS", + ]) + def test_tag_collection_names_accepted(self, name): + assert _is_tag_collection(name) + + @pytest.mark.parametrize("name", [ + "_TABLE_ARCHS", "_VALID_UNWIND_VERSIONS", "_REVISION_NAMES", + ]) + def test_non_tag_collection_names_rejected(self, name): + """ + _TABLE_ARCHS = ("amd64","arm64","arm") is structurally identical to + a priority list and produced three spurious phantoms. + """ + assert not _is_tag_collection(name) + + def test_arch_tuple_is_not_read_as_tags(self): + src = ''' +_TABLE_ARCHS = ("amd64", "arm64", "arm") +_ENTRY_ERROR_PRIORITY = ["real_tag"] +''' + matched = extract_validator_consumption(src).matched + assert matched == {"real_tag"} + assert "amd64" not in matched + + def test_membership_test_counts_as_consumption(self): + src = ''' +def _v(entry_errors): + if "ordinal_index_duplicate" in entry_errors: + pass +''' + assert "ordinal_index_duplicate" in extract_validator_consumption(src).matched + + def test_iterated_wholesale_detected(self): + src = ''' +def _v(imp, issues): + for tag in imp.get("truncations", []) or []: + issues.append(StructuralIssue(details={"table": tag})) +''' + assert extract_validator_consumption(src).iterated_sinks == {"truncations"} + + def test_iteration_that_ignores_the_loop_var_is_not_wholesale(self): + """Counting a list is not forwarding it.""" + src = ''' +def _v(imp, issues): + count = 0 + for tag in imp.get("truncations", []) or []: + count += 1 +''' + assert extract_validator_consumption(src).iterated_sinks == set() + + def test_forwarded_wholesale_detected(self): + src = ''' +def _v(imp, issues): + issues.append(StructuralIssue( + details={"sub_reason": "top_level_decode", + "errors": list(imp["errors"])})) +''' + assert extract_validator_consumption(src).forwarded_sinks == {"errors"} + + def test_literal_details_list_is_not_forwarding(self): + """A fixed payload names no sink.""" + src = ''' +def _v(issues): + issues.append(StructuralIssue(details={"errors": ["fixed", "payload"]})) +''' + assert extract_validator_consumption(src).forwarded_sinks == set() + + +# ================================================================= +# End-to-end verdicts +# ================================================================= + +_PARSER = ''' +def build_thing_structure(pe): + errors = [] + truncations = [] + if pe is None: + return _empty_result(errors=["top_decode_failed"]) + errors.append("top_direct") + truncations.append("table_truncated") + return {"errors": errors, "truncations": truncations} + +def _read_asciiz(pe, rva): + return None, "read_failed" + +def _decode_item(pe, descriptor): + descriptor["errors"].append("item_tag") + +def _decode_entry(buf, index): + return {"index": index, "errors": ["entry_unpack_failed"]} +''' + +_VALIDATOR = ''' +_ITEM_ERROR_PRIORITY = ["item_tag", "read_failed", "entry_unpack_failed"] + +def validate_thing(internal): + st = internal.get("thing_struct") + if st.get("errors"): + issues.append(StructuralIssue( + details={"sub_reason": "top_level_decode", + "errors": list(st["errors"])})) + return issues + for tag in st.get("truncations", []) or []: + issues.append(StructuralIssue(details={"table": tag})) +''' + +@pytest.mark.contract +class TestEndToEnd: + + def test_healthy_pair_is_clean(self): + result = check_contract(_PARSER, _VALIDATOR) + assert result.dropped == set() + assert result.phantom == set() + assert result.unexpanded == set() + assert result.ok + + def test_dropped_item_tag_detected(self): + bad = _VALIDATOR.replace('"item_tag", ', '') + result = check_contract(_PARSER, bad) + assert result.dropped == {"item_tag"} + + def test_top_level_tags_exempt_via_forwarding(self): + """ + Both top-level tags are forwarded wholesale, so neither needs a + priority entry. Removing the forward must expose them. + """ + no_forward = _VALIDATOR.replace( + '"errors": list(st["errors"])', '"count": 1') + result = check_contract(_PARSER, no_forward) + assert {"top_direct", "top_decode_failed"} <= result.dropped + + def test_truncations_exempt_via_iteration(self): + result = check_contract(_PARSER, _VALIDATOR) + assert "table_truncated" not in result.dropped + + def test_phantom_detected(self): + bad = _VALIDATOR.replace( + '"entry_unpack_failed"]', '"entry_unpack_failed", "never_emitted"]') + assert check_contract(_PARSER, bad).phantom == {"never_emitted"} + + def test_report_is_readable(self): + text = check_contract(_PARSER, _VALIDATOR, "pe_thing", "thing").report() + assert "pe_thing -> thing" in text + assert "OK" in text + + +# ================================================================= +# Convention dependencies +# ================================================================= + +@pytest.mark.contract +class TestNamingConventionDependencies: + """ + The checker cannot verify these conventions; it silently misclassifies + when they are broken. Pinned so the dependency is explicit. + """ + + @pytest.mark.parametrize("name,expected", [ + ("build_import_structure", True), + ("build_certificate_structure", True), + ("_read_descriptors", False), + ("build_something_else", False), + ("make_import_structure", False), + ]) + def test_entry_point_recognition(self, name, expected): + import ast as _ast + fn = _ast.parse(f"def {name}(pe): pass").body[0] + assert _is_entry_point(fn) is expected + + def test_renamed_entry_point_misclassifies_silently(self): + """ + A parser whose entry point is not build_*_structure has its + top-level locals read as per-item, producing spurious drops. This + fails as a wrong VERDICT, never as an exception. + """ + renamed = ''' +def make_thing(pe): + errors = [] + errors.append("top_tag") + return {"errors": errors} +''' + tags = extract_parser_tags(renamed) + assert tags.item_errors == {"top_tag"} # wrong, but silent + assert tags.top_errors == set() + + def test_unconventional_list_name_is_ignored(self): + """A tag list named outside the convention is not read at all.""" + src = '_SOMETHING = ["real_tag"]' + assert extract_validator_consumption(src).matched == set() + + +# ================================================================= +# Robustness +# ================================================================= + +@pytest.mark.contract +class TestRobustness: + + def test_empty_source(self): + assert extract_parser_tags("").errors == set() + assert extract_validator_consumption("").matched == set() + + def test_source_with_no_tags(self): + src = 'def f(x):\n return x + 1\n' + tags = extract_parser_tags(src) + assert tags.errors == set() and tags.truncations == set() + + def test_nested_functions_are_walked(self): + src = ''' +def build_thing_structure(pe): + def inner(errors): + errors.append("nested_tag") + return {} +''' + assert "nested_tag" in extract_parser_tags(src).errors + + def test_append_to_unrelated_list_is_ignored(self): + src = ''' +def _walk(pe, results): + results.append("not_a_tag") +''' + tags = extract_parser_tags(src) + assert tags.errors == set() and tags.truncations == set() + + def test_non_string_append_is_ignored(self): + src = ''' +def _walk(pe, errors): + errors.append(42) + errors.append(some_variable) +''' + assert extract_parser_tags(src).errors == set() diff --git a/tests/integration/test_franken_malformed_pe.py b/tests/integration/test_franken_malformed_pe.py index 5822033..522e52a 100644 --- a/tests/integration/test_franken_malformed_pe.py +++ b/tests/integration/test_franken_malformed_pe.py @@ -47,7 +47,8 @@ def test_franken_expected_heuristics(franken_result): "section_overlap", "section_raw_overlap", "exception_directory_size_not_multiple", - "exception_table_truncated" + "exception_table_truncated", + "import_table_truncated" } assert heur == expected diff --git a/tests/unit/cli/test_cli_ext.py b/tests/unit/cli/test_cli_ext.py index ea34269..7d121e0 100644 --- a/tests/unit/cli/test_cli_ext.py +++ b/tests/unit/cli/test_cli_ext.py @@ -5,7 +5,7 @@ import sys from pathlib import Path import json -import pytest +import pytest, re def run_cli(*args, input=None): @@ -54,8 +54,14 @@ def test_cli_list_detectors(): def test_cli_version(): result = run_cli("--version") assert result.returncode == 0 - # Version should look like "0.1.0" or similar - assert result.stdout.strip()[0].isdigit() + # Find the line starting with "iocx" + match = re.search(r"^iocx\s+([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)$", result.stdout, re.MULTILINE) + assert match, "iocx version line not found in banner" + + version = match.group(1) + + # Validate semantic version format + assert re.match(r"^\d+\.\d+\.\d+\.\d+$", version), "iocx version is not numeric semantic version" def test_cli_help(): diff --git a/tests/unit/parsers/_generate_vs_versioninfo_fixtures.py b/tests/unit/parsers/_generate_vs_versioninfo_fixtures.py new file mode 100644 index 0000000..3722d5a --- /dev/null +++ b/tests/unit/parsers/_generate_vs_versioninfo_fixtures.py @@ -0,0 +1,239 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 +""" +Byte-level VS_VERSIONINFO blob builder for parser stress tests. + +A resource compiler will only ever emit well-formed version-info, so the +malformed cases cannot come from a C fixture. This builds the structure +directly, with each mutation expressed as a named, single-fault deviation +from a known-good baseline. +""" + +import struct + +_VS_FFI_SIGNATURE = 0xFEEF04BD +_VS_FFI_STRUCT_VERSION = 0x00010000 + + +def _u16(v): + return struct.pack(" header_ok False.""" + children = _pad4(string_file_info([string_table(strings=_BASELINE_STRINGS)])) + return version_info(key="VS_VERSION_BAD", children=children) + + +def length_inconsistent(): + """wLength larger than the buffer -> length_consistent False.""" + return version_info(length_override=0xFFFF) + + +def truncated_header(): + """Fewer than 6 bytes -> too_short.""" + return b"\x40\x01\x34" + + +def ffi_bad_signature(): + return version_info(ffi=fixed_file_info(signature=0xDEADBEEF)) + + +def ffi_bad_struct_version(): + return version_info(ffi=fixed_file_info(struct_version=0x00020000)) + + +def ffi_truncated(): + """wValueLength non-zero but under 52 -> fixed_file_info_truncated.""" + return version_info(ffi=b"\x00" * 16, value_length=16) + + +def ffi_absent(): + """wValueLength == 0 is a legitimate omission, not a defect.""" + children = _pad4(string_file_info([string_table(strings=_BASELINE_STRINGS)])) + return version_info(ffi=b"", children=children) + + +def unknown_child(): + """A child that is neither StringFileInfo nor VarFileInfo.""" + bogus = _node("NotAKnownChild", children=_pad4(_node("x")), w_type=1) + return version_info(children=_pad4(bogus)) + + +def child_length_invalid(): + """A child claiming more bytes than the envelope holds.""" + sfi = string_file_info([string_table(strings=_BASELINE_STRINGS)]) + bad = _u16(0xFFFE) + sfi[2:] + return version_info(children=_pad4(bad)) + + +def child_max_exceeded(count=300): + """More children than the 256 hard cap.""" + one = _pad4(_node("NotAKnownChild", children=_pad4(_node("x")), w_type=1)) + return version_info(children=one * count) + + +def lang_codepage_key(): + """StringTable key that is not 8 hex characters.""" + tbl = string_table(lang_codepage="ENGLISHX", strings=_BASELINE_STRINGS) + return version_info(children=_pad4(string_file_info([tbl]))) + + +def string_length_invalid(): + """A String node whose wLength overruns its table.""" + entry = string_entry("CompanyName", "MalX Labs") + bad = _u16(0xFFFE) + entry[2:] + tbl = _node("040904B0", children=_pad4(bad), w_type=1) + return version_info(children=_pad4(string_file_info([tbl]))) + + +def translation_not_dword_aligned(): + """Translation wValueLength not a DWORD multiple.""" + return version_info(children=_pad4(var_file_info(value_length=6))) + + +def bidi_in_company_name(): + """ + Not a structural defect - the parser accepts it. Exercises the + projection's control/bidi stripping, which is where it matters. + """ + strings = dict(_BASELINE_STRINGS) + strings["CompanyName"] = "MalX \u202eLabs\u202c\nInc" + return version_info(children=_pad4(string_file_info([string_table(strings=strings)]))) + + +def deeply_nested_tables(count=200): + """Many string tables - exercises the projection's table cap.""" + tables = [string_table(lang_codepage=f"{i:04X}04B0", + strings={"CompanyName": f"T{i}"}) + for i in range(count)] + return version_info(children=_pad4(string_file_info(tables))) + + +CASES = { + "baseline": baseline, + "szkey_mismatch": szkey_mismatch, + "length_inconsistent": length_inconsistent, + "truncated_header": truncated_header, + "ffi_bad_signature": ffi_bad_signature, + "ffi_bad_struct_version": ffi_bad_struct_version, + "ffi_truncated": ffi_truncated, + "ffi_absent": ffi_absent, + "unknown_child": unknown_child, + "child_length_invalid": child_length_invalid, + "child_max_exceeded": child_max_exceeded, + "lang_codepage_key": lang_codepage_key, + "string_length_invalid": string_length_invalid, + "translation_not_dword_aligned": translation_not_dword_aligned, + "bidi_in_company_name": bidi_in_company_name, + "deeply_nested_tables": deeply_nested_tables, +} diff --git a/tests/unit/parsers/test_pe_certificates.py b/tests/unit/parsers/test_pe_certificates.py index 40c397e..0699eb7 100644 --- a/tests/unit/parsers/test_pe_certificates.py +++ b/tests/unit/parsers/test_pe_certificates.py @@ -316,7 +316,7 @@ def boom(fmt, buf, off=0): errs: List[str] = [] out = _read_certificates(data, 0x800, 0x18, len(data), [], errs) assert out == [] - assert any(e.startswith("certificate_header_unpack_failed_at_") + assert any(e.startswith("certificate_header_unpack_failed") for e in errs) diff --git a/tests/unit/parsers/test_pe_certificates_gf.py b/tests/unit/parsers/test_pe_certificates_gf.py new file mode 100644 index 0000000..3bceb50 --- /dev/null +++ b/tests/unit/parsers/test_pe_certificates_gf.py @@ -0,0 +1,409 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 + +""" +Gap-filling tests for pe_certificates and validators.signature. + +These cover paths the existing suites leave unexercised. Each class states +what a regression would look like, because several of these behaviours are +deliberate and read as bugs without that context. +""" + +from __future__ import annotations + +import struct +from typing import Any, Dict, List, Optional + +import pytest + +from iocx.parsers.pe_certificates import ( + build_certificate_structure, _align_up, _decode_certificate, + _read_certificates, _CERT_ALIGNMENT, _WIN_CERT_HEADER_SIZE, + _SECURITY_DIRECTORY_INDEX, +) +from iocx.reason_codes import ReasonCodes +from iocx.validators.signature import validate_signature + + +# ================================================================= +# Builders +# ================================================================= + +def _header(dw_length: int, revision: int = 0x0200, + cert_type: int = 0x0002) -> bytes: + return struct.pack(" bytes: + """ + A WIN_CERTIFICATE. `pad` appends the inter-entry padding a conformant + file carries - dwLength counts only the header plus blob, so an entry + whose dwLength is not a multiple of 8 is followed by padding bytes that + the walk skips via _align_up. + """ + entry = _header(_WIN_CERT_HEADER_SIZE + len(blob), revision, cert_type) + blob + if pad: + entry += b"\x00" * ((-len(entry)) % _CERT_ALIGNMENT) + return entry + + +class _DataDir: + def __init__(self, va, size): + self.VirtualAddress = va + self.Size = size + +class _OptHdr: + def __init__(self, dd): + self.DATA_DIRECTORY = [None] * 16 + if dd is not None: + self.DATA_DIRECTORY[_SECURITY_DIRECTORY_INDEX] = dd + +class _Section: + def __init__(self, ptr, raw_size): + self.PointerToRawData = ptr + self.SizeOfRawData = raw_size + +class _FakePE: + def __init__(self, file_bytes=b"", dd=None, sections=()): + self.__data__ = file_bytes + self.OPTIONAL_HEADER = _OptHdr(dd) + self.sections = sections + + +# ================================================================= +# PARSER - 8-byte alignment advance +# ================================================================= + +class TestAlignmentAdvance: + """ + dwLength counts the header plus blob only; entries are QWORD-aligned, so + the walk must advance by _align_up(dwLength, 8). + + The existing suite's multi-entry fixture uses blob sizes that are + already 8-aligned, so removing _align_up entirely would not fail any + test - the walk would read the next header at the same offset either + way. These fixtures use dwLength values that are NOT multiples of 8. + """ + + @pytest.mark.parametrize("blob_size,expected_advance", [ + (1, 16), # dwLength 9 + (5, 16), # dwLength 13 + (7, 16), # dwLength 15 + (13, 24), # dwLength 21 + ]) + def test_second_entry_found_at_aligned_offset(self, blob_size, expected_advance): + first = _entry(b"\xAA" * blob_size) + second = _entry(b"\xBB" * 4) + data = bytes(0x800) + first + second + out = _read_certificates(data, 0x800, len(first) + len(second), + len(data), [], []) + assert len(out) == 2 + assert out[1]["offset"] == 0x800 + expected_advance + assert out[1]["length"] == _WIN_CERT_HEADER_SIZE + 4 + + def test_already_aligned_length_advances_unchanged(self): + """Control: an 8-aligned dwLength must not be padded further.""" + first = _entry(b"\xAA" * 8) # dwLength 16 + second = _entry(b"\xBB" * 4) + data = bytes(0x800) + first + second + out = _read_certificates(data, 0x800, len(first) + len(second), + len(data), [], []) + assert out[1]["offset"] == 0x800 + 16 + + def test_three_unaligned_entries_walk_correctly(self): + """ + Cumulative alignment: a single mis-advance desynchronises every + subsequent entry, so a chain catches an off-by-one the pair misses. + """ + blobs = [b"\xAA" * 5, b"\xBB" * 13, b"\xCC" * 1] + table = b"".join(_entry(b) for b in blobs) + data = bytes(0x800) + table + out = _read_certificates(data, 0x800, len(table), len(data), [], []) + assert [c["length"] for c in out] == [13, 21, 9] + assert [c["offset"] for c in out] == [0x800, 0x810, 0x828] + + +# ================================================================= +# PARSER - non-advancing cursor +# ================================================================= + +class TestNonAdvancingLength: + """ + A dwLength below the 8-byte header cannot advance the cursor. The walk + must record the malformed entry and stop; the existing suite tests only + dwLength == 4. + """ + + @pytest.mark.parametrize("dw_length", [0, 1, 7]) + def test_walk_stops_and_tags(self, dw_length): + data = bytes(0x800) + _header(dw_length) + bytes(0x40) + out = _read_certificates(data, 0x800, 0x40, len(data), [], []) + assert len(out) == 1 + assert out[0]["errors"] == ["length_too_small"] + assert out[0]["data_length"] == 0 + + def test_zero_length_does_not_hang(self): + """ + dwLength == 0 is the classic infinite-loop shape: advance by zero, + read the same header forever. The `< header size` break is what + prevents it. + """ + data = bytes(0x800) + _header(0) * 4 + bytes(0x40) + out = _read_certificates(data, 0x800, 0x60, len(data), [], []) + assert len(out) == 1 + + def test_minimum_valid_length_continues(self): + """Control: dwLength exactly 8 is valid and the walk proceeds.""" + data = bytes(0x800) + _header(8) + _entry(b"\xBB" * 4) + out = _read_certificates(data, 0x800, 0x20, len(data), [], []) + assert len(out) == 2 + assert out[0]["errors"] == [] + + +# ================================================================= +# PARSER - offset / file-size boundary +# ================================================================= + +class TestOffsetBoundary: + """ + The guard is `base_offset > file_size`, not `>=`. An offset exactly at + EOF is therefore not tagged past-EOF; the walk simply finds nothing. + """ + + def test_offset_past_eof_tagged(self): + errors: List[str] = [] + out = _read_certificates(bytes(0x100), 0x101, 0x40, 0x100, [], errors) + assert out == [] + assert errors == ["certificate_offset_past_eof"] + + def test_offset_exactly_at_eof_is_not_tagged(self): + """ + Boundary: offset == file_size yields an empty walk rather than a + past-EOF error. Pinned so a change from `>` to `>=` is deliberate. + """ + errors: List[str] = [] + truncations: List[str] = [] + out = _read_certificates(bytes(0x100), 0x100, 0x40, 0x100, + truncations, errors) + assert out == [] + assert errors == [] + assert truncations == ["certificate_table_truncated"] + + +# ================================================================= +# PARSER - blob clamp +# ================================================================= + +class TestBlobTruncationClamp: + + def test_data_length_clamped_to_available(self): + truncations: List[str] = [] + cert = _decode_certificate(0, 0x100, 0x0200, 0x0002, + entry_offset=0x800, dir_end=0x818, + truncations=truncations) + assert truncations == ["certificate_blob_truncated"] + assert cert["data_length"] == 0x818 - (0x800 + _WIN_CERT_HEADER_SIZE) + + def test_negative_available_clamps_to_zero(self): + """ + When dir_end falls before the end of the header itself, `available` + is negative and must clamp to 0 rather than producing a negative + data_length in output. + """ + truncations: List[str] = [] + cert = _decode_certificate(0, 0x100, 0x0200, 0x0002, + entry_offset=0x800, dir_end=0x804, + truncations=truncations) + assert cert["data_length"] == 0 + assert truncations == ["certificate_blob_truncated"] + + +# ================================================================= +# PARSER - overlaps_image semantics +# ================================================================= + +class TestOverlapsImageSemantics: + """ + overlaps_image is False both when the table genuinely sits after the + image AND when image_raw_end could not be computed. The validator's + `is True` test treats them alike, so False means "not known to overlap", + not "verified outside the image". + """ + + def test_false_when_genuinely_outside(self): + cert = _entry(b"\x00" * 8) + pe = _FakePE(bytes(0x800) + cert, _DataDir(0x800, len(cert)), + sections=[_Section(0x200, 0x600)]) # raw end 0x800 + out = build_certificate_structure(pe) + assert out["image_raw_end"] == 0x800 + assert out["overlaps_image"] is False + + def test_false_when_image_raw_end_unknown(self): + """ + Same value, different meaning: no sections means the invariant could + not be evaluated. Pinned so the ambiguity is a recorded decision + rather than an accident. + """ + cert = _entry(b"\x00" * 8) + pe = _FakePE(bytes(0x800) + cert, _DataDir(0x800, len(cert)), + sections=[]) + out = build_certificate_structure(pe) + assert out["image_raw_end"] is None + assert out["overlaps_image"] is False + + def test_true_when_before_image_end(self): + cert = _entry(b"\x00" * 8) + data = bytearray(0x1000) + data[0x400:0x400 + len(cert)] = cert + pe = _FakePE(bytes(data), _DataDir(0x400, len(cert)), + sections=[_Section(0x200, 0x600)]) + out = build_certificate_structure(pe) + assert out["overlaps_image"] is True + + +# ================================================================= +# VALIDATOR - field value sets +# ================================================================= + +def _cert(offset=0x800, length=0x40, revision=0x0200, cert_type=0x0002): + return {"offset": offset, "length": length, "revision": revision, + "cert_type": cert_type, "errors": []} + + +def _struct(certs, **kw): + d = {"offset": 0x800, "size": 0x200, "file_size": None, + "image_raw_end": None, "overlaps_image": False, + "certificates": certs, "certificate_count": len(certs), + "truncations": [], "errors": []} + d.update(kw) + return d + + +def _run(cert_struct, has_signature=True, analysis=None): + return validate_signature({"certificate_struct": cert_struct}, + {"has_signature": has_signature}, + analysis if analysis is not None else {}) + + +def _codes(issues): + return [i["issue"] for i in issues] + + +class TestFieldValueSets: + """ + The parser and validator use DIFFERENT accepted-value sets for + cert_type, deliberately: the parser NAMES 0x0003/0x0004 (so output shows + RESERVED_1 / TS_STACK_SIGNED) while the validator REJECTS them, because + Authenticode uses only X509 and PKCS_SIGNED_DATA in practice. + + A future edit aligning the two sets would silently stop flagging + reserved types, so both halves are pinned here. + """ + + @pytest.mark.parametrize("revision", [0x0100, 0x0200]) + def test_both_valid_revisions_accepted(self, revision): + issues = _run(_struct([_cert(revision=revision)]), + analysis={"file_size": 0x10000}) + assert ReasonCodes.SIGNATURE_INVALID_REVISION not in _codes(issues) + + @pytest.mark.parametrize("cert_type", [0x0001, 0x0002]) + def test_both_valid_cert_types_accepted(self, cert_type): + issues = _run(_struct([_cert(cert_type=cert_type)]), + analysis={"file_size": 0x10000}) + assert ReasonCodes.SIGNATURE_INVALID_TYPE not in _codes(issues) + + @pytest.mark.parametrize("cert_type", [0x0003, 0x0004]) + def test_parser_named_but_validator_rejected_types(self, cert_type): + """ + 0x0003 and 0x0004 appear in the parser's _CERT_TYPE_NAMES but are + not accepted by the validator. The divergence is intentional. + """ + issues = _run(_struct([_cert(cert_type=cert_type)]), + analysis={"file_size": 0x10000}) + assert ReasonCodes.SIGNATURE_INVALID_TYPE in _codes(issues) + + +# ================================================================= +# VALIDATOR - suppression and break behaviour +# ================================================================= + +class TestSuppressionBehaviour: + """ + Two deliberate narrowings that read as under-reporting. Both are pinned + so a change is a decision rather than a drift. + """ + + def test_out_of_bounds_suppresses_overlay_and_section_checks(self): + """ + The bounds check `continue`s, so a certificate that is out of bounds + AND overlaps both the overlay and a section reports once. + """ + issues = _run( + _struct([_cert(offset=900, length=200)]), + analysis={"file_size": 1000, "overlay_offset": 950, + "sections": [{"name": ".text", "raw_address": 900, + "raw_size": 100}]}) + assert _codes(issues) == [ReasonCodes.SIGNATURE_OUT_OF_FILE_BOUNDS] + + def test_section_overlap_reports_only_the_first_section(self): + """The section loop breaks on the first hit, so a certificate + spanning several sections names one.""" + issues = _run( + _struct([_cert(offset=100, length=200)]), + analysis={"file_size": 1000, + "sections": [{"name": ".a", "raw_address": 150, "raw_size": 50}, + {"name": ".b", "raw_address": 250, "raw_size": 50}]}) + overlaps = [i["details"]["section"] for i in issues + if i["issue"] == ReasonCodes.SIGNATURE_OVERLAPS_OTHER_DATA] + assert overlaps == [".a"] + + def test_overlay_and_section_can_both_fire(self): + """ + Control for the first test: without the bounds failure, the overlay + and section checks are independent and both report. + """ + issues = _run( + _struct([_cert(offset=100, length=200)]), + analysis={"file_size": 1000, "overlay_offset": 150, + "sections": [{"name": ".a", "raw_address": 150, + "raw_size": 50}]}) + assert _codes(issues).count( + ReasonCodes.SIGNATURE_OVERLAPS_OTHER_DATA) == 2 + + +class TestOverlayBoundary: + """`offset < overlay < offset + length` excludes both endpoints.""" + + @pytest.mark.parametrize("overlay,fires", [ + (100, False), # == offset + (101, True), # just inside + (299, True), # just inside + (300, False), # == offset + length + ]) + def test_endpoints_excluded(self, overlay, fires): + issues = _run(_struct([_cert(offset=100, length=200)]), + analysis={"file_size": 1000, "overlay_offset": overlay}) + assert (ReasonCodes.SIGNATURE_OVERLAPS_OTHER_DATA in _codes(issues)) is fires + + +class TestFileSizeFallback: + """ + file_size prefers the analysis layer and falls back to the parser's + value. The fallback branch is otherwise unexercised. + """ + + def test_falls_back_to_parser_file_size(self): + issues = _run(_struct([_cert(offset=900, length=200)], file_size=1000), + analysis={}) + assert ReasonCodes.SIGNATURE_OUT_OF_FILE_BOUNDS in _codes(issues) + + def test_analysis_value_takes_precedence(self): + """Parser says 1000 (would fail), analysis says 2000 (passes).""" + issues = _run(_struct([_cert(offset=900, length=200)], file_size=1000), + analysis={"file_size": 2000}) + assert ReasonCodes.SIGNATURE_OUT_OF_FILE_BOUNDS not in _codes(issues) + + def test_no_file_size_anywhere_skips_bounds_check(self): + issues = _run(_struct([_cert(offset=900, length=200)]), analysis={}) + assert ReasonCodes.SIGNATURE_OUT_OF_FILE_BOUNDS not in _codes(issues) diff --git a/tests/unit/parsers/test_pe_delay_imports.py b/tests/unit/parsers/test_pe_delay_imports.py index cae4df6..b0d1ebb 100644 --- a/tests/unit/parsers/test_pe_delay_imports.py +++ b/tests/unit/parsers/test_pe_delay_imports.py @@ -334,7 +334,7 @@ def fake_unpack_from(fmt, buf, offset=0): # Defensive path: error appended to errors[], walk broken assert any( - e.startswith("descriptor_unpack_failed_at_") for e in result["errors"] + e.startswith("descriptor_unpack_failed") for e in result["errors"] ) # Walk should have broken before producing any descriptors assert result["descriptors"] == [] @@ -788,6 +788,54 @@ def fake_read_import_by_name(pe, rva): assert "name_read_failed" in entry["errors"] +class TestNameValidation: + + def test_long_printable_import_name_is_valid(self): + """A 600-char printable symbol (mangled C++) must NOT be flagged.""" + pe = _FakePE(delay_rva=0x1000, delay_size=64, data_by_rva={ + 0x1000: _build_descriptor() + _zero_descriptor(), + 0x2000: _asciiz("test.dll"), + 0x4000: _build_int_iat_array_64([0x6000]), + 0x5000: _build_int_iat_array_64([0x6000]), + 0x6000: _build_import_by_name(0x10, "A" * 600)}) + entry = build_delay_import_structure(pe)["descriptors"][0]["imports"][0] + assert entry["name_valid"] is True + assert entry["errors"] == [] + + def test_empty_import_name_flagged(self): + pe = _FakePE(delay_rva=0x1000, delay_size=64, data_by_rva={ + 0x1000: _build_descriptor() + _zero_descriptor(), + 0x2000: _asciiz("test.dll"), + 0x4000: _build_int_iat_array_64([0x6000]), + 0x5000: _build_int_iat_array_64([0x6000]), + 0x6000: struct.pack(" validator: "format" + assert entry["forwarder_valid"] is False + + @pytest.mark.parametrize("blob", [None, b"A" * 2000, b""]) + def test_unreadable_forwarder_yields_none(self, blob): + """The three failures that DO return None -> validator: "unreadable".""" + entry = build_export_structure( + _pe_with_forwarder(blob))["functions"][0] + assert entry["forwarder"] is None + + def test_non_forwarder_entries_have_empty_errors(self): + """errors[] must be present and empty on ordinary entries.""" + pe = _FakePEForwader(0x1000, 200, { + 0x1000: _hdr(nf=2, af=0x1100), + 0x1100: _dw([0x9000, 0]), # normal RVA, then unused slot + }) + for entry in build_export_structure(pe)["functions"]: + assert entry["errors"] == [] + assert entry["is_forwarder"] is False + + +# ================================================================= +# (v0.7.6.2) FunctionEntry.name_rva +# ================================================================= + +class TestFunctionNameRva: + """ + name_rva was documented but hard-coded to None. It now carries the RVA + of the resolved name string, sourced from the name-pointer cross-reference. + """ + + def test_named_function_carries_name_rva(self): + pe = _pe_with_names([("Foo", 0)], num_funcs=1) + entry = build_export_structure(pe)["functions"][0] + assert entry["name"] == "Foo" + assert entry["name_rva"] == 0x1400 + + def test_unnamed_function_has_none(self): + pe = _FakePEForwader(0x1000, 200, { + 0x1000: _hdr(nf=1, af=0x1100), 0x1100: _dw([0x2000])}) + entry = build_export_structure(pe)["functions"][0] + assert entry["name"] is None + assert entry["name_rva"] is None + + def test_name_rva_matches_the_name_pointer_entry(self): + """ + The function's name_rva must equal the name_rva recorded on the + name-pointer entry that resolved it - they are the same string. + """ + pe = _pe_with_names([("Alpha", 0), ("Beta", 1)], num_funcs=2) + out = build_export_structure(pe) + by_index = {np["ordinal_index"]: np["name_rva"] + for np in out["name_pointers"]} + for fn in out["functions"]: + if fn["name"] is not None: + assert fn["name_rva"] == by_index[fn["index"]] + + def test_name_rva_only_set_when_name_resolves(self): + """An unreadable name leaves both name and name_rva unset.""" + pe = _FakePEForwader(0x1000, 200, { + 0x1000: _hdr(nf=1, nn=1, af=0x1100, an=0x1200, ano=0x1300), + 0x1100: _dw([0x2000]), 0x1200: _dw([0x1400]), + 0x1300: _wd([0]), + 0x1400: b"A" * 2000, # unterminated + }) + entry = build_export_structure(pe)["functions"][0] + assert entry["name"] is None + assert entry["name_rva"] is None + + +# ================================================================= +# (v0.7.6.2) ordinal_index_duplicate +# ================================================================= + +class TestOrdinalIndexDuplicate: + """ + Two name pointers resolving to the same EAT index silently lost one name + from the function view. The collision is now tagged on the entry that + overwrites. + """ + + def test_duplicate_is_tagged_on_the_colliding_entry(self): + pe = _pe_with_names([("First", 0), ("Second", 0)], num_funcs=2) + out = build_export_structure(pe) + assert out["name_pointers"][0]["errors"] == [] + assert out["name_pointers"][1]["errors"] == ["ordinal_index_duplicate"] + + def test_last_write_wins_in_the_function_view(self): + """ + Documents the consequence: the FIRST name becomes unreachable through + the resolved-function list. This is what makes the tag worth emitting. + """ + pe = _pe_with_names([("First", 0), ("Second", 0)], num_funcs=2) + out = build_export_structure(pe) + assert out["functions"][0]["name"] == "Second" + assert out["functions"][1]["name"] is None + + def test_three_way_collision_tags_each_subsequent_entry(self): + pe = _pe_with_names([("A", 0), ("B", 0), ("C", 0)], num_funcs=2) + out = build_export_structure(pe) + errs = [np["errors"] for np in out["name_pointers"]] + assert errs == [[], ["ordinal_index_duplicate"], + ["ordinal_index_duplicate"]] + assert out["functions"][0]["name"] == "C" + + def test_distinct_indices_are_not_flagged(self): + pe = _pe_with_names([("Alpha", 0), ("Beta", 1)], num_funcs=2) + out = build_export_structure(pe) + assert all(np["errors"] == [] for np in out["name_pointers"]) + assert [f["name"] for f in out["functions"]] == ["Alpha", "Beta"] + + def test_invalid_name_does_not_collide_or_overwrite(self): + """ + An entry whose name failed validation never reaches the collision + check (the elif chain short-circuits), so it neither raises + ordinal_index_duplicate nor displaces the valid name already recorded + against that index. + """ + pe = _pe_with_names([("Good", 0), (b"Bad\x01\x00", 0)], num_funcs=2) + out = build_export_structure(pe) + assert out["name_pointers"][1]["errors"] == ["name_not_printable_ascii"] + assert "ordinal_index_duplicate" not in out["name_pointers"][1]["errors"] + assert out["functions"][0]["name"] == "Good" # not overwritten + + def test_out_of_range_index_does_not_collide(self): + """ + An out-of-range ordinal index is caught by the earlier elif, so it can + never reach the duplicate branch even if two entries share the value. + """ + pe = _pe_with_names([("A", 5), ("B", 5)], num_funcs=2) + out = build_export_structure(pe) + for np in out["name_pointers"]: + assert np["errors"] == ["ordinal_index_out_of_range"] + + +# ================================================================= +# (v0.7.6.2) Forwarder regex +# ================================================================= + +class TestForwarderRegex: + """ + The symbol branch excludes a leading '#' so an over-long ordinal cannot + fall through and be accepted as an ordinary symbol name. Tightening the + digit count alone was insufficient - '#' is a printable character, so + "Dll.#99999999999" matched the symbol alternative. + """ + + @pytest.mark.parametrize("s", [ + "KERNEL32.LoadLibraryA", + "KERNEL32.#42", + "KERNEL32.#0", + "KERNEL32.#65535", # max 16-bit ordinal + "A.B", + "api-ms-win-core.dll.Func", # multiple dots + ]) + def test_valid_forwarders(self, s): + assert _FORWARDER_RE.match(s) is not None + + @pytest.mark.parametrize("s", [ + "NoDotInThisString", + ".LeadingDot", # empty DLL part + "Dll.", # empty symbol part + "KERNEL32.#123456", # 6 digits - exceeds u16 + "KERNEL32.#99999999999", # the case the first fix missed + ]) + def test_invalid_forwarders(self, s): + assert _FORWARDER_RE.match(s) is None + + def test_over_long_ordinal_not_accepted_as_symbol_name(self): + """ + Direct regression guard: '#' must not be a valid first character of + the symbol branch, or the ordinal digit cap is unenforceable. + """ + assert _FORWARDER_RE.match("Dll.#999999") is None + assert _FORWARDER_RE.match("Dll.$999999") is not None # other punct ok + + # ================================================================= # Output contract # ================================================================= @@ -812,6 +1083,32 @@ def test_lists_are_lists_even_when_empty(self): assert isinstance(result["errors"], list) +class TestFunctionEntryContract: + + def test_function_entry_key_set(self): + pe = _pe_with_names([("Foo", 0)], num_funcs=1) + entry = build_export_structure(pe)["functions"][0] + assert set(entry) == { + "index", "ordinal", "address_rva", "is_forwarder", "forwarder", + "forwarder_valid", "name", "name_rva", "errors", + } + + def test_errors_key_present_on_every_entry(self): + pe = _FakePEForwader(0x1000, 200, { + 0x1000: _hdr(nf=3, af=0x1100), + 0x1100: _dw([0x2000, 0, 0x1050]), + 0x1050: _az("K.F"), + }) + for entry in build_export_structure(pe)["functions"]: + assert "errors" in entry + assert isinstance(entry["errors"], list) + + def test_json_serialisable_with_new_fields(self): + import json + pe = _pe_with_names([("Foo", 0), ("Foo", 0)], num_funcs=2) + json.dumps(build_export_structure(pe)) + + # ================================================================= # Determinism # ================================================================= diff --git a/tests/unit/parsers/test_pe_imports.py b/tests/unit/parsers/test_pe_imports.py new file mode 100644 index 0000000..ab5913d --- /dev/null +++ b/tests/unit/parsers/test_pe_imports.py @@ -0,0 +1,822 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 + +""" +Unit tests for iocx.parsers.pe_imports. + +Strategy: +- Byte-level builders construct 20-byte IMAGE_IMPORT_DESCRIPTOR structures, + NULL-terminated thunk arrays, and IMAGE_IMPORT_BY_NAME blobs via struct.pack. +- A fake PE exposes OPTIONAL_HEADER.Magic / DATA_DIRECTORY[1] and get_data(), + with a base-RVA lookup so thunk walks (which increment the RVA one element + at a time) resolve correctly, and a raise_at hook for read failures. + +The two divergences from pe_delay_imports carry their own test classes, +because both are cases where a plausible implementation misreports rather +than fails loudly: + + * OriginalFirstThunk == 0 is LEGAL here (delay-load treats a zero INT as + an anomaly). Getting this wrong flags a large fraction of legitimate + binaries, so TestOriginalFirstThunkFallback asserts the absence of a tag + as well as the successful fallback. + + * TimeDateStamp selects how FirstThunk should be read. Only the old-style + bound case makes the fallback unusable. +""" + +from __future__ import annotations + +import struct +from typing import Any, Dict, List, Optional + +import pytest + +from iocx.parsers.pe_imports import ( + build_import_structure, + _decode_descriptor, + _decode_import_entry, + _is_pe32_plus, + _is_zero_descriptor, + _locate_import_directory, + _read_asciiz, + _read_import_by_name, + _read_thunk_array, + _BOUND_NEW_STYLE, + _DESCRIPTOR_SIZE, + _DLL_NAME_MAX_CHARS, + _IMPORT_DIRECTORY_INDEX, + _MAGIC_PE32, + _MAGIC_PE32_PLUS, + _MAX_IMPORTS_PER_DESCRIPTOR, + _MAX_DESCRIPTORS, +) + + +# ================================================================= +# Byte-level builders +# ================================================================= + +def _descriptor(original_first_thunk: int = 0x3000, + timestamp: int = 0, + forwarder_chain: int = 0, + name_rva: int = 0x2000, + first_thunk: int = 0x4000) -> bytes: + """A 20-byte IMAGE_IMPORT_DESCRIPTOR.""" + return struct.pack(" bytes: + """ + The cheapest descriptor that is non-zero yet triggers no further reads: + name_rva = 0 -> dll_name_rva_zero, no string read + both thunk RVAs = 0 -> no_thunk_array, _enrich returns immediately + timestamp = 1 -> makes the descriptor non-zero + Keeps a 4096-iteration walk to one get_data call per descriptor. + """ + return _descriptor(original_first_thunk=0, timestamp=1, + forwarder_chain=0, name_rva=0, first_thunk=0) + + +def _zero_descriptor() -> bytes: + return b"\x00" * _DESCRIPTOR_SIZE + + +def _thunks64(values: List[int]) -> bytes: + return b"".join(struct.pack(" bytes: + return b"".join(struct.pack(" bytes: + return struct.pack(" bytes: + return s.encode("ascii") + b"\x00" + + +def _ordinal64(o: int) -> int: + return (1 << 63) | o + + +def _ordinal32(o: int) -> int: + return 0x80000000 | o + + +# ================================================================= +# Fake PE +# ================================================================= + +class _DataDir: + def __init__(self, rva: int, size: int): + self.VirtualAddress = rva + self.Size = size + + +class _OptHdr: + def __init__(self, d: Optional[_DataDir], magic: int = _MAGIC_PE32_PLUS, + has_magic: bool = True): + self.DATA_DIRECTORY = [None] * 16 + if d is not None: + self.DATA_DIRECTORY[_IMPORT_DIRECTORY_INDEX] = d + if has_magic: + self.Magic = magic + + +class _FakePE: + """ + get_data resolves an RVA against the nearest preceding buffer base, so a + thunk walk stepping 8 bytes at a time reads successive elements of one + stored array. A read past a buffer's end returns a short slice (as pefile + does at a section edge); an unmapped RVA raises. + """ + def __init__(self, import_rva: int = 0, import_size: int = 0, + data: Optional[Dict[int, bytes]] = None, + is_64bit: bool = True, + raise_at: Optional[int] = None, + has_optional_header: bool = True, + has_magic: bool = True): + magic = _MAGIC_PE32_PLUS if is_64bit else _MAGIC_PE32 + if has_optional_header: + dd = _DataDir(import_rva, import_size) if (import_rva or import_size) else None + self.OPTIONAL_HEADER = _OptHdr(dd, magic, has_magic) + self._data = data or {} + self._raise_at = raise_at + + def get_data(self, rva: int, size: int) -> bytes: + if self._raise_at is not None and rva == self._raise_at: + raise RuntimeError("simulated read failure") + for base in sorted((b for b in self._data if b <= rva), reverse=True): + buf = self._data[base] + offset = rva - base + if offset <= len(buf): + return buf[offset:offset + size] + raise ValueError(f"no fixture data covers rva {rva:#x}") + + +def _pe(descriptors: bytes, extra: Optional[Dict[int, bytes]] = None, + size: Optional[int] = None, is_64bit: bool = True, + raise_at: Optional[int] = None) -> _FakePE: + data = {0x1000: descriptors} + data.update(extra or {}) + return _FakePE(0x1000, size if size is not None else len(descriptors), + data, is_64bit=is_64bit, raise_at=raise_at) + + +def _simple(**kw) -> _FakePE: + """One well-formed descriptor: KERNEL32.dll, one named + one ordinal.""" + return _pe(_descriptor(**kw) + _zero_descriptor(), { + 0x2000: _asciiz("KERNEL32.dll"), + 0x3000: _thunks64([0x5000, _ordinal64(42)]), + 0x4000: _thunks64([0x5000, _ordinal64(42)]), + 0x5000: _import_by_name(0x10, "LoadLibraryA"), + }, size=60) + + +# ================================================================= +# Absence / locator +# ================================================================= + +class TestLocator: + + def test_absent_directory_returns_none(self): + assert build_import_structure(_FakePE(0, 0)) is None + + def test_zero_rva_returns_none(self): + assert _locate_import_directory(_FakePE(0, 100)) is None + + def test_zero_size_returns_none(self): + assert _locate_import_directory(_FakePE(0x1000, 0)) is None + + def test_missing_optional_header_returns_none(self): + assert _locate_import_directory( + _FakePE(has_optional_header=False)) is None + + def test_missing_entry_returns_none(self): + pe = _FakePE(0x1000, 100) + pe.OPTIONAL_HEADER.DATA_DIRECTORY[_IMPORT_DIRECTORY_INDEX] = None + assert _locate_import_directory(pe) is None + + def test_non_int_fields_return_none(self): + pe = _FakePE(0x1000, 100) + pe.OPTIONAL_HEADER.DATA_DIRECTORY[ + _IMPORT_DIRECTORY_INDEX].VirtualAddress = "nope" + assert _locate_import_directory(pe) is None + + +class TestPe32Plus: + + def test_pe32_plus_true(self): + assert _is_pe32_plus(_FakePE(is_64bit=True)) is True + + def test_pe32_false(self): + assert _is_pe32_plus(_FakePE(is_64bit=False)) is False + + def test_missing_magic_defaults_false(self): + """Conservative default: narrower thunks, lower over-read risk.""" + assert _is_pe32_plus(_FakePE(has_magic=False)) is False + + def test_missing_optional_header_defaults_false(self): + assert _is_pe32_plus(_FakePE(has_optional_header=False)) is False + + +# ================================================================= +# Descriptor decode +# ================================================================= + +class TestDescriptorDecode: + + def test_fields_decoded(self): + raw = _descriptor(original_first_thunk=0x1111, timestamp=0x2222, + forwarder_chain=0x3333, name_rva=0x4444, + first_thunk=0x5555) + d = _decode_descriptor(raw, 7) + assert d["index"] == 7 + assert d["original_first_thunk"] == 0x1111 + assert d["timestamp"] == 0x2222 + assert d["forwarder_chain"] == 0x3333 + assert d["name_rva"] == 0x4444 + assert d["first_thunk"] == 0x5555 + + def test_zero_descriptor_recognised(self): + assert _is_zero_descriptor(_decode_descriptor(_zero_descriptor(), 0)) is True + + def test_non_zero_descriptor_not_terminator(self): + assert _is_zero_descriptor(_decode_descriptor(_descriptor(), 0)) is False + + def test_terminator_is_not_emitted_as_an_entry(self): + out = build_import_structure(_simple()) + assert out["descriptor_count"] == 1 + + def test_multiple_descriptors_walked_in_order(self): + table = (_descriptor(original_first_thunk=0x3000, name_rva=0x2000) + + _descriptor(original_first_thunk=0, first_thunk=0x4000, + name_rva=0x2010) + + _zero_descriptor()) + out = build_import_structure(_pe(table, { + 0x2000: _asciiz("A.dll"), 0x2010: _asciiz("B.dll"), + 0x3000: _thunks64([0x5000]), 0x4000: _thunks64([_ordinal64(9)]), + 0x5000: _import_by_name(1, "Fn"), + }, size=80)) + assert [d["index"] for d in out["descriptors"]] == [0, 1] + assert [d["dll_name"] for d in out["descriptors"]] == ["A.dll", "B.dll"] + + +# ================================================================= +# DIVERGENCE 1 - OriginalFirstThunk == 0 is legal +# ================================================================= + +class TestOriginalFirstThunkFallback: + """ + Unlike the delay-load INT, a zero OriginalFirstThunk is legal and common: + older linkers emit only FirstThunk, which then holds INT-style thunks on + disk. Treating it as an anomaly would misreport a large fraction of + legitimate binaries, so the absence of an error tag is asserted as + explicitly as the successful fallback. + """ + + def test_fallback_resolves_names_from_first_thunk(self): + out = build_import_structure(_pe( + _descriptor(original_first_thunk=0, first_thunk=0x4000) + + _zero_descriptor(), { + 0x2000: _asciiz("OLD.dll"), + 0x4000: _thunks64([0x5000]), + 0x5000: _import_by_name(1, "Foo"), + }, size=60)) + d = out["descriptors"][0] + assert d["thunk_source"] == "iat_fallback" + assert [e["name"] for e in d["imports"]] == ["Foo"] + + def test_fallback_is_not_flagged_as_an_error(self): + """The regression guard: a zero INT must NOT produce a tag.""" + out = build_import_structure(_pe( + _descriptor(original_first_thunk=0, first_thunk=0x4000) + + _zero_descriptor(), { + 0x2000: _asciiz("OLD.dll"), + 0x4000: _thunks64([_ordinal64(1)]), + }, size=60)) + assert out["descriptors"][0]["errors"] == [] + + def test_int_preferred_when_both_present(self): + """ + With both arrays populated the INT wins - it is the on-disk name + source, and the IAT may already hold bound addresses. + """ + out = build_import_structure(_pe( + _descriptor(original_first_thunk=0x3000, first_thunk=0x4000) + + _zero_descriptor(), { + 0x2000: _asciiz("A.dll"), + 0x3000: _thunks64([0x5000]), + 0x4000: _thunks64([0x6000]), + 0x5000: _import_by_name(1, "FromINT"), + 0x6000: _import_by_name(2, "FromIAT"), + }, size=60)) + d = out["descriptors"][0] + assert d["thunk_source"] == "int" + assert d["imports"][0]["name"] == "FromINT" + + def test_neither_array_present_is_flagged(self): + out = build_import_structure(_pe( + _descriptor(original_first_thunk=0, first_thunk=0) + + _zero_descriptor(), {0x2000: _asciiz("X.dll")}, size=60)) + d = out["descriptors"][0] + assert d["errors"] == ["no_thunk_array"] + assert d["thunk_source"] is None + assert d["imports"] == [] + + def test_truncation_tag_names_the_array_actually_read(self): + """ + The tag prefix follows thunk_source, so a consumer can tell whether + the INT or the fallback IAT was short. + """ + int_short = build_import_structure(_pe( + _descriptor() + _zero_descriptor(), + {0x2000: _asciiz("A.dll"), 0x3000: b"\x01\x02\x03"}, size=60)) + fallback_short = build_import_structure(_pe( + _descriptor(original_first_thunk=0) + _zero_descriptor(), + {0x2000: _asciiz("A.dll"), 0x4000: b"\x01\x02\x03"}, size=60)) + assert int_short["truncations"] == ["int_truncated"] + assert fallback_short["truncations"] == ["iat_fallback_truncated"] + + +# ================================================================= +# DIVERGENCE 2 - TimeDateStamp selects how FirstThunk reads +# ================================================================= + +class TestBoundState: + + @pytest.mark.parametrize("timestamp,expected", [ + (0, "unbound"), + (_BOUND_NEW_STYLE, "bound_new_style"), + (0x5F000000, "bound_old_style"), + (1, "bound_old_style"), + ]) + def test_bound_state_classification(self, timestamp, expected): + out = build_import_structure(_pe( + _descriptor(timestamp=timestamp) + _zero_descriptor(), { + 0x2000: _asciiz("A.dll"), + 0x3000: _thunks64([_ordinal64(1)]), + }, size=60)) + assert out["descriptors"][0]["bound_state"] == expected + + def test_old_style_bound_without_int_cannot_recover_names(self): + """ + Old-style bound means FirstThunk holds resolved ADDRESSES on disk. + With no INT there is no name source at all - a structural fact, not a + parse failure, so no thunks are walked. + """ + out = build_import_structure(_pe( + _descriptor(original_first_thunk=0, timestamp=0x5F000000, + first_thunk=0x4000) + _zero_descriptor(), { + 0x2000: _asciiz("BOUND.dll"), + 0x4000: _thunks64([0x7FF800001234]), + }, size=60)) + d = out["descriptors"][0] + assert d["errors"] == ["names_unrecoverable_bound_no_int"] + assert d["thunk_source"] is None + assert d["imports"] == [] + + def test_old_style_bound_with_int_still_parses(self): + """The INT is unaffected by binding, so names remain readable.""" + out = build_import_structure(_pe( + _descriptor(original_first_thunk=0x3000, timestamp=0x5F000000) + + _zero_descriptor(), { + 0x2000: _asciiz("B.dll"), + 0x3000: _thunks64([0x5000]), + 0x5000: _import_by_name(1, "Bar"), + }, size=60)) + d = out["descriptors"][0] + assert d["errors"] == [] + assert d["imports"][0]["name"] == "Bar" + + def test_new_style_bound_without_int_uses_the_fallback(self): + """ + New-style bound keeps real timestamps in the BOUND_IMPORT directory, + so FirstThunk still holds thunks on disk and the fallback is valid. + """ + out = build_import_structure(_pe( + _descriptor(original_first_thunk=0, timestamp=_BOUND_NEW_STYLE, + first_thunk=0x4000) + _zero_descriptor(), { + 0x2000: _asciiz("NB.dll"), + 0x4000: _thunks64([0x5000]), + 0x5000: _import_by_name(1, "Baz"), + }, size=60)) + d = out["descriptors"][0] + assert d["errors"] == [] + assert d["thunk_source"] == "iat_fallback" + assert d["imports"][0]["name"] == "Baz" + + +# ================================================================= +# Thunk decode +# ================================================================= + +class TestThunkDecode: + + def test_named_import(self): + d = build_import_structure(_simple())["descriptors"][0] + e = d["imports"][0] + assert e["is_ordinal"] is False + assert e["hint"] == 0x10 + assert e["name"] == "LoadLibraryA" + assert e["name_rva"] == 0x5000 + assert e["name_valid"] is True + assert e["errors"] == [] + + def test_ordinal_import(self): + e = build_import_structure(_simple())["descriptors"][0]["imports"][1] + assert e["is_ordinal"] is True + assert e["ordinal"] == 42 + assert e["name"] is None + + def test_ordinal_masked_to_low_16_bits(self): + """Per spec the ordinal is bits 15-0; higher bits are discarded.""" + out = build_import_structure(_pe( + _descriptor() + _zero_descriptor(), { + 0x2000: _asciiz("A.dll"), + 0x3000: _thunks64([_ordinal64(42) | (1 << 40)]), + }, size=60)) + assert out["descriptors"][0]["imports"][0]["ordinal"] == 42 + + def test_ordinal_zero_flagged(self): + out = build_import_structure(_pe( + _descriptor() + _zero_descriptor(), { + 0x2000: _asciiz("A.dll"), + 0x3000: _thunks64([_ordinal64(0)]), + }, size=60)) + assert out["descriptors"][0]["imports"][0]["errors"] == ["ordinal_zero"] + + def test_pe32_uses_dword_thunks(self): + out = build_import_structure(_pe( + _descriptor() + _zero_descriptor(), { + 0x2000: _asciiz("K32.dll"), + 0x3000: _thunks32([0x5000, _ordinal32(7)]), + 0x5000: _import_by_name(2, "Baz"), + }, size=60, is_64bit=False)) + assert out["is_64bit"] is False + entries = out["descriptors"][0]["imports"] + assert entries[0]["name"] == "Baz" + assert entries[1]["ordinal"] == 7 + + @pytest.mark.parametrize("blob,tag", [ + (b"\x42", "name_too_short"), + (struct.pack(" reader err: None -> ['name_not_printable'] +Foo\xffBar -> reader err: name_non_ascii -> ['name_non_ascii'] + +_read_import_by_name decodes with "ascii", so a high byte raises UnicodeDecodeError and returns the name_non_ascii tag. The caller then takes the read_err is not None branch and never reaches the printability check. +""" + +from __future__ import annotations +import struct +from typing import Dict, List, Optional +import pytest + +from iocx.parsers.pe_imports import ( + build_import_structure, _decode_import_entry, _read_import_by_name, + _IMPORT_DIRECTORY_INDEX, _MAGIC_PE32_PLUS, +) + +_HIGH_BIT_64 = 1 << 63 + + +# ================================================================= +# Builders +# ================================================================= + +def _import_by_name(hint: int, name_bytes: bytes) -> bytes: + """IMAGE_IMPORT_BY_NAME with a raw byte name, so non-ASCII is expressible.""" + return struct.pack(" bytes: + return struct.pack(" bytes: + return b"".join(struct.pack(" Dict: + """Decode a single by-name import whose symbol is the given raw bytes.""" + pe = _FakePE({0x5000: _import_by_name(1, name_bytes)}) + return _decode_import_entry(pe, 0, 0x5000, _HIGH_BIT_64) + + +# ================================================================= +# The branch +# ================================================================= + +class TestNameNotPrintable: + """ + `name_not_printable` is reachable ONLY by control characters. + + A byte >= 0x80 never gets here: _read_import_by_name decodes with + "ascii", raises UnicodeDecodeError, and returns the tag `name_non_ascii` + - so the caller takes the `read_err is not None` branch instead. A test + written with \\xff (the obvious "non-printable" choice) would therefore + exercise a completely different branch and pass while proving nothing + about this one. + """ + + @pytest.mark.parametrize("byte,label", [ + (0x01, "SOH"), + (0x09, "TAB"), + (0x0A, "LF"), + (0x0D, "CR"), + (0x1F, "US - last control char below the printable range"), + (0x7F, "DEL - just above the printable range"), + ]) + def test_control_characters_are_flagged(self, byte, label): + entry = _entry_for(b"Foo" + bytes([byte]) + b"Bar") + assert entry["errors"] == ["name_not_printable"], label + assert entry["name_valid"] is False + + def test_name_is_still_recorded(self): + """ + The decoded string is preserved rather than discarded - a consumer + needs it to see WHAT was malformed, and the control byte survives + the round trip intact. + """ + entry = _entry_for(b"Foo\x01Bar") + assert entry["name"] == "Foo\x01Bar" + assert entry["name_rva"] == 0x5000 + assert entry["hint"] == 1 + + def test_wholly_non_printable_name(self): + entry = _entry_for(b"\x01\x02\x03") + assert entry["errors"] == ["name_not_printable"] + + def test_leading_and_trailing_control_chars(self): + for raw in (b"\x01Foo", b"Foo\x01"): + assert _entry_for(raw)["errors"] == ["name_not_printable"] + + +class TestPrintableBoundary: + """ + The regex range is [\\x20-\\x7E]. These pin both edges, so widening or + narrowing it by one is caught. + """ + + @pytest.mark.parametrize("byte,label", [ + (0x20, "SPACE - first printable"), + (0x7E, "TILDE - last printable"), + ]) + def test_boundary_bytes_are_accepted(self, byte, label): + entry = _entry_for(b"Foo" + bytes([byte]) + b"Bar") + assert entry["errors"] == [], label + assert entry["name_valid"] is True + + @pytest.mark.parametrize("byte,label", [ + (0x1F, "one below SPACE"), + (0x7F, "one above TILDE"), + ]) + def test_bytes_just_outside_are_rejected(self, byte, label): + entry = _entry_for(b"Foo" + bytes([byte]) + b"Bar") + assert entry["errors"] == ["name_not_printable"], label + + +class TestNotConfusedWithNonAscii: + """ + The negative controls. Without these, a mutation collapsing + name_not_printable into name_non_ascii (or vice versa) would pass. + """ + + @pytest.mark.parametrize("byte", [0x80, 0xC3, 0xFF]) + def test_high_bytes_yield_non_ascii_not_not_printable(self, byte): + entry = _entry_for(b"Foo" + bytes([byte]) + b"Bar") + assert entry["errors"] == ["name_non_ascii"] + assert "name_not_printable" not in entry["errors"] + + def test_the_two_tags_come_from_different_layers(self): + """ + name_non_ascii originates in _read_import_by_name (a read fault); + name_not_printable in _decode_import_entry (a content check). The + first short-circuits the second. + """ + pe_high = _FakePE({0x5000: _import_by_name(1, b"Foo\xffBar")}) + _, _, err_high = _read_import_by_name(pe_high, 0x5000) + assert err_high == "name_non_ascii" + + pe_ctrl = _FakePE({0x5000: _import_by_name(1, b"Foo\x01Bar")}) + _, _, err_ctrl = _read_import_by_name(pe_ctrl, 0x5000) + assert err_ctrl is None # reader is satisfied + assert _entry_for(b"Foo\x01Bar")["errors"] == ["name_not_printable"] + + def test_empty_name_takes_its_own_branch(self): + """name_empty is checked before printability, so an empty string + never reports as non-printable.""" + entry = _entry_for(b"") + assert entry["errors"] == ["name_empty"] + + def test_ordinal_imports_never_reach_the_check(self): + pe = _FakePE({}) + entry = _decode_import_entry(pe, 0, _HIGH_BIT_64 | 42, _HIGH_BIT_64) + assert entry["errors"] == [] + assert entry["name"] is None + + +class TestEndToEnd: + """The branch reached through the full parse, not just the helper.""" + + def test_non_printable_import_name_surfaces_in_the_structure(self): + pe = _FakePE({ + 0x1000: _descriptor() + b"\x00" * 20, + 0x2000: b"KERNEL32.dll\x00", + 0x3000: _thunks64([0x5000]), + 0x5000: _import_by_name(0x10, b"Load\x01Library"), + }) + out = build_import_structure(pe) + entry = out["descriptors"][0]["imports"][0] + assert entry["errors"] == ["name_not_printable"] + assert entry["name"] == "Load\x01Library" + assert entry["name_valid"] is False + # The descriptor itself is otherwise clean - single anomaly. + assert out["descriptors"][0]["errors"] == [] + assert out["truncations"] == [] + + def test_mixed_valid_and_non_printable_entries(self): + pe = _FakePE({ + 0x1000: _descriptor() + b"\x00" * 20, + 0x2000: b"KERNEL32.dll\x00", + 0x3000: _thunks64([0x5000, 0x5020, 0x5040]), + 0x5000: _import_by_name(1, b"GoodName"), + 0x5020: _import_by_name(2, b"Bad\x01Name"), + 0x5040: _import_by_name(3, b"AlsoGood"), + }) + out = build_import_structure(pe) + imports = out["descriptors"][0]["imports"] + assert [e["errors"] for e in imports] == [[], ["name_not_printable"], []] + assert [e["name_valid"] for e in imports] == [True, False, True] diff --git a/tests/unit/parsers/test_pe_parser.py b/tests/unit/parsers/test_pe_parser.py index b6ab634..d81c85e 100644 --- a/tests/unit/parsers/test_pe_parser.py +++ b/tests/unit/parsers/test_pe_parser.py @@ -1,17 +1,104 @@ # Copyright (c) 2026 MalX Labs and contributors # SPDX-License-Identifier: MPL-2.0 -import pytest +import pytest, pefile, json from types import SimpleNamespace +from typing import Dict, Any, Optional, List -from iocx.parsers.pe_parser import parse_pe, _walk_resources, analyse_pe_sections, _parse_data_directories, _parse_data_directories_raw +from iocx.parsers.pe_parser import (parse_pe, _walk_resources, analyse_pe_sections, _parse_data_directories, + _parse_data_directories_raw, _parse_resources, _MAX_RESOURCE_STRINGS, + _MAX_RESOURCE_ENTRIES, _MAX_RESOURCE_DEPTH) from iocx.parsers.string_extractor import extract_strings_from_bytes +from iocx.parsers.pe_resources import build_resource_structure + + +class _DataStruct: + def __init__(self, offset, size, codepage=0): + self.OffsetToData = offset + self.Size = size + self.CodePage = codepage + + +class _Data: + def __init__(self, offset, size): + self.struct = _DataStruct(offset, size) + + +class _Node: + def __init__(self, entries): + self.entries = entries + + +class _Entry: + def __init__(self, **kw): + self.name = None + self.id = None + for k, v in kw.items(): + setattr(self, k, v) + + +class _FakePE: + def __init__(self, root, file_size=0x10000000): + self.DIRECTORY_ENTRY_RESOURCE = root + self.__data__ = type("D", (), {"size": file_size})() + + def get_data(self, rva, size): + return b"A" * size + + def get_memory_mapped_image(self): + return b"\x00" * 0x10000 + + def get_offset_from_rva(self, rva): + return rva + + + +def _leaf(offset=0x100, size=0x40): + e = _Entry(id=0x409) + e.data = _Data(offset, size) + return e + + +def _name_entry(langs, name_id=1): + e = _Entry(id=name_id) + e.directory = _Node(langs) + return e + + +def _type_entry(names, type_id=16): + e = _Entry(id=type_id) + e.directory = _Node(names) + return e + + +def _tree(name_entries, type_id=16): + return _Node([_type_entry(name_entries, type_id)]) # ------------------------------------------------------------ # Fake PE builder with full interface required by parse_pe() # ------------------------------------------------------------ +@pytest.fixture +def fake_strings(monkeypatch): + """ + Replace extract_strings_from_bytes with a fixed-rate fake: one + fabricated string per 4 input bytes, regardless of content. + + Patches the name as bound INSIDE iocx.parsers.pe_parser's own + namespace - `from .string_extractor import extract_strings_from_bytes` + binds a local reference there, so patching the origin module + (string_extractor) would not affect it. + """ + import iocx.parsers.pe_parser as pe_parser_module + + def fake(data: bytes): + return [f"S{i}" for i in range(len(data) // 4)] + + monkeypatch.setattr(pe_parser_module, "extract_strings_from_bytes", fake) + return fake + + def fake_pe( imports=None, sections=None, @@ -85,6 +172,28 @@ def fake_loader(path, fast_load=True): yield +# ------------------------------------------------------------ +# Helpers +# ------------------------------------------------------------ + +def _walk(node: Dict[str, Any], acc: Optional[List] = None) -> List[Dict[str, Any]]: + """Every directory node in the tree, root first.""" + acc = acc if acc is not None else [] + acc.append(node) + for e in node["entries"]: + if e["directory"] is not None: + _walk(e["directory"], acc) + return acc + + +def _data_leaves(root: Dict[str, Any]) -> List[Dict[str, Any]]: + return [e for d in _walk(root) for e in d["entries"] if not e["is_directory"]] + + +def _all_directory_errors(root: Dict[str, Any]) -> List: + return [tag for d in _walk(root) for tag in d["errors"]] + + # ------------------------------------------------------------ # Tests for parse_pe() using pure mocks # ------------------------------------------------------------ @@ -202,7 +311,6 @@ class FakeDir: def test_parse_pe_handles_peformaterror(monkeypatch): - import pefile # Override the autouse patch for this test only def raise_peformaterror(path, fast_load=True): raise pefile.PEFormatError("bad file") @@ -303,6 +411,219 @@ def size(self): assert strings == [] +class TestResourceStringsCap: + + # The cap is the unit under test, not the extractor. Make the fake + # unconditional for the class so no test can silently fall back to + # the real scanner. + @pytest.fixture(autouse=True) + def _always_fake(self, fake_strings): + return fake_strings + + def test_many_small_resources_bounded(self): + tree = _tree([_name_entry([_leaf(0x100, 4000)], name_id=i) + for i in range(500)]) + _, strings, truncated = _parse_resources(_FakePE(tree)) + assert len(strings) == _MAX_RESOURCE_STRINGS + assert "resource_strings" in truncated + + def test_single_large_resource_bounded(self): + tree = _tree([_name_entry([_leaf(0x100, 400_000)])]) + _, strings, truncated = _parse_resources(_FakePE(tree)) + assert len(strings) == _MAX_RESOURCE_STRINGS + assert "resource_strings" in truncated + + @pytest.mark.parametrize("blob_size", [4, 400, 40_000, 400_000]) + def test_never_exceeds_the_cap(self, blob_size): + tree = _tree([_name_entry([_leaf(0x100, blob_size)], name_id=i) + for i in range(200)]) + _, strings, _ = _parse_resources(_FakePE(tree)) + assert len(strings) <= _MAX_RESOURCE_STRINGS + + def test_under_cap_not_flagged(self): + tree = _tree([_name_entry([_leaf(0x100, 0x40)])]) + _, strings, truncated = _parse_resources(_FakePE(tree)) + assert len(strings) < _MAX_RESOURCE_STRINGS + assert "resource_strings" not in truncated + + def test_fake_pe_satisfies_the_parser_interface(self): + """_parse_resources returns early and silently when + get_memory_mapped_image is absent, so an incomplete fake yields + empty resources rather than an error.""" + pe = _FakePE(_tree([_name_entry([_leaf(0x100, 0x40)])])) + for attr in ("get_data", "get_memory_mapped_image", "get_offset_from_rva"): + assert callable(getattr(pe, attr, None)), attr + + +class TestResourceEntriesCap: + + @pytest.mark.parametrize("count,expected_capped", [ + (_MAX_RESOURCE_ENTRIES - 1, False), + (_MAX_RESOURCE_ENTRIES, False), + (_MAX_RESOURCE_ENTRIES + 1, True), + ]) + def test_entry_cap_boundary(self, count, expected_capped): + """Small blobs keep the string budget clear so the entry cap is + the only one under test.""" + tree = _tree([_name_entry([_leaf(0x100, 4)], name_id=i) + for i in range(count)]) + resources, _, truncated = _parse_resources(_FakePE(tree)) + assert len(resources) == min(count, _MAX_RESOURCE_ENTRIES) + assert ("resources" in truncated) is expected_capped + + def test_far_over_cap_bounded(self): + tree = _tree([_name_entry([_leaf(0x100, 4)], name_id=i) + for i in range(50_000)]) + resources, _, truncated = _parse_resources(_FakePE(tree)) + assert len(resources) == _MAX_RESOURCE_ENTRIES + assert "resources" in truncated + + def test_output_size_does_not_scale_with_input(self): + """The property that matters downstream.""" + def sized(n): + tree = _tree([_name_entry([_leaf(0x100, 4)], name_id=i) + for i in range(n)]) + return len(json.dumps(_parse_resources(_FakePE(tree))[0])) + assert sized(2_000) == sized(50_000) + assert sized(2_000) > 2 + + def test_cap_flag_appears_once(self): + tree = _tree([_name_entry([_leaf(0x100, 4)], name_id=i) + for i in range(5_000)]) + _, _, truncated = _parse_resources(_FakePE(tree)) + assert truncated.count("resources") == 1 + + def test_outer_loop_breaks_after_cap(self): + """ + The cap fires inside the first type's name loop, which breaks the + inner loop only. The outer `if entries_capped: break` is what stops + the walk from advancing to the next type entry. + + Without it the second type is still blocked by the inner + `len(resources) >= _MAX_RESOURCE_ENTRIES` check - but that check + appends "resources" again, so the flag is duplicated. + """ + over_cap = [_name_entry([_leaf(0x100, 4)], name_id=i) + for i in range(_MAX_RESOURCE_ENTRIES + 1)] + root = _Node([ + _type_entry(over_cap, type_id=16), + _type_entry([_name_entry([_leaf(0x100, 4)], name_id=9000)], + type_id=3), + ]) + + resources, _, truncated = _parse_resources(_FakePE(root)) + + assert len(resources) == _MAX_RESOURCE_ENTRIES + assert truncated == ["resources"] + assert all(r["type"] != "RT_ICON" for r in resources) + + +class TestRecursionDepthCap: + + def test_deep_tree_does_not_raise(self): + """ + Before the cap this raised RecursionError out of _parse_resources, + which parse_pe does not catch - so one malformed file aborted the + whole analysis. + """ + node = _Node([_leaf()]) + for _ in range(2_000): + e = _Entry(id=1) + e.directory = node + node = _Node([e]) + strings = [] + _walk_resources(_FakePE(node), node, strings) # must not raise + + def test_normal_depth_unaffected(self): + """A well-formed tree is Type -> Name -> Language, three levels.""" + tree = _tree([_name_entry([_leaf(0x100, 0x40)])]) + strings = [] + _walk_resources(_FakePE(tree), tree, strings) + assert strings + + def test_depth_cap_is_generous_enough_for_real_trees(self): + assert _MAX_RESOURCE_DEPTH >= 3 + + +class TestCapsAreIndependent: + + def test_both_caps_can_fire_together(self, fake_strings): + tree = _tree([_name_entry([_leaf(0x100, 4000)], name_id=i) + for i in range(5_000)]) + resources, strings, truncated = _parse_resources(_FakePE(tree)) + assert len(resources) == _MAX_RESOURCE_ENTRIES + assert len(strings) == _MAX_RESOURCE_STRINGS + assert set(truncated) == {"resources", "resource_strings"} + + def test_entry_cap_alone(self, fake_strings): + tree = _tree([_name_entry([_leaf(0x100, 4)], name_id=i) + for i in range(5_000)]) + _, _, truncated = _parse_resources(_FakePE(tree)) + assert truncated == ["resources"] + + +class TestUnaffectedBehaviour: + + def test_clean_file_has_no_truncation_flags(self): + tree = _tree([_name_entry([_leaf(0x100, 0x40)])]) + resources, strings, truncated = _parse_resources(_FakePE(tree)) + assert truncated == [] + assert len(resources) == 1 + assert strings + + def test_no_resource_directory(self): + pe = _FakePE(None) + pe.DIRECTORY_ENTRY_RESOURCE = None + assert _parse_resources(pe) == ([], [], []) + + def test_ordering_still_deterministic(self): + def build(): + return _tree([_name_entry([_leaf(0x100, 64)], name_id=i) + for i in range(2_000)]) + first = json.dumps(_parse_resources(_FakePE(build()))[0], sort_keys=True) + for _ in range(5): + assert json.dumps(_parse_resources(_FakePE(build()))[0], + sort_keys=True) == first + + def test_caps_are_positive_and_bounded(self): + assert 0 < _MAX_RESOURCE_STRINGS <= 1_000_000 + assert 0 < _MAX_RESOURCE_ENTRIES <= 65_536 + assert 0 < _MAX_RESOURCE_DEPTH <= 64 + + +class TestResourcesUnavailable: + + def test_missing_method_is_tombstoned(self): + """An empty `resources` list must be distinguishable from a + binary that genuinely has none.""" + class _NoMMI(_FakePE): + @property + def get_memory_mapped_image(self): + raise AttributeError("get_memory_mapped_image") + + pe = _NoMMI(_tree([_name_entry([_leaf(0x100, 0x40)])])) + resources, strings, truncated = _parse_resources(pe) + assert resources == [] + assert "resources_unavailable" in truncated + assert strings # the walk still ran + + def test_raising_method_is_tombstoned_not_propagated(self): + class _RaisingMMI(_FakePE): + def get_memory_mapped_image(self): + raise struct.error("truncated image") + + pe = _RaisingMMI(_tree([_name_entry([_leaf(0x100, 0x40)])])) + resources, _, truncated = _parse_resources(pe) + assert resources == [] + assert "resources_map_read_failed" in truncated + + def test_tombstone_and_cap_are_mutually_exclusive(self): + pe = _FakePE(_tree([_name_entry([_leaf(0x100, 4)], name_id=i) + for i in range(5_000)])) + _, _, truncated = _parse_resources(pe) + assert "resources" in truncated + assert "resources_unavailable" not in truncated + # ------------------------------------------------------------ # Analyse PE sections # ------------------------------------------------------------ @@ -363,6 +684,78 @@ class FakePE: assert result == [] # early return path +@pytest.mark.parametrize("raw, label", [ + (None, "attribute absent"), + (b"", "zero-length buffer"), +]) +def test_parse_data_directories_raw_no_raw_bytes(raw, label): + """ + `if not raw` catches an empty buffer as well as a missing attribute - + the other parsers use `is None`, which would fall through to + struct.unpack_from on b"" here. A zero-length file genuinely has no + optional header, so both must return early. + """ + class FakeOptHdr: + Magic = 0x10B + def get_file_offset(self): + return 0xE8 + + pe = SimpleNamespace(OPTIONAL_HEADER=FakeOptHdr()) + if raw is not None: + pe.__data__ = raw + + from iocx.parsers.pe_parser import _parse_data_directories_raw + assert _parse_data_directories_raw(pe) == [], label + + +@pytest.mark.parametrize("readable", [0, 1, 3, 15]) +def test_parse_data_directories_raw_truncated_header(readable): + """ + A short optional header must yield the entries that were readable, + not an empty list and not an exception. `break` rather than `return + dirs` is what makes the partial result survive. + """ + import struct + opt_offset, data_dir_offset = 0, 96 + + class FakeOptHdr: + Magic = 0x10B # PE32 -> 96-byte offset + def get_file_offset(self): + return opt_offset + + # Fill each readable slot with a recognisable (rva, size) pair, then + # stop one byte short of completing the next. + payload = b"".join( + struct.pack(" Name -> Language, so the + root's first entry is a directory, not the data leaf. """ - import pytest as _pytest - from iocx.parsers.pe_resources import build_resource_structure + pe = self._make_pe_with_data_leaf_raising(exc) + out = build_resource_structure(pe) + leaves = _data_leaves(out["root"]) + assert len(leaves) == 1 + assert leaves[0]["raw_offset"] == -1 + assert _all_directory_errors(out["root"]) == [] + + def test_uncaught_type_falls_through_to_the_per_entry_guard(self): + """ + The inner except is narrow: a RuntimeError is NOT handled there, so + it reaches the per-entry guard and the leaf is dropped from its + containing directory. + + Widening the inner except to `Exception` would keep the leaf with + raw_offset = -1 instead. Before the never-raises patch this was + asserted as propagation; the per-entry guard now catches everything, + so the observable difference moved from "does it raise" to "is the + entry kept". + """ + pe = self._make_pe_with_data_leaf_raising(RuntimeError("not caught")) + out = build_resource_structure(pe) + assert _data_leaves(out["root"]) == [] + assert _all_directory_errors(out["root"]) == ["entry_decode_failed"] - pe = self._make_pe_with_data_leaf_raising( - RuntimeError("not a caught exception type") - ) - with _pytest.raises(RuntimeError): - build_resource_structure(pe) + def test_declared_size_survives_the_drop(self): + """ + The containing directory still reports the size its header implies, + so the gap between `size` and len(entries) remains interpretable. + """ + pe = self._make_pe_with_data_leaf_raising(RuntimeError("not caught")) + out = build_resource_structure(pe) + dropped = [d for d in _walk(out["root"]) if d["errors"]][0] + assert dropped["size"] == 24 + assert dropped["entries"] == [] diff --git a/tests/unit/parsers/test_pe_parser_extended.py b/tests/unit/parsers/test_pe_parser_extended.py index c7190d3..d220f82 100644 --- a/tests/unit/parsers/test_pe_parser_extended.py +++ b/tests/unit/parsers/test_pe_parser_extended.py @@ -344,10 +344,11 @@ class FakePE: pass from iocx.parsers.pe_parser import _parse_resources - resources, strings = _parse_resources(FakePE()) + resources, strings, truncated = _parse_resources(FakePE()) assert resources == [] assert strings == [] + assert truncated == [] def test_parse_resources_missing_memory_map(): @@ -359,10 +360,11 @@ class FakePE: # Crucially: NO get_memory_mapped_image attribute from iocx.parsers.pe_parser import _parse_resources - resources, strings = _parse_resources(FakePE()) + resources, strings, truncated = _parse_resources(FakePE()) assert resources == [] assert strings == [] + assert truncated == ["resources_unavailable"] assert hasattr(FakePE(), "DIRECTORY_ENTRY_RESOURCE") assert not hasattr(FakePE(), "get_memory_mapped_image") diff --git a/tests/unit/parsers/test_pe_relocations.py b/tests/unit/parsers/test_pe_relocations.py index 28293d7..f07c2f1 100644 --- a/tests/unit/parsers/test_pe_relocations.py +++ b/tests/unit/parsers/test_pe_relocations.py @@ -30,6 +30,7 @@ _MAX_ENTRIES_PER_BLOCK, _RELOC_DIRECTORY_INDEX, _RELOC_TYPE_NAMES, + _MAX_BLOCKS, ) @@ -223,7 +224,51 @@ def test_entries_truncated_when_block_exceeds_window(self): pe = _pe_with_reloc(partial, dir_size=10) # header + 1 entry trunc: List[str] = [] _read_blocks(pe, 0x1000, 10, trunc, []) - assert "relocation_entries_truncated" in trunc + assert "relocation_entries_exceed_directory" in trunc + + def test_short_header_read_is_tagged(self): + """get_data returns fewer than 8 bytes without raising.""" + trunc = [] + pe = _FakePE(bytes(0x1004), _FakeDataDir(0x1000, 0x40)) + out = _read_blocks(pe, 0x1000, 0x40, trunc, []) + assert out == [] + assert trunc == ["relocation_block_header_truncated"] + + def test_entries_read_failure_is_tagged(self): + """get_data raises on the entry region but not the header.""" + class _RaisingPE(_FakePE): + def get_data(self, rva, size): + if rva >= 0x1008: + raise ValueError("unmapped") + return super().get_data(rva, size) + blocks = _build_block(0x2000, [(3, 0x10), (3, 0x20)]) + image = bytearray(0x8000) + image[0x1000:0x1000 + len(blocks)] = blocks + pe = _RaisingPE(bytes(image), _FakeDataDir(0x1000, len(blocks))) + out = build_relocation_structure(pe) + assert "relocation_entries_read_failed" in out["truncations"] + assert out["blocks"][0]["entries"] == [] + + def test_entries_short_read_is_tagged(self): + """Header fits the window, but the image ends mid-entry-array.""" + hdr = _build_block_raw(0x2000, _BLOCK_HEADER_SIZE + 0x10) + image = bytearray(0x1010) # ends 8 bytes into the entries + image[0x1000:0x1008] = hdr + pe = _FakePE(bytes(image), _FakeDataDir(0x1000, _BLOCK_HEADER_SIZE + 0x10)) + out = build_relocation_structure(pe) + assert "relocation_entries_truncated" in out["truncations"] + assert out["blocks"][0]["entry_count"] == 4 # only what was readable + + def test_block_count_cap_is_tagged(self): + """65536 minimal blocks exhaust the walk limit.""" + one = _build_block_raw(0x2000, _BLOCK_HEADER_SIZE) + many = one * _MAX_BLOCKS + image = bytearray(0x1000 + len(many) + 0x10) + image[0x1000:0x1000 + len(many)] = many + pe = _FakePE(bytes(image), _FakeDataDir(0x1000, len(many))) + out = build_relocation_structure(pe) + assert out["block_count"] == _MAX_BLOCKS + assert "relocation_block_max_exceeded" in out["truncations"] # ================================================================= @@ -260,7 +305,7 @@ def test_never_raises_on_short_read(self): pe = _FakePE(bytes(0x100), _FakeDataDir(0x2000, 0x40)) out = build_relocation_structure(pe) assert out is not None - assert out["truncations"] # read failure recorded + assert out["truncations"] == ["relocation_block_read_failed"] # read failure recorded # ================================================================= diff --git a/tests/unit/parsers/test_pe_relocations_highadj.py b/tests/unit/parsers/test_pe_relocations_highadj.py new file mode 100644 index 0000000..f410c8b --- /dev/null +++ b/tests/unit/parsers/test_pe_relocations_highadj.py @@ -0,0 +1,221 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 + +from __future__ import annotations +import struct +import pytest +from iocx.parsers.pe_relocations import ( + build_relocation_structure, _decode_block, _read_blocks, + _BLOCK_HEADER_SIZE, _ENTRY_SIZE, _RELOC_TYPE_HIGHADJ, +) + +def _pack_entry(reloc_type, offset): + return struct.pack(" len(self._image): + raise ValueError("unmapped RVA") + return bytes(self._image[rva:rva + size]) + +def _pe_with_reloc(blocks, base_rva=0x1000, dir_size=None, image_size=0x8000): + if dir_size is None: + dir_size = len(blocks) + image = bytearray(image_size) + image[base_rva:base_rva + len(blocks)] = blocks + return _FakePE(bytes(image), _FakeDataDir(base_rva, dir_size)) + + +# ================================================================= +# Regression guard for the _RELOC_TYPE_HIGHADJ NameError +# ================================================================= + +class TestHighadjNameErrorRegression: + + @pytest.mark.parametrize("reloc_type", [t for t in range(16) if t != _RELOC_TYPE_HIGHADJ]) + def test_ordinary_entries_never_reference_undefined_name(self, reloc_type): + """ + Every non-HIGHADJ type must decode cleanly. This is the direct + regression guard for the bug where `_RELOC_TYPE_HIGHADJ` was + referenced without being defined: because the comparison ran + unconditionally at the end of every loop iteration (not gated behind + an `if reloc_type == HIGHADJ` branch), it crashed on ANY entry of ANY + type - not just HIGHADJ ones. A single entry of any type is enough to + reach the crashing line, so this is parametrised across all 16 + possible 4-bit type values rather than picking one representative. + """ + bb = _build_block(0x2000, [(reloc_type, 0x10)]) + pe = _pe_with_reloc(bb) + block = _decode_block(pe, 0, 0x1000, 0x2000, len(bb), 0x1000 + len(bb), []) + assert block["entry_count"] == 1 + assert block["entries"][0]["type"] == reloc_type + assert "adjustment" not in block["entries"][0] + + def test_full_page_of_all_types_never_raises(self): + """ + A single block containing every declared type (0-10) plus several + unknown types, repeated to page size, must decode without raising - + the broadest possible sweep of the crashing code path in one fixture. + """ + entries = [(t, (i * 4) & 0xFFF) for i, t in enumerate(list(range(16)) * 20)] + bb = _build_block(0x2000, entries) + out = build_relocation_structure(_pe_with_reloc(bb)) + assert out is not None + assert out["block_count"] == 1 + assert out["errors"] == [] + + def test_build_relocation_structure_does_not_raise_on_any_entry(self): + """ + End-to-end guard at the public API: the crash was reachable from + build_relocation_structure on the very first ordinary entry, so this + pins the full call path rather than only the internal helper. + """ + bb = _build_block(0x2000, [(3, 0x10)]) # ordinary HIGHLOW, not HIGHADJ + out = build_relocation_structure(_pe_with_reloc(bb)) + assert out is not None + assert out["entry_count"] == 1 + + +# ================================================================= +# skip_next state-machine coverage +# ================================================================= + +class TestHighadjSkipNextLogic: + """ + `skip_next` tracks whether the NEXT word is a raw adjustment value (the + second slot of a HIGHADJ pair) rather than an independent type+offset + entry. These tests drive every transition of that flag directly, rather + than relying on the single-pair case to imply the rest. + """ + + def test_highadj_sets_skip_next_and_consumes_following_word(self): + """HIGHADJ -> skip_next True -> next word consumed as `adjustment`, + never decoded as its own entry, and the flag resets to False.""" + bb = _build_block_raw(0x2000, _BLOCK_HEADER_SIZE + 4, + _pack_entry(4, 0x10) + struct.pack(" int: + if self._raise_on_offset: + raise pefile.PEFormatError("unmapped rva") + return rva - 0x1000 + 0x400 + + +# ---- tree builders ---- + +def _data_entry(offset: int = 0x1100, size: int = 0x40, + entry_id: Optional[int] = 0x409, + name: Optional[str] = None) -> _Entry: + e = _Entry(id=entry_id, name=name) + e.data = _Data(offset, size) + e.struct = _Struct(0) + return e + + +def _dir_entry(children, offset: int = 0x10, + entry_id: Optional[int] = 1, + name: Optional[str] = None) -> _Entry: + e = _Entry(id=entry_id, name=name) + e.directory = _Node(children) + e.struct = _Struct(0x80000000 | offset) + return e + + +def _hybrid_entry() -> _Entry: + """ + directory=None plus a valid .data. A data leaf to the main walk, a + directory to the string walk - the only shape that reaches the string + walk's exception handler while leaving the main tree intact. + """ + e = _Entry(id=1) + e.directory = None + e.data = _Data(0x1100, 0x40) + e.struct = _Struct(0) + return e + + +def _string_tree(tables=((0x1100, 0x40),)) -> _Node: + """A well-formed RT_STRING subtree: Type -> Name -> Language -> data.""" + langs = [_data_entry(off, size) for off, size in tables] + return _Node([_dir_entry([_dir_entry(langs, offset=0x10)], + offset=0x20, entry_id=_RT_STRING)]) + + +# ================================================================= +# Absence +# ================================================================= + +class TestAbsence: + + def test_no_resource_directory_returns_none(self): + pe = _FakePE(None, has_resource_dir=False) + assert build_resource_structure(pe) is None + + def test_missing_optional_header_returns_none(self): + """ + The data directory is needed for the base RVA; without it no RVA in + the tree can be derived, so returning None beats emitting a tree of + meaningless addresses. + """ + pe = _FakePE(_Node([]), has_optional_header=False) + assert build_resource_structure(pe) is None + + def test_empty_root_directory(self): + out = build_resource_structure(_FakePE(_Node([]))) + assert out["root"]["entries"] == [] + assert out["root"]["size"] == 16 # header only + assert out["root"]["errors"] == [] + assert out["string_tables"] == [] + assert out["errors"] == [] + + +# ================================================================= +# RVA derivation +# ================================================================= + +class TestRvaDerivation: + + def test_root_rva_is_the_directory_base(self): + out = build_resource_structure(_FakePE(_Node([]), base_rva=0x5000)) + assert out["root"]["rva"] == 0x5000 + + def test_subdirectory_rva_is_base_plus_offset(self): + tree = _Node([_dir_entry([], offset=0x20)]) + out = build_resource_structure(_FakePE(tree, base_rva=0x5000)) + assert out["root"]["entries"][0]["directory"]["rva"] == 0x5020 + + def test_high_bit_is_masked_off_the_offset(self): + """ + OffsetToData carries 0x80000000 to mark "is directory"; leaving it in + would place every subdirectory ~2GB past the image. + """ + tree = _Node([_dir_entry([], offset=0x30)]) + out = build_resource_structure(_FakePE(tree, base_rva=0x1000)) + assert out["root"]["entries"][0]["directory"]["rva"] == 0x1030 + + def test_nested_depth_derives_independently(self): + """Each level's RVA comes from its own entry struct, not by + accumulation, so a deep tree does not drift.""" + tree = _Node([_dir_entry([_dir_entry([], offset=0x10)], offset=0x20)]) + out = build_resource_structure(_FakePE(tree, base_rva=0x5000)) + type_dir = out["root"]["entries"][0]["directory"] + name_dir = type_dir["entries"][0]["directory"] + assert type_dir["rva"] == 0x5020 + assert name_dir["rva"] == 0x5010 + + +# ================================================================= +# Size derivation +# ================================================================= + +class TestSizeDerivation: + + @pytest.mark.parametrize("count,expected", [ + (0, 16), (1, 24), (2, 32), (5, 56), + ]) + def test_size_is_header_plus_eight_per_entry(self, count, expected): + tree = _Node([_data_entry() for _ in range(count)]) + out = build_resource_structure(_FakePE(tree)) + assert out["root"]["size"] == expected + + def test_size_uses_the_declared_count_not_the_decoded_count(self): + """ + A skipped entry must not shrink the reported size: the directory + header still claims that many entries, and the discrepancy between + declared size and decoded entries is the signal. + """ + tree = _Node([_data_entry(), _Entry(id=9)]) # second is undecodable + out = build_resource_structure(_FakePE(tree)) + assert out["root"]["size"] == 32 # 16 + 8*2 + assert len(out["root"]["entries"]) == 1 + assert out["root"]["errors"] == ["entry_decode_failed"] + + +# ================================================================= +# Entry decoding +# ================================================================= + +class TestEntryDecoding: + + def test_data_entry_fields(self): + tree = _Node([_data_entry(offset=0x1234, size=0x56)]) + entry = build_resource_structure(_FakePE(tree))["root"]["entries"][0] + assert entry["is_directory"] is False + assert entry["directory"] is None + assert entry["data_rva"] == 0x1234 + assert entry["data_size"] == 0x56 + assert entry["raw_offset"] == 0x1234 - 0x1000 + 0x400 + + def test_directory_entry_fields(self): + tree = _Node([_dir_entry([])]) + entry = build_resource_structure(_FakePE(tree))["root"]["entries"][0] + assert entry["is_directory"] is True + assert entry["directory"] is not None + assert entry["data_rva"] is None + assert entry["data_size"] is None + assert entry["raw_offset"] is None + + def test_named_entry_records_the_name(self): + tree = _Node([_data_entry(name="MYRESOURCE")]) + entry = build_resource_structure(_FakePE(tree))["root"]["entries"][0] + assert entry["name"] == "MYRESOURCE" + + def test_unnamed_entry_records_none(self): + tree = _Node([_data_entry(name=None)]) + assert build_resource_structure( + _FakePE(tree))["root"]["entries"][0]["name"] is None + + def test_entry_id_recorded(self): + tree = _Node([_data_entry(entry_id=0x0409)]) + assert build_resource_structure( + _FakePE(tree))["root"]["entries"][0]["id"] == 0x0409 + + def test_raw_offset_sentinel_on_unmapped_rva(self): + """ + A corrupt data RVA yields -1 rather than aborting. The validator + treats a negative raw offset as out-of-bounds, so the fact still + reaches output. + """ + tree = _Node([_data_entry()]) + out = build_resource_structure(_FakePE(tree, raise_on_offset=True)) + assert out["root"]["entries"][0]["raw_offset"] == -1 + + +# ================================================================= +# Per-entry guard - the never-raises contract +# ================================================================= + +class TestPerEntryGuard: + """ + Before this guard, a malformed entry propagated out of the parser and + took the whole analysis with it. Every other parser in the package holds + a never-raises contract; these pin it here. + """ + + def test_entry_with_neither_data_nor_directory_is_skipped(self): + tree = _Node([_Entry(id=9)]) + out = build_resource_structure(_FakePE(tree)) + assert out["root"]["entries"] == [] + assert out["root"]["errors"] == ["entry_decode_failed"] + + def test_sibling_entries_survive_a_bad_one(self): + tree = _Node([_data_entry(0x1100), _Entry(id=9), _data_entry(0x1200)]) + out = build_resource_structure(_FakePE(tree)) + assert [e["data_rva"] for e in out["root"]["entries"]] == [0x1100, 0x1200] + assert out["root"]["errors"] == ["entry_decode_failed"] + + def test_one_error_tag_per_failing_entry(self): + tree = _Node([_Entry(id=1), _Entry(id=2), _data_entry()]) + out = build_resource_structure(_FakePE(tree)) + assert out["root"]["errors"] == ["entry_decode_failed"] * 2 + + @pytest.mark.parametrize("broken", [_NoEntries, _RaisingEntries]) + def test_unreadable_subdirectory_is_contained(self, broken): + """ + A subtree whose entry list cannot be read costs that subtree only; + the parent still reports it as a directory entry. + """ + e = _Entry(id=1) + e.directory = broken() + e.struct = _Struct(0x80000010) + out = build_resource_structure(_FakePE(_Node([e]))) + subdir = out["root"]["entries"][0]["directory"] + assert subdir["errors"] == ["directory_entries_unavailable"] + assert subdir["entries"] == [] + assert subdir["size"] == 16 # header only; count unknowable + assert out["root"]["errors"] == [] + + def test_deep_failure_does_not_reach_the_root(self): + bad = _Entry(id=1) + bad.directory = _NoEntries() + bad.struct = _Struct(0x80000010) + tree = _Node([_dir_entry([_dir_entry([bad], offset=0x20)], offset=0x30)]) + out = build_resource_structure(_FakePE(tree)) + assert out["root"]["errors"] == [] + level1 = out["root"]["entries"][0]["directory"] + level2 = level1["entries"][0]["directory"] + level3 = level2["entries"][0]["directory"] + assert level3["errors"] == ["directory_entries_unavailable"] + + def test_errors_key_present_on_every_directory(self): + tree = _Node([_dir_entry([_dir_entry([_data_entry()])])]) + out = build_resource_structure(_FakePE(tree)) + + def _walk(node): + assert "errors" in node + assert isinstance(node["errors"], list) + for e in node["entries"]: + if e["directory"] is not None: + _walk(e["directory"]) + + _walk(out["root"]) + + +# ================================================================= +# RT_STRING walk +# ================================================================= + +class TestStringTableWalk: + + def test_healthy_walk_collects_tables(self): + out = build_resource_structure(_FakePE(_string_tree())) + assert out["string_tables"] == [{"rva": 0x1100, "size": 0x40}] + assert out["errors"] == [] + + def test_multiple_tables_collected_in_order(self): + tree = _string_tree(((0x1100, 0x40), (0x1200, 0x80))) + out = build_resource_structure(_FakePE(tree)) + assert out["string_tables"] == [{"rva": 0x1100, "size": 0x40}, + {"rva": 0x1200, "size": 0x80}] + + def test_walk_failure_is_tagged(self): + """ + The line that was `except Exception: pass`. Without the tag, an empty + list means both "no string resources" and "the walk broke". + """ + tree = _Node([_dir_entry([_hybrid_entry()], offset=0x20, + entry_id=_RT_STRING)]) + out = build_resource_structure(_FakePE(tree)) + assert out["errors"] == ["string_table_walk_failed"] + assert out["string_tables"] == [] + + def test_partial_collection_is_preserved(self): + """ + Tables gathered before the raise are kept, so the tag means "may be + incomplete" rather than "empty" - a non-empty list is not proof the + walk finished. + """ + good = _dir_entry([_data_entry(0x1100, 0x40)], offset=0x10) + tree = _Node([_dir_entry([good, _hybrid_entry()], offset=0x20, + entry_id=_RT_STRING)]) + out = build_resource_structure(_FakePE(tree)) + assert out["string_tables"] == [{"rva": 0x1100, "size": 0x40}] + assert out["errors"] == ["string_table_walk_failed"] + + def test_no_rt_string_resources_is_not_an_error(self): + """ + The distinction the tag exists for: an icon-only binary has an empty + list and a CLEAN errors list. + """ + tree = _Node([_dir_entry([_dir_entry([_data_entry()])], + offset=0x20, entry_id=_RT_ICON)]) + out = build_resource_structure(_FakePE(tree)) + assert out["string_tables"] == [] + assert out["errors"] == [] + + def test_walk_failure_does_not_break_the_main_tree(self): + tree = _Node([_dir_entry([_hybrid_entry()], offset=0x20, + entry_id=_RT_STRING)]) + out = build_resource_structure(_FakePE(tree)) + assert out["errors"] == ["string_table_walk_failed"] + assert len(out["root"]["entries"]) == 1 + assert out["root"]["errors"] == [] + + +# ================================================================= +# Output contract +# ================================================================= + +class TestOutputContract: + + def test_top_level_keys(self): + out = build_resource_structure(_FakePE(_string_tree())) + assert set(out) == {"root", "string_tables", "errors"} + + def test_directory_node_keys(self): + out = build_resource_structure(_FakePE(_Node([_data_entry()]))) + assert set(out["root"]) == {"rva", "size", "entries", "errors"} + + def test_entry_keys(self): + out = build_resource_structure(_FakePE(_Node([_data_entry()]))) + assert set(out["root"]["entries"][0]) == { + "name", "id", "is_directory", "directory", + "data_rva", "data_size", "raw_offset"} + + def test_json_serialisable(self): + import json + json.dumps(build_resource_structure(_FakePE(_string_tree()))) + + +# ================================================================= +# Robustness +# ================================================================= + +class TestNeverRaises: + + def _random_entry(self, rng, depth=0): + e = _Entry(id=rng.choice([None, 1, _RT_STRING, 0x409])) + r = rng.random() + if r < 0.20: + return e # neither branch + if r < 0.40: + e.data = _Data(rng.randint(0, 0xFFFF), rng.randint(0, 0xFF)) + e.struct = _Struct(0) + return e + if r < 0.50: + e.directory = None # hybrid + e.data = _Data(1, 1) + e.struct = _Struct(0) + return e + if r < 0.60: + e.directory = rng.choice([_NoEntries(), _RaisingEntries()]) + e.struct = _Struct(0x80000010) + return e + if depth > 2: + e.data = _Data(1, 1) + e.struct = _Struct(0) + return e + e.directory = _Node([self._random_entry(rng, depth + 1) + for _ in range(rng.randint(0, 3))]) + e.struct = _Struct(0x80000000 | rng.randint(0, 0xFFFF)) + return e + + def test_random_trees_never_raise(self): + rng = random.Random(11) + for _ in range(500): + tree = _Node([self._random_entry(rng) + for _ in range(rng.randint(0, 4))]) + out = build_resource_structure(_FakePE(tree)) + assert isinstance(out, dict) + + def test_deeply_nested_tree(self): + node = _Node([_data_entry()]) + for _ in range(50): + e = _Entry(id=1) + e.directory = node + e.struct = _Struct(0x80000010) + node = _Node([e]) + assert build_resource_structure(_FakePE(node)) is not None + + +# ================================================================= +# Determinism +# ================================================================= + +class TestDeterminism: + + def _tree(self): + return _Node([ + _dir_entry([_dir_entry([_data_entry(0x1100, 0x40)], offset=0x10)], + offset=0x20, entry_id=_RT_STRING), + _Entry(id=9), # undecodable + _dir_entry([_data_entry(0x1200, 0x80)], offset=0x30), + ]) + + def test_repeated_parse_identical(self): + import json + first = json.dumps(build_resource_structure(_FakePE(self._tree())), + sort_keys=True) + for _ in range(20): + assert json.dumps(build_resource_structure(_FakePE(self._tree())), + sort_keys=True) == first + + def test_error_tag_order_is_stable(self): + tree = _Node([_Entry(id=1), _data_entry(), _Entry(id=2)]) + for _ in range(10): + out = build_resource_structure(_FakePE(tree)) + assert out["root"]["errors"] == ["entry_decode_failed"] * 2 diff --git a/tests/unit/parsers/test_pe_version_info.py b/tests/unit/parsers/test_pe_version_info.py index 6745f9a..42fea3d 100644 --- a/tests/unit/parsers/test_pe_version_info.py +++ b/tests/unit/parsers/test_pe_version_info.py @@ -24,7 +24,7 @@ import pytest from iocx.parsers.pe_version_info import ( - build_version_info, + build_version_info_structure, _align4, _decode_string_file_info, _decode_var_file_info, @@ -36,8 +36,11 @@ _VS_FFI_STRUCT_VERSION, _VS_VERSION_INFO_KEY, RT_VERSION, + _MAX_CHILDREN, + _MAX_VERSION_BLOB ) +_PLACEMENT_TAG = "leaf_placement_implausible" # ================================================================= # Byte-level builders for VS_VERSIONINFO test fixtures @@ -54,6 +57,24 @@ def _pad4(buf: bytes) -> bytes: return buf + b"\x00" * pad +def _envelope(w_length: int = 0) -> bytes: + """VS_VERSIONINFO header with no FFI. w_length 0 leaves + length_consistent False, so the walk covers the whole buffer.""" + return _pad4(struct.pack(" bytes: + """A minimal well-formed child whose wLength covers header + key + pad.""" + key_bytes = _utf16_sz(key) + length = ((6 + len(key_bytes)) + 3) & ~3 + return _pad4(struct.pack(" bytes: + return _envelope() + _child(child_key) * count + + def _build_ffi( signature: int = _VS_FFI_SIGNATURE, struct_version: int = _VS_FFI_STRUCT_VERSION, @@ -130,17 +151,6 @@ def _build_var(key: str, translations: List[Tuple[int, int]]) -> bytes: return struct.pack(" bytes: - """Build a VarFileInfo child containing the given Vars.""" - key_bytes = _utf16_sz("VarFileInfo") - header_and_key = struct.pack(" _MAX_VERSION_BLOB is refused.""" + pe = _version_pe(0x1000, size) + pe.get_data = lambda rva, size: b"\x00" * size # allow the read + + out = build_version_info_structure(pe) + assert out["errors"] != [_PLACEMENT_TAG] + diff --git a/tests/unit/parsers/test_pe_version_info_fuzz.py b/tests/unit/parsers/test_pe_version_info_fuzz.py new file mode 100644 index 0000000..ad7ee68 --- /dev/null +++ b/tests/unit/parsers/test_pe_version_info_fuzz.py @@ -0,0 +1,197 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 +""" +Parser/validator coverage driven by synthesised VS_VERSIONINFO blobs. + +Each case is a single named deviation from a known-good baseline, so a +failure names the fault rather than "the blob is wrong somewhere". The +expectations below were verified against the real decoder. +""" + +import pytest + +from iocx.parsers.pe_version_info import _decode_vs_versioninfo +from _generate_vs_versioninfo_fixtures import CASES, baseline + + +def _tables(out): + return [t for s in out["string_file_info"] for t in s["tables"]] + + +def _all_errors(out): + """Every tag at every level - what _error_total counts.""" + tags = list(out["errors"]) + for s in out["string_file_info"]: + tags += s["errors"] + for t in s["tables"]: + tags += t["errors"] + for v in out["var_file_info"]: + tags += v["errors"] + return tags + + +# ===================================================================== +# Baseline +# ===================================================================== + +def test_baseline_is_clean(): + """If this fails, every expectation below is measuring the wrong thing.""" + out = _decode_vs_versioninfo(baseline()) + + assert out["decoded"] is True + assert out["header_ok"] is True + assert out["length_consistent"] is True + assert _all_errors(out) == [] + + ffi = out["fixed_file_info"] + assert ffi["signature_ok"] is True + assert ffi["struct_version_ok"] is True + + tables = _tables(out) + assert len(tables) == 1 + assert tables[0]["lang_codepage"] == "040904B0" + assert tables[0]["strings"]["CompanyName"] == "MalX Labs" + + assert out["var_file_info"][0]["vars"][0]["translations"] == [ + {"lang": 0x0409, "codepage": 0x04B0} + ] + + +def test_every_case_decodes_without_raising(): + """ + The parser's contract is to degrade, never to raise. A malformed blob + that escapes as an exception aborts the whole analysis. + """ + for name, fn in CASES.items(): + try: + _decode_vs_versioninfo(fn()) + except Exception as exc: # noqa: BLE001 + pytest.fail(f"{name} raised {type(exc).__name__}: {exc}") + + +# ===================================================================== +# Envelope faults +# ===================================================================== + +def test_szkey_mismatch(): + out = _decode_vs_versioninfo(CASES["szkey_mismatch"]()) + assert out["decoded"] is True + assert out["header_ok"] is False + # The fault is isolated: children still parse. + assert len(_tables(out)) == 1 + + +def test_length_inconsistent(): + out = _decode_vs_versioninfo(CASES["length_inconsistent"]()) + assert out["decoded"] is True + assert out["length_consistent"] is False + + +def test_truncated_header(): + out = _decode_vs_versioninfo(CASES["truncated_header"]()) + assert out["decoded"] is False + assert out["errors"] == ["too_short"] + + +# ===================================================================== +# VS_FIXEDFILEINFO +# ===================================================================== + +@pytest.mark.parametrize("case, flag", [ + ("ffi_bad_signature", "signature_ok"), + ("ffi_bad_struct_version", "struct_version_ok"), +]) +def test_ffi_field_faults(case, flag): + out = _decode_vs_versioninfo(CASES[case]()) + assert out["fixed_file_info"] is not None + assert out["fixed_file_info"][flag] is False + + +def test_ffi_truncated(): + out = _decode_vs_versioninfo(CASES["ffi_truncated"]()) + assert out["fixed_file_info"] is None + assert "fixed_file_info_truncated" in out["errors"] + + +def test_ffi_absent_is_not_a_defect(): + """wValueLength == 0 is a legitimate omission - no tag, no issue.""" + out = _decode_vs_versioninfo(CASES["ffi_absent"]()) + assert out["fixed_file_info"] is None + assert not any(e.startswith("fixed_file_info") for e in out["errors"]) + + +# ===================================================================== +# Child dispatch +# ===================================================================== + +def test_unknown_child(): + out = _decode_vs_versioninfo(CASES["unknown_child"]()) + assert "unknown_child" in out["errors"] + + +def test_child_length_invalid_stops_the_walk(): + out = _decode_vs_versioninfo(CASES["child_length_invalid"]()) + assert "child_length_invalid" in out["errors"] + assert _tables(out) == [] + + +def test_child_max_exceeded_bounds_the_errors_list(): + """ + unknown_child does not break the walk, so without the 256 cap the + errors list would scale with attacker-controlled input. + """ + out = _decode_vs_versioninfo(CASES["child_max_exceeded"]()) + assert "child_max_exceeded" in out["errors"] + assert out["errors"].count("unknown_child") <= 256 + + +def test_child_priority_resolution(): + """ + Both tags are present; _CHILD_ERROR_PRIORITY must report the + walk-terminating one, since the remaining children were never seen. + """ + out = _decode_vs_versioninfo(CASES["child_max_exceeded"]()) + errs = out["errors"] + assert "child_max_exceeded" in errs and "unknown_child" in errs + + +# ===================================================================== +# StringFileInfo / VarFileInfo +# ===================================================================== + +def test_lang_codepage_key(): + out = _decode_vs_versioninfo(CASES["lang_codepage_key"]()) + assert _tables(out)[0]["errors"] == ["lang_codepage_key"] + + +def test_string_length_invalid(): + out = _decode_vs_versioninfo(CASES["string_length_invalid"]()) + assert "string_length" in _tables(out)[0]["errors"] + + +def test_translation_not_dword_aligned(): + out = _decode_vs_versioninfo(CASES["translation_not_dword_aligned"]()) + assert out["var_file_info"][0]["errors"] == ["translation_not_dword_aligned"] + + +# ===================================================================== +# Projection-facing cases (parser accepts these; the projection acts) +# ===================================================================== + +def test_bidi_is_parsed_verbatim(): + """ + The parser must NOT sanitise - it records structural truth. Stripping + belongs in the projection, and this case proves the raw text reaches + it intact. + """ + out = _decode_vs_versioninfo(CASES["bidi_in_company_name"]()) + company = _tables(out)[0]["strings"]["CompanyName"] + assert "\u202e" in company + assert "\n" in company + assert _all_errors(out) == [] # not a structural defect + + +def test_many_tables_are_parsed_but_bounded_downstream(): + out = _decode_vs_versioninfo(CASES["deeply_nested_tables"]()) + assert len(_tables(out)) == 200 + assert _all_errors(out) == [] diff --git a/tests/unit/parsers/test_pe_version_info_projection.py b/tests/unit/parsers/test_pe_version_info_projection.py new file mode 100644 index 0000000..4dc4e3f --- /dev/null +++ b/tests/unit/parsers/test_pe_version_info_projection.py @@ -0,0 +1,539 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 +""" +Coverage for the public projection of version_info_struct. + +The projection is the last thing between an attacker-controlled resource +and public output, so the tests below lean on the two properties that +matter there: output size never scales with input size, and no key name +or character reaches the default payload unless it was explicitly +allowed. + +All expectations were verified against the implementation. +""" + +import pytest + +from iocx.parsers.version_info_projection import ( + project_version_info, + _clean, + _dotted_version, + _error_total, + _DEFAULT_KEYS, + _MAX_PUBLIC_TABLES, + _MAX_PUBLIC_STRINGS_PER_TABLE, + _MAX_PUBLIC_TRANSLATIONS, + _MAX_PUBLIC_KEY, + _MAX_PUBLIC_STRING, +) + + +# ===================================================================== +# Builders +# ===================================================================== + +def _struct(tables=None, translations=None, **kw): + """Minimal version_info_struct with the shape the projection expects.""" + vi = { + "decoded": True, + "errors": [], + "fixed_file_info": { + "file_version": (0x000A0000, 0x65F42308), + "product_version": (0x000A0000, 0x65F42308), + }, + "string_file_info": [{"errors": [], "tables": tables or []}], + "var_file_info": [{ + "errors": [], + "vars": [{"key": "Translation", + "translations": translations + if translations is not None + else [{"lang": 0x0409, "codepage": 0x04B0}]}], + }], + } + vi.update(kw) + return vi + + +def _table(lang="040904B0", strings=None, errors=None): + return {"lang_codepage": lang, + "strings": strings if strings is not None else {}, + "errors": errors or []} + + +_GOOD_STRINGS = { + "CompanyName": "MalX Labs", + "FileDescription": "IOCX test fixture", + "OriginalFilename": "FIXTURE.EXE", +} + + +# ===================================================================== +# Absence contract +# ===================================================================== + +def test_none_projects_to_none(): + """ + None means the binary carries no RT_VERSION resource. Because the + parse now runs on every file regardless of analysis level, this has + exactly one meaning and can never mean "this mode did not look". + """ + assert project_version_info(None) is None + + +def test_tombstone_is_projected_not_discarded(): + """ + A resource that exists but cannot be trusted must stay visible. + `decoded` plus `structural_error_count` are how a consumer tells a + tombstone apart from a healthy blob. + """ + out = project_version_info({ + "decoded": False, + "errors": ["leaf_placement_implausible"], + "fixed_file_info": None, + "string_file_info": [], + "var_file_info": [], + }) + + assert out is not None + assert out["decoded"] is False + assert out["structural_error_count"] == 1 + assert out["file_version"] is None + assert out["tables"] == [] + + +def test_empty_dict_is_not_treated_as_absence(): + """ + `if vi is None` rather than `if not vi`: an empty struct is a + degenerate blob, not a missing resource, and must still project. + """ + out = project_version_info({}) + assert out is not None + assert out["decoded"] is False + assert out["tables"] == [] + + +# ===================================================================== +# Version rendering +# ===================================================================== + +def test_dotted_version_splits_both_dwords(): + """MS = (major << 16) | minor; LS = (build << 16) | revision.""" + out = project_version_info(_struct()) + assert out["file_version"] == "10.0.26100.8968" + assert out["product_version"] == "10.0.26100.8968" + + +@pytest.mark.parametrize("pair", [ + None, + (1,), # wrong arity + (1, 2, 3), + ("a", 1), # non-int + (1, "b"), + (-1, 0), # negative + (0, -1), + (0x1_0000_0000, 0), # wider than a DWORD + (0, 0x1_0000_0000), + "10.0.0.1", # already-rendered string +]) +def test_dotted_version_rejects_malformed_pairs(pair): + """ + A malformed pair yields None rather than a partially-derived string: + a wrong version number is worse than an absent one. + """ + out = project_version_info(_struct( + fixed_file_info={"file_version": pair, "product_version": pair})) + assert out["file_version"] is None + assert out["product_version"] is None + + +def test_non_dict_ffi_does_not_raise(): + """A malformed struct must degrade, not propagate a TypeError.""" + for bad in (None, [], "x", 42): + out = project_version_info(_struct(fixed_file_info=bad)) + assert out["file_version"] is None + + +# ===================================================================== +# Sanitisation +# ===================================================================== + +def test_bidi_and_control_characters_are_stripped(): + """ + A bidi override in CompanyName renders text backwards in a terminal + or dashboard - the filename-spoofing trick arriving through a field + nobody inspects. Newlines break line-oriented consumers. + """ + out = project_version_info(_struct(tables=[_table(strings={ + "CompanyName": "MalX \u202eLabs\u202c\nInc\tX", + })])) + assert out["tables"][0]["strings"]["CompanyName"] == "MalX LabsIncX" + + +@pytest.mark.parametrize("raw", [ + "\x00", "\x1b[31m", "\x7f", "\u202a", "\u202e", "\u2066", "\u2069", + "\n", "\r", "\t", +]) +def test_each_unsafe_character_class_is_removed(raw): + assert raw not in (_clean(f"A{raw}B") or "") + + +def test_clean_rejects_non_strings(): + """A malformed struct cannot inject a non-string into public output.""" + for bad in (None, 42, [], {}, b"bytes"): + assert _clean(bad) is None + + +def test_value_length_is_bounded(): + out = project_version_info(_struct(tables=[_table( + strings={"CompanyName": "A" * (_MAX_PUBLIC_STRING + 500)})])) + assert len(out["tables"][0]["strings"]["CompanyName"]) == _MAX_PUBLIC_STRING + + +def test_key_length_is_bounded_separately(): + long_key = "C" * (_MAX_PUBLIC_KEY + 50) + out = project_version_info( + _struct(tables=[_table(strings={long_key: "v"})]), full=True) + projected = list(out["tables"][0]["strings"]) + assert all(len(k) <= _MAX_PUBLIC_KEY for k in projected) + + +def test_key_that_cleans_to_empty_is_dropped(): + """A control-character-only key would otherwise become "" in output.""" + out = project_version_info( + _struct(tables=[_table(strings={"\x00\x01": "v", + "CompanyName": "MalX"})]), + full=True) + assert "" not in out["tables"][0]["strings"] + assert out["tables"][0]["strings"]["CompanyName"] == "MalX" + + +def test_non_string_value_becomes_empty_string_not_none(): + """`_clean(v) or ""` - the key survives, the value degrades to "".""" + out = project_version_info( + _struct(tables=[_table(strings={"CompanyName": 12345})])) + assert out["tables"][0]["strings"]["CompanyName"] == "" + + +# ===================================================================== +# Closed key set +# ===================================================================== + +def test_default_projection_emits_only_shortlist_keys(): + """ + StringFileInfo keys are arbitrary strings from the file. The default + payload must not let a binary choose its own key NAMES, only values. + """ + strings = dict(_GOOD_STRINGS) + strings["EvilKey\u202e"] = "attacker chosen" + strings["Comments"] = "not in the shortlist" + + out = project_version_info(_struct(tables=[_table(strings=strings)])) + projected = set(out["tables"][0]["strings"]) + + assert projected <= _DEFAULT_KEYS + assert "Comments" not in projected + assert "keys_filtered" in out["truncated"] + + +def test_full_projection_opens_the_key_set(): + strings = dict(_GOOD_STRINGS) + strings["Comments"] = "now allowed" + + out = project_version_info(_struct(tables=[_table(strings=strings)]), + full=True) + projected = out["tables"][0]["strings"] + + assert projected["Comments"] == "now allowed" + assert "keys_filtered" not in out["truncated"] + + +def test_keys_filtered_not_flagged_when_nothing_is_dropped(): + out = project_version_info(_struct(tables=[_table(strings=_GOOD_STRINGS)])) + assert "keys_filtered" not in out["truncated"] + assert set(out["tables"][0]["strings"]) == set(_GOOD_STRINGS) + + +# ===================================================================== +# Per-table separation +# ===================================================================== + +def test_tables_stay_separate_so_keys_do_not_collide(): + """ + A flat merge would let a key present in two language blocks overwrite + itself, with the survivor decided by walk order - and a disagreement + between blocks is itself a repackaging signal. + """ + out = project_version_info(_struct(tables=[ + _table("040904B0", {"CompanyName": "Block A"}), + _table("040704B0", {"CompanyName": "Block B"}), + ])) + + assert len(out["tables"]) == 2 + assert out["tables"][0]["strings"]["CompanyName"] == "Block A" + assert out["tables"][1]["strings"]["CompanyName"] == "Block B" + + +def test_languages_are_deduped_but_tables_are_not(): + """Two blocks may legitimately share a lang_codepage.""" + out = project_version_info(_struct(tables=[ + _table("040904B0", {"CompanyName": "A"}), + _table("040904B0", {"CompanyName": "B"}), + ])) + assert out["languages"] == ["040904B0"] + assert len(out["tables"]) == 2 + + +def test_language_order_is_preserved(): + out = project_version_info(_struct(tables=[ + _table("080904B0"), _table("040904B0"), _table("080904B0"), + ])) + assert out["languages"] == ["080904B0", "040904B0"] + + +# ===================================================================== +# Bounds - output must not scale with input +# ===================================================================== + +def test_table_cap_is_inclusive(): + """Exactly _MAX_PUBLIC_TABLES must not be refused.""" + out = project_version_info(_struct( + tables=[_table(f"{i:04X}04B0") for i in range(_MAX_PUBLIC_TABLES)])) + assert len(out["tables"]) == _MAX_PUBLIC_TABLES + assert "tables" not in out["truncated"] + + +def test_table_cap_bounds_a_hostile_blob(): + """ + _decode_string_file_info advances by t_len >= 6, so a 1 MB blob can + yield ~174k tables. Without the cap, output scales with input. + """ + out = project_version_info(_struct( + tables=[_table(f"{i:04X}04B0") for i in range(5_000)])) + assert len(out["tables"]) == _MAX_PUBLIC_TABLES + assert "tables" in out["truncated"] + + +def test_output_size_does_not_scale_with_table_count(): + """The property that matters downstream.""" + import json + + def sized(n): + return len(json.dumps(project_version_info(_struct( + tables=[_table(f"{i:04X}04B0", dict(_GOOD_STRINGS)) + for i in range(n)])))) + + assert sized(500) == sized(50_000) + + +def test_strings_cap_is_inclusive(): + strings = {f"K{i}": "v" for i in range(_MAX_PUBLIC_STRINGS_PER_TABLE)} + out = project_version_info(_struct(tables=[_table(strings=strings)]), + full=True) + assert len(out["tables"][0]["strings"]) == _MAX_PUBLIC_STRINGS_PER_TABLE + assert "strings" not in out["truncated"] + + +def test_strings_cap_bounds_a_hostile_table(): + strings = {f"K{i}": "v" for i in range(5_000)} + out = project_version_info(_struct(tables=[_table(strings=strings)]), + full=True) + assert len(out["tables"][0]["strings"]) == _MAX_PUBLIC_STRINGS_PER_TABLE + assert "strings" in out["truncated"] + + +def test_translation_cap_is_inclusive(): + tr = [{"lang": i, "codepage": 0x04B0} + for i in range(_MAX_PUBLIC_TRANSLATIONS)] + out = project_version_info(_struct(translations=tr)) + assert len(out["translations"]) == _MAX_PUBLIC_TRANSLATIONS + assert "translations" not in out["truncated"] + + +def test_translation_cap_bounds_a_hostile_var_block(): + tr = [{"lang": i, "codepage": 0x04B0} for i in range(1_000)] + out = project_version_info(_struct(translations=tr)) + assert len(out["translations"]) == _MAX_PUBLIC_TRANSLATIONS + assert "translations" in out["truncated"] + + +def test_truncation_flags_appear_at_most_once(): + strings = {f"K{i}": "v" for i in range(200)} + out = project_version_info(_struct( + tables=[_table(f"{i:04X}04B0", strings) for i in range(50)]), + full=True) + for tag in out["truncated"]: + assert out["truncated"].count(tag) == 1 + + +def test_clean_input_carries_no_truncation_flags(): + out = project_version_info(_struct(tables=[_table(strings=_GOOD_STRINGS)])) + assert out["truncated"] == [] + + +# ===================================================================== +# Translations +# ===================================================================== + +def test_translations_render_as_eight_hex_chars(): + out = project_version_info(_struct( + translations=[{"lang": 0x0409, "codepage": 0x04B0}])) + assert out["translations"] == ["040904B0"] + + +def test_translations_are_deduped(): + out = project_version_info(_struct( + translations=[{"lang": 0x0409, "codepage": 0x04B0}] * 5)) + assert out["translations"] == ["040904B0"] + + +@pytest.mark.parametrize("bad", [ + {"lang": -1, "codepage": 0}, + {"lang": 0x10000, "codepage": 0}, + {"lang": 0, "codepage": -1}, + {"lang": 0, "codepage": 0x10000}, + {"lang": "0409", "codepage": 0x04B0}, + {"lang": 0x0409}, + {}, +]) +def test_out_of_range_translations_are_skipped(bad): + out = project_version_info(_struct( + translations=[bad, {"lang": 0x0409, "codepage": 0x04B0}])) + assert out["translations"] == ["040904B0"] + + +# ===================================================================== +# Structural error count +# ===================================================================== + +def test_error_count_sums_every_level(): + """ + The top-level list alone is not enough: a blob whose envelope decoded + cleanly but whose string tables are malformed would otherwise report + decoded=True with a zero count, reading as healthy when it is not. + """ + vi = { + "decoded": True, + "errors": ["top"], + "string_file_info": [{ + "errors": ["sfi"], + "tables": [{"lang_codepage": "040904B0", "strings": {}, + "errors": ["tbl_a", "tbl_b"]}], + }], + "var_file_info": [{"errors": ["var"], "vars": []}], + } + assert project_version_info(vi)["structural_error_count"] == 5 + + +def test_nested_errors_alone_are_still_counted(): + """decoded=True with a malformed table must not read as healthy.""" + out = project_version_info(_struct( + tables=[_table(strings=_GOOD_STRINGS, errors=["lang_codepage_key"])])) + assert out["decoded"] is True + assert out["structural_error_count"] == 1 + + +def test_error_total_tolerates_malformed_containers(): + for bad in ("string", 42, None): + assert _error_total({"string_file_info": [bad], + "var_file_info": [bad]}) == 0 + + +# ===================================================================== +# Robustness against a malformed struct +# ===================================================================== + +@pytest.mark.parametrize("field, value", [ + ("string_file_info", "not a list"), + ("string_file_info", [None, 42, "x"]), + ("string_file_info", [{"tables": "not a list"}]), + ("string_file_info", [{"tables": [None, 42]}]), + ("string_file_info", [{"tables": [{"strings": "not a dict"}]}]), + ("var_file_info", "not a list"), + ("var_file_info", [None, 42]), + ("var_file_info", [{"vars": "not a list"}]), + ("var_file_info", [{"vars": [None, 42]}]), + ("var_file_info", [{"vars": [{"translations": "not a list"}]}]), + ("var_file_info", [{"vars": [{"translations": [None, "x", 42]}]}]), +]) +def test_malformed_containers_do_not_raise(field, value): + """ + Every nested container is type-checked before use, so a malformed + struct yields empty output rather than a TypeError mid-projection. + """ + out = project_version_info(_struct(**{field: value})) + assert isinstance(out["tables"], list) + assert isinstance(out["translations"], list) + + +def test_missing_keys_throughout_do_not_raise(): + assert project_version_info({"decoded": True}) is not None + + +# ===================================================================== +# Output contract +# ===================================================================== + +def test_projection_has_a_stable_key_set(): + """The public shape must not vary with input, or consumers break.""" + expected = { + "file_version", "product_version", "tables", "languages", + "translations", "decoded", "structural_error_count", "truncated", + } + assert set(project_version_info(_struct())) == expected + assert set(project_version_info({})) == expected + assert set(project_version_info(_struct(), full=True)) == expected + + +def test_projection_is_json_serialisable(): + import json + json.dumps(project_version_info(_struct(tables=[ + _table(strings={"CompanyName": "\u00a9 MalX \u202eLabs"})]))) + + +def test_projection_does_not_mutate_the_input(): + vi = _struct(tables=[_table(strings=dict(_GOOD_STRINGS))]) + import copy + before = copy.deepcopy(vi) + project_version_info(vi) + assert vi == before + + +def test_decoded_is_always_a_bool(): + """`bool(...)` - a truthy non-bool must not leak into public output.""" + for raw in (1, "yes", [1], None, 0, ""): + assert isinstance(project_version_info({"decoded": raw})["decoded"], + bool) + + +def test_key_padding_cannot_suppress_the_shortlist(): + """ + Filtering happens before the cap is counted, so a table padded with + junk keys cannot push the triage fields out of default output. + Before that change, 64 junk keys ahead of CompanyName suppressed it + entirely - and OriginalFilename versus the on-disk name is exactly + the check that matters most on a suspicious file. + """ + strings = {f"Junk{i}": "x" for i in range(_MAX_PUBLIC_STRINGS_PER_TABLE)} + strings["CompanyName"] = "Microsoft Corporation" + strings["OriginalFilename"] = "NOTEPAD.EXE" + + out = project_version_info(_struct(tables=[_table(strings=strings)])) + + assert out["tables"][0]["strings"] == { + "CompanyName": "Microsoft Corporation", + "OriginalFilename": "NOTEPAD.EXE", + } + assert "keys_filtered" in out["truncated"] + assert "strings" not in out["truncated"] + + +def test_ordering_no_longer_decides_what_survives(): + """The same keys in either order must project identically.""" + junk = {f"Junk{i}": "x" for i in range(_MAX_PUBLIC_STRINGS_PER_TABLE)} + real = {"CompanyName": "Microsoft Corporation", + "OriginalFilename": "NOTEPAD.EXE"} + + junk_first = project_version_info(_struct(tables=[_table(strings={**junk, **real})])) + real_first = project_version_info(_struct(tables=[_table(strings={**real, **junk})])) + + assert junk_first["tables"] == real_first["tables"] diff --git a/tests/unit/validators/test_validator_exception_dir.py b/tests/unit/validators/test_validator_exception_dir.py index 6f93ba7..74d797e 100644 --- a/tests/unit/validators/test_validator_exception_dir.py +++ b/tests/unit/validators/test_validator_exception_dir.py @@ -441,7 +441,7 @@ def test_begin_rva_zero_flagged(self): assert _details_for(issues, ReasonCodes.EXCEPTION_ENTRY_INVALID)[0]["index"] == 1 @pytest.mark.parametrize("tag", [ - "entry_truncated", "entry_read_failed", "entry_unpack_failed", + "entry_unpack_failed", "begin_rva_zero", "end_rva_zero", "unwind_rva_zero", ]) def test_each_priority_tag_resolves(self, tag): @@ -453,10 +453,10 @@ def test_priority_first_match_wins(self): """entry_truncated outranks begin_rva_zero and unwind_rva_zero.""" ex = _make_ex(size=12, functions=[_make_entry( 0, None, None, None, unwind=None, - errors=["begin_rva_zero", "entry_truncated", "unwind_rva_zero"])]) + errors=["begin_rva_zero", "unwind_rva_zero"])]) issues = _run(ex) assert len(issues) == 1 - assert _has(issues, ReasonCodes.EXCEPTION_ENTRY_INVALID, "entry_truncated") + assert _has(issues, ReasonCodes.EXCEPTION_ENTRY_INVALID, "begin_rva_zero") def test_unknown_tag_not_flagged(self): """A future parser tag not in the priority list is skipped silently.""" @@ -998,8 +998,8 @@ def test_repeated_validation_produces_identical_issues(self): def test_priority_resolution_deterministic(self): ex = _make_ex(size=12, functions=[_make_entry( 0, None, None, None, unwind=None, - errors=["unwind_rva_zero", "entry_truncated", "begin_rva_zero"])]) + errors=["unwind_rva_zero", "begin_rva_zero"])]) results = [_run(ex) for _ in range(20)] for r in results[1:]: assert r == results[0] - assert results[0][0]["details"]["sub_reason"] == "entry_truncated" + assert results[0][0]["details"]["sub_reason"] == "begin_rva_zero" diff --git a/tests/unit/validators/test_validator_exports.py b/tests/unit/validators/test_validator_exports.py index 59b8d62..1f664d3 100644 --- a/tests/unit/validators/test_validator_exports.py +++ b/tests/unit/validators/test_validator_exports.py @@ -676,6 +676,198 @@ def test_multiple_pathology_classes_emit_independently(self): assert ReasonCodes.EXPORT_NAME_RVA_INVALID in codes +# ================================================================= +# (v0.7.6.2) The duplicate branch +# ================================================================= +_METADATA = {"optional_header": {"size_of_image": 0x100000}} + +def _np(index: int, + errors: List[str], + name: Optional[str] = "Alpha", + ordinal_index: Optional[int] = 0, + name_rva: int = 0x1400, + name_valid: bool = True) -> Dict[str, Any]: + """One NamePointerEntry as the parser emits it.""" + return {"index": index, "errors": errors, "name": name, + "ordinal_index": ordinal_index, "name_rva": name_rva, + "name_valid": name_valid} + + +def _internal(name_pointers: List[Dict[str, Any]], + num_functions: int = 2, + functions: Optional[List[Dict[str, Any]]] = None) -> Dict[str, Any]: + return {"export_struct": { + "rva": 0x1000, "size": 200, + "header": {"NumberOfFunctions": num_functions, + "NumberOfNames": len(name_pointers), + "Base": 1, "AddressOfFunctions": 0x1100, + "AddressOfNames": 0x1200, "AddressOfNameOrdinals": 0x1300}, + "functions": functions or [], + "name_pointers": name_pointers, + "truncations": [], "errors": []}} + + +def _ordinal_issues(issues): + return [i for i in issues + if i["issue"] == ReasonCodes.EXPORT_NAME_ORDINAL_INDEX_INVALID] + + +def _sub_reasons(issues): + return [i["details"].get("sub_reason") for i in _ordinal_issues(issues)] + + +class TestOrdinalIndexDuplicate: + + def test_duplicate_tag_is_emitted(self): + """ + Regression guard: before this branch existed the tag reached the + validator and produced nothing at all - the collision was invisible + downstream. + """ + issues = validate_exports(_internal([ + _np(0, [], "Alpha", 0), + _np(1, ["ordinal_index_duplicate"], "Beta", 0), + ]), _METADATA) + assert _sub_reasons(issues) == ["duplicate"] + + def test_details_payload(self): + issues = validate_exports(_internal([ + _np(0, [], "Alpha", 0), + _np(1, ["ordinal_index_duplicate"], "Beta", 0, name_rva=0x1408), + ]), _METADATA) + assert _ordinal_issues(issues)[0]["details"] == { + "index": 1, + "ordinal_index": 0, + "name": "Beta", + "sub_reason": "duplicate", + } + + def test_details_carry_the_shadowed_name(self): + """ + The name is what makes this finding actionable: it identifies the + export that became unreachable through the resolved-function list. + Without it a consumer knows only that *some* collision occurred. + """ + issues = validate_exports(_internal([ + _np(0, [], "Alpha", 0), + _np(1, ["ordinal_index_duplicate"], "SecretExport", 0), + ]), _METADATA) + assert _ordinal_issues(issues)[0]["details"]["name"] == "SecretExport" + + def test_no_num_functions_key(self): + """ + Unlike out_of_range, the duplicate branch does not carry + num_functions - the count is irrelevant to a collision. Pinned so the + two payload shapes stay deliberately distinct. + """ + issues = validate_exports(_internal([ + _np(0, [], "Alpha", 0), + _np(1, ["ordinal_index_duplicate"], "Beta", 0), + ]), _METADATA) + assert "num_functions" not in _ordinal_issues(issues)[0]["details"] + + def test_each_colliding_entry_emits_once(self): + issues = validate_exports(_internal([ + _np(0, [], "Alpha", 0), + _np(1, ["ordinal_index_duplicate"], "Beta", 0), + _np(2, ["ordinal_index_duplicate"], "Gamma", 0), + ]), _METADATA) + assert _sub_reasons(issues) == ["duplicate", "duplicate"] + assert [i["details"]["index"] for i in _ordinal_issues(issues)] == [1, 2] + + def test_clean_table_emits_nothing(self): + issues = validate_exports(_internal([ + _np(0, [], "Alpha", 0), + _np(1, [], "Beta", 1), + ]), _METADATA) + assert _ordinal_issues(issues) == [] + + +# ================================================================= +# (v0.7.6.2) Precedence within the elif chain +# ================================================================= + +class TestOrdinalBranchPrecedence: + """ + The three ordinal sub-reasons share one if/elif chain, so an entry + carrying several tags must still emit exactly one issue. + """ + + @pytest.mark.parametrize("errors,expected", [ + (["ordinal_index_missing", "ordinal_index_duplicate"], "missing"), + (["ordinal_index_out_of_range", "ordinal_index_duplicate"], "out_of_range"), + (["ordinal_index_duplicate"], "duplicate"), + (["ordinal_index_missing", "ordinal_index_out_of_range", + "ordinal_index_duplicate"], "missing"), + ]) + def test_first_branch_wins(self, errors, expected): + issues = validate_exports( + _internal([_np(0, errors, "Alpha", 0)]), _METADATA) + assert _sub_reasons(issues) == [expected] + + def test_duplicate_never_double_emits_with_a_sibling(self): + """One issue per entry per pathology class, regardless of tag count.""" + issues = validate_exports(_internal([ + _np(0, ["ordinal_index_out_of_range", "ordinal_index_duplicate"], + "Alpha", 0), + ]), _METADATA) + assert len(_ordinal_issues(issues)) == 1 + + +# ================================================================= +# (v0.7.6.2) Interaction with other pathology classes +# ================================================================= + +class TestCrossClassInteraction: + + def test_duplicate_co_fires_with_name_rva_class(self): + """ + The RVA, encoding and ordinal checks are independent classes, so an + entry may legitimately raise one of each. + """ + issues = validate_exports(_internal([ + _np(0, ["read_failed", "ordinal_index_duplicate"], + None, 0, name_valid=False), + ]), _METADATA) + codes = {i["issue"] for i in issues} + assert ReasonCodes.EXPORT_NAME_RVA_INVALID in codes + assert ReasonCodes.EXPORT_NAME_ORDINAL_INDEX_INVALID in codes + + def test_duplicate_co_fires_with_encoding_class(self): + issues = validate_exports(_internal([ + _np(0, ["name_not_printable_ascii", "ordinal_index_duplicate"], + "Bad\x01Name", 0, name_valid=False), + ]), _METADATA) + codes = {i["issue"] for i in issues} + assert ReasonCodes.EXPORT_NAME_NOT_ASCII in codes + assert ReasonCodes.EXPORT_NAME_ORDINAL_INDEX_INVALID in codes + + def test_ascending_names_do_not_trip_the_sortedness_check(self): + """ + Control for the fixture convention: with ascending names the duplicate + issue is the ONLY finding, so the assertions above are single-anomaly. + """ + issues = validate_exports(_internal([ + _np(0, [], "Alpha", 0), + _np(1, ["ordinal_index_duplicate"], "Beta", 0), + ]), _METADATA) + assert len(issues) == 1 + assert issues[0]["issue"] == ReasonCodes.EXPORT_NAME_ORDINAL_INDEX_INVALID + + def test_descending_names_additionally_raise_unsorted(self): + """ + Documents why the fixtures ascend: the sortedness check walks the same + table and is independent of the duplicate finding. + """ + issues = validate_exports(_internal([ + _np(0, [], "Zebra", 0), + _np(1, ["ordinal_index_duplicate"], "Alpha", 0), + ]), _METADATA) + codes = {i["issue"] for i in issues} + assert ReasonCodes.EXPORT_NAME_ORDINAL_INDEX_INVALID in codes + assert ReasonCodes.EXPORT_NAME_POINTER_TABLE_UNSORTED in codes + + # ================================================================= # Output contract # ================================================================= @@ -743,6 +935,35 @@ def test_top_level_decode_avoids_reserved_reason_key(self): assert issues assert "reason" not in issues[0]["details"] + def test_issue_shape(self): + issues = validate_exports(_internal([ + _np(0, [], "Alpha", 0), + _np(1, ["ordinal_index_duplicate"], "Beta", 0), + ]), _METADATA) + for issue in issues: + assert set(issue) == {"issue", "details"} + assert isinstance(issue["details"], dict) + + def test_no_reserved_reason_key(self): + """ + "reason" is reserved by the heuristics emission layer; a details key of + that name would overwrite the parent code. + """ + issues = validate_exports(_internal([ + _np(0, [], "Alpha", 0), + _np(1, ["ordinal_index_duplicate"], "Beta", 0), + ]), _METADATA) + assert issues + assert all("reason" not in i["details"] for i in issues) + + def test_json_serialisable(self): + import json + issues = validate_exports(_internal([ + _np(0, [], "Alpha", 0), + _np(1, ["ordinal_index_duplicate"], "Beta", 0), + ]), _METADATA) + json.dumps(issues) + # ================================================================= # Determinism @@ -787,3 +1008,15 @@ def test_priority_resolution_deterministic(self): # Confirm priority winner details = _details_for(results[0], ReasonCodes.EXPORT_NAME_RVA_INVALID) assert details[0]["sub_reason"] == "name_rva_missing" + + def test_deterministic_dedupe(self): + import json + internal = _internal([ + _np(0, [], "Alpha", 0), + _np(1, ["ordinal_index_duplicate"], "Beta", 0), + _np(2, ["ordinal_index_duplicate"], "Gamma", 0), + ]) + first = json.dumps(validate_exports(internal, _METADATA), sort_keys=True) + for _ in range(20): + assert json.dumps(validate_exports(internal, _METADATA), + sort_keys=True) == first diff --git a/tests/unit/validators/test_validator_imports.py b/tests/unit/validators/test_validator_imports.py new file mode 100644 index 0000000..8802a9d --- /dev/null +++ b/tests/unit/validators/test_validator_imports.py @@ -0,0 +1,625 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 + +""" +Unit tests for iocx.validators.imports.validate_imports. + +Layer note: @depends_on("internal") - ONE positional argument. Placement is +deliberately not checked here (the RVA-graph backbone owns it), so no +metadata layer is needed. + +Fixture note: the three pathology classes - DLL name, thunk source, and +per-entry - are fully independent. A descriptor carrying faults in two +classes emits one issue from each, so fixtures targeting one class must +leave the others clean or the assertions stop being single-anomaly. +""" + +from __future__ import annotations + +from typing import Any, Dict, List, Optional + +import pytest + +from iocx.reason_codes import ReasonCodes +from iocx.validators.imports import ( + validate_imports, + _DLL_NAME_ERROR_PRIORITY, + _ENTRY_ERROR_PRIORITY, + _MAX_ENTRY_ISSUES_PER_DESCRIPTOR, + _THUNK_SOURCE_ERROR_PRIORITY, +) + + +# ================================================================= +# Builders +# ================================================================= + +def _entry(index: int = 0, + errors: Optional[List[str]] = None, + **kw) -> Dict[str, Any]: + """One ImportEntry as the parser emits it.""" + e = {"index": index, "errors": errors or [], "is_ordinal": False, + "ordinal": None, "name": "Fn", "name_rva": 0x5000, + "name_valid": True, "hint": 1, "thunk_value": 0x5000} + e.update(kw) + return e + + +def _descriptor(index: int = 0, + errors: Optional[List[str]] = None, + dll_name: Optional[str] = "KERNEL32.dll", + imports: Optional[List[Dict[str, Any]]] = None, + **kw) -> Dict[str, Any]: + """One descriptor as the parser emits it.""" + d = {"index": index, "errors": errors or [], "dll_name": dll_name, + "dll_name_valid": True, "name_rva": 0x2000, + "bound_state": "unbound", "original_first_thunk": 0x3000, + "first_thunk": 0x4000, "thunk_source": "int", + "imports": imports or [], "timestamp": 0, "forwarder_chain": 0} + d.update(kw) + return d + + +def _internal(descriptors: Optional[List[Dict[str, Any]]] = None, + truncations: Optional[List[str]] = None, + errors: Optional[List[str]] = None) -> Dict[str, Any]: + descriptors = descriptors or [] + return {"import_struct": { + "rva": 0x1000, "size": 60, "is_64bit": True, + "descriptors": descriptors, + "descriptor_count": len(descriptors), + "truncations": truncations or [], + "errors": errors or [], + }} + + +def _codes(issues) -> List[str]: + return [i["issue"] for i in issues] + + +def _of(issues, code) -> List[Dict[str, Any]]: + return [i["details"] for i in issues if i["issue"] == code] + + +def _subs(issues, code) -> List[str]: + return [d.get("sub_reason") for d in _of(issues, code)] + + +# ================================================================= +# Priority-list content +# ================================================================= + +class TestPriorityListOrder: + """ + The ORDER of each priority list is a contract, not an implementation + detail: it decides which sub_reason a multi-fault entry reports. + + These assert the literal sequence, because a behavioural test cannot + catch a reorder - _first_matching is definitionally consistent with + whatever order the list happens to have, so fixtures derived from the + list move with it. The behavioural pair tests below still earn their + place by proving the lists are actually consulted; only these pin the + order itself. + """ + + def test_dll_name_priority_order(self): + assert _DLL_NAME_ERROR_PRIORITY == [ + # RVA-level + "dll_name_rva_zero", + # _read_asciiz read faults + "rva_zero", + "read_failed", + "empty_read", + "unterminated", + "non_ascii", + # content checks (three-way split, mutually exclusive) + "dll_name_empty", + "dll_name_not_printable", + "dll_name_too_long", + ] + + def test_thunk_source_priority_order(self): + assert _THUNK_SOURCE_ERROR_PRIORITY == [ + "names_unrecoverable_bound_no_int", + "no_thunk_array", + ] + + def test_entry_priority_order(self): + assert _ENTRY_ERROR_PRIORITY == [ + # ordinal path + "ordinal_zero", + # IMAGE_IMPORT_BY_NAME read faults + "name_rva_zero", + "name_read_failed", + "name_too_short", + "hint_unpack_failed", + "name_unterminated", + "name_non_ascii", + # name content checks (mutually exclusive) + "name_empty", + "name_not_printable", + ] + + def test_no_duplicates_within_a_list(self): + for name, lst in (("dll", _DLL_NAME_ERROR_PRIORITY), + ("source", _THUNK_SOURCE_ERROR_PRIORITY), + ("entry", _ENTRY_ERROR_PRIORITY)): + assert len(lst) == len(set(lst)), f"{name} list has duplicates" + + def test_descriptor_lists_are_disjoint(self): + """ + Both read descriptor["errors"]; an overlapping tag would emit two + issues for one fault. + """ + assert not (set(_DLL_NAME_ERROR_PRIORITY) + & set(_THUNK_SOURCE_ERROR_PRIORITY)) + + +# ================================================================= +# Absence / short-circuit +# ================================================================= + +class TestAbsence: + + def test_no_import_struct_returns_empty(self): + assert validate_imports({}) == [] + + def test_none_import_struct_returns_empty(self): + assert validate_imports({"import_struct": None}) == [] + + def test_empty_directory_emits_nothing(self): + assert validate_imports(_internal([])) == [] + + def test_clean_descriptor_emits_nothing(self): + assert validate_imports(_internal([_descriptor()])) == [] + + +class TestTopLevelShortCircuit: + + def test_top_level_error_emits_one_issue(self): + issues = validate_imports( + _internal(errors=["descriptor_unpack_failed"])) + assert _codes(issues) == [ReasonCodes.IMPORT_DIRECTORY_INVALID_HEADER] + assert _subs(issues, ReasonCodes.IMPORT_DIRECTORY_INVALID_HEADER) == [ + "top_level_decode"] + + def test_top_level_error_suppresses_everything_else(self): + """ + Without a usable descriptor array there is nothing further to say, so + truncations and per-descriptor faults are not reported. + """ + issues = validate_imports(_internal( + descriptors=[_descriptor(errors=["dll_name_empty"])], + truncations=["int_truncated"], + errors=["descriptor_unpack_failed"], + )) + assert len(issues) == 1 + assert issues[0]["issue"] == ReasonCodes.IMPORT_DIRECTORY_INVALID_HEADER + + def test_errors_list_forwarded_verbatim(self): + issues = validate_imports(_internal(errors=["a", "b"])) + assert _of(issues, ReasonCodes.IMPORT_DIRECTORY_INVALID_HEADER)[0][ + "errors"] == ["a", "b"] + + +# ================================================================= +# Truncations +# ================================================================= + +class TestTruncations: + + def test_one_issue_per_tag(self): + issues = validate_imports( + _internal(truncations=["int_truncated", "int_read_failed"])) + assert _codes(issues) == [ReasonCodes.IMPORT_TABLE_TRUNCATED] * 2 + + def test_tag_carried_in_table_key(self): + issues = validate_imports(_internal(truncations=["int_truncated"])) + assert _of(issues, ReasonCodes.IMPORT_TABLE_TRUNCATED)[0] == { + "table": "int_truncated"} + + @pytest.mark.parametrize("tag", [ + "import_descriptor_unterminated", + "import_descriptor_read_failed", + "import_descriptor_truncated", + "import_descriptor_max_exceeded", + "int_read_failed", "int_truncated", + "int_unpack_failed", "int_max_exceeded", + "iat_fallback_read_failed", "iat_fallback_truncated", + "iat_fallback_unpack_failed", "iat_fallback_max_exceeded", + ]) + def test_every_parser_truncation_tag_is_forwarded(self, tag): + """ + Truncations are forwarded wholesale, so no tag can be dropped. This + enumerates the parser's full vocabulary as a contract guard. + """ + issues = validate_imports(_internal(truncations=[tag])) + assert _of(issues, ReasonCodes.IMPORT_TABLE_TRUNCATED)[0]["table"] == tag + + def test_fallback_prefix_distinguishes_the_array_read(self): + """ + `int_*` vs `iat_fallback_*` tells a consumer which array was short - + the fallback path only exists on older-linker binaries. + """ + int_issues = validate_imports(_internal(truncations=["int_truncated"])) + fb_issues = validate_imports( + _internal(truncations=["iat_fallback_truncated"])) + assert _of(int_issues, ReasonCodes.IMPORT_TABLE_TRUNCATED)[0][ + "table"] == "int_truncated" + assert _of(fb_issues, ReasonCodes.IMPORT_TABLE_TRUNCATED)[0][ + "table"] == "iat_fallback_truncated" + + +# ================================================================= +# DLL name class +# ================================================================= + +class TestDllName: + + @pytest.mark.parametrize("tag", _DLL_NAME_ERROR_PRIORITY) + def test_every_priority_tag_emits(self, tag): + """Contract guard: no tag in the list may be silently dropped.""" + issues = validate_imports(_internal([_descriptor(errors=[tag])])) + assert _subs(issues, ReasonCodes.IMPORT_DLL_NAME_INVALID) == [tag] + + def test_details_payload(self): + issues = validate_imports(_internal([ + _descriptor(index=2, errors=["dll_name_empty"], + dll_name="", name_rva=0x2222)])) + assert _of(issues, ReasonCodes.IMPORT_DLL_NAME_INVALID)[0] == { + "index": 2, "dll_name_rva": 0x2222, "dll_name": "", + "sub_reason": "dll_name_empty"} + + def test_one_issue_per_descriptor(self): + """Priority-resolved: several tags still produce a single issue.""" + issues = validate_imports(_internal([ + _descriptor(errors=["dll_name_rva_zero", "read_failed", + "dll_name_empty"])])) + assert len(_of(issues, ReasonCodes.IMPORT_DLL_NAME_INVALID)) == 1 + + @pytest.mark.parametrize("higher,lower", list( + zip(_DLL_NAME_ERROR_PRIORITY, _DLL_NAME_ERROR_PRIORITY[1:]))) + def test_priority_order_adjacent_pairs(self, higher, lower): + """Every adjacent pair, both presentation orders - see the entry-class + equivalent for why distant pairs are insufficient.""" + for errors in ([higher, lower], [lower, higher]): + issues = validate_imports(_internal([_descriptor(errors=errors)])) + assert _subs(issues, + ReasonCodes.IMPORT_DLL_NAME_INVALID) == [higher] + + def test_each_descriptor_reported_separately(self): + issues = validate_imports(_internal([ + _descriptor(index=0, errors=["dll_name_empty"]), + _descriptor(index=1, errors=["dll_name_too_long"]), + ])) + details = _of(issues, ReasonCodes.IMPORT_DLL_NAME_INVALID) + assert [d["index"] for d in details] == [0, 1] + assert [d["sub_reason"] for d in details] == [ + "dll_name_empty", "dll_name_too_long"] + + +# ================================================================= +# Thunk-source class +# ================================================================= + +class TestThunkSource: + """ + A descriptor with no readable name source identifies the module but not + its symbols - a different fact from a malformed DLL name, hence its own + reason code. + """ + + @pytest.mark.parametrize("tag", _THUNK_SOURCE_ERROR_PRIORITY) + def test_every_priority_tag_emits(self, tag): + issues = validate_imports(_internal([_descriptor(errors=[tag])])) + assert _subs(issues, ReasonCodes.IMPORT_DESCRIPTOR_INVALID) == [tag] + + def test_bound_without_int_details(self): + """ + The details carry enough to explain WHY names are unrecoverable: + old-style bound means FirstThunk holds addresses, and there is no INT. + """ + issues = validate_imports(_internal([_descriptor( + index=1, errors=["names_unrecoverable_bound_no_int"], + dll_name="BOUND.dll", bound_state="bound_old_style", + original_first_thunk=0, first_thunk=0x4000, thunk_source=None)])) + assert _of(issues, ReasonCodes.IMPORT_DESCRIPTOR_INVALID)[0] == { + "index": 1, "dll_name": "BOUND.dll", + "bound_state": "bound_old_style", + "original_first_thunk": 0, "first_thunk": 0x4000, + "sub_reason": "names_unrecoverable_bound_no_int"} + + def test_no_thunk_array_details(self): + issues = validate_imports(_internal([_descriptor( + errors=["no_thunk_array"], original_first_thunk=0, + first_thunk=0, thunk_source=None)])) + d = _of(issues, ReasonCodes.IMPORT_DESCRIPTOR_INVALID)[0] + assert d["sub_reason"] == "no_thunk_array" + assert d["original_first_thunk"] == 0 + assert d["first_thunk"] == 0 + + def test_priority_order(self): + issues = validate_imports(_internal([_descriptor( + errors=["no_thunk_array", "names_unrecoverable_bound_no_int"])])) + assert _subs(issues, ReasonCodes.IMPORT_DESCRIPTOR_INVALID) == [ + "names_unrecoverable_bound_no_int"] + + def test_healthy_fallback_is_not_flagged(self): + """ + The critical negative: OriginalFirstThunk == 0 with a usable fallback + is legal and must produce nothing. Flagging it would misreport a large + fraction of legitimate binaries. + """ + issues = validate_imports(_internal([_descriptor( + errors=[], original_first_thunk=0, first_thunk=0x4000, + thunk_source="iat_fallback", + imports=[_entry()])])) + assert issues == [] + + +# ================================================================= +# Entry class +# ================================================================= + +class TestEntries: + + @pytest.mark.parametrize("tag", _ENTRY_ERROR_PRIORITY) + def test_every_priority_tag_emits(self, tag): + issues = validate_imports(_internal([ + _descriptor(imports=[_entry(errors=[tag])])])) + assert _subs(issues, ReasonCodes.IMPORT_ENTRY_INVALID) == [tag] + + def test_details_payload(self): + issues = validate_imports(_internal([_descriptor( + index=3, dll_name="USER32.dll", + imports=[_entry(index=7, errors=["ordinal_zero"], + is_ordinal=True, ordinal=0, name=None, + name_rva=None)])])) + assert _of(issues, ReasonCodes.IMPORT_ENTRY_INVALID)[0] == { + "descriptor_index": 3, "dll_name": "USER32.dll", + "entry_index": 7, "is_ordinal": True, "ordinal": 0, + "name": None, "name_rva": None, + "sub_reason": "ordinal_zero", "invalid_entry_count": 1} + + def test_clean_entries_emit_nothing(self): + issues = validate_imports(_internal([ + _descriptor(imports=[_entry(0), _entry(1), _entry(2)])])) + assert issues == [] + + def test_only_invalid_entries_reported(self): + issues = validate_imports(_internal([_descriptor(imports=[ + _entry(0), _entry(1, ["ordinal_zero"]), + _entry(2), _entry(3, ["name_empty"])])])) + details = _of(issues, ReasonCodes.IMPORT_ENTRY_INVALID) + assert [d["entry_index"] for d in details] == [1, 3] + + def test_invalid_entry_count_counts_invalid_not_total(self): + """ + The count is of INVALID entries, not of the whole import list - a + consumer reading it as a table size would be misled. + """ + issues = validate_imports(_internal([_descriptor(imports=[ + _entry(0), _entry(1, ["ordinal_zero"]), + _entry(2), _entry(3, ["name_empty"])])])) + assert all(d["invalid_entry_count"] == 2 + for d in _of(issues, ReasonCodes.IMPORT_ENTRY_INVALID)) + + @pytest.mark.parametrize("higher,lower", list( + zip(_ENTRY_ERROR_PRIORITY, _ENTRY_ERROR_PRIORITY[1:]))) + def test_priority_order_adjacent_pairs(self, higher, lower): + """ + Every ADJACENT pair, in both presentation orders. Pairing distant + tags would only prove that some ordering exists - swapping two + neighbours would go unnoticed. Driving the list itself also means a + newly inserted tag is covered automatically. + """ + for errors in ([higher, lower], [lower, higher]): + issues = validate_imports(_internal([ + _descriptor(imports=[_entry(errors=errors)])])) + assert _subs(issues, ReasonCodes.IMPORT_ENTRY_INVALID) == [higher] + + def test_entries_from_multiple_descriptors_are_attributed(self): + issues = validate_imports(_internal([ + _descriptor(index=0, dll_name="A.dll", + imports=[_entry(0, ["ordinal_zero"])]), + _descriptor(index=1, dll_name="B.dll", + imports=[_entry(0, ["name_empty"])]), + ])) + details = _of(issues, ReasonCodes.IMPORT_ENTRY_INVALID) + assert [(d["descriptor_index"], d["dll_name"]) for d in details] == [ + (0, "A.dll"), (1, "B.dll")] + + +class TestEntryEmissionCap: + + def test_below_cap_all_emitted(self): + n = _MAX_ENTRY_ISSUES_PER_DESCRIPTOR - 1 + issues = validate_imports(_internal([_descriptor( + imports=[_entry(i, ["ordinal_zero"]) for i in range(n)])])) + assert len(_of(issues, ReasonCodes.IMPORT_ENTRY_INVALID)) == n + + def test_at_cap_all_emitted(self): + n = _MAX_ENTRY_ISSUES_PER_DESCRIPTOR + issues = validate_imports(_internal([_descriptor( + imports=[_entry(i, ["ordinal_zero"]) for i in range(n)])])) + assert len(_of(issues, ReasonCodes.IMPORT_ENTRY_INVALID)) == n + + def test_above_cap_clamped_but_count_is_true(self): + n = _MAX_ENTRY_ISSUES_PER_DESCRIPTOR + 20 + issues = validate_imports(_internal([_descriptor( + imports=[_entry(i, ["ordinal_zero"]) for i in range(n)])])) + details = _of(issues, ReasonCodes.IMPORT_ENTRY_INVALID) + assert len(details) == _MAX_ENTRY_ISSUES_PER_DESCRIPTOR + assert all(d["invalid_entry_count"] == n for d in details) + + def test_cap_is_per_descriptor_not_global(self): + """ + A file with many malformed modules should not have later descriptors + silenced by earlier ones. + """ + n = _MAX_ENTRY_ISSUES_PER_DESCRIPTOR + 10 + bad = [_entry(i, ["ordinal_zero"]) for i in range(n)] + issues = validate_imports(_internal([ + _descriptor(index=0, imports=list(bad)), + _descriptor(index=1, imports=list(bad)), + ])) + details = _of(issues, ReasonCodes.IMPORT_ENTRY_INVALID) + per = {} + for d in details: + per[d["descriptor_index"]] = per.get(d["descriptor_index"], 0) + 1 + assert per == {0: _MAX_ENTRY_ISSUES_PER_DESCRIPTOR, + 1: _MAX_ENTRY_ISSUES_PER_DESCRIPTOR} + + +# ================================================================= +# Class independence +# ================================================================= + +class TestClassIndependence: + """ + The three pathology classes are orthogonal. These pin that they neither + suppress one another nor merge, which is what makes the single-anomaly + fixtures elsewhere in this file meaningful. + """ + + def test_dll_name_and_thunk_source_both_fire(self): + issues = validate_imports(_internal([ + _descriptor(errors=["dll_name_empty", "no_thunk_array"])])) + assert set(_codes(issues)) == { + ReasonCodes.IMPORT_DLL_NAME_INVALID, + ReasonCodes.IMPORT_DESCRIPTOR_INVALID} + + def test_descriptor_fault_does_not_suppress_entries(self): + issues = validate_imports(_internal([_descriptor( + errors=["dll_name_empty"], + imports=[_entry(errors=["ordinal_zero"])])])) + assert set(_codes(issues)) == { + ReasonCodes.IMPORT_DLL_NAME_INVALID, + ReasonCodes.IMPORT_ENTRY_INVALID} + + def test_truncations_do_not_suppress_descriptors(self): + issues = validate_imports(_internal( + descriptors=[_descriptor(errors=["dll_name_empty"])], + truncations=["int_truncated"])) + assert set(_codes(issues)) == { + ReasonCodes.IMPORT_TABLE_TRUNCATED, + ReasonCodes.IMPORT_DLL_NAME_INVALID} + + def test_all_three_classes_together(self): + issues = validate_imports(_internal( + descriptors=[_descriptor( + errors=["dll_name_empty", "no_thunk_array"], + imports=[_entry(errors=["ordinal_zero"])])], + truncations=["int_truncated"])) + assert set(_codes(issues)) == { + ReasonCodes.IMPORT_TABLE_TRUNCATED, + ReasonCodes.IMPORT_DLL_NAME_INVALID, + ReasonCodes.IMPORT_DESCRIPTOR_INVALID, + ReasonCodes.IMPORT_ENTRY_INVALID} + + +# ================================================================= +# Placement ownership +# ================================================================= + +class TestPlacementNotChecked: + """ + Placement is owned by the RVA-graph backbone. This validator must not + assert it, or the two would double-count on every malformed directory. + """ + + def test_no_metadata_dependency(self): + assert validate_imports._depends_on == ("internal",) + + def test_absurd_placement_emits_nothing(self): + internal = _internal([_descriptor()]) + internal["import_struct"]["rva"] = 0x99999999 + internal["import_struct"]["size"] = 0x99999999 + assert validate_imports(internal) == [] + + +# ================================================================= +# Output contract +# ================================================================= + +class TestOutputContract: + + def test_issue_shape(self): + issues = validate_imports(_internal( + descriptors=[_descriptor(errors=["dll_name_empty"], + imports=[_entry(errors=["ordinal_zero"])])], + truncations=["int_truncated"])) + assert issues + for issue in issues: + assert set(issue) == {"issue", "details"} + assert isinstance(issue["issue"], str) + assert isinstance(issue["details"], dict) + + def test_no_reserved_reason_key(self): + """ + "reason" is reserved by the heuristics emission layer; a details key + of that name would overwrite the parent code. + """ + issues = validate_imports(_internal( + descriptors=[_descriptor( + errors=["dll_name_empty", "no_thunk_array"], + imports=[_entry(errors=["ordinal_zero"])])], + truncations=["int_truncated"])) + assert issues + offenders = [i["issue"] for i in issues if "reason" in i["details"]] + assert not offenders + + def test_json_serialisable(self): + import json + json.dumps(validate_imports(_internal( + descriptors=[_descriptor(errors=["dll_name_empty"], + imports=[_entry(errors=["name_empty"])])], + truncations=["int_truncated"]))) + + def test_input_not_mutated(self): + import copy + internal = _internal( + descriptors=[_descriptor(errors=["dll_name_empty"], + imports=[_entry(errors=["ordinal_zero"])])], + truncations=["int_truncated"]) + snapshot = copy.deepcopy(internal) + validate_imports(internal) + assert internal == snapshot + + +# ================================================================= +# Determinism +# ================================================================= + +class TestDeterminism: + + def test_repeated_validation_identical(self): + import json + internal = _internal( + descriptors=[ + _descriptor(index=0, errors=["dll_name_empty"], + imports=[_entry(0, ["ordinal_zero"]), + _entry(1, ["name_empty"])]), + _descriptor(index=1, errors=["no_thunk_array"]), + ], + truncations=["int_truncated", "iat_fallback_read_failed"]) + first = json.dumps(validate_imports(internal), sort_keys=True) + for _ in range(20): + assert json.dumps(validate_imports(internal), + sort_keys=True) == first + + def test_emission_order_is_truncations_then_descriptors(self): + issues = validate_imports(_internal( + descriptors=[_descriptor(errors=["dll_name_empty"])], + truncations=["int_truncated"])) + assert _codes(issues) == [ReasonCodes.IMPORT_TABLE_TRUNCATED, + ReasonCodes.IMPORT_DLL_NAME_INVALID] + + def test_per_descriptor_order_is_name_then_source_then_entries(self): + issues = validate_imports(_internal([_descriptor( + errors=["dll_name_empty", "no_thunk_array"], + imports=[_entry(errors=["ordinal_zero"])])])) + assert _codes(issues) == [ + ReasonCodes.IMPORT_DLL_NAME_INVALID, + ReasonCodes.IMPORT_DESCRIPTOR_INVALID, + ReasonCodes.IMPORT_ENTRY_INVALID] diff --git a/tests/unit/validators/test_validator_imports_ru.py b/tests/unit/validators/test_validator_imports_ru.py new file mode 100644 index 0000000..06d411b --- /dev/null +++ b/tests/unit/validators/test_validator_imports_ru.py @@ -0,0 +1,249 @@ +# Copyright (c) 2026 MalX Labs and contributors +# SPDX-License-Identifier: MPL-2.0 + +""" +Coverage for the `reason == "unknown"` branch in _validate_entries. + +This branch IS the silent-drop mechanism this workstream has spent its time +closing: an entry carrying only tags absent from _ENTRY_ERROR_PRIORITY is +skipped entirely - no issue, and no contribution to invalid_entry_count. + +It is UNREACHABLE with current parser output. Every tag pe_imports can place +in entry["errors"] appears in the priority list, verified statically by the +tag-contract check. The branch exists to absorb a future parser tag that +someone forgets to register, which is precisely how `empty_read`, +`dll_name_empty`, `dll_name_too_long` and `ordinal_index_duplicate` were all +lost in other subsystems. + +These tests therefore pin behaviour that is defensible but not obviously +desirable. They exist so that: + * the branch is covered rather than dead; + * the consequence (total invisibility, including in the count) is written + down rather than discovered; + * a change to that behaviour is a deliberate decision, not a drift. + +The real guard against the drop is the tag-contract test, which fails CI +when a parser gains a tag no validator consumes. This suite is the +belt-and-braces record of what happens if that guard is ever bypassed. +""" + +from __future__ import annotations + +from typing import Any, Dict, List, Optional + +import pytest + +from iocx.reason_codes import ReasonCodes +from iocx.validators.imports import ( + validate_imports, + _ENTRY_ERROR_PRIORITY, + _MAX_ENTRY_ISSUES_PER_DESCRIPTOR, +) + + +# Every tag pe_imports can place in entry["errors"]. Kept as a literal so a +# parser change that adds a tag makes test_branch_is_unreachable_today fail, +# rather than the list silently tracking the parser. +_PARSER_ENTRY_TAGS = frozenset({ + "ordinal_zero", "name_rva_zero", "name_read_failed", "name_too_short", + "hint_unpack_failed", "name_unterminated", "name_non_ascii", + "name_empty", "name_not_printable", +}) + +_UNKNOWN = "some_future_parser_tag" + + +# ================================================================= +# Builders +# ================================================================= + +def _entry(index: int = 0, errors: Optional[List[str]] = None, + **kw) -> Dict[str, Any]: + e = {"index": index, "errors": errors or [], "is_ordinal": False, + "ordinal": None, "name": "Fn", "name_rva": 0x5000, + "name_valid": True, "hint": 1, "thunk_value": 0x5000} + e.update(kw) + return e + + +def _descriptor(index: int = 0, imports: Optional[List[Dict]] = None, + errors: Optional[List[str]] = None, + dll_name: str = "KERNEL32.dll") -> Dict[str, Any]: + return {"index": index, "errors": errors or [], "dll_name": dll_name, + "dll_name_valid": True, "name_rva": 0x2000, + "bound_state": "unbound", "original_first_thunk": 0x3000, + "first_thunk": 0x4000, "thunk_source": "int", + "imports": imports or []} + + +def _internal(descriptors: List[Dict[str, Any]]) -> Dict[str, Any]: + return {"import_struct": { + "rva": 0x1000, "size": 60, "is_64bit": True, + "descriptors": descriptors, "descriptor_count": len(descriptors), + "truncations": [], "errors": []}} + + +def _entry_issues(issues) -> List[Dict[str, Any]]: + return [i["details"] for i in issues + if i["issue"] == ReasonCodes.IMPORT_ENTRY_INVALID] + + +# ================================================================= +# Reachability +# ================================================================= + +class TestBranchReachability: + + def test_branch_is_unreachable_with_current_parser_tags(self): + """ + Guard on the guard: every tag pe_imports can emit must appear in the + priority list, so the unknown branch cannot fire in production. If + this fails, a parser tag has been added without registering it - and + the entries carrying it are being silently dropped right now. + """ + unregistered = _PARSER_ENTRY_TAGS - set(_ENTRY_ERROR_PRIORITY) + assert not unregistered, ( + f"parser tags missing from _ENTRY_ERROR_PRIORITY: " + f"{sorted(unregistered)} - entries carrying these are dropped") + + def test_priority_list_has_no_phantom_entries(self): + """The inverse: a listed tag no parser produces implies a routing + that does not exist.""" + phantom = set(_ENTRY_ERROR_PRIORITY) - _PARSER_ENTRY_TAGS + assert not phantom, f"unreachable tags in priority list: {sorted(phantom)}" + + +# ================================================================= +# The branch itself +# ================================================================= + +class TestUnknownEntryTagIsDropped: + + def test_entry_with_only_unknown_tags_emits_nothing(self): + issues = validate_imports(_internal([ + _descriptor(imports=[_entry(0, [_UNKNOWN])])])) + assert _entry_issues(issues) == [] + + def test_multiple_unknown_tags_on_one_entry(self): + issues = validate_imports(_internal([ + _descriptor(imports=[_entry(0, [_UNKNOWN, "another_unknown"])])])) + assert _entry_issues(issues) == [] + + def test_all_entries_unknown_takes_the_empty_return(self): + """ + With nothing collected, `if not invalid: return` fires before the + emission loop - a distinct path from an empty imports list. + """ + issues = validate_imports(_internal([ + _descriptor(imports=[_entry(0, [_UNKNOWN]), + _entry(1, ["yet_another"])])])) + assert _entry_issues(issues) == [] + + def test_dropped_entry_is_excluded_from_invalid_entry_count(self): + """ + The `continue` precedes the append, so a dropped entry is invisible + in every respect - not merely unreported, but uncounted. A consumer + reading invalid_entry_count cannot tell that anything was skipped. + """ + issues = validate_imports(_internal([_descriptor(imports=[ + _entry(0, ["ordinal_zero"]), + _entry(1, [_UNKNOWN]), + _entry(2, ["name_empty"]), + ])])) + details = _entry_issues(issues) + assert [d["entry_index"] for d in details] == [0, 2] + assert all(d["invalid_entry_count"] == 2 for d in details), ( + "the unknown entry must not inflate the count either") + + def test_clean_entries_and_unknown_entries_are_indistinguishable(self): + """ + An entry with no errors and an entry with only unknown errors produce + identical output. This is the property that makes the drop dangerous, + and the reason the tag-contract check exists upstream. + """ + with_unknown = validate_imports(_internal([ + _descriptor(imports=[_entry(0, [_UNKNOWN])])])) + with_clean = validate_imports(_internal([ + _descriptor(imports=[_entry(0, [])])])) + assert with_unknown == with_clean == [] + + +# ================================================================= +# Rescue: a known tag anywhere on the entry +# ================================================================= + +class TestKnownTagRescuesTheEntry: + """ + _first_matching scans the PRIORITY list, not the entry's error list, so + order within the entry is irrelevant - one recognised tag is enough. + """ + + @pytest.mark.parametrize("errors,expected", [ + ([_UNKNOWN, "ordinal_zero"], "ordinal_zero"), + (["ordinal_zero", _UNKNOWN], "ordinal_zero"), + ([_UNKNOWN, "name_empty", "another_unknown"], "name_empty"), + ]) + def test_known_tag_wins_regardless_of_position(self, errors, expected): + issues = validate_imports(_internal([ + _descriptor(imports=[_entry(0, errors)])])) + details = _entry_issues(issues) + assert len(details) == 1 + assert details[0]["sub_reason"] == expected + + def test_priority_still_applies_among_known_tags(self): + """An unknown tag does not disturb the ordering of the known ones.""" + issues = validate_imports(_internal([ + _descriptor(imports=[ + _entry(0, [_UNKNOWN, "name_empty", "ordinal_zero"])])])) + assert _entry_issues(issues)[0]["sub_reason"] == "ordinal_zero" + + +# ================================================================= +# Interaction with the emission cap +# ================================================================= + +class TestInteractionWithCap: + + def test_dropped_entries_do_not_consume_cap_budget(self): + """ + Because the drop happens during collection, unknown entries never + reach the capped slice - so a table padded with unknown tags cannot + squeeze out reportable ones. + """ + n = _MAX_ENTRY_ISSUES_PER_DESCRIPTOR + imports = [] + for i in range(n): + imports.append(_entry(len(imports), [_UNKNOWN])) + imports.append(_entry(len(imports), ["ordinal_zero"])) + issues = validate_imports(_internal([_descriptor(imports=imports)])) + details = _entry_issues(issues) + assert len(details) == n + assert all(d["sub_reason"] == "ordinal_zero" for d in details) + assert all(d["invalid_entry_count"] == n for d in details) + + +# ================================================================= +# Descriptor level uses the inverse shape +# ================================================================= + +class TestDescriptorLevelUnknown: + """ + _validate_descriptors tests `if reason != "unknown"` rather than + `continue`-ing, so an unrecognised descriptor tag is also dropped - but + via a different code shape. Pinned here so both are covered. + """ + + def test_unknown_descriptor_tag_emits_nothing(self): + issues = validate_imports(_internal([ + _descriptor(errors=[_UNKNOWN])])) + assert issues == [] + + def test_unknown_descriptor_tag_does_not_block_entry_reporting(self): + """The two levels are independent: a dropped descriptor tag must not + suppress a reportable entry beneath it.""" + issues = validate_imports(_internal([ + _descriptor(errors=[_UNKNOWN], + imports=[_entry(0, ["ordinal_zero"])])])) + details = _entry_issues(issues) + assert len(details) == 1 + assert details[0]["sub_reason"] == "ordinal_zero" diff --git a/tests/unit/validators/test_validator_resources.py b/tests/unit/validators/test_validator_resources.py index 19ec6d1..f524fba 100644 --- a/tests/unit/validators/test_validator_resources.py +++ b/tests/unit/validators/test_validator_resources.py @@ -8,7 +8,8 @@ The resource tree comes from internal["resources_struct"]; sections, file_size and overlay_offset from analysis. Note the validator reads those three with DIRECT SUBSCRIPTS (`analysis["sections"]`), so a missing key raises KeyError -rather than degrading - pinned below. +rather than degrading - pinned below. Errors on a node drives +RESOURCE_DIRECTORY_ENTRY_UNREADABLE. Fixture note: a data leaf is only well-formed at depth 2 (the Language layer). Any leaf hung directly off the root is at depth 0 and therefore ALSO trips @@ -26,7 +27,7 @@ import pytest -from iocx.validators.resources import validate_resources +from iocx.validators.resources import validate_resources, _DIRECTORY_ERROR_PRIORITY from iocx.reason_codes import ReasonCodes @@ -38,6 +39,10 @@ FILE_SIZE = 0x10000 OVERLAY = 0xF000 +UNREADABLE = ReasonCodes.RESOURCE_DIRECTORY_ENTRY_UNREADABLE +UNREADABLE_ST = ReasonCodes.RESOURCE_STRING_TABLE_UNREADABLE +CORRUPT_ST = ReasonCodes.RESOURCE_STRING_TABLE_CORRUPT + # ================================================================= # Builders @@ -71,9 +76,10 @@ def _leaf(data_rva: Any = 0x1100, data_size: Any = 0x50, def _node(rva: int, size: int = 24, - entries: Optional[List[Dict[str, Any]]] = None) -> Dict[str, Any]: + entries: Optional[List[Dict[str, Any]]] = None, + errors: Optional[List[str]] = None) -> Dict[str, Any]: """A directory NODE (the thing `directory` points at).""" - return {"rva": rva, "size": size, "entries": entries or []} + return {"rva": rva, "size": size, "entries": entries or [], "errors": errors or []} def _subdir(target: Dict[str, Any], name: Any = None, @@ -85,13 +91,13 @@ def _subdir(target: Dict[str, Any], name: Any = None, def _tree(leaves: List[Dict[str, Any]], - lang_rva: int = 0x1080) -> Dict[str, Any]: + lang_rva: int = 0x1080, lang_errors: Optional[List[str]] = None) -> Dict[str, Any]: """ A correctly-shaped Type -> Name -> Language tree whose Language directory (depth 2) holds `leaves`. Lets data-entry checks be tested without also tripping RESOURCE_DATA_AT_INVALID_DEPTH. """ - lang = _node(lang_rva, 24, leaves) + lang = _node(lang_rva, 24, leaves, errors=lang_errors) name = _node(0x1040, 24, [_subdir(lang)]) return _node(0x1000, 24, [_subdir(name)]) @@ -108,6 +114,21 @@ def _run(root: Dict[str, Any], analysis or _analysis()) +def _run_struct(root: Dict[str, Any], + errors: Optional[List[str]] = None, + string_tables: Optional[List[Dict[str, Any]]] = None, + analysis: Optional[Dict[str, Any]] = None): + """ + _run for tests needing a struct-level `errors` list - the sink + `string_table_walk_failed` lands in, distinct from a directory node's + own errors. + """ + resources = {"root": root, "errors": errors or []} + if string_tables is not None: + resources["string_tables"] = string_tables + return _run(root, analysis, resources=resources) + + def make_issue_list(result) -> List[str]: return [i["issue"] for i in result] @@ -116,6 +137,43 @@ def _details_for(issues, code) -> List[Dict[str, Any]]: return [i["details"] for i in issues if i["issue"] == code] +def _directory(entries: Optional[List[Dict[str, Any]]] = None, + rva: int = 0x1000, size: Optional[int] = None, + errors: Optional[List[str]] = None) -> Dict[str, Any]: + entries = entries or [] + return {"rva": rva, "size": size if size is not None else 16 + 8 * len(entries), + "entries": entries, "errors": errors or []} + + +def _dir_entry(directory: Dict[str, Any], entry_id: Optional[int] = 1, + name: Optional[str] = None) -> Dict[str, Any]: + return {"name": name, "id": entry_id, "is_directory": True, + "directory": directory, "data_rva": None, "data_size": None, + "raw_offset": None} + + +def _data_entry(data_rva: int = 0x1100, data_size: int = 0x40, + raw_offset: int = 0x500, entry_id: Optional[int] = 0x409, + name: Optional[str] = None) -> Dict[str, Any]: + return {"name": name, "id": entry_id, "is_directory": False, + "directory": None, "data_rva": data_rva, "data_size": data_size, + "raw_offset": raw_offset} + + +def _internal(root: Optional[Dict[str, Any]] = None, + string_tables: Optional[List[Dict[str, Any]]] = None, + errors: Optional[List[str]] = None) -> Dict[str, Any]: + return {"resources_struct": { + "root": root if root is not None else _directory(), + "string_tables": string_tables or [], + "errors": errors or [], + }} + + +def _codes(issues) -> List[str]: + return [i["issue"] for i in issues] + + # ================================================================= # Absence / early return # ================================================================= @@ -270,6 +328,145 @@ def test_sibling_directories_at_distinct_rvas_are_fine(self): assert _run(root) == [] +class TestDirectoryEntryUnreadable: + """ + The parser skips an entry it cannot decode rather than aborting. Without + these checks the skip is invisible: the directory reports fewer entries + than its declared size implies and nothing says why. + """ + + def test_entry_decode_failed_emits(self): + issues = _run(_node(0x1000, 24, [], errors=["entry_decode_failed"])) + assert make_issue_list(issues) == [UNREADABLE] + + def test_directory_entries_unavailable_emits(self): + issues = _run(_node(0x1000, 16, [], + errors=["directory_entries_unavailable"])) + assert make_issue_list(issues) == [UNREADABLE] + + def test_details_expose_the_declared_decoded_gap(self): + """ + `size` reflects the DECLARED entry count and `decoded_entries` the + number actually built, so their difference is the loss. Neither alone + conveys it. + """ + issues = _run(_node(0x1000, 32, [], errors=["entry_decode_failed"])) + assert _details_for(issues, UNREADABLE)[0] == { + "rva": 0x1000, "depth": 0, "sub_reason": "entry_decode_failed", + "failed_entry_count": 1, "declared_size": 32, + "decoded_entries": 0} + + def test_failed_entry_count_counts_repeats(self): + issues = _run(_node(0x1000, 40, [], + errors=["entry_decode_failed"] * 3)) + assert _details_for(issues, UNREADABLE)[0]["failed_entry_count"] == 3 + + def test_one_issue_per_directory(self): + """Priority-resolved: several tags still produce a single issue.""" + issues = _run(_node(0x1000, 16, [], + errors=["entry_decode_failed", + "directory_entries_unavailable"])) + assert len(_details_for(issues, UNREADABLE)) == 1 + + def test_priority_unavailable_beats_decode_failed(self): + """ + An unreadable entry LIST subsumes any per-entry failure - no entry + was reached at all. + """ + issues = _run(_node(0x1000, 16, [], + errors=["entry_decode_failed", + "directory_entries_unavailable"])) + assert _details_for(issues, UNREADABLE)[0]["sub_reason"] == ( + "directory_entries_unavailable") + + @pytest.mark.parametrize("higher,lower", list( + zip(_DIRECTORY_ERROR_PRIORITY, _DIRECTORY_ERROR_PRIORITY[1:]))) + def test_priority_order_adjacent_pairs(self, higher, lower): + """ + Adjacent pairs in both presentation orders. Pairing distant tags + would only prove that some ordering exists. + """ + for errors in ([higher, lower], [lower, higher]): + issues = _run(_node(0x1000, 16, [], errors=errors)) + assert _details_for(issues, UNREADABLE)[0]["sub_reason"] == higher + + def test_priority_list_content(self): + """ + Order is a contract, and a behavioural test cannot catch a reorder: + _first_matching is definitionally consistent with whatever order the + list happens to have, so fixtures derived from it move with it. + """ + assert _DIRECTORY_ERROR_PRIORITY == [ + "directory_entries_unavailable", + "entry_decode_failed", + ] + + def test_unknown_tag_is_ignored(self): + assert _run(_node(0x1000, 24, [], errors=["some_future_tag"])) == [] + + def test_missing_errors_key_tolerated(self): + """A struct predating the schema change must not raise.""" + node = _node(0x1000, 24, []) + del node["errors"] + assert _run(node) == [] + + def test_nested_directory_errors_reported_at_depth(self): + root = _tree([_leaf()], lang_errors=["entry_decode_failed"]) + details = _details_for(_run(root), UNREADABLE)[0] + assert details["depth"] == 2 + assert details["rva"] == 0x1080 + + def test_each_failing_directory_reports_separately(self): + lang = _node(0x1080, 24, [_leaf()], errors=["entry_decode_failed"]) + name = _node(0x1040, 24, [_subdir(lang)], + errors=["entry_decode_failed"]) + root = _node(0x1000, 24, [_subdir(name)]) + assert [d["depth"] for d in _details_for(_run(root), UNREADABLE)] == [ + 1, 2] + + +class TestDirectoryErrorSuppression: + """ + The errors report sits after the placement and loop checks, so a + directory the validator declines to process does not also report its + contents. + """ + + def test_out_of_bounds_directory_does_not_report_errors(self): + issues = _run(_node(0x9999, 16, [], errors=["entry_decode_failed"])) + assert make_issue_list(issues) == [ + ReasonCodes.RESOURCE_DIRECTORY_OUT_OF_BOUNDS] + + def test_looping_directory_does_not_report_errors_twice(self): + """ + A directory reached twice reports the loop on the second visit, not + a duplicate decode failure. + """ + shared = _node(0x1080, 24, [], errors=["entry_decode_failed"]) + root = _node(0x1000, 24, [_subdir(shared), _subdir(shared)]) + issues = _run(root) + assert len(_details_for(issues, UNREADABLE)) == 1 + assert ReasonCodes.RESOURCE_DIRECTORY_LOOP in make_issue_list(issues) + + def test_errors_do_not_suppress_the_entry_walk(self): + """ + A directory with a decode failure still has its SURVIVING entries + validated - the tag reports what was lost, not a reason to stop. + """ + root = _tree([_leaf(0x9999, 0x50)], + lang_errors=["entry_decode_failed"]) + codes = make_issue_list(_run(root)) + assert UNREADABLE in codes + assert ReasonCodes.RESOURCE_DATA_OUT_OF_BOUNDS in codes + + def test_directory_errors_precede_entry_findings(self): + """Emission order: the directory's own fault before its contents'.""" + root = _tree([_leaf(0x9999, 0x50)], + lang_errors=["entry_decode_failed"]) + codes = make_issue_list(_run(root)) + assert codes.index(UNREADABLE) < codes.index( + ReasonCodes.RESOURCE_DATA_OUT_OF_BOUNDS) + # ================================================================= # Language layer (depth 2) @@ -636,6 +833,89 @@ def test_string_tables_checked_even_when_tree_is_clean(self): ReasonCodes.RESOURCE_STRING_TABLE_CORRUPT] +class TestStringTableUnreadable: + + def test_walk_failure_emits(self): + """ + Regression guard: without the membership test the tag is dropped and + a malformed RT_STRING subtree reports clean. + """ + issues = _run_struct(_node(0x1000, 24, []), + errors=["string_table_walk_failed"]) + assert make_issue_list(issues) == [UNREADABLE_ST] + + def test_details_carry_the_collected_count(self): + """ + A non-empty list is not proof the walk finished, so the count is + reported alongside the failure. + """ + issues = _run_struct(_node(0x1000, 24, []), + errors=["string_table_walk_failed"], + string_tables=[{"rva": 0x1100, "size": 0x40}]) + assert _details_for(issues, UNREADABLE_ST)[0] == { + "sub_reason": "walk_failed", "collected_tables": 1} + + def test_clean_errors_emit_nothing(self): + assert _run_struct(_node(0x1000, 24, [])) == [] + + def test_empty_string_tables_without_the_tag_is_silent(self): + """ + The distinction the tag exists for: a binary with no string + resources must stay silent. + """ + assert _run_struct(_node(0x1000, 24, []), string_tables=[]) == [] + + def test_unrelated_tag_does_not_emit(self): + assert _run_struct(_node(0x1000, 24, []), + errors=["some_other_tag"]) == [] + + def test_missing_struct_errors_key_tolerated(self): + """`resources.get("errors") or []` - unlike the analysis keys.""" + assert _run(_node(0x1000, 24, [])) == [] + + +class TestStringTableIndependence: + """ + UNREADABLE (could not enumerate) and CORRUPT (a table is misplaced) are + different facts and must co-fire on a partial walk. + """ + + def test_both_fire_together(self): + issues = _run_struct(_node(0x1000, 24, []), + errors=["string_table_walk_failed"], + string_tables=[{"rva": 0x9999, "size": 0x40}]) + assert set(make_issue_list(issues)) == {UNREADABLE_ST, CORRUPT_ST} + + def test_unreadable_does_not_return_early(self): + """ + The tag check must not suppress the table bounds loop. Note the + in-bounds fixture here would pass either way; test_both_fire_together + above is what actually catches an early return. + """ + issues = _run_struct(_node(0x1000, 24, []), + errors=["string_table_walk_failed"], + string_tables=[{"rva": 0x1100, "size": 0x40}]) + assert make_issue_list(issues) == [UNREADABLE_ST] + + def test_unreadable_emitted_before_corrupt(self): + issues = _run_struct(_node(0x1000, 24, []), + errors=["string_table_walk_failed"], + string_tables=[{"rva": 0x9999, "size": 0x40}]) + assert make_issue_list(issues)[0] == UNREADABLE_ST + + def test_corrupt_alone_without_walk_failure(self): + issues = _run(_node(0x1000, 24, []), + string_tables=[{"rva": 0x9999, "size": 0x40}]) + assert make_issue_list(issues) == [CORRUPT_ST] + + def test_corrupt_breaks_after_the_first(self): + """One issue per file, not per table.""" + issues = _run(_node(0x1000, 24, []), + string_tables=[{"rva": 0x9999, "size": 0x40}, + {"rva": 0x8888, "size": 0x40}]) + assert len(_details_for(issues, CORRUPT_ST)) == 1 + + # ================================================================= # Output contract # ================================================================= @@ -737,3 +1017,13 @@ def test_entries_processed_in_order(self): rvas = [d["data_rva"] for d in _details_for( issues, ReasonCodes.RESOURCE_DATA_OUT_OF_BOUNDS)] assert rvas == [0x9991, 0x9992] + + def test_directory_errors_precede_entry_findings(self): + lang = _node(0x1080, 24, [_leaf(0x9999, 0x50)]) + lang["errors"] = ["entry_decode_failed"] + name = _node(0x1040, 24, [_subdir(lang)]) + root = _node(0x1000, 24, [_subdir(name)]) + codes = make_issue_list(_run(root)) + assert codes.index( + ReasonCodes.RESOURCE_DIRECTORY_ENTRY_UNREADABLE) < codes.index( + ReasonCodes.RESOURCE_DATA_OUT_OF_BOUNDS) diff --git a/tests/unit/validators/test_validator_version_info.py b/tests/unit/validators/test_validator_version_info.py index 3751416..95253d2 100644 --- a/tests/unit/validators/test_validator_version_info.py +++ b/tests/unit/validators/test_validator_version_info.py @@ -35,8 +35,9 @@ import pytest from iocx.reason_codes import ReasonCodes -from iocx.validators.version_info import validate_version_info +from iocx.validators.version_info import validate_version_info, _CHILD_ERROR_PRIORITY +_HEADER = ReasonCodes.RESOURCE_VERSIONINFO_INVALID_HEADER # ================================================================= # Input builders @@ -139,6 +140,15 @@ def _placement_details(issues) -> List[Dict[str, Any]]: ] +def _sub_reasons(issues): + return [d.get("sub_reason") for d in _details_for(issues, _HEADER)] + + +def _run(vi, analysis=None): + return validate_version_info({"version_info_struct": vi}, + analysis or _make_analysis()) + + # ================================================================= # Absence / clean cases # ================================================================= @@ -595,6 +605,153 @@ def test_placement_plus_sfi_plus_vfi_all_emit(self): assert ReasonCodes.RESOURCE_VERSIONINFO_INVALID_VARFILEINFO in codes +# ================================================================= +# Child dispatch +# ================================================================= + +class TestChildDispatch: + + @pytest.mark.parametrize("tag", _CHILD_ERROR_PRIORITY) + def test_every_child_tag_emits(self, tag): + """ + Regression guard: each of these previously produced NO issue at all, + because they are appended after decoded=True. + """ + issues = _run(_make_vi(errors=[tag])) + assert _sub_reasons(issues) == [tag] + + def test_details_carry_all_matching_tags(self): + """ + unknown_child can repeat, and the count is the signal - a blob with + five unrecognised children is more suspicious than one. + """ + issues = _run(_make_vi(errors=["unknown_child"] * 3)) + details = _details_for(issues, _HEADER) + assert len(details) == 1 + assert details[0]["errors"] == ["unknown_child"] * 3 + + def test_one_issue_per_blob(self): + issues = _run(_make_vi(errors=["unknown_child", "child_length_invalid"])) + assert len(_details_for(issues, _HEADER)) == 1 + + @pytest.mark.parametrize("errors,expected", [ + (["unknown_child", "child_length_invalid"], "child_length_invalid"), + (["child_length_invalid", "unknown_child"], "child_length_invalid"), + (["unknown_child", "child_header_unpack"], "child_header_unpack"), + (["child_header_unpack", "child_length_invalid"], "child_header_unpack"), + ]) + def test_walk_terminating_faults_win(self, errors, expected): + """ + A fault that stopped the walk outranks one that did not: the + remaining children were never examined. + """ + assert _sub_reasons(_run(_make_vi(errors=errors))) == [expected] + + @pytest.mark.parametrize("higher,lower", list( + zip(_CHILD_ERROR_PRIORITY, _CHILD_ERROR_PRIORITY[1:]))) + def test_priority_order_adjacent_pairs(self, higher, lower): + for errors in ([higher, lower], [lower, higher]): + assert _sub_reasons(_run(_make_vi(errors=errors))) == [higher] + + def test_priority_list_content(self): + """ + Order is a contract; a behavioural test cannot catch a reorder, + since _first_matching is definitionally consistent with whatever + order the list has. + """ + assert _CHILD_ERROR_PRIORITY == [ + "child_max_exceeded", + "child_header_unpack", + "child_length_invalid", + "unknown_child", + ] + + def test_clean_blob_emits_nothing(self): + assert _run(_make_vi()) == [] + + def test_unrelated_tags_ignored(self): + issues = _run(_make_vi(errors=["some_future_tag"])) + assert _codes(issues) == [] + + def test_ffi_tag_does_not_trigger_the_child_branch(self): + """ + The two branches read the same list with different filters and must + not overlap: fixed_file_info_* belongs to the FFI branch alone. + """ + issues = _run(_make_vi(errors=["fixed_file_info_truncated"], + fixed_file_info=None)) + assert _sub_reasons(issues) == [] + assert ReasonCodes.RESOURCE_VERSIONINFO_INVALID_FIXEDINFO in _codes(issues) + + def test_child_and_ffi_tags_both_reported(self): + issues = _run(_make_vi( + errors=["unknown_child", "fixed_file_info_truncated"], + fixed_file_info=None)) + assert "unknown_child" in _sub_reasons(issues) + assert ReasonCodes.RESOURCE_VERSIONINFO_INVALID_FIXEDINFO in _codes(issues) + + def test_child_details_exclude_non_child_tags(self): + issues = _run(_make_vi( + errors=["unknown_child", "fixed_file_info_truncated"], + fixed_file_info=None)) + child = [d for d in _details_for(issues, _HEADER) + if d.get("sub_reason") == "unknown_child"][0] + assert child["errors"] == ["unknown_child"] + + +class TestChildDispatchInteraction: + + def test_undecoded_short_circuits_the_child_branch(self): + """ + When decoded is False the undecoded branch forwards the whole list + and returns, so the child branch must not also fire. + """ + issues = _run(_make_vi(decoded=False, + errors=["unknown_child", "too_short"])) + assert _sub_reasons(issues) == ["undecoded"] + assert _details_for(issues, _HEADER)[0]["errors"] == [ + "unknown_child", "too_short"] + + def test_child_branch_coexists_with_header_checks(self): + issues = _run(_make_vi(header_ok=False, length_consistent=False, + errors=["unknown_child"])) + assert _sub_reasons(issues) == [ + "szkey_mismatch", "length_inconsistent", "unknown_child"] + + def test_child_branch_follows_placement(self): + """Emission order: placement, header checks, then child dispatch.""" + issues = _run(_make_vi(rva=0x5000, errors=["unknown_child"])) + assert _sub_reasons(issues) == ["placement", "unknown_child"] + + def test_child_branch_does_not_suppress_sfi_or_vfi(self): + issues = _run(_make_vi( + errors=["unknown_child"], + string_file_info=[{"tables": [], "errors": ["string_table_header"]}], + var_file_info=[{"vars": [], "errors": ["var_header"]}])) + codes = _codes(issues) + assert _HEADER in codes + assert ReasonCodes.RESOURCE_VERSIONINFO_INVALID_STRINGFILEINFO in codes + assert ReasonCodes.RESOURCE_VERSIONINFO_INVALID_VARFILEINFO in codes + + def test_missing_errors_key_tolerated(self): + vi = _make_vi() + del vi["errors"] + assert _run(vi) == [] + + def test_no_reserved_reason_key(self): + issues = _run(_make_vi(errors=["unknown_child"])) + assert issues + assert all("reason" not in i["details"] for i in issues) + + def test_deterministic(self): + import json + vi = _make_vi(errors=["unknown_child", "child_length_invalid"], + header_ok=False) + first = json.dumps(_run(vi), sort_keys=True) + for _ in range(20): + assert json.dumps(_run(vi), sort_keys=True) == first + + # ================================================================= # Output shape contract # =================================================================