Retire the Media Parser Fuzz Gate, Since Its Inputs Are Trusted

.github/actions/validate/action.yml

@@ -30,11 +30,6 @@ runs:
       shell: bash
       run: python3 checks/media_metadata_gate.py
 
-    # The carried tree is all well formed, so this feeds the same parsers seeded malformed variants.
-    - name: Fuzz media parsers step
-      shell: bash
-      run: python3 checks/media_parser_fuzz_gate.py
-
     # A redacted image replaced by its original passes every other check, so this compares hashes.
     - name: Check media redactions step
       shell: bash

README.md

@@ -142,7 +142,7 @@ A post is public the moment it deploys, and a photo or a pasted configuration ca
 - **Redactions.** A house number, a face, or a serial number is covered with a flat opaque fill, never a blur. A fill on an image already published is declared in [`checks/media-redactions.json`][media-redactions] and applied by a script. [`checks/media_redactions_gate.py`][media-redactions-gate] fails when a redacted file no longer matches its recorded hash, which is how a quietly restored original is caught.
 - **Text.** [`checks/text_pii_gate.py`][text-pii-gate] reads every page and post under `content/` outside the imported archive years, front matter and code blocks included. It reports email addresses, hardware addresses, public IP addresses, coordinates, street addresses, and phone numbers. It also reports a URL that carries coordinates or names one specific sensor or station. A finding names the file, the line, and the class, never the value, so the CI log does not republish it. A value a post prints on purpose is declared in an allow file with a reason, and an entry that stops matching fails the gate.
 
-A fuzzer feeds the metadata parsers malformed images in the same action, because the carried tree is all well formed and would never exercise a refusal. What no gate can judge, whether a picture locates the home or identifies a person, is the checklist in [CONTENT.md][content].
+What no gate can judge, whether a picture locates the home or identifies a person, is the checklist in [CONTENT.md][content].
 
 ### What Happens While the Site Runs
 

checks/README.md

@@ -13,7 +13,6 @@ This README covers the URL contract alone. The other checks in this directory ea
 | [`media_metadata_gate.py`][media-metadata-gate] | carried media is not in a form the gate can vouch for |
 | [`media_redactions_gate.py`][media-redactions-gate] | a file declared in `media-redactions.json` is not its redacted result |
 | [`text_pii_gate.py`][text-pii-gate] | a Markdown file under `content/`, outside the imported archive years, carries personal data in its text |
-| [`media_parser_fuzz_gate.py`][media-parser-fuzz-gate] | malformed input breaks the media gate or its normalizer |
 
 ## Why this exists
 
@@ -200,4 +199,3 @@ Three traps in that adjudication, each of which produced a wrong answer first an
 [url-parity-gate]: ./url_parity_gate.py
 [deploy-readme-redirects]: ../deploy/README.md#how-the-redirects-are-expressed
 [deploy-readme-security-headers]: ../deploy/README.md#security-headers
-[media-parser-fuzz-gate]: ./media_parser_fuzz_gate.py

checks/tests/media_fixtures.py

@@ -0,0 +1,123 @@
+"""Constructed media the media gate tests build their cases from."""
+
+import struct
+import zlib
+
+
+def jpeg_segment(marker: int, body: bytes) -> bytes:
+    return bytes((0xFF, marker)) + struct.pack(">H", len(body) + 2) + body
+
+
+def exif_segment() -> bytes:
+    """An Exif APP1 carrying only Orientation."""
+    entry = struct.pack("<HHIHH", 0x0112, 3, 1, 6, 0)
+    tiff = b"II" + struct.pack("<HI", 42, 8) + struct.pack("<H", 1) + entry
+    return jpeg_segment(0xE1, b"Exif\x00\x00" + tiff + struct.pack("<I", 0))
+
+
+def exif_ifd_segment(
+    entries: list[tuple[int, int, int, bytes]], pad: bytes = b""
+) -> bytes:
+    """An Exif APP1 with one IFD holding the given tag, type, count and value entries.
+
+    A pad goes between the IFD and the values it points to, as a writer aligning them leaves one.
+    """
+    table, values = b"", b""
+    start = 8 + 2 + 12 * len(entries) + 4 + len(pad)
+    for tag, kind, count, value in entries:
+        if len(value) > 4:
+            field = struct.pack("<I", start + len(values))
+            values += value
+        else:
+            field = value.ljust(4, b"\x00")
+        table += struct.pack("<HHI", tag, kind, count) + field
+    ifd = struct.pack("<H", len(entries)) + table + struct.pack("<I", 0)
+    tiff = b"II" + struct.pack("<HI", 42, 8) + ifd + pad + values
+    return jpeg_segment(0xE1, b"Exif\x00\x00" + tiff)
+
+
+DATE = b"2001:01:01 00:00:00\x00"
+ORIENTATION = (0x0112, 3, 1, struct.pack("<H", 6))
+
+JFIF = b"JFIF\x00\x01\x01\x00\x00\x01\x00\x01\x00\x00"
+ADOBE = b"Adobe\x00\x64\x00\x00\x00\x00\x01"
+
+
+def bend(fields: bytes, at: int, value: bytes) -> bytes:
+    """A fixed-shape segment's fields with the bytes at one offset replaced."""
+    return fields[:at] + value + fields[at + len(value) :]
+
+
+SQUARE = struct.pack(">IIB", 3779, 3779, 1)
+ICC = b"ICC_PROFILE\x00\x01\x01"
+
+
+def jpeg_fixture() -> bytes:
+    """A structurally complete JPEG, with stuffed bytes and a restart marker in its scan."""
+    out = b"\xff\xd8" + jpeg_segment(
+        0xE0, b"JFIF\x00\x01\x01\x00\x00\x01\x00\x01\x00\x00"
+    )
+    out += exif_segment()
+    out += jpeg_segment(0xDB, b"\x00" + bytes(range(64)))
+    out += jpeg_segment(0xC0, b"\x08\x00\x08\x00\x08\x01\x01\x11\x00")
+    out += jpeg_segment(0xC4, b"\x00" + bytes(16) + b"")
+    out += jpeg_segment(0xDA, b"\x01\x01\x00\x00\x3f\x00")
+    out += b"\x12\x34\xff\x00\x56\xff\xd0\x78\x9a"
+    return out + b"\xff\xd9"
+
+
+def png_chunk(name: bytes, body: bytes) -> bytes:
+    crc = zlib.crc32(name + body) & 0xFFFFFFFF
+    return struct.pack(">I", len(body)) + name + body + struct.pack(">I", crc)
+
+
+def png_fixture() -> bytes:
+    ihdr = struct.pack(">IIBBBBB", 1, 1, 8, 0, 0, 0, 0)
+    idat = zlib.compress(b"\x00\x7f")
+    return (
+        b"\x89PNG\r\n\x1a\n"
+        + png_chunk(b"IHDR", ihdr)
+        + png_chunk(b"gAMA", struct.pack(">I", 45455))
+        + png_chunk(b"IDAT", idat)
+        + png_chunk(b"IEND", b"")
+    )
+
+
+def palette_png_fixture() -> bytes:
+    ihdr = struct.pack(">IIBBBBB", 1, 1, 1, 3, 0, 0, 0)
+    idat = zlib.compress(b"\x00\x00")
+    return (
+        b"\x89PNG\r\n\x1a\n"
+        + png_chunk(b"IHDR", ihdr)
+        + png_chunk(b"PLTE", bytes(6))
+        + png_chunk(b"IDAT", idat)
+        + png_chunk(b"IEND", b"")
+    )
+
+
+def gif_fixture() -> bytes:
+    head = b"GIF89a" + struct.pack("<HH", 1, 1) + b"\x80\x00\x00" + bytes(6)
+    netscape = b"\x21\xff\x0bNETSCAPE2.0\x03\x01\x00\x00\x00"
+    control = b"\x21\xf9\x04\x00\x00\x00\x00\x00"
+    image = b"\x2c" + struct.pack("<HHHH", 0, 0, 1, 1) + b"\x00"
+    return head + netscape + control + image + b"\x02\x02\x44\x01\x00" + b"\x3b"
+
+
+def riff_chunk(name: bytes, body: bytes) -> bytes:
+    return (
+        name + struct.pack("<I", len(body)) + body + (b"\x00" if len(body) & 1 else b"")
+    )
+
+
+def webp_fixture() -> bytes:
+    body = riff_chunk(b"VP8X", bytes(10)) + riff_chunk(b"VP8L", b"\x2f\x00\x00\x00\x00")
+    return b"RIFF" + struct.pack("<I", len(body) + 4) + b"WEBP" + body
+
+
+def animated_webp_fixture() -> bytes:
+    """An animated WebP of one frame, which holds its own chunks inside an ANMF."""
+    frame = bytes(12) + b"\x64\x00\x00\x00"
+    frame += riff_chunk(b"VP8L", b"\x2f\x00\x00\x00\x00")
+    body = riff_chunk(b"VP8X", b"\x02" + bytes(9)) + riff_chunk(b"ANIM", bytes(6))
+    body += riff_chunk(b"ANMF", frame)
+    return b"RIFF" + struct.pack("<I", len(body) + 4) + b"WEBP" + body

checks/tests/test_cli_usage.py

@@ -44,7 +44,6 @@
         "checks/env_docs_gate.py",
         "checks/github_env_gate.py",
         "checks/media_metadata_gate.py",
-        "checks/media_parser_fuzz_gate.py",
         "checks/media_redactions_gate.py",
         "checks/text_pii_gate.py",
         "deploy/make-release.sh",