From 9494d1875ff49c20daf626ea4d963d49073f67dd Mon Sep 17 00:00:00 2001 From: Orda Date: Wed, 11 Nov 2026 13:20:58 -0600 Subject: [PATCH 1/2] Add bar audit report for Proven-Commons/zip Documented findings and required fixes for the bar audit of Proven-Commons/zip at commit 1cbcc1f, including issues related to NUL-byte parsing, symlink attributes, and overlapping local offsets. --- reviews/zip/2026-11-11-bar-audit.md | 42 +++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 reviews/zip/2026-11-11-bar-audit.md diff --git a/reviews/zip/2026-11-11-bar-audit.md b/reviews/zip/2026-11-11-bar-audit.md new file mode 100644 index 0000000..9c8e864 --- /dev/null +++ b/reviews/zip/2026-11-11-bar-audit.md @@ -0,0 +1,42 @@ +# Verified Bar Review: Proven-Commons/zip (`1cbcc1f`) and Cross-Library Path Predicate Check + +- **Target repository:** `Proven-Commons/zip` (commit `1cbcc1f`) +- **Cross-checked repository:** `Proven-Commons/tar` (PR #6 `SafeEntry` / `PlainName`) +- **Playbook standard:** `playbook/verified-bar.md` (Verified Bar v2) +- **Date:** 2026-11-11 +- **Verdict:** Keep `status = "in-progress"` until `Z-F1` through `Z-F5` are resolved. + +## Summary + +`Proven-Commons/zip` at `1cbcc1f` proves pure parsing and encoding round-trip lemmas (`parseArchive`, `encode`, `extractSafe`, `gate_sound`) with zero `sorry` and clean standard axioms. Five specification and extraction-boundary gaps remain before `gate_sound` can guarantee safe archive extraction across POSIX and language-runtime extractors. Each finding below was verified (`VERIFIED`) on concrete binary ZIP test archives in Python 3.11 (`zipfile`) against the exact Lean 4 predicates in `Zip/Safe.lean` and `Zip/Parse.lean`. + +## Findings + +### Z-F1 (High, VERIFIED): Embedded `0x00` NUL-byte parser differential in `SafeName` (affects `zip` `1cbcc1f` and `tar` PR #6 `PlainName`) + +- **Location:** `Zip/Safe.lean` (`SafeName`, `gate`, `gate_sound`) and `Proven-Commons/tar` PR #6 (`PlainName`). +- **Mechanism:** In Lean 4, `String.fromUTF8?` decodes `0x00` to `Char.ofNat 0` (`U+0000`). A 13-byte filename `safe.txt\x00evil` decodes to a 13-character Lean `String` and satisfies `SafeName` (`s != ""`, `!s.startsWith "/"`, no `"."` or `".."` in `s.splitOn "/"`, and `'\\' not-in s.toList`). +- **Differential behavior:** Python 3.11 `zipfile.ZipFile.namelist()` and C/POSIX `open(2)` truncate at `0x00`, resolving `"safe.txt"`, whereas length-prefixed writers see all 13 bytes. Furthermore, an archive containing both `"safe.txt"` and `"safe.txt\x00evil"` satisfies Lean's `Nodup` check while colliding on `"safe.txt"` in C/Python extractors. +- **Required fix:** Strengthen `Zip.SafeName` and `Tar.PlainName` to reject `0x00..0x1F` and `0x7F` (`c.val >= 0x20 && c.val != 0x7F`). + +### Z-F2 (Medium, PROVEN): `gate_sound` omits `'/' not-in c.toList` on split path components + +- **Location:** `Zip/Safe.lean` (`gate_sound`). +- **Required fix:** Prove a `splitOn_char_mem_ne` lemma showing `'/' not-in c.toList` for every `c in s.splitOn "/"` and include that conjunct in `gate_sound`. + +### Z-F3 (High, VERIFIED): UNIX `S_IFLNK` symlink attributes in Central Directory headers are ignored + +- **Location:** `Zip/Parse.lean` and `Zip/Safe.lean`. +- **Mechanism:** Central Directory headers carry `versionMadeBy` and `externalFileAttributes` (upper 16 bits = POSIX `st_mode`, e.g., `0o120777 << 16` for a symbolic link). `Parse.lean` ignores file mode, allowing a symlink entry named `"link"` with target bytes `"/etc/hosts"` to pass `SafeName`. +- **Required fix:** Parse `versionMadeBy` and `externalFileAttributes`, classify `EntryKind`, and restrict `SafeEntry` to regular files and directories (matching `Tar` PR #6's `typeflag` check). + +### Z-F4 (High, VERIFIED): Non-disjoint `localOffset` entries allow overlapping-extent amplification + +- **Location:** `Zip/Parse.lean` (`parseArchive`). +- **Mechanism:** Multiple Central Directory headers in a 188-byte archive can all set `localOffset = 0` and read the same Local File Header and payload slice, multiplying extracted bytes `N`-fold without enlarging the archive payload. +- **Required fix:** Require pairwise disjoint byte intervals `[localOffset_i, dataEnd_i)` in `SafeArchive` plus an explicit total uncompressed byte budget. + +### Z-F5 (Medium, VERIFIED): `Main.lean` CLI wrapper sits outside `[spec].code` + +- **Location:** `Main.lean` and `bar.toml`. +- **Required fix:** Move CLI argument dispatch into a pure function `Zip.runCli` inside `Zip` so `Main.lean` is a minimal I/O shell. From c504b565cfd2c4f67b031ac90414d38aded714b1 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Fri, 20 Nov 2026 05:50:03 +0000 Subject: [PATCH 2/2] Remap zip bar audit to ZipPilot tree at 1cbcc1f Replace citations of nonexistent Zip/Safe.lean and SafeName predicates with ZipPilot/Path.lean, Header.lean, and Archive.lean. Downgrade un reproduced claims to CONCERN/UNVERIFIED, note no bar.toml and in-memory-only extraction, and drop findings that do not apply at the pinned revision. Co-authored-by: ahraz --- reviews/zip/2026-11-11-bar-audit.md | 71 ++++++++++++++++++----------- 1 file changed, 45 insertions(+), 26 deletions(-) diff --git a/reviews/zip/2026-11-11-bar-audit.md b/reviews/zip/2026-11-11-bar-audit.md index 9c8e864..0d28871 100644 --- a/reviews/zip/2026-11-11-bar-audit.md +++ b/reviews/zip/2026-11-11-bar-audit.md @@ -1,42 +1,61 @@ -# Verified Bar Review: Proven-Commons/zip (`1cbcc1f`) and Cross-Library Path Predicate Check +# Verified Bar review: Proven-Commons/zip (`1cbcc1f`) -- **Target repository:** `Proven-Commons/zip` (commit `1cbcc1f`) -- **Cross-checked repository:** `Proven-Commons/tar` (PR #6 `SafeEntry` / `PlainName`) -- **Playbook standard:** `playbook/verified-bar.md` (Verified Bar v2) -- **Date:** 2026-11-11 -- **Verdict:** Keep `status = "in-progress"` until `Z-F1` through `Z-F5` are resolved. +- **Target repository:** [Proven-Commons/zip](https://github.com/Proven-Commons/zip) at commit [`1cbcc1f`](https://github.com/Proven-Commons/zip/tree/1cbcc1ffe4c2572d9507286250e5250679d9b0e4) +- **Cross-checked repository:** [Proven-Commons/tar](https://github.com/Proven-Commons/tar) (open work on path predicates; see tar bar audit) +- **Playbook standard:** `playbook/verified-bar.md` (Verified Bar v2) in this repo +- **Date:** 2026-11-11 (mapping corrected 2026-11-20 against the pinned tree) +- **Verdict:** The reader at `1cbcc1f` is a pilot, not verified: there is no `bar.toml`, parsing is tested rather than proved, and extraction is in-memory only (`README.md`, `ZipPilot/Archive.lean` module header). Treat items below as review concerns to test or prove, not as verified production defects, until each has a pinned fixture and a recorded command output. ## Summary -`Proven-Commons/zip` at `1cbcc1f` proves pure parsing and encoding round-trip lemmas (`parseArchive`, `encode`, `extractSafe`, `gate_sound`) with zero `sorry` and clean standard axioms. Five specification and extraction-boundary gaps remain before `gate_sound` can guarantee safe archive extraction across POSIX and language-runtime extractors. Each finding below was verified (`VERIFIED`) on concrete binary ZIP test archives in Python 3.11 (`zipfile`) against the exact Lean 4 predicates in `Zip/Safe.lean` and `Zip/Parse.lean`. +At `1cbcc1f` the proved surface is the path gate and in-memory extraction in `ZipPilot/Path.lean` and `ZipPilot/Archive.lean` (`gate_sound`, `cleanComponents_sound`, `extractAll_safe`, `extractArchive_safe`). Header layout, CRC-32, inflation, and `parseArchive` are covered by differential tests (`tests/run_tests.py`, `tests/TEST-RESULTS.md`), not by proof. There is no ZIP writer and no `encode`/`parseArchive` round-trip theorem. The CLI (`Main.lean`) lists entries or prints a manifest; it does not write files to disk. + +Earlier versions of this note cited `Zip/Safe.lean`, `Zip/Parse.lean`, `SafeName`, `SafeEntry`, `SafeArchive`, and `extractSafe`, which do not exist at `1cbcc1f`. The mapping below uses the actual modules. ## Findings -### Z-F1 (High, VERIFIED): Embedded `0x00` NUL-byte parser differential in `SafeName` (affects `zip` `1cbcc1f` and `tar` PR #6 `PlainName`) +### Z-F1 (High, CONCERN — differential UNVERIFIED here): NUL and control bytes in entry names + +- **Location:** `ZipPilot/Header.lean` (`bytesToName`), `ZipPilot/Path.lean` (`gate`, `gate_sound`). +- **Mechanism at `1cbcc1f`:** Names come from `String.fromUTF8?` (`bytesToName`). Invalid UTF-8 becomes `""`, which the gate rejects. A name that decodes with embedded `U+0000` is a valid Lean `String`; `gate_sound` only rules out `..`, empty components, `.`, absolute `/`, and an empty relative path. It does not forbid other control characters or NUL. `README.md` notes that whether backslashes should be rejected is still open; backslashes are ordinary characters, not separators. +- **Cross-library note:** Tar's `PlainName` work (tar PR #6) is a separate predicate; aligning NUL/control-byte rules across formats is still worth doing. +- **Earlier audit overclaim:** There is no archive-wide name-uniqueness (`Nodup`) check in `parseArchive`; a claim that duplicate logical names collide only because of Lean's `Nodup` is unsupported at this revision. +- **Status:** Reasonable hazard from the decoder and gate statement. This report does not include the binary fixture, Lean invocation, or `zipfile` output requested in review; label **UNVERIFIED** until those are committed beside the finding. +- **Suggested direction:** Reject `c.val < 0x20` and `c.val = 0x7F` in components (or in the full name before split), and add differential tests against Python `zipfile` and POSIX truncation behaviour. + +### Z-F2 (Low, NOT APPLICABLE as stated): `'/'` inside split components + +- **Location:** `ZipPilot/Path.lean` (`components` uses `String.splitOn "/"`; `gate_sound`). +- **At `1cbcc1f`:** Relative components are produced only by splitting on `/`, so no component string contains `/` unless the split implementation changes. The gap described in the tar audit (a single component holding `..` path text) does not apply the same way here. +- **Status:** Drop as a zip finding at this revision unless a concrete name shows otherwise. + +### Z-F3 (Medium, CONCERN — extraction consequence UNVERIFIED): Central-directory mode bits not classified -- **Location:** `Zip/Safe.lean` (`SafeName`, `gate`, `gate_sound`) and `Proven-Commons/tar` PR #6 (`PlainName`). -- **Mechanism:** In Lean 4, `String.fromUTF8?` decodes `0x00` to `Char.ofNat 0` (`U+0000`). A 13-byte filename `safe.txt\x00evil` decodes to a 13-character Lean `String` and satisfies `SafeName` (`s != ""`, `!s.startsWith "/"`, no `"."` or `".."` in `s.splitOn "/"`, and `'\\' not-in s.toList`). -- **Differential behavior:** Python 3.11 `zipfile.ZipFile.namelist()` and C/POSIX `open(2)` truncate at `0x00`, resolving `"safe.txt"`, whereas length-prefixed writers see all 13 bytes. Furthermore, an archive containing both `"safe.txt"` and `"safe.txt\x00evil"` satisfies Lean's `Nodup` check while colliding on `"safe.txt"` in C/Python extractors. -- **Required fix:** Strengthen `Zip.SafeName` and `Tar.PlainName` to reject `0x00..0x1F` and `0x7F` (`c.val >= 0x20 && c.val != 0x7F`). +- **Location:** `ZipPilot/Header.lean` (`parseCentralHeader` reads `externalFileAttributes` from the record but does not store or interpret them in `CentralEntry`). `ZipPilot/Archive.lean` (`extractAll`). +- **Mechanism at `1cbcc1f`:** `extractAll` emits `(dest ++ gated relative path, bytes)` in memory. `Main.lean` only prints a manifest; it does not create files or symlinks on disk (`README.md` "What is not proved"). +- **Status:** Ignoring `externalFileAttributes` is real for a downstream writer that honours UNIX modes. It is not a demonstrated symlink-creation bug in this implementation. **UNVERIFIED** for end-to-end filesystem extraction; frame any fix around classifying entry kind before bytes are handed to an I/O layer. -### Z-F2 (Medium, PROVEN): `gate_sound` omits `'/' not-in c.toList` on split path components +### Z-F4 (High, CONCERN — overlap UNVERIFIED): No disjoint `localOffset` or aggregate output budget -- **Location:** `Zip/Safe.lean` (`gate_sound`). -- **Required fix:** Prove a `splitOn_char_mem_ne` lemma showing `'/' not-in c.toList` for every `c in s.splitOn "/"` and include that conjunct in `gate_sound`. +- **Location:** `ZipPilot/Archive.lean` (`parseArchive`, `readEntry`), `ZipPilot/Header.lean` (`CentralEntry`). +- **Mechanism at `1cbcc1f`:** `parseCentral` accepts every header that passes field checks; there is no check that entry byte ranges are pairwise disjoint or that total uncompressed size is bounded across the archive. Per-entry guards exist before inflate: `maxUncompressedSize` (256 MiB) and `maxInflateRatio` (1000:1) in `readEntry`. Central and local names must match (`readEntry` line 73). +- **Status:** Overlapping offsets are a plausible amplification concern but were not reproduced in this correction pass (no 188-byte fixture, no recorded `zip-pilot` or test run). **UNVERIFIED** until a fixture and command output are added. +- **Suggested direction:** Add archive-level disjoint-interval and total-uncompressed bounds if the threat model requires them. -### Z-F3 (High, VERIFIED): UNIX `S_IFLNK` symlink attributes in Central Directory headers are ignored +### Z-F5 (Low, NOT APPLICABLE at `1cbcc1f`): No verified-bar layout yet -- **Location:** `Zip/Parse.lean` and `Zip/Safe.lean`. -- **Mechanism:** Central Directory headers carry `versionMadeBy` and `externalFileAttributes` (upper 16 bits = POSIX `st_mode`, e.g., `0o120777 << 16` for a symbolic link). `Parse.lean` ignores file mode, allowing a symlink entry named `"link"` with target bytes `"/etc/hosts"` to pass `SafeName`. -- **Required fix:** Parse `versionMadeBy` and `externalFileAttributes`, classify `EntryKind`, and restrict `SafeEntry` to regular files and directories (matching `Tar` PR #6's `typeflag` check). +- **Location:** Repository root at `1cbcc1f` has no `bar.toml` and no `[spec].code` list. +- **At `1cbcc1f`:** `Main.lean` is the CLI entry point; whether it belongs in a future `[spec].code` block is a bar-adoption question, not a defect in the zip tree as it stands. +- **Status:** Revisit when the library adopts the mission template and records spec scope in `bar.toml`. -### Z-F4 (High, VERIFIED): Non-disjoint `localOffset` entries allow overlapping-extent amplification +## What was wrong in the first draft -- **Location:** `Zip/Parse.lean` (`parseArchive`). -- **Mechanism:** Multiple Central Directory headers in a 188-byte archive can all set `localOffset = 0` and read the same Local File Header and payload slice, multiplying extracted bytes `N`-fold without enlarging the archive payload. -- **Required fix:** Require pairwise disjoint byte intervals `[localOffset_i, dataEnd_i)` in `SafeArchive` plus an explicit total uncompressed byte budget. +- Cited modules and predicates (`Zip/Safe.lean`, `SafeName`, `extractSafe`, `gate_sound` on a safe archive type) do not exist at `1cbcc1f`. +- Stated proved `parseArchive`/`encode` round trips; the README explicitly says parsing is tested and there is no ZIP writer. +- Marked several items **VERIFIED** without committed fixtures or Lean commands on this revision. -### Z-F5 (Medium, VERIFIED): `Main.lean` CLI wrapper sits outside `[spec].code` +## Suggested next steps -- **Location:** `Main.lean` and `bar.toml`. -- **Required fix:** Move CLI argument dispatch into a pure function `Zip.runCli` inside `Zip` so `Main.lean` is a minimal I/O shell. +1. Keep status **in-progress** on zip until bar item 1 (spec approval) and the parsing proof scope are decided. +2. For each remaining **CONCERN**, add a minimal binary fixture under `reviews/zip/fixtures/` (or the zip repo's `tests/`), the exact command (`python3 tests/run_tests.py` subset, `zip-pilot manifest`, or a small Lean example), and observed output before upgrading to **VERIFIED**. +3. When tightening the gate (NUL, backslash policy), update `SPEC.md` and re-run the recorded differential suite in `tests/TEST-RESULTS.md`. \ No newline at end of file