Skip to content

First slice: Go's path.Clean and path.Join, proved (bar v2) - #2

Merged
ahrazzle merged 3 commits into
mainfrom
claude/path-first-slice
Oct 28, 2026
Merged

ahrazzle merged 3 commits into
mainfrom
claude/path-first-slice

Conversation

@zero-aska

@zero-aska zero-aska commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

This is the first slice of the plan in #1, which I claimed there. It is built to bar v2, in the same shape as hostname, punycode, bech32 and hpack. A self-review (one reviewer, each finding checked by a second agent) has run on it; the changes it led to are listed at the end.

What it does

Go's path.Clean and path.Join (go1.24.7), in Lean, on String:

  • Path.clean p: the path cleaned by lexical processing alone;
  • Path.join es: the elements joined by slashes and cleaned, with the empty ones left out, and the empty string when nothing is left.

Path/Basic.lean writes Go's loop line by line. Go writes into a buffer and keeps its length w; here the buffer is a list kept reversed, so writing and removing at the end take one step. The loop reads the input once, from the front.

What is proved

Spec.lean has 10 statements, each about Path.clean or Path.join. lake comparator --paranoid accepts them under all six kernels bundled with Lean 4.35.0, using only propext, Classical.choice and Quot.sound, in about 45 seconds. There is no sorry outside the challenge file.

Spec/Defs.lean uses nothing from the code. It states Go's four rules on the elements of a path, as a rewriting relation Step:

  1. drop an empty element (from //, the root's slash, or a slash at the end);
  2. drop a .;
  3. drop a .. and the name before it (Go says "the non-.. element"; read literally, with an empty element or . allowed, abc/./../def could end as abc/def or def, and Go's own tests give def);
  4. drop a .. at the start of a rooted path.

Normal says no rule applies, and render writes elements back: a slash first when rooted, . for nothing. Names s r es says s is what render r writes for es, that no rule applies to es, and that no element holds a slash.

  • clean_rules: whatever order the rules are applied in, once none applies, the elements left are those clean p names. This is the documentation's "applies the following rules iteratively until no further processing can be done", taken literally.
  • rules_end: the rules can always be applied until none applies, and leave the elements of clean p.
  • clean_names: clean p names exactly one list of elements, rooted when p is. With clean_rules, every order of the rules ends at the same elements. (Each rule drops one element or two, as Step shows, so every order comes to an end.)
  • clean_shape: the elements of clean p are some .. (none if rooted), then names. So the result has no empty element or ., and .. only at its start. This is the plan's shape claim.
  • clean_idem: cleaning is idempotent.
  • join_eq: join is the documentation of Join.
  • join_inside: a clean relative path q with no .., joined to any b, gives the elements of clean b and then those of q. This is the plan's "stays inside any folder it is joined to".
  • join_root_inside: any u, cleaned as rooted (clean ("/" ++ u)) and joined to a b that is not empty, gives the elements of clean b and then elements with no ... This is the usual way to join an untrusted path. It needs b ≠ "": join ["", "/etc"] is /etc, which go_join_tests records.
  • go_clean_tests, go_join_tests: Go's own cleantests (each path, and each result cleaned again) and jointests, by decide +kernel.

The leading //. As the plan asked, the spec chooses a rule and records it: two slashes at the start are one slash, as in Go (//abc cleans to /abc, one of Go's tests). POSIX lets a system give them a meaning of their own, and Python keeps them.

The proofs go through a stack on the elements (Pike's algorithm, in Proofs/Rules.lean): every rule leaves the stack's result unchanged, the rules lead to it, and it is in normal form. Proofs/Loop.lean shows Go's loop keeps the stack in its buffer, element by element.

Differential tests

tests/TEST-RESULTS.md: 167,263 inputs, seed 20261027, 0 failures, with no oracle left out.

  • A model in Python, written from the documentation, in two parts that must agree: the four rules applied one at a time, at random places, until none applies; and a stack. They agree on all 147,239 paths checked. The model must give Go's cleantests and jointests. Every answer of the tool equals the model's.
  • Three oracles:
    • Go's path.Clean and path.Join (go1.24.7): agrees on all 167,263;
    • Python's posixpath.normpath (3.13.17);
    • Node's path.posix.normalize and path.posix.join (v22.22.0).
  • Named differences, each confirmed by the model with the oracle's rule:
    • Python keeps two slashes at the start (9,230);
    • Node keeps a slash at the end (41,482 in normalize, 3,236 in join);
    • Node's join gives . for nothing (5,099).
  • Inputs:
    • every path of up to 10 characters over a . / (88,573) and up to 7 over a b . /;
    • random paths of up to 80 characters with .., names, characters that are not ASCII, NUL, tab, CR and LF;
    • random joins of 0 to 5 elements;
    • Go's tests and the paths of Pike's paper;
    • 12 inputs of about a million characters each, answered in 3.3 s;
    • 31 malformed requests, among them hex that is not UTF-8;
    • the command-line modes, among them arguments that are not UTF-8.
  • Planted bugs: tests/plant_bugs.py plants 22 bugs and compares with the model alone; all 22 are caught (tests/PLANTED-BUGS.md). Among them:
    • backtracking past dotdot or past a slash;
    • a .. kept in a rooted path;
    • dotdot not moved past a kept ..;
    • a trailing . kept, or a trailing .. dropped;
    • the root's dotdot set to 0;
    • join giving . for nothing, or keeping leading empty elements;
    • in Main.lean, a CR kept and wrong arguments exiting 0.

Not proved

These are listed on the page:

  • Go's code: agreement with go1.24.7's path.Clean and path.Join is tested, not proved; only Go's own test tables are proved;
  • the file system: the rules are lexical, and a symbolic link can point anywhere;
  • the leading //, a recorded choice;
  • the empty folder in join_root_inside;
  • Go's other path functions (Split, Base, Dir, Ext, IsAbs, Match), and Windows paths;
  • strings that are not UTF-8: the library cannot take them, and on the command line the Lean runtime replaces them with U+FFFD, so the tool then prints a path Go would not;
  • time and memory;
  • the request handling in Main.lean, which the tests cover.

Re-check

lake build
lake comparator --paranoid
python3 tests/run_tests.py      # Go and Node for the oracles
python3 tests/plant_bugs.py

Changes from the self-review

The review found nothing blocking. Its reviewer also compared 802,041 more inputs with Go (every path over a . / up to 12 characters, and joins of up to five tricky elements) and found no difference.

  • The page's "in any order, the rules end at the same elements" is now a statement: clean_rules gives the elements of clean p, and clean_names says there is one such list.
  • CI's bar check then asked every approved statement to mention the code under proof; rules_end, normal_iff, and the step_shortens and rules_unique first added for this, spoke of the rules alone. They are restated about clean p: rules_end leaves its elements, clean_names replaces rules_unique, clean_shape replaces normal_iff, and that the rules end is read off Step.
  • Step.parent and SPEC.md say why rule 3 is read as taking a name.
  • Path.clean and SPEC.md no longer call the result the shortest path, which nothing proves.
  • The page lists Go's code as tested, not proved, and says what the CLI does with arguments that are not UTF-8; the tests now run the CLI on such arguments.
  • The record gives the extra join case its own label.

Kept as is: join_inside takes a clean q. A form for any q with no .. also holds and follows from join_eq, clean_rules and rules_end; it needs care at b = q = "", where join gives the empty string.

Review asks

  • The rules as a rewriting relation. Is Step a fair reading of Go's four rules? In particular, rule 1 is stated on elements as "drop an empty element", which also covers the slash at the end and the root's slash; the root is kept apart as rooted.
  • Rule 3's "non-.. element". Step.parent takes only a name: not empty, . or ... Taking an empty element or . there would let a//.. clean to a, which Go does not do. Is that the right reading?
  • The containment statements. Are join_inside and join_root_inside the claims a user needs, and is b ≠ "" stated clearly enough?

🤖 Generated with Claude Code

https://claude.ai/code/session_01AovXVgJBHGiN44TxQM4dcd

Path.clean is Go's loop, read once; Path.join is Go's Join. Spec/Defs.lean
states Go's four rules on the elements of a path, as a rewriting
relation, and Spec.lean proves that whatever order the rules are applied
in, clean gives what they leave; that the result has no empty element or
`.`, and `..` only at the start; that cleaning is idempotent; that join is
what Go's documentation says; and that a clean relative path with no
`..`, or any path cleaned as rooted, joined to a folder stays inside it.
Go's own tests of Clean and Join are checked by the kernel.

lake comparator --paranoid accepts all 10 statements under the six
kernels. The differential tests compare 167,263 inputs with a model of
the rules and with Go, Python and Node (0 failures, the differences
named and confirmed by the model), and catch 22 of 22 planted bugs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AovXVgJBHGiN44TxQM4dcd
- Spec.step_shortens (every rule drops an element, so the rules end in
  any order) and Spec.rules_unique (every order ends at the same
  elements) back what the page said of the order; 12 statements now.
- Step.parent and SPEC.md say why rule 3's "non-.. element" is read as
  a name: read literally, `abc/./../def` could end as `abc/def` or `def`.
- Path.clean and SPEC.md no longer call the result the shortest path,
  which nothing proves.
- README lists Go's code as tested, not proved, and says that the CLI
  replaces argument octets that are not UTF-8 with U+FFFD; the tests now
  run the CLI on such arguments.
- The record gives the extra join case its own label.

lake comparator --paranoid accepts all 12 statements. 167,263 inputs,
0 failures; 22 of 22 planted bugs caught.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AovXVgJBHGiN44TxQM4dcd
The bar check asks every approved statement to mention the code under
proof, and CI failed item 3 on rules_end, normal_iff, step_shortens and
rules_unique, which spoke of the rules alone. They are restated:

- clean_rules: the elements the rules leave are those `clean p` names
  (`Names`: rendered as `clean p`, no rule applies, no slash inside);
- rules_end: the rules can always be applied until none applies, and
  leave the elements of `clean p`;
- clean_names: `clean p` names exactly one list of elements, so every
  order of the rules ends at the same elements (was rules_unique);
- clean_shape: those elements are some `..`, none if rooted, then names
  (was normal_iff).

That the rules always end is read off `Spec.Step` on the page, not
claimed as a statement. Proofs/Unique.lean proves that a path splits
one way, that `Names` fixes its elements, and that the rules only drop
elements. lake comparator --paranoid accepts all 10 statements, and
bar/check.py passes item 3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AovXVgJBHGiN44TxQM4dcd
@ahrazzle
ahrazzle marked this pull request as ready for review October 28, 2026 04:00
@ahrazzle
ahrazzle merged commit 6768b74 into main Oct 28, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants