plawk: compile END-only programs (no main rule)

docs/design/PLAWK_AWK_FEATURE_AUDIT.md

@@ -17,7 +17,7 @@ only (runtime pending) · ❌ missing.
 
 | Feature | Status | Notes |
 |---|---|---|
-| `BEGIN` / `END` | ✅ | incl. constant `print` (BEGIN/END literal print). **`$0` / `$N` in END now read the LAST RECORD** (`END { print $1 }` → `c` on `a 1/b 2/c 3`, like gawk), in a straight-line END print, a statement list (`END { print $1; print $2 }`) and a concatenation (`print $1 " / " $2`). The last record is *gone* by END — at `end_print` the transient buffer holds the `end_of_file` sentinel — so retention is explicit: the record loop copies each record into a reused, geometrically grown buffer (`@plawk_lastrec_store`, the same shape as `@wam_rt_set`, so **constant memory and one memcpy per record**; interning per record would grow the atom table with every distinct record), and END re-materialises those bytes in the shared transient buffer and hands the reserved transient atom id to `llvm_emit_atom_field_slice/5` — the *same* slicer every in-loop field read uses. Honours FS/OFS/ORS/RS; `$N` past NF is empty; empty input gives an empty `$0`. **Pay-per-use**: the globals and defines are emitted as program-level IR only when `plawk_end_term_mentions_field/1` (the #4100 END-loop safety gate, *inverted* rather than restated) fires, so every program without an END field read is byte-identical (15/15 golden corpus). **Also in an END `if` branch** (`END { if (n == 3) print $1 }` — which had been silently printing `end_of_file`, a pre-existing wrong output surfaced and fixed by this line) **and an END loop body** (`while` / `do-while` / C-`for`, nested, inside an `if` inside a loop, with `break` — flipping #4100's gate). Those two needed no emitter parameterisation: `plawk_end_lastrec_rewrite/2` rewrites `field(N)` → `end_lastrec_field(N)` in the END actions (a structural walk, matching the gate) and **two clauses** on the shared `plawk_emit_print_expr_for_context/6` know the record source, so the rule-body print emitter and `plawk_scalar_action_sequence_pairs//15` are untouched. The rewrite covers conditions too, which is fail-safe: no condition emitter has a clause for `end_lastrec_field(_)`, so `END { if ($1 == "c") … }` declines rather than miscompiling. **`NF` in END counts the retained record** (`END { print NF }` → 2), in all three contexts — and in the `if`-branch and loop-body ones it had been printing **1**, the field count of the `end_of_file` sentinel, since those drivers existed: the gate was named `plawk_end_term_mentions_field/1` and matched `field(_)` only, so `NF` walked past the gate built to catch record reads. It is now `plawk_end_term_reads_record/1`, named for the property, matching `field(_)`, `special('NF')` and `special(length)` (`NR`/`RT` are process state and stay out). **`printf` arguments too** — `printf "%s\n", $1`, `printf "%s\n", $0`, `printf "%d\n", NF`, and mixes with scalars — producing the same `string_ptr` / `slice_len`+`slice_ptr` / `i64` call-argument vocabulary a record-context printf produces, so the format rewriter needed no new cases. **`length` / `length($N)` in END counts the retained record too**, in every route at once — a straight-line print, a concatenation (`print "L=" length`), a statement list, the mixed and assoc routes, a `printf` argument, an `if` branch and a loop body. It cost ONE clause, because the change that carried it first collapsed the duplication that had been making every previous cell cost one per route. Two collapses: (a) `NF` had an `end_lastrec_nf` expression row that was its in-loop `nf` row with `%line` swapped for the retained record Value — `length` had no such row, which is the whole reason every END form of it declined — so both became entries in `plawk_record_i64_read/5`, ONE table of record-reading i64 leaves parameterised on *which record they read*, with ONE retained-record wrapper (`end_lastrec_read(Kind)`) covering every entry, and `plawk_end_lastrec_nf_lines//2` generalised to `plawk_end_lastrec_i64_lines//4` with a NameBase parameter so NF keeps its own temporaries and its IR stays byte-identical; (b) the THREE END print walkers each enumerated the print-field vocabulary as clause heads, and each clause was three steps — separator, one per-kind call, recurse — around exactly the call `plawk_end_field_print_lines/4` already makes for that kind inside a concat, so "what may be printed in END" was FOUR lists with nothing keeping them equal (they had already drifted twice: a string scalar in a concat printed its atom id, and NF reached the routes one at a time). **24 clauses became 13**, each walker keeping only what is genuinely its own — `concat`, the one kind that is not a single value, and for the assoc route the table reads. Two smaller instances of the same shape went with it: `plawk_end_list_printf_arg_ok/2`'s two chains each listed the record-reading printf arguments (they differ only in `var` and `NR`), factored so `length` was one row not two; and `plawk_rule_body_print_field/1`, a THIRD list reached by an END loop body, needed one row. **32 pre-existing golden-corpus programs are byte-identical** across both collapses, which is the check that says a collapse was a collapse and not a rewrite. Bare `length` needed no case of its own: it parses to `length(field(0))` and field 0 measures the whole record, so the awk shorthand is already the arity-1 instance of the general form. One unintended-but-correct side effect, pinned rather than left unremarked: delegating the assoc walker passes the EMPTY scalar plan (as it already did for concat parts), which makes the shared generic expression clause reachable, so `END { print 1 + 2, c["x"] }` now builds and agrees with gawk. Tests: `tests/test_plawk_end_length.pl`. Still declined: a field or `NF` in a loop/`if` **condition**, **builtins over the record** (`substr($0, …)`, `toupper($1)`), an **END-only** program (different driver, no retain), a record read in a **for-in CHAIN** body (`END { for (k in c) print k, NF }` — `NF` and `length` decline there alike, so it is the chain body's print vocabulary rather than either feature), the **associative** END-`if` branch (`plawk_assoc_end_if_branch_prints_ok/2` allows only string literals there — an unrelated pre-existing restriction), and under a **binary descriptor** |
+| `BEGIN` / `END` | ✅ | incl. constant `print` (BEGIN/END literal print). **`$0` / `$N` in END now read the LAST RECORD** (`END { print $1 }` → `c` on `a 1/b 2/c 3`, like gawk), in a straight-line END print, a statement list (`END { print $1; print $2 }`) and a concatenation (`print $1 " / " $2`). The last record is *gone* by END — at `end_print` the transient buffer holds the `end_of_file` sentinel — so retention is explicit: the record loop copies each record into a reused, geometrically grown buffer (`@plawk_lastrec_store`, the same shape as `@wam_rt_set`, so **constant memory and one memcpy per record**; interning per record would grow the atom table with every distinct record), and END re-materialises those bytes in the shared transient buffer and hands the reserved transient atom id to `llvm_emit_atom_field_slice/5` — the *same* slicer every in-loop field read uses. Honours FS/OFS/ORS/RS; `$N` past NF is empty; empty input gives an empty `$0`. **Pay-per-use**: the globals and defines are emitted as program-level IR only when `plawk_end_term_mentions_field/1` (the #4100 END-loop safety gate, *inverted* rather than restated) fires, so every program without an END field read is byte-identical (15/15 golden corpus). **Also in an END `if` branch** (`END { if (n == 3) print $1 }` — which had been silently printing `end_of_file`, a pre-existing wrong output surfaced and fixed by this line) **and an END loop body** (`while` / `do-while` / C-`for`, nested, inside an `if` inside a loop, with `break` — flipping #4100's gate). Those two needed no emitter parameterisation: `plawk_end_lastrec_rewrite/2` rewrites `field(N)` → `end_lastrec_field(N)` in the END actions (a structural walk, matching the gate) and **two clauses** on the shared `plawk_emit_print_expr_for_context/6` know the record source, so the rule-body print emitter and `plawk_scalar_action_sequence_pairs//15` are untouched. The rewrite covers conditions too, which is fail-safe: no condition emitter has a clause for `end_lastrec_field(_)`, so `END { if ($1 == "c") … }` declines rather than miscompiling. **`NF` in END counts the retained record** (`END { print NF }` → 2), in all three contexts — and in the `if`-branch and loop-body ones it had been printing **1**, the field count of the `end_of_file` sentinel, since those drivers existed: the gate was named `plawk_end_term_mentions_field/1` and matched `field(_)` only, so `NF` walked past the gate built to catch record reads. It is now `plawk_end_term_reads_record/1`, named for the property, matching `field(_)`, `special('NF')` and `special(length)` (`NR`/`RT` are process state and stay out). **`printf` arguments too** — `printf "%s\n", $1`, `printf "%s\n", $0`, `printf "%d\n", NF`, and mixes with scalars — producing the same `string_ptr` / `slice_len`+`slice_ptr` / `i64` call-argument vocabulary a record-context printf produces, so the format rewriter needed no new cases. **`length` / `length($N)` in END counts the retained record too**, in every route at once — a straight-line print, a concatenation (`print "L=" length`), a statement list, the mixed and assoc routes, a `printf` argument, an `if` branch and a loop body. It cost ONE clause, because the change that carried it first collapsed the duplication that had been making every previous cell cost one per route. Two collapses: (a) `NF` had an `end_lastrec_nf` expression row that was its in-loop `nf` row with `%line` swapped for the retained record Value — `length` had no such row, which is the whole reason every END form of it declined — so both became entries in `plawk_record_i64_read/5`, ONE table of record-reading i64 leaves parameterised on *which record they read*, with ONE retained-record wrapper (`end_lastrec_read(Kind)`) covering every entry, and `plawk_end_lastrec_nf_lines//2` generalised to `plawk_end_lastrec_i64_lines//4` with a NameBase parameter so NF keeps its own temporaries and its IR stays byte-identical; (b) the THREE END print walkers each enumerated the print-field vocabulary as clause heads, and each clause was three steps — separator, one per-kind call, recurse — around exactly the call `plawk_end_field_print_lines/4` already makes for that kind inside a concat, so "what may be printed in END" was FOUR lists with nothing keeping them equal (they had already drifted twice: a string scalar in a concat printed its atom id, and NF reached the routes one at a time). **24 clauses became 13**, each walker keeping only what is genuinely its own — `concat`, the one kind that is not a single value, and for the assoc route the table reads. Two smaller instances of the same shape went with it: `plawk_end_list_printf_arg_ok/2`'s two chains each listed the record-reading printf arguments (they differ only in `var` and `NR`), factored so `length` was one row not two; and `plawk_rule_body_print_field/1`, a THIRD list reached by an END loop body, needed one row. **32 pre-existing golden-corpus programs are byte-identical** across both collapses, which is the check that says a collapse was a collapse and not a rewrite. Bare `length` needed no case of its own: it parses to `length(field(0))` and field 0 measures the whole record, so the awk shorthand is already the arity-1 instance of the general form. One unintended-but-correct side effect, pinned rather than left unremarked: delegating the assoc walker passes the EMPTY scalar plan (as it already did for concat parts), which makes the shared generic expression clause reachable, so `END { print 1 + 2, c["x"] }` now builds and agrees with gawk. Tests: `tests/test_plawk_end_length.pl`. Still declined: a field or `NF` in a loop/`if` **condition**, **builtins over the record** (`substr($0, …)`, `toupper($1)`), ~~an **END-only** program~~ **now compiles** (`END { print ... }`, printf, END-`if`, and the empty program `''` — the rules+END driver admits an empty rule chain via an empty scalar state plan + a dedicated `plawk_scalar_rule_chain_ir([], ...)` clause emitting `br label %continue_loop`; the record loop still runs so NR/$0/$N/NF/length are the last record, byte-identical on 20/20 existing programs; remaining END-only follow-ons are a **scalar-variable read/write** `END { print x }` (declines cleanly at exit 3 -- an ultra review caught that admitting it as a slot made the loop's next-slot phi reference `%rule_-1_*`, a miscompile; and an unset scalar is context-dependent, the uninitialised-scalar problem, so it needs that settled first), a **loop** `END { while ... }` (clean exit-3 decline, guarded so it never miscompiles), a scalar **assignment**, and for-in/getline; tests `tests/test_plawk_end_only.pl` 29), a record read in a **for-in CHAIN** body (`END { for (k in c) print k, NF }` — `NF` and `length` decline there alike, so it is the chain body's print vocabulary rather than either feature), the **associative** END-`if` branch (`plawk_assoc_end_if_branch_prints_ok/2` allows only string literals there — an unrelated pre-existing restriction), and under a **binary descriptor** |
 | `/regex/` | ✅ | a bare `/re/` pattern matches the whole record with **full POSIX ERE**: a body with metacharacters lowers to `field_match(0, Regex)` and runs through the same regex engine as `~`/`!~` (`/foo.*bar/`, `/^(err\|warn):/`, character classes, alternation, groups). A metachar-free body keeps its fast native lowering (`^prefix` → `prefix/1`, all-literal → `contains/1`); `\/` is a literal slash |
 | `$N == "v"`, `$3 > 100` | ✅ | field-equality + numeric field guards; **string inequality `$N != "v"`** now parses too (the skip-a-value idiom `$1 != "header"`) — rewritten to `not_pat(field_eq(N,"v"))`, so it reuses the `==` guard codegen and composes with `!`/`&&`/`||`, `if` conditions, and for-in-END group-by. Numeric `$N != K` keeps the integer path. **Field string ordering `$N < "v"` / `<=` / `>` / `>=` landed too** — awk compares a field against a string constant lexically (never numeric, so `$1 < "10"` is byte-wise while `$1 < 10` stays the numeric path); lowered to a memcmp of the field slice against the interned literal (`@wam_atom_field_str_cmp_value`), composing with `!`/`&&`/`||`, `if` conditions, and for-in-END group-by |
 | `&& \|\| !` combinators | ✅ | awk precedence, parens, single-block lowering |

docs/design/PLAWK_CAMPAIGN_HANDOFF.md

@@ -367,6 +367,44 @@ parse, not the gate you expected to fire.
 
 ## Verification practices (do not skip)
 
+- **Relaxing a shared gate can turn a clean decline into a MISCOMPILE in a
+  neighbouring driver that relied on that gate failing.** END-only support needed
+  `plawk_scalar_state_plan/3` to admit an empty plan (`; Rules == []`). That made
+  `END { print ... }` compile — but the END-LOOP driver clause (`END { while ... }`)
+  cuts as soon as it sees a loop, then had been relying on `state_plan` *failing* for
+  a rule-less program to decline after the cut. With the guard relaxed, state_plan
+  succeeded, the clause committed past its cut, and it emitted a malformed loop
+  driver: exit 4, a miscompile, on a program that used to decline cleanly at exit 3.
+  The fix was a targeted `Rules0 \== []` guard on that clause so END-loop-with-no-rule
+  keeps declining. **When you relax a gate, enumerate every clause that used that
+  gate's failure as its own decline mechanism** — a cut-then-rely-on-a-later-goal-
+  failing clause is silently converted from "declines" to "commits and may miscompile".
+  A broad exit-4 sweep of the whole surface the relaxation touches is mandatory, not
+  optional: here it found exactly one such site, and the campaign's worst outcome
+  (invalid LLVM on a supported-looking program) was one un-run probe away.
+
+  **And the exit-4 sweep must vary the DATA the relaxation admits, not just the
+  program shapes.** My own sweep enumerated END-only shapes (print / printf / if /
+  loop / for-in) and driver clauses, and passed -- but every print case used a
+  literal, a field, NR, NF, or length. An ULTRA REVIEW caught what I missed:
+  `END { print x }`, a scalar VARIABLE, still exit-4'd. The reason is one level below
+  the dispatch clause: admitting an empty rule chain let plawk_scalar_state_plan
+  collect `x` as a slot, and the next-slot phi emitter then computed
+  LastRuleIndex = RuleCount - 1 = -1 and referenced `%rule_-1_*` -- undefined SSA.
+  The miscompile lived in the interaction between an empty rule chain and a NON-empty
+  state plan, a combination no shape-only sweep reaches unless it includes a program
+  whose END reads a scalar variable. Two lessons: (1) when a relaxation admits a new
+  input CLASS, enumerate the class's data dimensions (here: does the END read a
+  literal, a field, a special, or a scalar VAR?), because the defect can hide in a
+  data-dependent downstream (a phi index) that the dispatch-clause enumeration never
+  sees; (2) a second reviewer with fresh eyes found in one pass what my own thorough
+  verification did not -- the value of external review scales with how confident and
+  polished the change already looks. The fix guards the empty rule chain to fire only
+  when the state plan has no slots, so a scalar-var END-only program declines cleanly
+  (exit 3) as a follow-on; a correct pass-through phi also needs the
+  uninitialised-scalar representation settled (`print x` of an unset var is empty in
+  string context, 0 in numeric), which is why declining, not a quick phi patch, was
+  the right scope.
 - **gawk 5.2 is the oracle.** Compare output *and* exit status. Probe harness
   pattern: write the program, `rm -f` the binary first (a declining build must not
   run a stale one), build, run, diff against `gawk`.
@@ -652,6 +690,18 @@ the implementation changed about the design. The load-bearing facts:
 
 ## Remaining follow-ons
 
+**EOF-sentinel: a literal `end_of_file` input line is mistaken for EOF (PRE-EXISTING,
+not END-only).** `END { print NR, $0 }` on input `a\nend_of_file\nb\n` prints `1 a`;
+gawk prints `3 b`. Two independent reviews (terra, astra) surfaced this. The shared
+stream driver (`src/unifyweaver/targets/wam_llvm_target.pl`, ~line 23909) detects
+end-of-input by comparing the record TEXT against `"end_of_file"` instead of by atom
+identity -- the runtime already has a distinct EOF atom, so the fix is an
+identity-based comparison plus a regression test. It predates the END-only work and
+affects EVERY program (the rule-bearing `{ n++ } END { print NR, $0 }` is equally
+wrong); END-only just made it reachable in an END-only shape. A runtime fix of its
+own, deliberately out of scope for the END-only PR.
+
+
 **END record reads, what is left** — each pinned as a decline in
 `tests/test_plawk_end_field_reads.pl`. A field or `NF` in a loop / `if`
 **condition** (a fail-safe decline: the rewrite reaches conditions, and no
@@ -690,8 +740,13 @@ condition emitter has a clause for `end_lastrec_field(_)` / `end_lastrec_nf`) ·
   at parse time (#4169). The two share a spelling and nothing else: one needs the
   retained record, the other needs no runtime at all. · **builtins over the
 record** in END (`substr($0, …)`, `toupper($1)` — the gate retains for them, the
-emitters have no clause; the *literal* forms of all five are done) · **END-only**
-programs (a driver with no retain) ·
+emitters have no clause; the *literal* forms of all five are done) · ~~**END-only**
+programs~~ **DONE** (see below; `END { print ... }`, printf, END-`if`, and the empty
+program now compile — remaining END-only follow-ons are an END-only **loop**
+(`END { while ... }`, pinned as a clean exit-3 decline, NOT a miscompile), an END-only
+**scalar-variable read/write** (`END { print x }` -- see the review-found miscompile below),
+an END-only scalar **assignment** (`END { x = 5; print x }`), and for-in / getline in
+END-only) ·
 `printf` field args in the **assoc / mixed END chain** (a different driver, passes
 `no_end_record`) · the **associative** END-`if` branch (refused by
 `plawk_assoc_end_if_branch_prints_ok/2`, which allows only string literals there —