HTML autoescape: add html_escape_mode (legacy switch) and an unmarked-fragment diagnostic, both non-breaking

[randlee]

Summary

sc-compose 1.6 turned on HTML autoescape for .html/.xhtml outputs. The default itself matches Jinja/Flask/Django, but it shipped in a minor release with no compatibility switch and no diagnostic, so every consumer template that inserted a pre-built fragment ({{ body_html }}) silently started rendering its markup as literal text. The JSON side of the same feature got json_escape_mode: legacy / --json-escape-mode; the HTML side did not.

Impact in one consumer (randlee/atm-core): five templates, 31 committed evidence sets rendered blank for three weeks before anyone opened one. Fixed on the consumer side in randlee/atm-core#1430 by adding | safe, plus a regression test that renders through the real binary.

This issue proposes two additive, non-breaking changes so this cannot recur.

A. html_escape_mode — a compatibility switch mirroring json_escape_mode

Surface

Where	Form	Values	Default
Template frontmatter	html_escape_mode: auto | legacy	auto, legacy	auto
CLI	--html-escape-mode <auto|legacy>	same	unset (defer to frontmatter, then auto)

Precedence: CLI flag > frontmatter > default auto. Identical to how --json-escape-mode is documented today ("default is auto unless frontmatter declares one").

Semantics

• auto (current 1.6 behavior): for outputs whose format resolves to HTML or XHTML (by output extension .html/.xhtml, or an explicit format: frontmatter key), every interpolated value is HTML-escaped unless the placeholder carries | safe.
• legacy (pre-1.6 behavior): no automatic escaping for HTML/XHTML output. | e / | escape continue to escape explicitly; | safe is accepted and is a no-op.
• Non-HTML formats are untouched by either value.

Implementation notes

• Resolve the mode in the same place the renderer already resolves json_escape_mode, so both live in one "escape policy" struct derived from (CLI, frontmatter, output format). Suggested shape:

  pub struct EscapePolicy { pub json: JsonEscapeMode, pub html: HtmlEscapeMode }
  pub enum HtmlEscapeMode { Auto, Legacy }

• Feed policy.html == Auto into the environment's autoescape decision for that render; Legacy disables autoescape for the run. Do not special-case individual filters.
• Frontmatter validation: reject unknown values with the same error style as an unknown json_escape_mode.
• Tests: (1) frontmatter legacy renders <b> verbatim into .html; (2) CLI flag overrides frontmatter both directions; (3) auto still escapes and | safe still passes through; (4) .md/.txt output unaffected by either value; (5) --help lists the flag next to --json-escape-mode.
• Docs: one paragraph next to the json_escape_mode docs, plus a CHANGELOG line that names 1.6.0 as the release that changed the HTML default (so readers of old templates know what to set).

B. Unmarked-fragment diagnostic

Heuristic

When rendering to HTML/XHTML in auto mode, for each interpolated value that was autoescaped (i.e. not safe, not already Markup), flag it if the pre-escape string matches either:

• a tag opener: <[A-Za-z][A-Za-z0-9-]*[\s/>] (catches <h1>, <tr class=…>, <br/>), or
• an entity reference: &(#[0-9]+|#x[0-9A-Fa-f]+|[A-Za-z][A-Za-z0-9]+);

The second pattern is what identifies "already-escaped HTML being escaped again" (& → &), which is exactly the failure mode from #1430. Values that are pure text never match either pattern, so false positives are limited to prose that literally contains something like a < b > c; — rare, and silenceable with | e (explicit escape suppresses the diagnostic because the author has stated intent).

Surface

Where	Form	Values	Default
CLI	--escape-diagnostics <off|warn|error>		warn
Frontmatter	escape_diagnostics: off | warn | error		inherits CLI

• warn (default): print one line per offending placeholder to stderr, exit code unchanged. Non-breaking.
• error: fail the render with exit code 2 (same code as other template errors). Opt-in.
• off: silence, for templates that intentionally display markup as text.
• Do not fold error into the existing --strict in this release; consumers already pass --strict (atm-core's view-site builder does) and would break. Promote it into --strict in the next minor after one release of warnings, with a CHANGELOG notice.

Message shape

warning: html-escape: variable `body_html` in templates/smoke-report/inbound-peer-pane.xhtml.j2:17 contains markup and was autoescaped; add `| safe` for a trusted fragment, `| e` to silence, or set html_escape_mode: legacy

Include template path and line, variable name, and the three remedies. One line per placeholder, deduplicated per (template, line, variable).

Implementation notes

• Hook the check where the autoescape wrapper converts a plain string to escaped output; that is the only point that sees both the raw value and the "was this marked safe" bit. Keep the regexes compiled once per render.
• | e on a placeholder should mark the value so the check skips it (explicit intent). | safe never reaches the escaper, so it is skipped naturally.
• Tests: (1) {{ body_html }} with <h1> warns; (2) {{ body_html | safe }} does not; (3) {{ body_html | e }} does not; (4) & in a scalar warns (double-escape case); (5) error mode exits 2; (6) off prints nothing; (7) .md output never warns; (8) dedup across a loop rendering the same placeholder.

Rollout

1. Ship A and B(warn) together in one minor release (additive; no existing render changes output or exit code).
2. Next minor: --strict implies --escape-diagnostics error; CHANGELOG entry.
3. Consumers that want the old behavior set html_escape_mode: legacy in frontmatter and never see the diagnostic.

References

• Consumer fix and evidence of impact: fix(smoke-report): pass evidence fragments through sc-compose unescaped atm-core#1430
• Existing JSON precedent: --json-escape-mode, frontmatter json_escape_mode: legacy
• Related closed issues on escaping: format: xml declares XML output but renderer performs no XML-escaping of interpolated variables #268, Control characters survive the explicit pipe-e escape filter, producing not-well-formed XML even on an explicitly-escaped path #278, Fuzz: template-init followed by render silently corrupts round-trip JSON via double-quoting auto-escape #385, Homebrew formula.rb.j2 install_block still raw-interpolated (same class as #478, lower severity) #483

🤖 Generated with Claude Code

https://claude.ai/code/session_019kon2YPcoTpu2ok838FrgZ

[randlee]

Reference templates from randlee/atm-core, the consumer this was found in. "Before" links are pinned to develop 11b6fcf0f (the broken form, plain {{ x }}); "after" links are pinned to the fix branch commit 63afd2470 ({{ x | safe }}).

Template	Fragment variables	Before	After
templates/smoke-report/inbound-peer-pane.xhtml.j2	body_html	L17	L17
templates/smoke-report/inbound-peer-review.html.j2	pane_html	L28	L28
templates/smoke-report/inbound-peer-review.xhtml.j2	pane_html	L16	L16
.just/templates/view-report.html.j2	summary_html, sections_html, recommendations_html, footer_html	L68-L82	L68-L82
.just/templates/view-panel.xhtml.j2	body_html	L45	L45

Notes for the diagnostic heuristic:

• These templates already carry the frontmatter block the proposal would extend (required_variables, defaults, metadata), e.g. view-report.html.j2 L1-L16. html_escape_mode: would sit alongside those keys.
• templates/smoke-report/inbound-peer-frame.html.j2 is the control case: scalars only (title, generated_at, pane_src), correctly autoescaped, must not warn.
• Fragment producers that do the escaping upstream, so the values are already &-bearing HTML when they reach the renderer (this is the double-escape signature the diagnostic keys on): feature_smoke_report.render_feature_pane, run_inbound_peer_smoke.render_host_pane, read_benchmark._report_variables.
• Consumer regression test that renders all five through the real binary, usable as a fixture for the proposed sc-compose tests: scripts/smoke/test_report_templates.py.
• An already-safe template in the same repo (never affected, shows the intended end state): .claude/skills/html-report/templates/fuzz-run-report.html.j2 L101-L155.

[randlee]

The same escaping applies to .xml.j2 templates, not only .html/.xhtml. Measured with sc-compose 1.6.1:

Template extension	{{ v }} with v = <b> & "q"
.xml.j2	&lt;b&gt; &amp; &quot;q&quot;
.html.j2	&lt;b&gt; &amp; &quot;q&quot;
.md.j2, .j2	<b> & "q"

atm-core uses .xml.j2 for agent task bodies. They are read by agents and never parsed as XML, and several embed a JSON variable, so assignees were receiving {"status":"PASS".... The HTML sweep missed them because nothing fails: the render exits 0.

Fixed downstream in randlee/atm-core#1538 by wrapping each body in {% autoescape false %}, which sc-compose honours. Two requests for this issue's scope:

1. Whatever switch or diagnostic lands here should cover .xml as well as .html/.xhtml.
2. Please document which extensions autoescape. The README and render --help do not say.