Add rewriteModuleStandalone, a TypeScript-free entry to the transform - #1226
NullVoxPopuli merged 13 commits into
Conversation
`rewriteModule` parses the content-tag-preprocessed script with the `TSLib` it is given and reads template placement, `this` binding and import bindings off the AST. Embedders that pair the transform with a compiler that has no JS API (TypeScript 7) had to carry a second TypeScript install just for that parse. `rewriteModuleStandalone(input, environment)` derives the same information from content-tag's parse output (template ranges and the class-member flag) plus a small scanner over the script that tracks strings, comments, template literals, regexes and bracket nesting to find import declarations, class bodies, static blocks and statement starts. The template emit itself is shared: `calculateTemplateSpans` now takes a plain `EmbeddedTemplate` and `calculateTaggedTemplateSpans` is a thin adapter over the TS node. `rewriteModule` keeps its signature and behaviour for every current caller. A parity test runs both entry points over every .gts/.gjs in test-packages and over targeted edge cases and asserts identical output, errors, directives and span mappings. Two preprocess fixes surfaced by that parity check, both of which also affect `rewriteModule`: - content-tag reports a class's own template before a template that appears earlier in its heritage clause; sort the parse output so the splice happens in source order. - A class member template after a field initializer with no trailing semicolon became `value[___T`...`]`, an element access, so the class's own template was emitted as a context-bound expression. Emit `;[...]` (a `;` is an empty class element) so it always parses as a member. Fixes typed-ember#1224 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
ember-estree (content-tag + oxc-parser) gives a real ESTree for the file. Walking its outer tree, with an ancestor stack, before templates are spliced in reproduces `templateThisBinding` and `collectImportedBindings` directly, so the hand-written tokenizer goes. Known divergence: oxc yields an empty program on an unrecoverable script syntax error, where TypeScript's parser keeps going. Such a file is emitted without import bindings or placement (bare identifiers still resolve lexically); the checker reports the syntax error either way. Two parity cases depend on ember-estree fixes (NullVoxPopuli/ember-estree#77 and typed-ember#78) and are marked `test.fails` until those ship. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
`gts-virtual-code.ts` imports `transform/index.js`, so re-exporting the standalone entry there made tsserver load ember-estree and oxc-parser's native binary at plugin activation. A dedicated entrypoint keeps the language server, tsserver plugin and `ember-tsc` free of it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
0.7.1 sorts content-tag results into source order and uses `void` placeholders, so heritage-clause templates and statement-start templates after an unterminated statement now match `rewriteModule`. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
| 'No interpolated values in template strings', | ||
| ); | ||
|
|
||
| // environment-specific transforms may emit templateLocation in meta, in |
There was a problem hiding this comment.
we only have one environment specific transform -- this is not a generic behavior
There was a problem hiding this comment.
Right, that fallback was pre-existing and dead. Replaced with an assert in be745d1.
| }, | ||
| }); | ||
| } catch (error) { | ||
| // After the visitor factory has run, ember-estree parses each template's |
There was a problem hiding this comment.
is this actually a bug? I don't know what this comment is trying to say\
why do we set analyzed true on the preparation of the visitors?
and why do we collect importing bindings? does oxc not have a fault tolerant parse mode?
does ember-estree need to add fault-tolerance?
There was a problem hiding this comment.
Not a bug in glint, a workaround for ember-estree throwing out of toTree mid-walk on a template syntax error (the factory had run, so the analysis was complete, hence the flag). Gone in be745d1: NullVoxPopuli/ember-estree#80 adds onTemplateError, so the walk continues and this path just gets the failing template's location like any other.
Import bindings are needed for the emit itself, not tolerance: import { on } from '@ember/modifier' must win over the on keyword global (#1113), and imports of special forms (fn, hash, ...) need resolving to their source. Same as collectImportedBindings on the TS path.
oxc has no tolerant mode: on an unrecoverable script error program.body is [] (TypeScript's parser keeps going). That's the one documented divergence (covered by a test); the type-checker reports the syntax error either way.
|
|
||
| const SKIPPED_KEYS = new Set(['parent', 'loc', 'range', 'tokens', 'comments']); | ||
|
|
||
| function walk(node: Node, ancestors: Array<Ancestor>, placements: Map<number, Placement>): void { |
There was a problem hiding this comment.
why are we walking here? we already have a means to do this with ember-estree
There was a problem hiding this comment.
Agreed. be745d1 uses visitors (ImportDeclaration, GlimmerTemplate) plus onTemplateError's path; the walk is gone. That needed two ember-estree fixes (NullVoxPopuli/ember-estree#80): keep the ExportDefaultDeclaration wrapper (today export default <template> loses the export in the tree and in print()), and don't abort on one bad template.
| // ember-estree stands each expression template in with a same-length | ||
| // template literal (`void `...`` from 0.7.1, a bare backtick literal before) and | ||
| // each class member with a `static{`...`}` block. | ||
| function isPlaceholderShape(node: Node): boolean { |
There was a problem hiding this comment.
this function name implies that we get something definite out of it, however, it will match many things that are not placeholders (just real code)
| return parent?.type === 'ExpressionStatement'; | ||
| } | ||
|
|
||
| // Mirrors `collectImportedBindings` in `inlining/tagged-strings.ts`. |
There was a problem hiding this comment.
why mirror? we don't like duplicate code if we don't need it
There was a problem hiding this comment.
Shared now: thisBindingFromAncestors(kinds) in inlining/index.ts is the one decision; templateThisBinding (TS AST) and ancestorsOf (ESTree path) only map their own AST to static-block | heritage-clause | class. Import reading stays per-AST since the node shapes differ, but both produce the same ImportedBindings.
Address review: no hand-rolled tree walk. Import bindings and template placement come from `ImportDeclaration` / `GlimmerTemplate` visitor callbacks, with `onTemplateError` (NullVoxPopuli/ember-estree#80) supplying the path of a template that fails to parse so it is placed like any other. The `this`-binding resolution is shared with the TS path (`thisBindingFromAncestors`); each path only maps its own AST to the ancestor kinds. Also drop the generic `templateLocation` fallback in `calculateTaggedTemplateSpans`: the only environment transform always records it, so assert instead. Needs ember-estree with typed-ember#80 (export default wrapper kept; onTemplateError). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
oxc has no error recovery, so `rewriteModuleStandalone` is a build-time entry: a file with a script syntax error is emitted without import bindings or placement and the type-checker reports the error. Say so. An editor host (a content mapper on TypeScript 7's native compiler) has a tolerant parse of the file already. `rewriteModuleWithAnalysis` takes the same `ScriptAnalysis` (import bindings + template placements) from the caller instead of from ember-estree; `rewriteModuleStandalone` is now that with ember-estree's analysis plugged in. The analysis types are exported from the standalone entrypoint, and a file content-tag finds no templates in is not analyzed at all. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
The TypeScript content-mapper protocol hands a mapper only the file's text, never the host's parse, so there is no caller with a tolerant analysis to supply. Keep the seam internal and say plainly that an editor still needs `rewriteModule` for mid-keystroke recovery. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
Mid-keystroke, a broken script already goes one of two ways on the TS path: content-tag fails and the raw source is kept with verification off so the type-checker's recovering parser powers completions, or swc tolerates it and the templates are emitted normally. oxc tolerates less than swc, so the standalone entry had a third mode in between: templates emitted with no import bindings or placement. Collapse it into the first: when ember-estree reports script errors (ast.errors, NullVoxPopuli/ember-estree#81), record one as a content-tag-style error and keep the raw source, exactly as on content-tag failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
Only the script is analyzed there; the templates are parsed (and their errors reported) by calculateTemplateSpans. Replacing each body with same-length whitespace keeps every offset, spares ember-estree a second handlebars parse per template, and means a template that does not parse cannot abort the analysis, so the onTemplateError option is not needed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
Released with the export default wrapper fix (typed-ember#82) and ast.errors (typed-ember#81). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
`@glint/ember-tsc/transform/standalone` (typed-ember/glint#1226) analyzes the script parts of a .gts file with ember-estree instead of the TypeScript JavaScript API, so the mapper no longer carries TypeScript 5 at runtime. It stays a devDependency for `lint:types` and the `CompilerOptions` type import in protocol.ts. Script parse errors now take the same path as content-tag errors: the error is reported and no transformed text is returned for the file. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN
Fixes #1224
Adds
rewriteModuleStandalone(input, environment), exported from a new@glint/ember-tsc/transform/standaloneentrypoint. It produces the sameTransformedModuleasrewriteModule(ts, input, environment)without a TypeScript JS API, so embedders that type-check with TypeScript 7 (content mappers, build plugins) no longer need a private TypeScript 5 install just to run the transform.How it works.
rewriteModulereads three things off the TS AST: template placement, the template'sthisbinding, and import bindings. The standalone entry gets them from ember-estree (content-tag + oxc-parser) through itsvisitors:ImportDeclarationfor bindings,GlimmerTemplatefor placement, walking the visitor path for thethisbinding. Template bodies are blanked (same length) before the call: only the script is analyzed there, the templates are parsed by glint itself, and a template that does not parse then cannot abort the analysis. That decision is shared with the TS path (thisBindingFromAncestors). Template ranges and the class-member flag still come from content-tag viapreprocess. ember-estree is a regular dependency of@glint/ember-tsc, but only the./transform/standaloneentrypoint imports it, so the language server, tsserver plugin andember-tscnever load oxc (verified: importing the main and./transformentries loads neither ember-estree nor oxc-parser).Compatibility.
rewriteModulekeeps its signature and behaviour. Internally,calculateTemplateSpansnow takes a plainEmbeddedTemplateandcalculateTaggedTemplateSpansis a thin adapter over the TS node, so both entries share the emit.Verification.
rewrite-standalone.test.tsruns both entry points over every.gts/.gjsintest-packages(56 files) plus targeted cases (nested classes, arrow fields, static blocks, heritage clauses,satisfies, ASI, confusing tokens in strings/regexes/template literals, gjs) and asserts identicaltransformedContents, errors, directives andtoDebugString(). Same comparison over ember-primitives (132 files), kolay (48) and limber (360): all identical except limber's two deliberately broken tutorial prompts, see below.Mid-keystroke behaviour. Completions while a file is invalid never depended on the transform's parser. Two things carry them: glint's template-level tolerance (handlebars errors inside
templateToTypescriptstill emit the template) and, when the script is broken so content-tag fails, keeping the raw source with verification off so the type-checker's own recovering parser handles completions (regression-tested today). Both apply unchanged to the standalone entry, and TS 7's parser recovers like TS 5's.oxc tolerates less than swc, so there was a third mode in between (templates emitted with no imports/placement). Removed: when ember-estree reports script errors (
ast.errors, NullVoxPopuli/ember-estree#81) the standalone entry records a content-tag-style error and keeps the raw source, exactly like content-tag failure. One rule: script didn't parse → type-checker recovers, templates wait. Verified on{{this.me,{{this., unclosed{{this.,go = () => this.before a template, and an unclosed call before a class: identical outcome torewriteModulein every case.Upstream. Depends on ember-estree 0.8.0, which carries the fixes the parity check found: #77 (content-tag result ordering), #78 (
voidplaceholders for ASI), #82 (keep theexport defaultwrapper) and #81 (ast.errors).Two preprocess fixes the parity check surfaced (both also affect
rewriteModuletoday):preprocessassumed source order and spliced garbage. The parse output is now sorted.isChecked = (v) => v === 1\n<template>…) preprocessed to1[___T\…`], an element access, so the class's own template was emitted as a context-boundtemplateExpressioninstead oftemplateForBackingValue(this, …). Four files in limber's tutorial hit this.preprocessnow emits;[…](a;is an empty class element). Regression test added inrewrite.test.ts`.🤖 Generated with Claude Code
https://claude.ai/code/session_01FsXgxh6vNRWMCNcAQ3g6TN