[leonidaz]

Replaces #183, rebuilt on top of #198 and #199. Follows the rule from #198: positions come from the AST or from the parser, never from arithmetic.

Problem

Since #198 every node's start and end is mapped, so a token that opens a node ([ of a computed Property, { of an object, if, class) is covered. Tokens inside a node have no node boundary to anchor them and stay unmapped:

• the closing ] of a computed key, and both brackets of a computed class member ([ Symbol.iterator ]())
• the ( and ) of parameter lists, calls, and if / for / while / switch / do … while heads
• the { and } of import and export specifier lists and import attributes, of static blocks, switch and enum bodies
• ${ and } in template literals and template-literal types
• every keyword after a node's first token: function after async or declare, else, finally, while in do … while, default after export, type after import / export, class after abstract, the second and later modifiers of a class member, extends, implements, in, of, as, satisfies, enum, interface, namespace

TypeScript reports on ranges that start or end on these: TS2416 / TS2464 / TS1170 on [ Symbol.iterator ] (brackets included), TS1040 on async, TS1029 / TS1030 on a modifier, TS1103 on await in for await, and return-type inlay hints right after the parameter list's ), which with prettier's default trailing commas sits alone on its line. The repro from #183 (comment) on 2.3.11 maps the class member to [ Symbol.iterator and drops the inlay hint. This is also the exact fix for #190: for /* foo */ await /* bar */ (…) maps await to the token instead of to for plus four characters.

Change (minor)

tokens option on ts() / tsx(): the parser's token list, in any of the common shapes (Acorn onToken with type.label / type.keyword, Babel tokens, ESLint-style { type: 'Punctuator' | 'Keyword' | 'Identifier', value }). Its presence is the switch; there is no other flag.

src/languages/ts/tokens.js pulls the delimiters and the keywords listed above out of the token list on first use, sorts them if the parser emitted them out of order (acorn-typescript does when it backtracks), and looks one up by a bounded binary search from a position the AST does know: the ( after the callee's end, the ] after the key's end, the ) before the body's start, the else after the consequent's end, the first function after the node's start (or after its last decorator). Never leaving the node being printed keeps nested tokens apart. Without tokens every lookup returns undefined and the token is written unmapped, exactly as today.

Closers at a node's end () of a call, ] of an array, tuple or computed member, } of an object, switch, enum, mapped type or interface body) are mapped from node.loc.end without tokens, the way BlockStatement already maps its }. Those nodes end with that character by grammar. new Foo may have no parentheses, so NewExpression uses the token lookup instead.

In the printer this is one-line changes of the form

- context.write('[');
+ write_at(context, '[', enclosing(node, node.key, '['));

- context.write('else ');
+ write_keyword(context, node, 'else', node.consequent.loc?.end);
+ context.write(' ');

plus write_params, which replaces nine copies of "write (, print the parameters, write )". Printed output is byte-identical with and without tokens; 62 sample maps gain segments and no code snapshot changes.

Coverage on the samples from the #183 thread (86 delimiters; 107 keyword positions):

	delimiters	keywords
main (2.3.13)	43	86
this PR, no tokens	54	86
this PR, with tokens	86	105

What is left with tokens: the in / instanceof operators, which are out of scope here.

One estimate removed on the way

handle_var_declaration writes the leading keywords with a single call:

token(child_context, node.declare ? `declare ${node.kind}` : node.kind, node);

token() maps the start from node.loc.start, which is exact because the node begins with the keyword, and the end at start + text.length. For a plain const that is exact. For declare const the text is "declare const", so the end lands at start + 13, which is only right when there is exactly one space between the two words; declare const x or declare /* c */ const x puts that end segment in the wrong place. The same kind of estimate #192 removed elsewhere.

Now:

if (node.declare) {
	token(child_context, 'declare', node); // start and end exact: the node begins with it
	child_context.write(' ');
	write_keyword(child_context, node, node.kind); // found in the tokens
} else {
	token(child_context, node.kind, node); // unchanged
}

declare keeps its exact start and end. The kind after it comes from the tokens, so with tokens const maps to its real position wherever it sits, and without tokens it is unmapped. That is the only place where the no-tokens map differs from main (the ts-declare sample loses that one end segment). Plain const / let / var, including in for heads, are unchanged without tokens.

Decorated declarations (after #227)

Follows up #227 (comment). Two things, in the second commit:

• export of a decorated class is mapped without tokens. The export node starts at export with both Acorn and typescript-estree, whichever side of it the decorators were written on, so once the decorators are printed the visitor emits that start itself. The root visitor skips it for an export whose declaration has decorators, so nothing maps the @ column to export any more (@dec export class, export @dec class, and the default variants are tested).
• class, abstract, declare and the member modifiers after decorators come from the tokens. visit_without_decorators no longer clears the declaration's loc and decorators; it records that the decorators were printed, and block_decorators and the root visitor consult that. The node keeps its location, so write_keyword can locate class after the last decorator. The root visitor maps the node's start when that start is the class keyword and skips it when it is a decorator. The two parsers differ here: for @dec export class K {} typescript-estree ranges the class from class (12-22), Acorn from the decorator (0-22, before its parent export node 5-22). Filed as Class range starts at decorators written before export, ahead of the parent export node (typescript-estree starts it at class) acorn-typescript#117. With Acorn the class position is therefore only known from the tokens, which is the case Rich described in that comment.

Tests

sourcemap-keywords.test.js: for each construct, every delimiter or keyword of a kind in the output maps to the matching source token and the output is identical with and without tokens (brackets: 18 cases, parentheses: 27, braces: 11, keywords: 32, each with comments between the tokens; decorated exports and members: 8), plus: outer parentheses chosen for if ((x)), internal tokens unmapped without tokens, closers mapped without tokens, tokens without locations, empty tokens, out-of-order tokens, ESLint-style tokens.

Validation:

• pnpm test — 518 tests passed
• pnpm lint — passed
• pnpm check — the two pre-existing TS2580 Buffer errors in src/index.js on main, nothing new

Commits: the squashed feature, merges of main (mine after #212 / #214 / #215 / #227, then Rich's after #217–#223 and #233), the decorated-declaration follow-up, the catch parentheses, and the braces of an empty type-only import (import type {} from, which #219 prints as one string).

🤖 Generated with Claude Code

[pkg-svelte-dev]

Install the latest version of esrap from a4d5642:

pnpm add https://pkg.svelte.dev/esrap/c/a4d5642a56f4f4e57fa195e2f002c00ecce7f073

Open in pkg.svelte.dev: https://pkg.svelte.dev/repos/esrap/pr/213

Note

This PR is from a fork. A maintainer must approve approve each commit before it can be built and installed.

[changeset-bot]

🦋 Changeset detected

Latest commit: a4d5642

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
esrap	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[Rich-Harris]

Closing since #235 was merged