diff --git a/docs/content/5.reference/1.parse.md b/docs/content/5.reference/1.parse.md index e99b9c6d..f46c5ef8 100644 --- a/docs/content/5.reference/1.parse.md +++ b/docs/content/5.reference/1.parse.md @@ -360,6 +360,7 @@ console.timeEnd('createMarkdownParser x1000') | `comark` | `parseMarkdown()`, `createMarkdownParser()`, `createSerializedMarkdownParser()` with the default plugins and auto-close, plus `autoCloseMarkdown()`, `createComarkContext()`, `defineComarkPlugin()`, `parseFrontmatter()` | | `comark/core` | The same three parser functions with no default plugins and no built-in auto-closer | | `comark/auto-close` | `autoCloseMarkdown()` and its placeholders, without the parser | +| `comark/auto-close/core` | A CommonMark/GFM-only `autoCloseMarkdown()` and `dropTrailingOpeners()`, without Comark syntax or table completion | | `comark/context` | `createComarkContext()` and the live-update types, without the parser | | `comark/parse` | **Deprecated.** Alias of `comark`. Import from `comark` instead | @@ -368,7 +369,7 @@ console.timeEnd('createMarkdownParser x1000') Use `comark/core` when you want to choose every plugin yourself and keep the default plugins out of your bundle. It accepts the same options as `comark`, with two differences: - Only the plugins in `plugins` run. `registerDefaultPlugins` is not accepted; add [`linkify()`](/plugins/defaults/linkify) to link bare URLs. -- `autoClose` is a function or `undefined` (the default, which skips auto-closing). Pass `autoCloseMarkdown` from `comark/auto-close` to opt in. +- `autoClose` is a function or `undefined` (the default, which skips auto-closing). Pass `autoCloseMarkdown` from `comark/auto-close` to opt in, or from `comark/auto-close/core` when you register none of the components, attributes, math or frontmatter plugins and do not need incomplete tables completed. ```typescript [core.ts] import { createMarkdownParser } from 'comark/core' diff --git a/docs/content/5.reference/2.auto-close.md b/docs/content/5.reference/2.auto-close.md index c663aa2a..3575e0c7 100644 --- a/docs/content/5.reference/2.auto-close.md +++ b/docs/content/5.reference/2.auto-close.md @@ -107,6 +107,25 @@ During incremental parsing with `{ streaming: true }`, the custom function recei To heal text without loading the parser, import from `comark/auto-close` instead of `comark`. :: +### CommonMark/GFM only + +`comark/auto-close/core` exports an `autoCloseMarkdown()` that heals CommonMark and GFM inline syntax and leaves components, `{...}` attributes, math, frontmatter and tables as text. Its result matches `comark/auto-close` with `syntax`, `attributes`, `math`, `frontmatter` and `tables` set to `false`, and it is about a third smaller. Use it with a parser that has none of those plugins, or with a renderer that completes tables itself. + +It accepts the inline and link options above. `dropTrailingOpeners` is a separate function: run it first, with the same `inlineCode` option. + +```typescript [core.ts] +import { autoCloseMarkdown, dropTrailingOpeners } from 'comark/auto-close/core' + +autoCloseMarkdown('Some **bold and `code') +// 'Some **bold and `code`**' + +autoCloseMarkdown('::alert\nSome **bold') +// '::alert\nSome **bold**' (the component stays open) + +autoCloseMarkdown(dropTrailingOpeners('hello *')) +// 'hello' +``` + ### Supported syntax #### Inline Markdown diff --git a/packages/comark/SPEC/auto-close.md b/packages/comark/SPEC/auto-close.md index 156e0254..227dbecd 100644 --- a/packages/comark/SPEC/auto-close.md +++ b/packages/comark/SPEC/auto-close.md @@ -45,6 +45,8 @@ Self-healing markdown for streaming. Completes incomplete syntax so partial AI o Legacy aliases accepted for one release: `katex` → `blockMath`, `math` and `inlineKatex` → `inlineMath`. +An implementation may leave out `blockMath`, `inlineMath` and `tables`, treating them as `false`. Comark's `comark/auto-close/core` does, and matches `comark/auto-close` with those options off. + ## Case format Each case is a `diff` fence with one `-` line (input) and one `+` line (expected output). `\n`, `\r`, `\t` are literal escape sequences for the corresponding characters; every other character is verbatim (so trailing spaces on the `+` line are part of the expected output). Options for a single case go in the fence info string, never in the `+` line: diff --git a/packages/comark/package.json b/packages/comark/package.json index 93663ac0..d1e6401f 100644 --- a/packages/comark/package.json +++ b/packages/comark/package.json @@ -54,6 +54,7 @@ "./parse": "./dist/index.js", "./core": "./dist/core.js", "./auto-close": "./dist/auto-close.js", + "./auto-close/core": "./dist/auto-close/core.js", "./context": "./dist/context.js", "./render": "./dist/render/index.js", "./render/html": "./dist/render/html.js" diff --git a/packages/comark/src/auto-close.ts b/packages/comark/src/auto-close.ts index c4cd74f5..16a8d745 100644 --- a/packages/comark/src/auto-close.ts +++ b/packages/comark/src/auto-close.ts @@ -1,36 +1,22 @@ /** * Auto-closes unclosed markdown and Comark component syntax. * - * Two O(n) passes, no backtracking: - * - * 1. `scanBlocks` — one walk over the document on offsets only (no per-line - * strings): fences, raw HTML, frontmatter, block math, component stack, - * trailing table, and the range eligible for inline healing (the last - * soft-wrapped paragraph). - * 2. `healRegion` — one walk over that range with a single delimiter stack. - * Unclosed frames are closed LIFO at EOF; edits (tilde escapes, dropped - * brackets) are collected as positions and applied once at the end, so the - * common "nothing to escape" path never copies the string. - * * Behavioral contract: `packages/comark/SPEC/auto-close.md`. */ +import type { AutoCloseCoreOptions, Frame, Syntax } from './internal/auto-close.ts' +import { autoClose, dropTrailingOpeners, F_MATH, LINE_DROP, LINE_SKIP, resolveOptions } from './internal/auto-close.ts' import { findBracketClose, isBracketMath } from './internal/bracket-math.ts' -export const INCOMPLETE_LINK_PLACEHOLDER = 'comark:incomplete-link' -export const INCOMPLETE_IMAGE_PLACEHOLDER = 'comark:incomplete-image' +export { INCOMPLETE_IMAGE_PLACEHOLDER, INCOMPLETE_LINK_PLACEHOLDER } from './internal/auto-close.ts' +export type { LinkMode } from './internal/auto-close.ts' -export type LinkMode = 'protocol' | 'text-only' - -export interface AutoCloseOptions { +export interface AutoCloseOptions extends AutoCloseCoreOptions { frontmatter?: boolean /** Component fences (`::`). Default true. */ syntax?: boolean /** Attached `{...}` attribute scopes. Defaults to `syntax`. */ attributes?: boolean - linkMode?: LinkMode - incompleteLinkPlaceholder?: string - incompleteImagePlaceholder?: string /** * Auto-close math. When set, enables both block and inline unless the more * specific flags override. Enabled automatically when the math plugin is used @@ -50,251 +36,88 @@ export interface AutoCloseOptions { * Enabled automatically when `parseMarkdown(..., { streaming: true })`. */ dropTrailingOpeners?: boolean - /** Auto-close incomplete links (`[text`). Default true. */ - links?: boolean - /** Auto-close incomplete images (`![alt`). Default true. */ - images?: boolean - /** Auto-close `**bold**`. Default true. */ - bold?: boolean - /** Auto-close `*italic*` / `_italic_`. Default true. */ - italic?: boolean - /** Auto-close `~~strikethrough~~`. Default true. */ - strikethrough?: boolean - /** Auto-close inline `` `code` ``. Default true. */ - inlineCode?: boolean - /** Auto-close `***bold-italic***`. Default true. */ - boldItalic?: boolean - /** Escape mid-word single `~` (`20~25` → `20\~25`). Default true. */ - singleTilde?: boolean - /** - * Escape list-item comparison openers (`- > 25` → `- \> 25`) so they are not - * parsed as blockquotes. Default true. - */ - comparisonOperators?: boolean - /** Strip incomplete HTML tags at EOF (`Hello = 0) - - return result + return autoClose(source, o, new ComarkSyntax(options, syntax)) } -// --------------------------------------------------------------------------- -// Block pass -// --------------------------------------------------------------------------- - interface Component { depth: number indent: string yaml: boolean } -interface DocState { - /** Offsets of the region to heal, or `start < 0` for "nothing to heal". */ - start: number - end: number - /** Effective document end — a trailing bare `::` is dropped. */ - docEnd: number - fmOpen: boolean - fmContent: boolean - mathOpen: boolean - /** The document ends inside a table block. */ - table: boolean - comps: Component[] -} - -const RAW_TEXT_OPEN_RE = /^<(script|pre|style|textarea)(\s|>|$)/i -const RAW_TEXT_CLOSE_RE = { - script: /<\/script\s*>/i, - pre: /<\/pre\s*>/i, - style: /<\/style\s*>/i, - textarea: /<\/textarea\s*>/i, -} as const - -type RawTag = keyof typeof RAW_TEXT_CLOSE_RE - -/** - * One walk over the document, on offsets only: no per-line strings, so a long - * streaming document costs a single scan plus one slice of the healed region. - * - * Every line is examined as three offsets: `ls` (start), `st`…`en` (the trimmed - * content) and `le` (end, before the newline). - */ -function scanBlocks(src: string, o: Opts): DocState { - const len = src.length - const starts: number[] = [0] - for (let i = src.indexOf('\n'); i !== -1; i = src.indexOf('\n', i + 1)) starts.push(i + 1) - let n = starts.length - let docEnd = len - - const comps: Component[] = [] - let fenceLen = 0 - let fenceCh = 0 - let raw: RawTag | null = null - let fmOpen = false - let fmContent = false - let mathOpen = false - let table = false - let shielded = -1 // last line that structurally shields what precedes it. - - for (let i = 0; i < n; i++) { - const ls = starts[i] - const le = i + 1 < n ? starts[i + 1] - 1 : len - let st = ls - while (st < le && isIndentCode(src.charCodeAt(st))) st++ - let en = le - while (en > st && isSpaceCode(src.charCodeAt(en - 1))) en-- - const c0 = st < en ? src.charCodeAt(st) : -1 - - if (raw !== null) { - if (RAW_TEXT_CLOSE_RE[raw].test(src.slice(ls, le))) raw = null - continue - } - if (c0 === 60 /* < */) { - const m = RAW_TEXT_OPEN_RE.exec(src.slice(st, en)) - if (m !== null) { - const tag = m[1].toLowerCase() as RawTag - if (!RAW_TEXT_CLOSE_RE[tag].test(src.slice(ls, le))) raw = tag - continue - } - } +class ComarkSyntax implements Syntax { + open = false + #components: boolean + #frontmatter: boolean + #blockMath: boolean + #inlineMath: boolean + #bracketMath: boolean + #tables: boolean + #comps: Component[] = [] + #fmOpen = false + #fmContent = false + #mathOpen = false + // Line end up to which no `\)` / `\]` was found (bracket math), or -1. + #noParenClose = -1 + #noBracketClose = -1 - if (c0 === 96 /* ` */ || c0 === 126 /* ~ */) { - let run = 1 - while (st + run < en && src.charCodeAt(st + run) === c0) run++ - if (run >= 3) { - if (fenceLen === 0) { - // `` ```code`` `` is an incomplete inline code span, not a fence opener. - if (!(c0 === 96 && isIncompleteInlineFence(src, st, en))) { - fenceLen = run - fenceCh = c0 - shielded = i - continue - } - } else if (c0 === fenceCh && run >= fenceLen && st + run === en) { - // A closer is the same character, at least as long, and followed only by - // whitespace. `` ```js `` inside an open block is code content, not a closer. - fenceLen = 0 - shielded = i - continue - } else { - shielded = i - continue - } - } - } - if (fenceLen !== 0) continue + constructor(options: AutoCloseOptions, components: boolean) { + const math = options.math === true + this.#components = components + this.#frontmatter = options.frontmatter === true + this.#blockMath = options.blockMath ?? math + this.#inlineMath = options.inlineMath ?? math + this.#bracketMath = options.bracketMath === true + this.#tables = options.tables !== false + } + line(src: string, i: number, ls: number, st: number, en: number, c0: number, last: boolean): number { + // The closer shields a `$$` line itself; only the open block is tracked here. if (c0 === 36 /* $ */ && en - st === 2 && src.charCodeAt(st + 1) === 36) { - if (o.blockMath) mathOpen = !mathOpen - shielded = i - continue + if (this.#blockMath) { + this.#mathOpen = !this.#mathOpen + this.open = this.#fmOpen || this.#mathOpen + } + return 0 } const dashes = c0 === 45 /* - */ && en - st === 3 && src.charCodeAt(st + 1) === 45 && src.charCodeAt(st + 2) === 45 - if (i === 0 && o.frontmatter && dashes) { - fmOpen = true - continue + if (i === 0 && this.#frontmatter && dashes) { + this.#fmOpen = this.open = true + return LINE_SKIP } - if (fmOpen) { - if (dashes) fmOpen = false - else if (c0 !== -1) fmContent = true - continue + if (this.#fmOpen) { + if (dashes) { + this.#fmOpen = false + this.open = this.#mathOpen + } else if (c0 !== -1) this.#fmContent = true + return LINE_SKIP } + const comps = this.#comps if (dashes && comps.length > 0) { const top = comps[comps.length - 1] top.yaml = !top.yaml - continue + return LINE_SKIP } - table = c0 === 124 /* | */ - - if (o.syntax && c0 === 58 /* : */) { + if (this.#components && c0 === 58 /* : */) { let colons = 1 while (st + colons < en && src.charCodeAt(st + colons) === 58) colons++ const bare = st + colons === en - if (bare && i === n - 1 && comps.length === 0) { - // Trailing bare `::` — drop it so it does not flash as text. - docEnd = ls - n-- - break - } + // Trailing bare `::` — drop it so it does not flash as text. + if (bare && last && comps.length === 0) return LINE_DROP if (colons >= 2) { if (!bare && isNameStart(src.charCodeAt(st + colons))) { comps.push({ depth: colons, indent: src.slice(ls, st), yaml: false }) @@ -303,478 +126,71 @@ function scanBlocks(src: string, o: Opts): DocState { } } } - } - - const doc: DocState = { start: -1, end: -1, docEnd, fmOpen, fmContent, mathOpen, table, comps } - - // Nothing is healed while a shielded region is still open at EOF. - if (fenceLen !== 0 || raw !== null || fmOpen || mathOpen) return doc - - const lineEnd = (i: number) => (i + 1 < n ? starts[i + 1] - 1 : docEnd) - - let e = n - 1 - while (e > 0 && lineEnd(e) === starts[e]) e-- - - // A half-typed list item at the end of a list is a marker with no content yet - // (`- item\n- ` → `- item`). Only inside a list — under a paragraph the same line - // is a setext candidate and gets the U+200B guard instead. - while ( - e > shielded + 1 && - isEmptyListItem(src, starts[e], lineEnd(e)) && - isListItem(src, starts[e - 1], lineEnd(e - 1)) - ) { - docEnd = starts[e] - 1 // drop the line and the newline before it - n = e - e-- - while (e > 0 && lineEnd(e) === starts[e]) e-- - } - doc.docEnd = docEnd - - if (lineEnd(e) === starts[e] || e <= shielded) return doc - - // Widen to the soft-wrapped paragraph: stop at a blank line or a new block. - let s = e - while (s > shielded + 1 && lineEnd(s - 1) !== starts[s - 1] && !isBlockStart(src, starts[s], lineEnd(s))) s-- - if (s !== e) { - let join = !isBlockStart(src, starts[s], lineEnd(s)) || isListItem(src, starts[s], lineEnd(s)) - for (let i = s + 1; join && i <= e; i++) if (isBlockStart(src, starts[i], lineEnd(i))) join = false - if (!join) s = e - } - - // Indented code is literal. - if (s === e && isIndentedCode(src, starts[e], lineEnd(e)) && !isListItem(src, starts[e], lineEnd(e))) return doc - - doc.start = starts[s] - doc.end = lineEnd(e) - return doc -} - -/** `` ```python code`` `` — a code span that merely looks like a fence. */ -function isIncompleteInlineFence(src: string, st: number, en: number): boolean { - if (en - st < 5) return false - if (src.charCodeAt(en - 1) !== 96 || src.charCodeAt(en - 2) !== 96 || src.charCodeAt(en - 3) === 96) return false - return !src.slice(st + 3, en).includes('```') -} - -const ATX_HEADING_RE = /^#{1,6}(\s|$)/ -const THEMATIC_BREAK_RE = /^(\*{3,}|_{3,}|-{3,})\s*$/ -const ORDERED_LIST_RE = /^\d{1,9}[.)](\s|$)/ - -function isIndentCode(c: number): boolean { - return c === 32 || c === 9 -} - -/** True when a line opens a new block, so paragraph joining must not cross it. */ -function isBlockStart(src: string, ls: number, le: number): boolean { - if (ls >= le) return false - if (isIndentedCode(src, ls, le)) return true - let st = ls - while (st < le && isIndentCode(src.charCodeAt(st))) st++ - if (st >= le) return false - const c = src.charCodeAt(st) - if (c === 96 || c === 126) { - let run = 1 - while (st + run < le && src.charCodeAt(st + run) === c) run++ - if (run >= 3) return true - } - if (c === 35 /* # */) return ATX_HEADING_RE.test(src.slice(st, le)) - if (c === 62 /* > */ || c === 124 /* | */) return true - if ((c === 42 || c === 95 || c === 45) && THEMATIC_BREAK_RE.test(src.slice(st, le))) return true - return isListItem(src, st, le) -} - -/** CommonMark indented code: ≥ 4 leading spaces (or a tab) plus content. */ -function isIndentedCode(src: string, ls: number, le: number): boolean { - if (ls >= le) return false - if (src.charCodeAt(ls) === 9) return true - let i = ls - while (i < le && src.charCodeAt(i) === 32) i++ - return i - ls >= 4 && i < le -} - -function isListItem(src: string, ls: number, le: number): boolean { - let st = ls - while (st < le && isIndentCode(src.charCodeAt(st))) st++ - if (st >= le) return false - const c = src.charCodeAt(st) - if (c === 45 || c === 43 || c === 42) { - const c1 = st + 1 < le ? src.charCodeAt(st + 1) : -1 - return c1 === 32 || c1 === 9 - } - return c >= 48 && c <= 57 && ORDERED_LIST_RE.test(src.slice(st, le)) -} - -/** A list marker with nothing after it: `-`, `- `, `1.`, `2) ` — but not `- x` or `---`. */ -function isEmptyListItem(src: string, ls: number, le: number): boolean { - let st = ls - while (st < le && isIndentCode(src.charCodeAt(st))) st++ - let en = le - while (en > st && isSpaceCode(src.charCodeAt(en - 1))) en-- - if (st >= en) return false - const c = src.charCodeAt(st) - if (c === 45 || c === 43 || c === 42) return st + 1 === en - let i = st - while (i < en && src.charCodeAt(i) >= 48 && src.charCodeAt(i) <= 57) i++ - if (i === st || i === en) return false - const mark = src.charCodeAt(i) - return (mark === 46 /* . */ || mark === 41) /* ) */ && i + 1 === en -} - -function isNameStart(c: number): boolean { - return (c >= 97 && c <= 122) || (c >= 65 && c <= 90) || c === 36 /* $ */ -} - -// --------------------------------------------------------------------------- -// Inline pass -// --------------------------------------------------------------------------- - -const F_STAR = 1 -const F_UNDER = 2 -const F_TILDE = 3 -const F_CODE = 4 -const F_MATH = 5 -const F_LINK = 6 - -/** An open construct. `len` is the delimiter run still to be closed. */ -interface Frame { - k: number - len: number - pos: number -} - -type Edit = [pos: number, replacement: string] - -const LIST_COMPARE_PREFIX_RE = /^\s*(?:[-*+]|\d+[.)]) +$/ -const LIST_COMPARE_VALUE_RE = /^=?\s*\$?\d/ - -/** - * Heals one paragraph-sized region: closes every construct still open at EOF, - * escapes mid-word tildes and list comparison operators, drops an incomplete - * HTML tag, and completes an incomplete link or image. - */ -function healRegion(text: string, o: Opts): string { - let len = text.length - // One trailing space is streaming noise; two are a hard line break. - if (text.charCodeAt(len - 1) === 32 && text.charCodeAt(len - 2) !== 32) len-- - if (len === 0) return '' - - const frames: Frame[] = [] - let edits: Edit[] | null = null - - let lineStart = 0 - let escEnd = -1 // index just past the last escape sequence - let attr = 0 - let htmlLt = -1 - let starInWord = false - let starDisabled = false - // The last word-internal `*` run left literal, as its length and the frame - // depth it was seen at. Until the next whitespace or the next `*` frame - // opened or closed, a same-length run that would open pairs with it - // instead, so `这是**中文。**后续` gets no closer. - let starSkippedLen = 0 - let starSkippedDepth = -1 - // Line end up to which no `\)` / `\]` was found (bracket math), or -1. - let noParenClose = -1 - let noBracketClose = -1 - - for (let i = 0; i < len; i++) { - const c = text.charCodeAt(i) - const top = frames.length === 0 ? null : frames[frames.length - 1] - - if (c === 10 /* \n */) { - lineStart = i + 1 - starInWord = false - starSkippedLen = 0 - htmlLt = -1 - continue - } - - // A code span shields everything but its own closer. - if (top !== null && top.k === F_CODE) { - if (c === 96) { - let n = 1 - while (text.charCodeAt(i + n) === 96) n++ - if (n >= top.len) frames.pop() - else if (i + n === len) top.len -= n - i += n - 1 - } - continue - } - - if (c === 32 || c === 9 || c === 13) { - starInWord = false - starSkippedLen = 0 - continue - } - if (c === 92 /* \ */) { - // A closed `\(…\)` or `\[…\]` is math under `math({ brackets: true })`, so it - // isn't healed. Mirrors the parser: `\(` and inline `\[` close on the same - // line, `\[` after only indentation may span lines (no blank line, nothing - // after `\]`), and citations or `\[text\](url)` stay text. - if (o.bracketMath) { - const next = text.charCodeAt(i + 1) - if (next === 40 /* ( */ || next === 91 /* [ */) { - const block = next === 91 && isBlankRange(text, lineStart, i) - let end = len - if (!block) { - const nl = text.indexOf('\n', i + 2) - if (nl !== -1 && nl < len) end = nl - } - // A search that failed up to `end` fails from any later opener too. - const failed = next === 40 ? noParenClose : noBracketClose - const close = failed === end ? -1 : findBracketClose(text, i + 2, next === 40 ? 41 : 93, end) - if (close === -1) { - if (next === 40) noParenClose = end - else noBracketClose = end - } - if ( - close !== -1 && - isBracketMath(text.slice(i + 2, close)) && - !(next === 91 && text.charCodeAt(close + 2) === 40 /* ( */) && - (!block || isBracketBlock(text, i + 2, close, len)) - ) { - i = close + 1 - escEnd = i + 1 - continue - } - } - } - i++ - escEnd = i + 1 - continue - } - if (attr > 0) { - if (c === 125 /* } */) attr-- - continue - } - if (htmlLt >= 0) { - if (c === 62 /* > */) htmlLt = -1 - continue - } - - if (c === 96 /* ` */) { - let n = 1 - while (text.charCodeAt(i + n) === 96) n++ + return 0 + } + + // A closed `\(…\)` or `\[…\]` is math under `math({ brackets: true })`, so it + // isn't healed. Mirrors the parser: `\(` and inline `\[` close on the same + // line, `\[` after only indentation may span lines (no blank line, nothing + // after `\]`), and citations or `\[text\](url)` stay text. + bracket(text: string, i: number, lineStart: number, len: number): number { + const next = text.charCodeAt(i + 1) + if (!this.#bracketMath || (next !== 40 /* ( */ && next !== 91) /* [ */) return -1 + const block = next === 91 && isBlankRange(text, lineStart, i) + let end = len + if (!block) { + const nl = text.indexOf('\n', i + 2) + if (nl !== -1 && nl < len) end = nl + } + // A search that failed up to `end` fails from any later opener too. + const failed = next === 40 ? this.#noParenClose : this.#noBracketClose + const close = failed === end ? -1 : findBracketClose(text, i + 2, next === 40 ? 41 : 93, end) + if (close === -1) { + if (next === 40) this.#noParenClose = end + else this.#noBracketClose = end + return -1 + } + if ( + isBracketMath(text.slice(i + 2, close)) && + !(next === 91 && text.charCodeAt(close + 2) === 40 /* ( */) && + (!block || isBracketBlock(text, i + 2, close, len)) + ) + return close + return -1 + } + + dollar(text: string, i: number, len: number, frames: Frame[], top: Frame | null): number { + let n = 1 + while (text.charCodeAt(i + n) === 36) n++ + if (top !== null && top.k === F_MATH) { + if (n >= top.len) frames.pop() + else if (i + n === len) top.len -= n + } else { const after = i + n < len ? text.charCodeAt(i + n) : -1 - if (o.inlineCode && after !== -1 && after !== 10) frames.push({ k: F_CODE, len: n, pos: i }) - i += n - 1 - continue - } - - if (c === 36 /* $ */) { - let n = 1 - while (text.charCodeAt(i + n) === 36) n++ - if (top !== null && top.k === F_MATH) { - if (n >= top.len) frames.pop() - else if (i + n === len) top.len -= n - } else { - const after = i + n < len ? text.charCodeAt(i + n) : -1 - if (n === 1) { - // `$50` is currency and `::$name` is a component name, not math. - const attached = after > 13 && after !== 32 - if (o.inlineMath && attached && !(after >= 48 && after <= 57) && text.charCodeAt(i - 1) !== 58) - frames.push({ k: F_MATH, len: 1, pos: i }) - } else if (n === 2 && o.blockMath) { - // `$$` may be padded (`$$ x + y $$`); only EOL/EOF means "not an opener". - if (after !== -1 && after !== 10 && after !== 13) frames.push({ k: F_MATH, len: 2, pos: i }) - } - } - i += n - 1 - continue - } - - // Inside math only code spans and the math closer are syntax. - if (top !== null && top.k === F_MATH) continue - - if (c === 42 /* * */ || c === 95 /* _ */) { - let n = 1 - while (text.charCodeAt(i + n) === c) n++ - const prev = codePointBefore(text, i) - const next = i + n < len ? codePointAt(text, i + n) : '' - const attachedLeft = prev !== '' && !isSpaceCode(prev.charCodeAt(0)) && i !== escEnd - // `_` only opens at a boundary: whitespace, an escape or another marker. - // That keeps `func(_arg` literal while `**_text` and `\___bold` still open. - const openLeft = !attachedLeft || (c === 95 && (prev === '*' || prev === '_' || prev === '~')) - const openRight = next !== '' && !isSpaceCode(next.charCodeAt(0)) - const enabled = n === 1 ? o.italic : n === 2 ? o.bold : o.boldItalic - - if (c === 42) { - // Word-internal `*` only counts once the word already holds a delimiter - // (`*foo*bar*baz` pairs up, `abc*123` stays literal). - const inWord = attachedLeft && isAlnum(prev) && isAlnum(next) - if (inWord && !starInWord) { - starSkippedLen = n - starSkippedDepth = frames.length - } else { - if (attachedLeft && top !== null && top.k === F_STAR && (!inWord || top.len === n)) { - closeRun(frames, F_STAR, n) - starSkippedLen = 0 - starInWord = true - i += n - 1 - continue - } - // An attached run after punctuation (`中文。**后续`) that would open - // instead pairs with a word-internal run left literal earlier in the - // same whitespace-free stretch, with no frame opened or closed in - // between. CommonMark cannot pair the two, and the second run is a - // would-be closer far more often than an opener (CJK bold that ends - // in punctuation), so it is left literal instead of being closed. - // After opening punctuation (`bar(**baz`) it is an opener, as usual. - if ( - attachedLeft && - !inWord && - starSkippedLen === n && - starSkippedDepth === frames.length && - !isOpeningPunct(prev) - ) { - starSkippedLen = 0 - i += n - 1 - continue - } - if (openRight && enabled) { - frames.push({ k: F_STAR, len: n, pos: i }) - starSkippedLen = 0 - starInWord = true - i += n - 1 - continue - } - if (openRight && n === 1) starDisabled = true - } - } else { - // `_` closes only against a non-word on the right, so `snake_case` and - // `some__field` stay literal mid-word. - if (attachedLeft && !isAlnum(next) && top !== null && top.k === F_UNDER) { - closeRun(frames, F_UNDER, n) - i += n - 1 - continue - } - // A `_` run after punctuation and followed by a word char (`__下線、__次`, - // `__a.__b`, `_a(__b`) can only open in CommonMark, so any `_` closer - // appended at EOF would pair with this run instead of the open frames. - if (attachedLeft && isAlnum(next) && isPunct(prev) && top !== null && top.k === F_UNDER) { - while (frames.length > 0 && frames[frames.length - 1].k === F_UNDER) frames.pop() - // The scope a skipped `*` run was recorded in is gone with the frame. - starSkippedLen = 0 - i += n - 1 - continue - } - if (openLeft && openRight && enabled) frames.push({ k: F_UNDER, len: n, pos: i }) + if (n === 1) { + // `$50` is currency and `::$name` is a component name, not math. + const attached = after > 13 && after !== 32 + if (this.#inlineMath && attached && !(after >= 48 && after <= 57) && text.charCodeAt(i - 1) !== 58) + frames.push({ k: F_MATH, len: 1, pos: i }) + } else if (n === 2 && this.#blockMath) { + // `$$` may be padded (`$$ x + y $$`); only EOL/EOF means "not an opener". + if (after !== -1 && after !== 10 && after !== 13) frames.push({ k: F_MATH, len: 2, pos: i }) } - i += n - 1 - continue - } - - if (c === 126 /* ~ */) { - let n = 1 - while (text.charCodeAt(i + n) === 126) n++ - const prev = codePointBefore(text, i) - const next = i + n < len ? codePointAt(text, i + n) : '' - const attachedLeft = prev !== '' && !isSpaceCode(prev.charCodeAt(0)) - if (top !== null && top.k === F_TILDE && attachedLeft) { - closeRun(frames, F_TILDE, n) - } else if (n === 2 && o.strikethrough && next !== '' && !isSpaceCode(next.charCodeAt(0))) { - frames.push({ k: F_TILDE, len: 2, pos: i }) - } else if (n === 1 && o.singleTilde && isAlnum(prev) && isAlnum(next) && !isSubscriptTilde(text, i)) { - // A lone mid-word `~` would become GFM strikethrough — keep it literal. - edits = push(edits, [i, '\\~']) - } - i += n - 1 - continue - } - - if (c === 91 /* [ */) { - const image = text.charCodeAt(i - 1) === 33 /* ! */ - if (image ? o.images : o.links) frames.push({ k: F_LINK, len: image ? 2 : 1, pos: image ? i - 1 : i }) - continue - } - - if (c === 93 /* ] */) { - let b = frames.length - 1 - while (b >= 0 && frames[b].k !== F_LINK) b-- - if (b < 0) continue - const label = frames[b] - frames.length = b // the label closed: markers opened inside it stay literal - const after = i + 1 < len ? text.charCodeAt(i + 1) : -1 - - if (after === 40 /* ( */) { - const end = scanDestination(text, i + 2, len) - if (end > 0) { - i = end // complete `[text](url)` - continue - } - if (end < 0) continue // spaces inside: not a destination, keep scanning - } else if (after === 91 /* [ */) { - let j = i + 2 - while (j < len && text.charCodeAt(j) !== 93) j++ - if (j < len) { - i = j // complete `[text][ref]` - continue - } - } else if (!(label.len === 2 && i + 1 === len && hasContent(text, label.pos + 2, i))) { - continue // `[text]` may be a shortcut reference; only images are healed - } - - // Incomplete destination / reference: the link wins over everything inside. - if (label.len === 2 || o.linkMode === 'protocol') { - return apply(text, edits, i + 1, '(' + (label.len === 2 ? o.imagePh : o.linkPh) + ')') - } - return apply(text, push(edits, [label.pos, '']), i, '') - } - - if (c === 60 /* < */) { - const next = text.charCodeAt(i + 1) - if (next === 47 /* / */ || isAsciiAlpha(next)) htmlLt = i - continue - } - - if (c === 62 /* > */) { - if ( - o.comparisonOperators && - LIST_COMPARE_PREFIX_RE.test(text.slice(lineStart, i)) && - LIST_COMPARE_VALUE_RE.test(text.slice(i + 1)) - ) { - edits = push(edits, [i, '\\>']) - } - continue - } - - if (c === 123 /* { */ && o.attributes) { - // Only an attached `{` is an attribute scope (`text{.cls`), never a bare line. - const prev = text.charCodeAt(i - 1) - if (i > 0 && prev !== 32 && prev !== 9 && prev !== 10) attr++ - continue } + return n } - let end = len - - // Incomplete HTML tag at EOF, plus the whitespace in front of it. - if (htmlLt >= 0 && o.htmlTags) { - end = htmlLt - while (end > 0 && (text.charCodeAt(end - 1) === 32 || text.charCodeAt(end - 1) === 9)) end-- - while (frames.length > 0 && frames[frames.length - 1].pos >= htmlLt) frames.pop() - } - - // Unclosed `[` / `![` at EOF. An empty label is a half-typed marker, not a link: - // there is nothing to wrap, so `hello [` stays literal. - let b = frames.length - 1 - while (b >= 0 && frames[b].k !== F_LINK) b-- - if (b >= 0 && hasContent(text, frames[b].pos + frames[b].len, end)) { - const label = frames[b] - if (label.len === 2 || o.linkMode === 'protocol') { - return apply(text, edits, end, '](' + (label.len === 2 ? o.imagePh : o.linkPh) + ')') - } - // text-only: unwrap link labels from the inside out, stop at an image. - for (let k = frames.length - 1; k >= 0; k--) { - const f = frames[k] - if (f.k !== F_LINK) continue - frames.length = k - if (f.len === 2) return apply(text, edits, end, '](' + o.imagePh + ')') - edits = push(edits, [f.pos, '']) - } + finish(result: string, source: string, healed: boolean, table: boolean): string { + if (this.#tables && table) result = closeTables(result) + if (this.#blockMath && this.#mathOpen) result += result.endsWith('\n') ? '$$' : '\n$$' + if (this.#fmOpen && this.#fmContent) result = closeDelimiterLine(result, '-', 3) + if (this.#components) result = closeComponents(result, source, this.#comps, healed) + return result } +} - return apply(text, edits, end, closers(frames, text, o, starDisabled)) +function isNameStart(c: number): boolean { + return (c >= 97 && c <= 122) || (c >= 65 && c <= 90) || c === 36 /* $ */ } function isBlankRange(text: string, from: number, to: number): boolean { @@ -799,121 +215,6 @@ function isBracketBlock(text: string, from: number, close: number, len: number): return text.slice(close + 2, le).trim() === '' } -/** Consume `n` delimiters against the open frames of `kind`, innermost first. */ -function closeRun(frames: Frame[], kind: number, n: number): void { - let m = n - while (m > 0 && frames.length > 0) { - const f = frames[frames.length - 1] - if (f.k !== kind) break - if (m >= f.len) { - m -= f.len - frames.pop() - } else { - // Partial closer: `**bold*` leaves one `*` to complete. - f.len -= m - m = 0 - } - } -} - -/** - * End of an inline link destination: index of the closing `)`, `0` when the - * destination is unterminated, `-1` when it holds whitespace (so it is not a - * destination at all and the text must keep flowing). - */ -function scanDestination(text: string, from: number, len: number): number { - let depth = 1 - for (let i = from; i < len; i++) { - const c = text.charCodeAt(i) - if (c === 92) i++ - else if (c === 40) depth++ - else if (c === 41) { - if (--depth === 0) return i - } else if (c === 32 || c === 9 || c === 10) return -1 - } - return 0 -} - -/** LIFO closers for everything still open. */ -function closers(frames: Frame[], text: string, o: Opts, starDisabled: boolean): string { - const skipStar = starDisabled && !o.italic - let starSum = 0 - let starCount = 0 - let innerStar = -1 - for (let i = frames.length - 1; i >= 0; i--) { - if (frames[i].k !== F_STAR) continue - starSum += frames[i].len - starCount++ - if (innerStar < 0) innerStar = i - } - // A cross-family construct outside the innermost `*` cannot be nested through: - // `~~strike **bold *italic` closes only the `*` (SPEC), never `***~~`. - let crossOutside = false - for (let i = innerStar - 1; i >= 0; i--) { - const k = frames[i].k - if (k === F_TILDE || k === F_UNDER || k === F_MATH) { - crossOutside = true - break - } - } - - let out = '' - let starDone = false - for (let i = frames.length - 1; i >= 0; i--) { - const f = frames[i] - switch (f.k) { - case F_STAR: - if (starDone || skipStar) break - starDone = true - out += '*'.repeat(starCount >= 2 && crossOutside ? f.len : starSum) - break - case F_UNDER: - out += '_'.repeat(f.len) - break - case F_TILDE: - out += '~'.repeat(f.len) - break - case F_CODE: - out += '`'.repeat(f.len) - break - case F_MATH: - // A `$$` block opened on an earlier line closes on its own line. - out += f.len === 2 && text.lastIndexOf('\n') > f.pos ? '\n$$' : '$'.repeat(f.len) - break - } - } - return out -} - -function push(edits: Edit[] | null, edit: Edit): Edit[] { - if (edits === null) return [edit] - edits.push(edit) - return edits -} - -/** `text[0, end)` with every edit applied, then `suffix`. */ -function apply(text: string, edits: Edit[] | null, end: number, suffix: string): string { - if (edits === null) return end === text.length ? (suffix === '' ? text : text + suffix) : text.slice(0, end) + suffix - if (edits.length > 1) edits.sort(byPos) - let out = '' - let copied = 0 - for (let i = 0; i < edits.length; i++) { - const [pos, replacement] = edits[i] - if (pos >= end) break - out += text.slice(copied, pos) + replacement - copied = pos + 1 - } - return out + text.slice(copied, end) + suffix -} - -function byPos(a: Edit, b: Edit): number { - return a[0] - b[0] -} - -// --------------------------------------------------------------------------- -// Document-level closers -// --------------------------------------------------------------------------- - /** * Appends a `---` delimiter line, completing a half-typed one in place * (`title: x\n-` → `title: x\n---`). @@ -981,232 +282,6 @@ function closeComponents(result: string, source: string, comps: Component[], hea const ZWSP_RE = /\u200B/g const TRAILING_ZWSP_RE = /\u200B+$/ -const SETEXT_RE = /^(?:-{1,2}|={1,2})$/ -const YAML_KEY_RE = /^[A-Za-z_][\w.-]*\s*:/ - -/** - * A 1–2 char `-`/`=` line under a paragraph would flash as a setext heading - * while the list marker or rule is still being typed — park it behind U+200B. - */ -function applySetextGuard(text: string): string { - const lastNl = text.lastIndexOf('\n') - if (lastNl === -1) return text - const last = text.slice(lastNl + 1) - const t = last.trim() - if (!SETEXT_RE.test(t)) return text - // Trailing whitespace other than a stripped single space means "still typing". - if (last !== t && /\s$/.test(last)) return text - const head = text.slice(0, lastNl) - const prev = head.slice(head.lastIndexOf('\n') + 1).trim() - if (prev === '' || prev === '---' || YAML_KEY_RE.test(prev)) return text - return text + '\u200B' -} - -// --------------------------------------------------------------------------- -// Trailing openers (streaming) -// --------------------------------------------------------------------------- - -function isTrailingOpener(c: number): boolean { - // * _ $ : ` ~ [ { ! - return c === 42 || c === 95 || c === 36 || c === 58 || c === 96 || c === 126 || c === 91 || c === 123 || c === 33 -} - -/** - * Drops a trailing opener run at EOF unless it is attached to a word - * (`hello *` → `hello`), so a half-typed marker never flashes. Markers with content - * after them (`**bold`) are not trailing runs and stay for the heal; a bare `$` - * after a word (`text123$`) is dropped anyway, being only a math opener. - */ -function dropTrailingOpeners(text: string, inlineCode: boolean): string { - let ws = text.length - while (ws > 0 && isSpaceCode(text.charCodeAt(ws - 1))) ws-- - if (ws === 0) return text - - let i = ws - while (i > 0 && isTrailingOpener(text.charCodeAt(i - 1))) { - if (text.charCodeAt(i - 2) === 92 /* \ */) break - i-- - } - if (i === ws) return text - - let allDollar = true - let allTick = true - for (let k = i; k < ws; k++) { - const c = text.charCodeAt(k) - if (c !== 36) allDollar = false - if (c !== 96) allTick = false - } - if (allDollar && isAlnum(codePointBefore(text, i))) return text.slice(0, i) + text.slice(ws) - - // A trailing backtick run may belong to an open code span rather than start a - // new one. Then it is the heal pass's job, not ours: it either closes the span - // (`space `` ``) or completes a short closer (`spaces `` ` ` → `spaces `` ``). - // Only a span with no content yet is dropped, opener and all. - if (allTick && inlineCode) { - const span = openCodeSpan(text, i) - if (span !== null) { - if (ws - i >= span.len) return text // a closer — leave it to the heal - const gap = i - span.end - const pad = gap === 1 ? text.charCodeAt(span.end) : -1 - if (gap > 1 || (gap === 1 && pad !== 32 && pad !== 9)) return text // real content - return text.slice(0, trimBack(text, span.start)) + text.slice(ws) - } - } - - // Attached to a word the run is prose or a closer (`text**`, `word_`, `20~`), so it - // stays. After whitespace or punctuation it can only be a half-typed opener - // (`hello *`, ``escape lone `~` (` ``) and would flash. - if (isAlnum(codePointBefore(text, i))) return text - - return text.slice(0, trimBack(text, i)) + text.slice(ws) -} - -/** True when `text[from, to)` holds anything other than whitespace. */ -function hasContent(text: string, from: number, to: number): boolean { - for (let i = from; i < to; i++) if (!isSpaceCode(text.charCodeAt(i))) return true - return false -} - -/** `from`, minus one space or tab of padding before it. */ -function trimBack(text: string, from: number): number { - const prev = text.charCodeAt(from - 1) - return prev === 32 || prev === 9 ? from - 1 : from -} - -/** - * The inline code span still open at `limit` on its line, or null. Mirrors the - * heal pass's matching rule: a run closes the span when it is at least as long - * as the opener, otherwise it is content. - */ -function openCodeSpan(text: string, limit: number): { start: number; len: number; end: number } | null { - let start = -1 - let len = 0 - let end = -1 - let i = text.lastIndexOf('\n', limit - 1) + 1 - while (i < limit) { - const c = text.charCodeAt(i) - if (c === 92 /* \ */) { - i += 2 - continue - } - if (c !== 96) { - i++ - continue - } - let n = 1 - while (i + n < limit && text.charCodeAt(i + n) === 96) n++ - if (len === 0) { - len = n - start = i - end = i + n - } else if (n >= len) { - len = 0 - start = -1 - end = -1 - } - i += n - } - return len === 0 ? null : { start, len, end } -} - -// --------------------------------------------------------------------------- -// Characters -// --------------------------------------------------------------------------- - -const ALNUM_CP_RE = /\p{L}|\p{N}/u - -/** Letter or digit. Everything else (including `_`) is CommonMark punctuation. */ -function isAlnum(cp: string): boolean { - if (cp === '') return false - const c = cp.charCodeAt(0) - // ASCII fast path covers nearly all markdown without a Unicode test. - if (c <= 127) return (c >= 48 && c <= 57) || (c >= 65 && c <= 90) || (c >= 97 && c <= 122) - return ALNUM_CP_RE.test(cp) -} - -const PUNCT_CP_RE = /\p{P}|\p{S}/u - -/** CommonMark punctuation. A combining mark is neither this nor alphanumeric. */ -function isPunct(cp: string): boolean { - if (cp === '') return false - const c = cp.charCodeAt(0) - if (c <= 127) return c > 32 && c < 127 && !isAlnum(cp) - return PUNCT_CP_RE.test(cp) -} - -const OPENING_PUNCT_RE = /\p{Ps}|\p{Pi}/u - -/** Opening bracket or initial quote: `(`, `[`, `{`, `「`, `“`, ... */ -function isOpeningPunct(cp: string): boolean { - if (cp === '') return false - const c = cp.charCodeAt(0) - if (c <= 127) return c === 40 || c === 91 || c === 123 - return OPENING_PUNCT_RE.test(cp) -} - -function isAsciiAlpha(c: number): boolean { - return (c >= 97 && c <= 122) || (c >= 65 && c <= 90) -} - -/** Space, tab, line feed, form feed or carriage return. */ -function isSpaceCode(c: number): boolean { - return c === 32 || c === 9 || c === 10 || c === 12 || c === 13 -} - -/** Full code point before `i` (surrogate aware), `''` at the start. */ -function codePointBefore(text: string, i: number): string { - if (i <= 0) return '' - const c = text.charCodeAt(i - 1) - if (c >= 0xdc00 && c <= 0xdfff && i >= 2) { - const hi = text.charCodeAt(i - 2) - if (hi >= 0xd800 && hi <= 0xdbff) return text.slice(i - 2, i) - } - return text[i - 1] -} - -/** Full code point at `i` (surrogate aware). */ -function codePointAt(text: string, i: number): string { - const c = text.charCodeAt(i) - if (c >= 0xd800 && c <= 0xdbff) { - const lo = text.charCodeAt(i + 1) - if (lo >= 0xdc00 && lo <= 0xdfff) return text.slice(i, i + 2) - } - return text[i] -} - -/** - * True when the `~` at `i` closes or opens a subscript pair (`markdown-it-sub`), - * the one paired use of single tildes that must stay unescaped. - */ -function isSubscriptTilde(text: string, i: number): boolean { - for (let j = i - 1; j >= 0 && text.charCodeAt(j) !== 10; j--) { - if (text.charCodeAt(j) === 126) { - if (isSubscriptPair(text, j, i)) return true - break - } - } - for (let j = i + 1; j < text.length && text.charCodeAt(j) !== 10; j++) { - if (text.charCodeAt(j) === 126) return isSubscriptPair(text, i, j) - } - return false -} - -/** - * `word~text~` with no whitespace inside (`a~note~`, `x~ab~.`). A word right - * after the closer is only allowed for digits (`H~2~O`), so `foo~bar~baz` and - * `20~25°C。20~25°C` stay prose. - */ -function isSubscriptPair(text: string, open: number, close: number): boolean { - if (close - open < 2) return false - if (text.charCodeAt(open - 1) === 126 || text.charCodeAt(open + 1) === 126) return false - if (text.charCodeAt(close + 1) === 126) return false - const wordAfter = close + 1 < text.length && isAlnum(codePointAt(text, close + 1)) - for (let k = open + 1; k < close; k++) { - const c = text.charCodeAt(k) - if (isSpaceCode(c) || (wordAfter && (c < 48 || c > 57))) return false - } - return isAlnum(codePointBefore(text, open)) -} // region -- table auto close /** diff --git a/packages/comark/src/auto-close/core.ts b/packages/comark/src/auto-close/core.ts new file mode 100644 index 00000000..2ebfef13 --- /dev/null +++ b/packages/comark/src/auto-close/core.ts @@ -0,0 +1,17 @@ +/** CommonMark/GFM auto-close without Comark syntax or table completion; contract in `SPEC/auto-close.md`. */ + +import type { AutoCloseCoreOptions } from '../internal/auto-close.ts' +import { autoClose, dropTrailingOpeners as dropOpeners, resolveOptions } from '../internal/auto-close.ts' + +export { INCOMPLETE_IMAGE_PLACEHOLDER, INCOMPLETE_LINK_PLACEHOLDER } from '../internal/auto-close.ts' +export type { AutoCloseCoreOptions, LinkMode } from '../internal/auto-close.ts' + +export function autoCloseMarkdown(markdown: string, options: AutoCloseCoreOptions = {}): string { + if (!markdown) return markdown + return autoClose(markdown, resolveOptions(options, false), null) +} + +/** Drops a half-typed trailing opener (`hello *` → `hello`); run it before `autoCloseMarkdown`. */ +export function dropTrailingOpeners(markdown: string, options: Pick = {}): string { + return dropOpeners(markdown, options.inlineCode !== false) +} diff --git a/packages/comark/src/internal/auto-close.ts b/packages/comark/src/internal/auto-close.ts new file mode 100644 index 00000000..012fcacb --- /dev/null +++ b/packages/comark/src/internal/auto-close.ts @@ -0,0 +1,994 @@ +/** + * The CommonMark/GFM auto-closer shared by `comark/auto-close` and `comark/auto-close/core`. + * + * Two O(n) passes, no backtracking: + * + * 1. `scanBlocks` — one walk over the document on offsets only (no per-line + * strings): fences, raw HTML, trailing table, and the range eligible for + * inline healing (the last soft-wrapped paragraph). + * 2. `healRegion` — one walk over that range with a single delimiter stack. + * Unclosed frames are closed LIFO at EOF; edits (tilde escapes, dropped + * brackets) are collected as positions and applied once at the end, so the + * common "nothing to escape" path never copies the string. + * + * Behavioral contract: `packages/comark/SPEC/auto-close.md`. + */ + +export const INCOMPLETE_LINK_PLACEHOLDER = 'comark:incomplete-link' +export const INCOMPLETE_IMAGE_PLACEHOLDER = 'comark:incomplete-image' + +export type LinkMode = 'protocol' | 'text-only' + +export interface AutoCloseCoreOptions { + linkMode?: LinkMode + incompleteLinkPlaceholder?: string + incompleteImagePlaceholder?: string + /** Auto-close incomplete links (`[text`). Default true. */ + links?: boolean + /** Auto-close incomplete images (`![alt`). Default true. */ + images?: boolean + /** Auto-close `**bold**`. Default true. */ + bold?: boolean + /** Auto-close `*italic*` / `_italic_`. Default true. */ + italic?: boolean + /** Auto-close `~~strikethrough~~`. Default true. */ + strikethrough?: boolean + /** Auto-close inline `` `code` ``. Default true. */ + inlineCode?: boolean + /** Auto-close `***bold-italic***`. Default true. */ + boldItalic?: boolean + /** Escape mid-word single `~` (`20~25` → `20\~25`). Default true. */ + singleTilde?: boolean + /** + * Escape list-item comparison openers (`- > 25` → `- \> 25`) so they are not + * parsed as blockquotes. Default true. + */ + comparisonOperators?: boolean + /** Strip incomplete HTML tags at EOF (`Hello = 0, doc.table) +} + +// --------------------------------------------------------------------------- +// Block pass +// --------------------------------------------------------------------------- + +interface DocState { + /** Offsets of the region to heal, or `start < 0` for "nothing to heal". */ + start: number + end: number + /** Effective document end — `LINE_DROP` cuts the document short. */ + docEnd: number + /** The document ends inside a table block. */ + table: boolean +} + +const RAW_TEXT_OPEN_RE = /^<(script|pre|style|textarea)(\s|>|$)/i +const RAW_TEXT_CLOSE_RE = { + script: /<\/script\s*>/i, + pre: /<\/pre\s*>/i, + style: /<\/style\s*>/i, + textarea: /<\/textarea\s*>/i, +} as const + +type RawTag = keyof typeof RAW_TEXT_CLOSE_RE + +/** + * One walk over the document, on offsets only: no per-line strings, so a long + * streaming document costs a single scan plus one slice of the healed region. + * + * Every line is examined as three offsets: `ls` (start), `st`…`en` (the trimmed + * content) and `le` (end, before the newline). + */ +function scanBlocks(src: string, syntax: Syntax | null): DocState { + const len = src.length + const starts: number[] = [0] + for (let i = src.indexOf('\n'); i !== -1; i = src.indexOf('\n', i + 1)) starts.push(i + 1) + let n = starts.length + let docEnd = len + + let fenceLen = 0 + let fenceCh = 0 + let raw: RawTag | null = null + let table = false + let shielded = -1 // last line that structurally shields what precedes it. + + for (let i = 0; i < n; i++) { + const ls = starts[i] + const le = i + 1 < n ? starts[i + 1] - 1 : len + let st = ls + while (st < le && isIndentCode(src.charCodeAt(st))) st++ + let en = le + while (en > st && isSpaceCode(src.charCodeAt(en - 1))) en-- + const c0 = st < en ? src.charCodeAt(st) : -1 + + if (raw !== null) { + if (RAW_TEXT_CLOSE_RE[raw].test(src.slice(ls, le))) raw = null + continue + } + if (c0 === 60 /* < */) { + const m = RAW_TEXT_OPEN_RE.exec(src.slice(st, en)) + if (m !== null) { + const tag = m[1].toLowerCase() as RawTag + if (!RAW_TEXT_CLOSE_RE[tag].test(src.slice(ls, le))) raw = tag + continue + } + } + + if (c0 === 96 /* ` */ || c0 === 126 /* ~ */) { + let run = 1 + while (st + run < en && src.charCodeAt(st + run) === c0) run++ + if (run >= 3) { + if (fenceLen === 0) { + // `` ```code`` `` is an incomplete inline code span, not a fence opener. + if (!(c0 === 96 && isIncompleteInlineFence(src, st, en))) { + fenceLen = run + fenceCh = c0 + shielded = i + continue + } + } else if (c0 === fenceCh && run >= fenceLen && st + run === en) { + // A closer is the same character, at least as long, and followed only by + // whitespace. `` ```js `` inside an open block is code content, not a closer. + fenceLen = 0 + shielded = i + continue + } else { + shielded = i + continue + } + } + } + if (fenceLen !== 0) continue + + const r = + syntax !== null && (c0 === 36 /* $ */ || c0 === 45 /* - */ || c0 === 58 /* : */ || syntax.open) + ? syntax.line(src, i, ls, st, en, c0, i === n - 1) + : 0 + if (r === LINE_SKIP) continue + + if (c0 === 36 /* $ */ && en - st === 2 && src.charCodeAt(st + 1) === 36) { + shielded = i + continue + } + + table = c0 === 124 /* | */ + + if (r === LINE_DROP) { + docEnd = ls + n-- + break + } + } + + const doc: DocState = { start: -1, end: -1, docEnd, table } + + // Nothing is healed while a shielded region is still open at EOF. + if (fenceLen !== 0 || raw !== null || (syntax !== null && syntax.open)) return doc + + const lineEnd = (i: number) => (i + 1 < n ? starts[i + 1] - 1 : docEnd) + + let e = n - 1 + while (e > 0 && lineEnd(e) === starts[e]) e-- + + // A half-typed list item at the end of a list is a marker with no content yet + // (`- item\n- ` → `- item`). Only inside a list — under a paragraph the same line + // is a setext candidate and gets the U+200B guard instead. + while ( + e > shielded + 1 && + isEmptyListItem(src, starts[e], lineEnd(e)) && + isListItem(src, starts[e - 1], lineEnd(e - 1)) + ) { + docEnd = starts[e] - 1 // drop the line and the newline before it + n = e + e-- + while (e > 0 && lineEnd(e) === starts[e]) e-- + } + doc.docEnd = docEnd + + if (lineEnd(e) === starts[e] || e <= shielded) return doc + + // Widen to the soft-wrapped paragraph: stop at a blank line or a new block. + let s = e + while (s > shielded + 1 && lineEnd(s - 1) !== starts[s - 1] && !isBlockStart(src, starts[s], lineEnd(s))) s-- + if (s !== e) { + let join = !isBlockStart(src, starts[s], lineEnd(s)) || isListItem(src, starts[s], lineEnd(s)) + for (let i = s + 1; join && i <= e; i++) if (isBlockStart(src, starts[i], lineEnd(i))) join = false + if (!join) s = e + } + + // Indented code is literal. + if (s === e && isIndentedCode(src, starts[e], lineEnd(e)) && !isListItem(src, starts[e], lineEnd(e))) return doc + + doc.start = starts[s] + doc.end = lineEnd(e) + return doc +} + +/** `` ```python code`` `` — a code span that merely looks like a fence. */ +function isIncompleteInlineFence(src: string, st: number, en: number): boolean { + if (en - st < 5) return false + if (src.charCodeAt(en - 1) !== 96 || src.charCodeAt(en - 2) !== 96 || src.charCodeAt(en - 3) === 96) return false + return !src.slice(st + 3, en).includes('```') +} + +const ATX_HEADING_RE = /^#{1,6}(\s|$)/ +const THEMATIC_BREAK_RE = /^(\*{3,}|_{3,}|-{3,})\s*$/ +const ORDERED_LIST_RE = /^\d{1,9}[.)](\s|$)/ + +function isIndentCode(c: number): boolean { + return c === 32 || c === 9 +} + +/** True when a line opens a new block, so paragraph joining must not cross it. */ +function isBlockStart(src: string, ls: number, le: number): boolean { + if (ls >= le) return false + if (isIndentedCode(src, ls, le)) return true + let st = ls + while (st < le && isIndentCode(src.charCodeAt(st))) st++ + if (st >= le) return false + const c = src.charCodeAt(st) + if (c === 96 || c === 126) { + let run = 1 + while (st + run < le && src.charCodeAt(st + run) === c) run++ + if (run >= 3) return true + } + if (c === 35 /* # */) return ATX_HEADING_RE.test(src.slice(st, le)) + if (c === 62 /* > */ || c === 124 /* | */) return true + if ((c === 42 || c === 95 || c === 45) && THEMATIC_BREAK_RE.test(src.slice(st, le))) return true + return isListItem(src, st, le) +} + +/** CommonMark indented code: ≥ 4 leading spaces (or a tab) plus content. */ +function isIndentedCode(src: string, ls: number, le: number): boolean { + if (ls >= le) return false + if (src.charCodeAt(ls) === 9) return true + let i = ls + while (i < le && src.charCodeAt(i) === 32) i++ + return i - ls >= 4 && i < le +} + +function isListItem(src: string, ls: number, le: number): boolean { + let st = ls + while (st < le && isIndentCode(src.charCodeAt(st))) st++ + if (st >= le) return false + const c = src.charCodeAt(st) + if (c === 45 || c === 43 || c === 42) { + const c1 = st + 1 < le ? src.charCodeAt(st + 1) : -1 + return c1 === 32 || c1 === 9 + } + return c >= 48 && c <= 57 && ORDERED_LIST_RE.test(src.slice(st, le)) +} + +/** A list marker with nothing after it: `-`, `- `, `1.`, `2) ` — but not `- x` or `---`. */ +function isEmptyListItem(src: string, ls: number, le: number): boolean { + let st = ls + while (st < le && isIndentCode(src.charCodeAt(st))) st++ + let en = le + while (en > st && isSpaceCode(src.charCodeAt(en - 1))) en-- + if (st >= en) return false + const c = src.charCodeAt(st) + if (c === 45 || c === 43 || c === 42) return st + 1 === en + let i = st + while (i < en && src.charCodeAt(i) >= 48 && src.charCodeAt(i) <= 57) i++ + if (i === st || i === en) return false + const mark = src.charCodeAt(i) + return (mark === 46 /* . */ || mark === 41) /* ) */ && i + 1 === en +} + +// --------------------------------------------------------------------------- +// Inline pass +// --------------------------------------------------------------------------- + +const F_STAR = 1 +const F_UNDER = 2 +const F_TILDE = 3 +const F_CODE = 4 +export const F_MATH = 5 +const F_LINK = 6 + +/** An open construct. `len` is the delimiter run still to be closed. */ +export interface Frame { + k: number + len: number + pos: number +} + +type Edit = [pos: number, replacement: string] + +const LIST_COMPARE_PREFIX_RE = /^\s*(?:[-*+]|\d+[.)]) +$/ +const LIST_COMPARE_VALUE_RE = /^=?\s*\$?\d/ + +/** + * Heals one paragraph-sized region: closes every construct still open at EOF, + * escapes mid-word tildes and list comparison operators, drops an incomplete + * HTML tag, and completes an incomplete link or image. + */ +function healRegion(text: string, o: Opts, syntax: Syntax | null): string { + let len = text.length + // One trailing space is streaming noise; two are a hard line break. + if (text.charCodeAt(len - 1) === 32 && text.charCodeAt(len - 2) !== 32) len-- + if (len === 0) return '' + + const frames: Frame[] = [] + let edits: Edit[] | null = null + + let lineStart = 0 + let escEnd = -1 // index just past the last escape sequence + let attr = 0 + let htmlLt = -1 + let starInWord = false + let starDisabled = false + // The last word-internal `*` run left literal, as its length and the frame + // depth it was seen at. Until the next whitespace or the next `*` frame + // opened or closed, a same-length run that would open pairs with it + // instead, so `这是**中文。**后续` gets no closer. + let starSkippedLen = 0 + let starSkippedDepth = -1 + + for (let i = 0; i < len; i++) { + const c = text.charCodeAt(i) + const top = frames.length === 0 ? null : frames[frames.length - 1] + + if (c === 10 /* \n */) { + lineStart = i + 1 + starInWord = false + starSkippedLen = 0 + htmlLt = -1 + continue + } + + // A code span shields everything but its own closer. + if (top !== null && top.k === F_CODE) { + if (c === 96) { + let n = 1 + while (text.charCodeAt(i + n) === 96) n++ + if (n >= top.len) frames.pop() + else if (i + n === len) top.len -= n + i += n - 1 + } + continue + } + + if (c === 32 || c === 9 || c === 13) { + starInWord = false + starSkippedLen = 0 + continue + } + if (c === 92 /* \ */) { + const close = syntax === null ? -1 : syntax.bracket(text, i, lineStart, len) + if (close !== -1) { + i = close + 1 + escEnd = i + 1 + continue + } + i++ + escEnd = i + 1 + continue + } + if (attr > 0) { + if (c === 125 /* } */) attr-- + continue + } + if (htmlLt >= 0) { + if (c === 62 /* > */) htmlLt = -1 + continue + } + + if (c === 96 /* ` */) { + let n = 1 + while (text.charCodeAt(i + n) === 96) n++ + const after = i + n < len ? text.charCodeAt(i + n) : -1 + if (o.inlineCode && after !== -1 && after !== 10) frames.push({ k: F_CODE, len: n, pos: i }) + i += n - 1 + continue + } + + if (c === 36 /* $ */ && syntax !== null) { + i += syntax.dollar(text, i, len, frames, top) - 1 + continue + } + + // Inside math only code spans and the math closer are syntax. + if (top !== null && top.k === F_MATH) continue + + if (c === 42 /* * */ || c === 95 /* _ */) { + let n = 1 + while (text.charCodeAt(i + n) === c) n++ + const prev = codePointBefore(text, i) + const next = i + n < len ? codePointAt(text, i + n) : '' + const attachedLeft = prev !== '' && !isSpaceCode(prev.charCodeAt(0)) && i !== escEnd + // `_` only opens at a boundary: whitespace, an escape or another marker. + // That keeps `func(_arg` literal while `**_text` and `\___bold` still open. + const openLeft = !attachedLeft || (c === 95 && (prev === '*' || prev === '_' || prev === '~')) + const openRight = next !== '' && !isSpaceCode(next.charCodeAt(0)) + const enabled = n === 1 ? o.italic : n === 2 ? o.bold : o.boldItalic + + if (c === 42) { + // Word-internal `*` only counts once the word already holds a delimiter + // (`*foo*bar*baz` pairs up, `abc*123` stays literal). + const inWord = attachedLeft && isAlnum(prev) && isAlnum(next) + if (inWord && !starInWord) { + starSkippedLen = n + starSkippedDepth = frames.length + } else { + if (attachedLeft && top !== null && top.k === F_STAR && (!inWord || top.len === n)) { + closeRun(frames, F_STAR, n) + starSkippedLen = 0 + starInWord = true + i += n - 1 + continue + } + // An attached run after punctuation (`中文。**后续`) that would open + // instead pairs with a word-internal run left literal earlier in the + // same whitespace-free stretch, with no frame opened or closed in + // between. CommonMark cannot pair the two, and the second run is a + // would-be closer far more often than an opener (CJK bold that ends + // in punctuation), so it is left literal instead of being closed. + // After opening punctuation (`bar(**baz`) it is an opener, as usual. + if ( + attachedLeft && + !inWord && + starSkippedLen === n && + starSkippedDepth === frames.length && + !isOpeningPunct(prev) + ) { + starSkippedLen = 0 + i += n - 1 + continue + } + if (openRight && enabled) { + frames.push({ k: F_STAR, len: n, pos: i }) + starSkippedLen = 0 + starInWord = true + i += n - 1 + continue + } + if (openRight && n === 1) starDisabled = true + } + } else { + // `_` closes only against a non-word on the right, so `snake_case` and + // `some__field` stay literal mid-word. + if (attachedLeft && !isAlnum(next) && top !== null && top.k === F_UNDER) { + closeRun(frames, F_UNDER, n) + i += n - 1 + continue + } + // A `_` run after punctuation and followed by a word char (`__下線、__次`, + // `__a.__b`, `_a(__b`) can only open in CommonMark, so any `_` closer + // appended at EOF would pair with this run instead of the open frames. + if (attachedLeft && isAlnum(next) && isPunct(prev) && top !== null && top.k === F_UNDER) { + while (frames.length > 0 && frames[frames.length - 1].k === F_UNDER) frames.pop() + // The scope a skipped `*` run was recorded in is gone with the frame. + starSkippedLen = 0 + i += n - 1 + continue + } + if (openLeft && openRight && enabled) frames.push({ k: F_UNDER, len: n, pos: i }) + } + i += n - 1 + continue + } + + if (c === 126 /* ~ */) { + let n = 1 + while (text.charCodeAt(i + n) === 126) n++ + const prev = codePointBefore(text, i) + const next = i + n < len ? codePointAt(text, i + n) : '' + const attachedLeft = prev !== '' && !isSpaceCode(prev.charCodeAt(0)) + if (top !== null && top.k === F_TILDE && attachedLeft) { + closeRun(frames, F_TILDE, n) + } else if (n === 2 && o.strikethrough && next !== '' && !isSpaceCode(next.charCodeAt(0))) { + frames.push({ k: F_TILDE, len: 2, pos: i }) + } else if (n === 1 && o.singleTilde && isAlnum(prev) && isAlnum(next) && !isSubscriptTilde(text, i)) { + // A lone mid-word `~` would become GFM strikethrough — keep it literal. + edits = push(edits, [i, '\\~']) + } + i += n - 1 + continue + } + + if (c === 91 /* [ */) { + const image = text.charCodeAt(i - 1) === 33 /* ! */ + if (image ? o.images : o.links) frames.push({ k: F_LINK, len: image ? 2 : 1, pos: image ? i - 1 : i }) + continue + } + + if (c === 93 /* ] */) { + let b = frames.length - 1 + while (b >= 0 && frames[b].k !== F_LINK) b-- + if (b < 0) continue + const label = frames[b] + frames.length = b // the label closed: markers opened inside it stay literal + const after = i + 1 < len ? text.charCodeAt(i + 1) : -1 + + if (after === 40 /* ( */) { + const end = scanDestination(text, i + 2, len) + if (end > 0) { + i = end // complete `[text](url)` + continue + } + if (end < 0) continue // spaces inside: not a destination, keep scanning + } else if (after === 91 /* [ */) { + let j = i + 2 + while (j < len && text.charCodeAt(j) !== 93) j++ + if (j < len) { + i = j // complete `[text][ref]` + continue + } + } else if (!(label.len === 2 && i + 1 === len && hasContent(text, label.pos + 2, i))) { + continue // `[text]` may be a shortcut reference; only images are healed + } + + // Incomplete destination / reference: the link wins over everything inside. + if (label.len === 2 || o.linkMode === 'protocol') { + return apply(text, edits, i + 1, '(' + (label.len === 2 ? o.imagePh : o.linkPh) + ')') + } + return apply(text, push(edits, [label.pos, '']), i, '') + } + + if (c === 60 /* < */) { + const next = text.charCodeAt(i + 1) + if (next === 47 /* / */ || isAsciiAlpha(next)) htmlLt = i + continue + } + + if (c === 62 /* > */) { + if ( + o.comparisonOperators && + LIST_COMPARE_PREFIX_RE.test(text.slice(lineStart, i)) && + LIST_COMPARE_VALUE_RE.test(text.slice(i + 1)) + ) { + edits = push(edits, [i, '\\>']) + } + continue + } + + if (c === 123 /* { */ && o.attributes) { + // Only an attached `{` is an attribute scope (`text{.cls`), never a bare line. + const prev = text.charCodeAt(i - 1) + if (i > 0 && prev !== 32 && prev !== 9 && prev !== 10) attr++ + continue + } + } + + let end = len + + // Incomplete HTML tag at EOF, plus the whitespace in front of it. + if (htmlLt >= 0 && o.htmlTags) { + end = htmlLt + while (end > 0 && (text.charCodeAt(end - 1) === 32 || text.charCodeAt(end - 1) === 9)) end-- + while (frames.length > 0 && frames[frames.length - 1].pos >= htmlLt) frames.pop() + } + + // Unclosed `[` / `![` at EOF. An empty label is a half-typed marker, not a link: + // there is nothing to wrap, so `hello [` stays literal. + let b = frames.length - 1 + while (b >= 0 && frames[b].k !== F_LINK) b-- + if (b >= 0 && hasContent(text, frames[b].pos + frames[b].len, end)) { + const label = frames[b] + if (label.len === 2 || o.linkMode === 'protocol') { + return apply(text, edits, end, '](' + (label.len === 2 ? o.imagePh : o.linkPh) + ')') + } + // text-only: unwrap link labels from the inside out, stop at an image. + for (let k = frames.length - 1; k >= 0; k--) { + const f = frames[k] + if (f.k !== F_LINK) continue + frames.length = k + if (f.len === 2) return apply(text, edits, end, '](' + o.imagePh + ')') + edits = push(edits, [f.pos, '']) + } + } + + return apply(text, edits, end, closers(frames, text, o, starDisabled)) +} + +/** Consume `n` delimiters against the open frames of `kind`, innermost first. */ +function closeRun(frames: Frame[], kind: number, n: number): void { + let m = n + while (m > 0 && frames.length > 0) { + const f = frames[frames.length - 1] + if (f.k !== kind) break + if (m >= f.len) { + m -= f.len + frames.pop() + } else { + // Partial closer: `**bold*` leaves one `*` to complete. + f.len -= m + m = 0 + } + } +} + +/** + * End of an inline link destination: index of the closing `)`, `0` when the + * destination is unterminated, `-1` when it holds whitespace (so it is not a + * destination at all and the text must keep flowing). + */ +function scanDestination(text: string, from: number, len: number): number { + let depth = 1 + for (let i = from; i < len; i++) { + const c = text.charCodeAt(i) + if (c === 92) i++ + else if (c === 40) depth++ + else if (c === 41) { + if (--depth === 0) return i + } else if (c === 32 || c === 9 || c === 10) return -1 + } + return 0 +} + +/** LIFO closers for everything still open. */ +function closers(frames: Frame[], text: string, o: Opts, starDisabled: boolean): string { + const skipStar = starDisabled && !o.italic + let starSum = 0 + let starCount = 0 + let innerStar = -1 + for (let i = frames.length - 1; i >= 0; i--) { + if (frames[i].k !== F_STAR) continue + starSum += frames[i].len + starCount++ + if (innerStar < 0) innerStar = i + } + // A cross-family construct outside the innermost `*` cannot be nested through: + // `~~strike **bold *italic` closes only the `*` (SPEC), never `***~~`. + let crossOutside = false + for (let i = innerStar - 1; i >= 0; i--) { + const k = frames[i].k + if (k === F_TILDE || k === F_UNDER || k === F_MATH) { + crossOutside = true + break + } + } + + let out = '' + let starDone = false + for (let i = frames.length - 1; i >= 0; i--) { + const f = frames[i] + switch (f.k) { + case F_STAR: + if (starDone || skipStar) break + starDone = true + out += '*'.repeat(starCount >= 2 && crossOutside ? f.len : starSum) + break + case F_UNDER: + out += '_'.repeat(f.len) + break + case F_TILDE: + out += '~'.repeat(f.len) + break + case F_CODE: + out += '`'.repeat(f.len) + break + case F_MATH: + // A `$$` block opened on an earlier line closes on its own line. + out += f.len === 2 && text.lastIndexOf('\n') > f.pos ? '\n$$' : '$'.repeat(f.len) + break + } + } + return out +} + +function push(edits: Edit[] | null, edit: Edit): Edit[] { + if (edits === null) return [edit] + edits.push(edit) + return edits +} + +/** `text[0, end)` with every edit applied, then `suffix`. */ +function apply(text: string, edits: Edit[] | null, end: number, suffix: string): string { + if (edits === null) return end === text.length ? (suffix === '' ? text : text + suffix) : text.slice(0, end) + suffix + if (edits.length > 1) edits.sort(byPos) + let out = '' + let copied = 0 + for (let i = 0; i < edits.length; i++) { + const [pos, replacement] = edits[i] + if (pos >= end) break + out += text.slice(copied, pos) + replacement + copied = pos + 1 + } + return out + text.slice(copied, end) + suffix +} + +function byPos(a: Edit, b: Edit): number { + return a[0] - b[0] +} + +// --------------------------------------------------------------------------- +// Setext guard +// --------------------------------------------------------------------------- + +const SETEXT_RE = /^(?:-{1,2}|={1,2})$/ +const YAML_KEY_RE = /^[A-Za-z_][\w.-]*\s*:/ + +/** + * A 1–2 char `-`/`=` line under a paragraph would flash as a setext heading + * while the list marker or rule is still being typed — park it behind U+200B. + */ +function applySetextGuard(text: string): string { + const lastNl = text.lastIndexOf('\n') + if (lastNl === -1) return text + const last = text.slice(lastNl + 1) + const t = last.trim() + if (!SETEXT_RE.test(t)) return text + // Trailing whitespace other than a stripped single space means "still typing". + if (last !== t && /\s$/.test(last)) return text + const head = text.slice(0, lastNl) + const prev = head.slice(head.lastIndexOf('\n') + 1).trim() + if (prev === '' || prev === '---' || YAML_KEY_RE.test(prev)) return text + return text + '\u200B' +} + +// --------------------------------------------------------------------------- +// Trailing openers (streaming) +// --------------------------------------------------------------------------- + +function isTrailingOpener(c: number): boolean { + // * _ $ : ` ~ [ { ! + return c === 42 || c === 95 || c === 36 || c === 58 || c === 96 || c === 126 || c === 91 || c === 123 || c === 33 +} + +/** + * Drops a trailing opener run at EOF unless it is attached to a word + * (`hello *` → `hello`), so a half-typed marker never flashes. Markers with content + * after them (`**bold`) are not trailing runs and stay for the heal; a bare `$` + * after a word (`text123$`) is dropped anyway, being only a math opener. + */ +export function dropTrailingOpeners(text: string, inlineCode: boolean): string { + let ws = text.length + while (ws > 0 && isSpaceCode(text.charCodeAt(ws - 1))) ws-- + if (ws === 0) return text + + let i = ws + while (i > 0 && isTrailingOpener(text.charCodeAt(i - 1))) { + if (text.charCodeAt(i - 2) === 92 /* \ */) break + i-- + } + if (i === ws) return text + + let allDollar = true + let allTick = true + for (let k = i; k < ws; k++) { + const c = text.charCodeAt(k) + if (c !== 36) allDollar = false + if (c !== 96) allTick = false + } + if (allDollar && isAlnum(codePointBefore(text, i))) return text.slice(0, i) + text.slice(ws) + + // A trailing backtick run may belong to an open code span rather than start a + // new one. Then it is the heal pass's job, not ours: it either closes the span + // (`space `` ``) or completes a short closer (`spaces `` ` ` → `spaces `` ``). + // Only a span with no content yet is dropped, opener and all. + if (allTick && inlineCode) { + const span = openCodeSpan(text, i) + if (span !== null) { + if (ws - i >= span.len) return text // a closer — leave it to the heal + const gap = i - span.end + const pad = gap === 1 ? text.charCodeAt(span.end) : -1 + if (gap > 1 || (gap === 1 && pad !== 32 && pad !== 9)) return text // real content + return text.slice(0, trimBack(text, span.start)) + text.slice(ws) + } + } + + // Attached to a word the run is prose or a closer (`text**`, `word_`, `20~`), so it + // stays. After whitespace or punctuation it can only be a half-typed opener + // (`hello *`, ``escape lone `~` (` ``) and would flash. + if (isAlnum(codePointBefore(text, i))) return text + + return text.slice(0, trimBack(text, i)) + text.slice(ws) +} + +/** True when `text[from, to)` holds anything other than whitespace. */ +function hasContent(text: string, from: number, to: number): boolean { + for (let i = from; i < to; i++) if (!isSpaceCode(text.charCodeAt(i))) return true + return false +} + +/** `from`, minus one space or tab of padding before it. */ +function trimBack(text: string, from: number): number { + const prev = text.charCodeAt(from - 1) + return prev === 32 || prev === 9 ? from - 1 : from +} + +/** + * The inline code span still open at `limit` on its line, or null. Mirrors the + * heal pass's matching rule: a run closes the span when it is at least as long + * as the opener, otherwise it is content. + */ +function openCodeSpan(text: string, limit: number): { start: number; len: number; end: number } | null { + let start = -1 + let len = 0 + let end = -1 + let i = text.lastIndexOf('\n', limit - 1) + 1 + while (i < limit) { + const c = text.charCodeAt(i) + if (c === 92 /* \ */) { + i += 2 + continue + } + if (c !== 96) { + i++ + continue + } + let n = 1 + while (i + n < limit && text.charCodeAt(i + n) === 96) n++ + if (len === 0) { + len = n + start = i + end = i + n + } else if (n >= len) { + len = 0 + start = -1 + end = -1 + } + i += n + } + return len === 0 ? null : { start, len, end } +} + +// --------------------------------------------------------------------------- +// Characters +// --------------------------------------------------------------------------- + +const ALNUM_CP_RE = /\p{L}|\p{N}/u + +/** Letter or digit. Everything else (including `_`) is CommonMark punctuation. */ +function isAlnum(cp: string): boolean { + if (cp === '') return false + const c = cp.charCodeAt(0) + // ASCII fast path covers nearly all markdown without a Unicode test. + if (c <= 127) return (c >= 48 && c <= 57) || (c >= 65 && c <= 90) || (c >= 97 && c <= 122) + return ALNUM_CP_RE.test(cp) +} + +const PUNCT_CP_RE = /\p{P}|\p{S}/u + +/** CommonMark punctuation. A combining mark is neither this nor alphanumeric. */ +function isPunct(cp: string): boolean { + if (cp === '') return false + const c = cp.charCodeAt(0) + if (c <= 127) return c > 32 && c < 127 && !isAlnum(cp) + return PUNCT_CP_RE.test(cp) +} + +const OPENING_PUNCT_RE = /\p{Ps}|\p{Pi}/u + +/** Opening bracket or initial quote: `(`, `[`, `{`, `「`, `“`, ... */ +function isOpeningPunct(cp: string): boolean { + if (cp === '') return false + const c = cp.charCodeAt(0) + if (c <= 127) return c === 40 || c === 91 || c === 123 + return OPENING_PUNCT_RE.test(cp) +} + +function isAsciiAlpha(c: number): boolean { + return (c >= 97 && c <= 122) || (c >= 65 && c <= 90) +} + +/** Space, tab, line feed, form feed or carriage return. */ +function isSpaceCode(c: number): boolean { + return c === 32 || c === 9 || c === 10 || c === 12 || c === 13 +} + +/** Full code point before `i` (surrogate aware), `''` at the start. */ +function codePointBefore(text: string, i: number): string { + if (i <= 0) return '' + const c = text.charCodeAt(i - 1) + if (c >= 0xdc00 && c <= 0xdfff && i >= 2) { + const hi = text.charCodeAt(i - 2) + if (hi >= 0xd800 && hi <= 0xdbff) return text.slice(i - 2, i) + } + return text[i - 1] +} + +/** Full code point at `i` (surrogate aware). */ +function codePointAt(text: string, i: number): string { + const c = text.charCodeAt(i) + if (c >= 0xd800 && c <= 0xdbff) { + const lo = text.charCodeAt(i + 1) + if (lo >= 0xdc00 && lo <= 0xdfff) return text.slice(i, i + 2) + } + return text[i] +} + +/** + * True when the `~` at `i` closes or opens a subscript pair (`markdown-it-sub`), + * the one paired use of single tildes that must stay unescaped. + */ +function isSubscriptTilde(text: string, i: number): boolean { + for (let j = i - 1; j >= 0 && text.charCodeAt(j) !== 10; j--) { + if (text.charCodeAt(j) === 126) { + if (isSubscriptPair(text, j, i)) return true + break + } + } + for (let j = i + 1; j < text.length && text.charCodeAt(j) !== 10; j++) { + if (text.charCodeAt(j) === 126) return isSubscriptPair(text, i, j) + } + return false +} + +/** + * `word~text~` with no whitespace inside (`a~note~`, `x~ab~.`). A word right + * after the closer is only allowed for digits (`H~2~O`), so `foo~bar~baz` and + * `20~25°C。20~25°C` stay prose. + */ +function isSubscriptPair(text: string, open: number, close: number): boolean { + if (close - open < 2) return false + if (text.charCodeAt(open - 1) === 126 || text.charCodeAt(open + 1) === 126) return false + if (text.charCodeAt(close + 1) === 126) return false + const wordAfter = close + 1 < text.length && isAlnum(codePointAt(text, close + 1)) + for (let k = open + 1; k < close; k++) { + const c = text.charCodeAt(k) + if (isSpaceCode(c) || (wordAfter && (c < 48 || c > 57))) return false + } + return isAlnum(codePointBefore(text, open)) +} diff --git a/packages/comark/test/auto-close-core.test.ts b/packages/comark/test/auto-close-core.test.ts new file mode 100644 index 00000000..f8a64186 --- /dev/null +++ b/packages/comark/test/auto-close-core.test.ts @@ -0,0 +1,101 @@ +import { gzipSync } from 'node:zlib' +import { fileURLToPath } from 'node:url' +import { build } from 'vite' +import { describe, expect, it } from 'vitest' +import { autoCloseMarkdown as autoCloseFull, type AutoCloseOptions } from '../src/auto-close.ts' +import { autoCloseMarkdown, dropTrailingOpeners, type AutoCloseCoreOptions } from '../src/auto-close/core.ts' +import { loadSpecCases, sectionAwareMapOptions, toGenericAutoCloseOptions } from './spec-runner.mjs' + +const SPEC_PATH = fileURLToPath(new URL('../SPEC/auto-close.md', import.meta.url)) + +const COMARK_OFF: AutoCloseOptions = { + syntax: false, + attributes: false, + frontmatter: false, + math: false, + blockMath: false, + inlineMath: false, + bracketMath: false, + tables: false, +} + +function core(markdown: string, options: AutoCloseOptions): string { + const source = + options.dropTrailingOpeners === true ? dropTrailingOpeners(markdown, { inlineCode: options.inlineCode }) : markdown + return autoCloseMarkdown(source, options as AutoCloseCoreOptions) +} + +function full(markdown: string, options: AutoCloseOptions): string { + return autoCloseFull(markdown, { ...options, ...COMARK_OFF }) +} + +describe('comark/auto-close/core', () => { + it('matches comark/auto-close with Comark syntax and tables off on every SPEC case', () => { + const cases = loadSpecCases(SPEC_PATH) + expect(cases.length).toBeGreaterThan(0) + for (const c of cases) { + const options = toGenericAutoCloseOptions(sectionAwareMapOptions(c.options, c)) as AutoCloseOptions + expect(core(c.input, options), JSON.stringify(c.input)).toBe(full(c.input, options)) + for (let end = 1; end < c.input.length; end++) { + const prefix = c.input.slice(0, end) + expect(core(prefix, options), JSON.stringify(prefix)).toBe(full(prefix, options)) + } + } + }) + + it('matches comark/auto-close on generated documents', () => { + const tokens = ( + '*,**,***,_,__,~,~~,`,``,```,$,$$,\\(,\\),\\[,\\],[,],](,![,(,),{,},{.cls,",<,>,,:,::,:::,' + + '::alert,::card{a="b,---,--,-,=,- ,1. ,> ,# ,| , |,|---,\n,\n,\n\n, , , ,\t, ,a,word,20,中文,。,😀,title: x,=25' + ).split(',') + const flags = ['dropTrailingOpeners', 'italic', 'inlineCode', 'links', 'htmlTags'] + let seed = 1 + const random = () => (seed = (seed * 1103515245 + 12345) & 0x7fffffff) / 0x7fffffff + for (let n = 0; n < 5000; n++) { + let markdown = '' + for (let k = 1 + Math.floor(random() * 12); k > 0; k--) markdown += tokens[Math.floor(random() * tokens.length)] + const options: Record = { linkMode: random() < 0.5 ? 'protocol' : 'text-only' } + for (const flag of flags) if (random() < 0.3) options[flag] = random() < 0.5 + expect(core(markdown, options), JSON.stringify([markdown, options])).toBe(full(markdown, options)) + } + }) + + it('leaves Comark syntax and tables as text', () => { + expect(autoCloseMarkdown('::alert\nSome **bold')).toBe('::alert\nSome **bold**') + expect(autoCloseMarkdown('::card{title="x')).toBe('::card{title="x') + expect(autoCloseMarkdown('Text{.cls **bold')).toBe('Text{.cls **bold**') + expect(autoCloseMarkdown('$x + y')).toBe('$x + y') + expect(autoCloseMarkdown('---\ntitle: x')).toBe('---\ntitle: x') + expect(autoCloseMarkdown('| a | b |\n| c')).toBe('| a | b |\n| c') + }) + + it('heals CommonMark and GFM like comark/auto-close', () => { + expect(autoCloseMarkdown('Some **bold and `code')).toBe('Some **bold and `code`**') + expect(autoCloseMarkdown('[link](http://a')).toBe('[link](comark:incomplete-link)') + expect(autoCloseMarkdown('[link', { linkMode: 'text-only' })).toBe('link') + expect(autoCloseMarkdown('```js\nconst x = *1')).toBe('```js\nconst x = *1') + expect(autoCloseMarkdown('- item\n- ')).toBe('- item') + expect(autoCloseMarkdown('Hello { + const entry = fileURLToPath(new URL('../src/auto-close/core.ts', import.meta.url)) + const result = await build({ + configFile: false, + logLevel: 'silent', + build: { lib: { entry, formats: ['es'] }, minify: true, write: false }, + }) + const bundles = Array.isArray(result) ? result : [result] + const chunks = bundles.flatMap((bundle) => + 'output' in bundle ? bundle.output.filter((output) => output.type === 'chunk') : [] + ) + expect(chunks).toHaveLength(1) + const retained = Object.entries(chunks[0]!.modules) + .filter(([, module]) => module.renderedLength > 0) + .map(([id]) => id.slice(id.indexOf('/src/'))) + expect(retained).toEqual(['/src/internal/auto-close.ts', '/src/auto-close/core.ts']) + expect(gzipSync(chunks[0]!.code).length).toBeLessThan(5 * 1024) + }) +}) diff --git a/test/bundle.test.ts b/test/bundle.test.ts index b0d64568..e36f2efe 100644 --- a/test/bundle.test.ts +++ b/test/bundle.test.ts @@ -77,7 +77,7 @@ describe('package bundle size', { timeout: 60_000 }, () => { "@comark/react": "42.9k (83 files)", "@comark/svelte": "50.3k (91 files)", "@comark/vue": "62.6k (89 files)", - "comark": "439k (171 files)", + "comark": "444k (175 files)", } `) })