Skip to content

normalize display math fences attached to content / nested in lists - #332

Open
dykhgit wants to merge 2 commits into
agegr:mainfrom
dykhgit:main
Open

normalize display math fences attached to content / nested in lists#332
dykhgit wants to merge 2 commits into
agegr:mainfrom
dykhgit:main

Conversation

@dykhgit

@dykhgit dykhgit commented Jul 31, 2026

Copy link
Copy Markdown

Fix display math swallowing text after the formula

Problem

normalizeDisplayMath in lib/markdown.ts rewrites display math so that
remark-math (which requires $$ to sit on their own lines) can parse it.
Three common LLM output shapes are currently normalized incorrectly, and each
makes the text after the formula render as a giant red katex-error block
(ParseError: Can't use function '$' in math mode):

  1. Single-line $$…$$ nested in a GFM list item

    - 嵌入模型几乎一律用**余弦相似度**,因为要对向量做 L2 归一化:
      $$\mathrm{sim}(q, x) = \frac{q \cdot x}{\|q\| \, \|x\|}$$
    - 归一化后相似度范围 $[-1, 1]$,再除以 $\tau$ 放大后过 softmax。…

    The split emits $$ / content / $$. Inside a list item the
    column-0 content line is a lazy continuation, which breaks remark-math's
    math-flow tokenizer: the opening $$ becomes an empty math node and the
    closing one becomes a new opener that swallows all following text.

  2. Multi-line block with delimiters glued to content

    $$\frac{1}{x_{2023}^2}\le 4+4044+\frac52\sum_{k=2}^{2023}\frac1k
    <4048+\frac52\cdot\ln 2023<4048+\frac52\cdot 8=4068<4096=64^2.$$

    remark-math never finds a valid fence (delimiters must start their own
    lines), so the whole block — and everything after it — is consumed as math.

  3. Indented bare $$ fences inside a list item with column-0 content:
    same lazy-continuation breakage as (1).

Fix

  • Re-indent content lines together with the $$ fence when the block is
    indented, keeping it inside the list item (no lazy continuation).
  • Normalize multi-line $$ blocks with glued delimiters to
    $$\n<content>\n$$.
  • Detach a glued closing $$ and re-indent lazy content of indented bare
    fences.
  • Fenced code, inline code, raw HTML tags and \[…\] handling are unchanged;
    a top-level block whose delimiters are already on their own lines is left
    untouched.

Tests

Adds lib/markdown.test.mjs (12 cases) using the built-in node:test runner —
no new dependencies, no lockfile changes. Run with npm test.

Verified end-to-end through the app's render pipeline
(remark-gfm + remark-math → rehype-raw → rehype-sanitize → rehype-katex):
both affected real-world answers drop from 3 katex-error spans and dozens of
stray $ to 0; a regression pass over all local session texts reports 0 new
failures. tsc --noEmit is clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant