Skip to content

Cap LaTeX math conversion depth so nested frac/scripts cannot overflow the stack - #357

Merged
tylerreckart merged 3 commits into
mainfrom
fix/latex-math-recursion-depth
Sep 21, 2026
Merged

tylerreckart merged 3 commits into
mainfrom
fix/latex-math-recursion-depth

Conversation

@cursor

@cursor cursor Bot commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Problem

latex_math_to_plain recursed through convert() for every nested \frac, ^, _, \sqrt, and \text with no depth limit. TUI and HTTP markdown both call this on model and user text. A few hundred nested groups — easy to emit from a model or paste — can overflow an 8 MiB worker thread stack and kill the process. The JSON parser already caps nesting for the same reason (json.cpp::kMaxDepth).

Fix

Thread a depth counter through convert / convert_script. After 64 levels, emit the remaining raw fragment instead of recursing. 64 is deeper than any legitimate equation and far below the stack limit. Shallow nested fractions and scripts still convert as before.

Tests

unit_markdown_diff:

  • Modest nested \frac{\frac{a}{b}}{c} and a^{b^{2}} still convert.
  • 400 nested \frac groups return a finite string (slash form on the converted prefix, raw \frac past the cap) instead of overflowing.
  • 400 nested superscripts and a \[...\] display-math path also survive.

Suite 50/50 locally (281 assertions) + ASan + UBSan. arbiter compiles.

Independently mergeable against main (0456350). Does not touch files owned by #321–#356 except CHANGELOG [Unreleased], which sibling-conflicts after the first of those merges, same as the rest of the series.

Open in Web View Automation 

Note

Low Risk
Localized defensive change to LaTeX-to-plain conversion with a documented depth cap; normal equations unchanged, only extreme nesting degrades to raw fragments instead of crashing.

Overview
Fixes a stack-overflow risk when the TUI/API render LaTeX in markdown: latex_math_to_plain now tracks recursion depth and stops converting after 64 nested \frac, ^, _, \sqrt, or \text groups, emitting the rest as raw TeX instead of recursing further.

Depth is threaded through internal convert / convert_script calls (same entry point as before for callers). Shallow math still converts to Unicode as today; pathological model/user input no longer blows the worker thread stack. Header and [Unreleased] changelog document the cap.

Tests add modest nested fraction/script cases, 400-level \frac and superscript nests (finite output with raw \frac past the cap), and a MarkdownRenderer \[...\] path over deeply nested math.

Reviewed by Cursor Bugbot for commit 6e41648. Bugbot is set up for automated code reviews on this repo. Configure here.

…w the stack

latex_math_to_plain recursed through convert() for every nested \frac, ^, _,
\sqrt, and \text with no depth limit. Model or user markdown with a few
hundred nested groups can blow an 8 MiB TUI/API thread stack. Stop converting
after 64 levels and emit the remaining raw fragment, matching json.cpp.

Co-authored-by: Tyler Reckart <tylerreckart@users.noreply.github.com>
@tylerreckart
tylerreckart marked this pull request as ready for review September 21, 2026 12:58
@tylerreckart
tylerreckart enabled auto-merge (squash) September 21, 2026 12:58
Keep the LaTeX depth-cap Unreleased bullet alongside the secret-write
and --connect URL fixes that landed on main (#365, #366).

Co-authored-by: Tyler Reckart <tylerreckart@users.noreply.github.com>
@tylerreckart
tylerreckart merged commit fbd3c76 into main Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants