Skip to content

perf(html): avoid duplicate UTF-16 buffers and transient serialization allocations - #21

Merged
palmoni5 merged 5 commits into
mainfrom
palmoni5/html-stream-allocations-20261110
Nov 10, 2026
Merged

palmoni5 merged 5 commits into
mainfrom
palmoni5/html-stream-allocations-20261110

Conversation

@palmoni5

@palmoni5 palmoni5 commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

Problem

The HTML input stream made a full-size duplicate list of code units even when it had no CRLF or invalid surrogate to normalize, and charsUntil* created temporary sublists. The serializer read one-character strings and wrote unescaped text character by character.

Changes

  • HtmlInputStream keeps the decoded String (or the String passed in) and copies its code units only on CR/CRLF conversion or malformed-surrogate substitution. Raw input is never mutated, so reset() and errors stay deterministic.
  • Spans are read with substring on that String when no copy was made, and with String.fromCharCodes(chars, start, end) (no sublist) otherwise. String.fromCharCodes on String.codeUnits must be avoided: on the VM it iterates from index 0 for non-builtin lists, which made parsing quadratic (an 86 KB Hebrew document took ~3.3 s instead of ~12 ms, and noah_ark_clause_limit_test timed out).
  • htmlSerializeEscape scans code units, copies unchanged runs at once and returns the original String when nothing needs escaping.
  • Tests for Hebrew, decoded bytes, surrogates, CRLF + reset, source spans and escaping; CHANGELOG entry.

Verification

  • Differential vs main (outerHtml, text, parse errors with spans, element/attribute spans, fragments, byte input, CRLF variant, both escape modes) over 1352 inputs (all html5lib tree-construction #data, Hebrew documents with nikud/entities/comments/malformed markup, surrogates, empty/CR-only input, 3000-deep nesting): byte-identical.
  • AOT benchmark on Windows x64, 86 KB Hebrew document, median of 15 (3 runs): parse(String) 11.1–11.3 ms vs 11.8–12.0 ms on main; parse(bytes) 9.1–12.1 vs 11.3–13.0; outerHtml 1.2–2.1 ms vs 4.3–5.3 ms; htmlSerializeEscape on the whole document (attribute mode, x20) 47–52 ms vs 116–120 ms; on text without escapable characters (x200) 207–327 ms vs 862–910 ms.
  • dart format, dart analyze --fatal-infos, dart test and dart test -p chrome pass, except parse error spans - full, which fails identically on main on Windows (the source_span glyph set prints ,/| instead of ╷/│).

Otzaria impact

Fewer allocations for repeated HTML parse/serialize calls in text rendering and HTML/EPUB import/export. No Otzaria version-pin change.

@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

PR Health

Publish ✔️
Package Version Status Publish tag (post-merge)
package:html ⭐ 0.15.8-wip WIP (no publish necessary)
  • 16 already published.
  • 26 WIP (no publish necessary).

Documentation at https://github.com/dart-lang/ecosystem/wiki/Publishing-automation.

This check can be disabled by tagging the PR with skip-publish-check.

Changelog Entry ✔️
Package Changed Files

Changes to files need to be accounted for in their respective changelogs.

This check can be disabled by tagging the PR with skip-changelog-check (for example, for non-user-visible changes when the package version is already at -wip).

Breaking changes ✔️
Package Change Current Version New Version Needed Version Looking good?
html None 0.15.7 0.15.8-wip 0.15.8-wip ✔️

This check can be disabled by tagging the PR with skip-breaking-check.

API leaks ⚠️

The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.

Package Leaked API symbol Leaking sources
html HtmlTokenizer html/parser.dart::HtmlParser::tokenizer
html Token tokenizer.dart::HtmlTokenizer
tokenizer.dart::HtmlTokenizer::tokenQueue
tokenizer.dart::HtmlTokenizer::currentToken
tokenizer.dart::HtmlTokenizer::currentToken
token.dart::TagToken
token.dart::DoctypeToken
token.dart::StringToken
tokenizer.dart::HtmlTokenizer::current
token.dart::StartTagToken
token.dart::CommentToken
html/parser.dart::Phase::processComment
html/parser.dart::Phase::processDoctype
token.dart::CharactersToken
html/parser.dart::Phase::processCharacters
token.dart::SpaceCharactersToken
html/parser.dart::Phase::processSpaceCharacters
html/parser.dart::Phase::processStartTag
html/parser.dart::Phase::startTagHtml
token.dart::EndTagToken
html/parser.dart::Phase::processEndTag
html/parser.dart::HtmlParser::inForeignContent::token
html/parser.dart::HtmlParser::parseRCDataRawtext::token
html/parser.dart::BeforeHeadPhase::startTagOther
html/parser.dart::BeforeHeadPhase::endTagImplyHead
html/parser.dart::InHeadPhase::startTagOther
html/parser.dart::InHeadPhase::endTagHtmlBodyBr
html/parser.dart::AfterHeadPhase::startTagOther
html/parser.dart::AfterHeadPhase::endTagHtmlBodyBr
html/parser.dart::InBodyPhase::startTagProcessInHead
html/parser.dart::InBodyPhase::startTagButton
html/parser.dart::InBodyPhase::startTagOther
html/parser.dart::InBodyPhase::endTagHtml
html/parser.dart::InTablePhase::startTagCol
html/parser.dart::InTablePhase::startTagImplyTbody
html/parser.dart::InTablePhase::startTagTable
html/parser.dart::InTablePhase::startTagStyleScript
html/parser.dart::InCaptionPhase::startTagTableElement
html/parser.dart::InCaptionPhase::startTagOther
html/parser.dart::InCaptionPhase::endTagTable
html/parser.dart::InCaptionPhase::endTagOther
html/parser.dart::InColumnGroupPhase::startTagOther
html/parser.dart::InColumnGroupPhase::endTagOther
html/parser.dart::InTableBodyPhase::startTagTableCell
html/parser.dart::InTableBodyPhase::startTagTableOther
html/parser.dart::InTableBodyPhase::startTagOther
html/parser.dart::InTableBodyPhase::endTagTable
html/parser.dart::InTableBodyPhase::endTagOther
html/parser.dart::InRowPhase::startTagTableOther
html/parser.dart::InRowPhase::startTagOther
html/parser.dart::InRowPhase::endTagTable
html/parser.dart::InRowPhase::endTagTableRowGroup
html/parser.dart::InRowPhase::endTagOther
html/parser.dart::InCellPhase::startTagTableOther
html/parser.dart::InCellPhase::startTagOther
html/parser.dart::InCellPhase::endTagImply
html/parser.dart::InCellPhase::endTagOther
html/parser.dart::InSelectPhase::startTagInput
html/parser.dart::InSelectPhase::startTagScript
html/parser.dart::InSelectPhase::startTagOther
html/parser.dart::InSelectInTablePhase::startTagTable
html/parser.dart::InSelectInTablePhase::startTagOther
html/parser.dart::InSelectInTablePhase::endTagTable
html/parser.dart::InSelectInTablePhase::endTagOther
html/parser.dart::AfterBodyPhase::startTagOther
html/parser.dart::AfterBodyPhase::endTagHtml::token
html/parser.dart::AfterBodyPhase::endTagOther
html/parser.dart::InFramesetPhase::startTagNoframes
html/parser.dart::InFramesetPhase::startTagOther
html/parser.dart::AfterFramesetPhase::startTagNoframes
html/parser.dart::AfterAfterBodyPhase::startTagOther
html/parser.dart::AfterAfterFramesetPhase::startTagNoFrames
html HtmlInputStream tokenizer.dart::HtmlTokenizer::stream
html TagToken tokenizer.dart::HtmlTokenizer::currentTagToken
token.dart::StartTagToken
token.dart::EndTagToken
html/parser.dart::InTableBodyPhase::startTagTableOther::token
html/parser.dart::InTableBodyPhase::endTagTable::token
html DoctypeToken tokenizer.dart::HtmlTokenizer::currentDoctypeToken
treebuilder.dart::TreeBuilder::insertDoctype::token
html/parser.dart::Phase::processDoctype::token
html StringToken tokenizer.dart::HtmlTokenizer::currentStringToken
token.dart::StringToken::add
treebuilder.dart::TreeBuilder::insertComment::token
token.dart::CommentToken
token.dart::CharactersToken
token.dart::SpaceCharactersToken
html/parser.dart::InBodyPhase::processSpaceCharactersDropNewline::token
html/parser.dart::InTableTextPhase::characterTokens
html TreeBuilder html/parser.dart::HtmlParser::tree
html/parser.dart::Phase::tree
html/parser.dart::HtmlParser::new::tree
html ActiveFormattingElements treebuilder.dart::TreeBuilder::activeFormattingElements
html StartTagToken treebuilder.dart::TreeBuilder::insertRoot::token
treebuilder.dart::TreeBuilder::createElement::token
treebuilder.dart::TreeBuilder::insertElement::token
treebuilder.dart::TreeBuilder::insertElementNormal::token
treebuilder.dart::TreeBuilder::insertElementTable::token
html/parser.dart::Phase::processStartTag::token
html/parser.dart::Phase::startTagHtml::token
html/parser.dart::HtmlParser::adjustMathMLAttributes::token
html/parser.dart::HtmlParser::adjustSVGAttributes::token
html/parser.dart::HtmlParser::adjustForeignAttributes::token
html/parser.dart::BeforeHeadPhase::startTagHead::token
html/parser.dart::BeforeHeadPhase::startTagOther::token
html/parser.dart::InHeadPhase::startTagHead::token
html/parser.dart::InHeadPhase::startTagBaseLinkCommand::token
html/parser.dart::InHeadPhase::startTagMeta::token
html/parser.dart::InHeadPhase::startTagTitle::token
html/parser.dart::InHeadPhase::startTagNoScriptNoFramesStyle::token
html/parser.dart::InHeadPhase::startTagScript::token
html/parser.dart::InHeadPhase::startTagOther::token
html/parser.dart::AfterHeadPhase::startTagBody::token
html/parser.dart::AfterHeadPhase::startTagFrameset::token
html/parser.dart::AfterHeadPhase::startTagFromHead::token
html/parser.dart::AfterHeadPhase::startTagHead::token
html/parser.dart::AfterHeadPhase::startTagOther::token
html/parser.dart::InBodyPhase::addFormattingElement::token
html/parser.dart::InBodyPhase::startTagProcessInHead::token
html/parser.dart::InBodyPhase::startTagBody::token
html/parser.dart::InBodyPhase::startTagFrameset::token
html/parser.dart::InBodyPhase::startTagCloseP::token
html/parser.dart::InBodyPhase::startTagPreListing::token
html/parser.dart::InBodyPhase::startTagForm::token
html/parser.dart::InBodyPhase::startTagListItem::token
html/parser.dart::InBodyPhase::startTagPlaintext::token
html/parser.dart::InBodyPhase::startTagHeading::token
html/parser.dart::InBodyPhase::startTagA::token
html/parser.dart::InBodyPhase::startTagFormatting::token
html/parser.dart::InBodyPhase::startTagNobr::token
html/parser.dart::InBodyPhase::startTagButton::token
html/parser.dart::InBodyPhase::startTagAppletMarqueeObject::token
html/parser.dart::InBodyPhase::startTagXmp::token
html/parser.dart::InBodyPhase::startTagTable::token
html/parser.dart::InBodyPhase::startTagVoidFormatting::token
html/parser.dart::InBodyPhase::startTagInput::token
html/parser.dart::InBodyPhase::startTagParamSource::token
html/parser.dart::InBodyPhase::startTagHr::token
html/parser.dart::InBodyPhase::startTagImage::token
html/parser.dart::InBodyPhase::startTagIsIndex::token
html/parser.dart::InBodyPhase::startTagTextarea::token
html/parser.dart::InBodyPhase::startTagIFrame::token
html/parser.dart::InBodyPhase::startTagRawtext::token
html/parser.dart::InBodyPhase::startTagOpt::token
html/parser.dart::InBodyPhase::startTagSelect::token
html/parser.dart::InBodyPhase::startTagRpRt::token
html/parser.dart::InBodyPhase::startTagMath::token
html/parser.dart::InBodyPhase::startTagSvg::token
html/parser.dart::InBodyPhase::startTagMisplaced::token
html/parser.dart::InBodyPhase::startTagOther::token
html/parser.dart::InTablePhase::startTagCaption::token
html/parser.dart::InTablePhase::startTagColgroup::token
html/parser.dart::InTablePhase::startTagCol::token
html/parser.dart::InTablePhase::startTagRowGroup::token
html/parser.dart::InTablePhase::startTagImplyTbody::token
html/parser.dart::InTablePhase::startTagTable::token
html/parser.dart::InTablePhase::startTagStyleScript::token
html/parser.dart::InTablePhase::startTagInput::token
html/parser.dart::InTablePhase::startTagForm::token
html/parser.dart::InTablePhase::startTagOther::token
html/parser.dart::InCaptionPhase::startTagTableElement::token
html/parser.dart::InCaptionPhase::startTagOther::token
html/parser.dart::InColumnGroupPhase::startTagCol::token
html/parser.dart::InColumnGroupPhase::startTagOther::token
html/parser.dart::InTableBodyPhase::startTagTr::token
html/parser.dart::InTableBodyPhase::startTagTableCell::token
html/parser.dart::InTableBodyPhase::startTagOther::token
html/parser.dart::InRowPhase::startTagTableCell::token
html/parser.dart::InRowPhase::startTagTableOther::token
html/parser.dart::InRowPhase::startTagOther::token
html/parser.dart::InCellPhase::startTagTableOther::token
html/parser.dart::InCellPhase::startTagOther::token
html/parser.dart::InSelectPhase::startTagOption::token
html/parser.dart::InSelectPhase::startTagOptgroup::token
html/parser.dart::InSelectPhase::startTagSelect::token
html/parser.dart::InSelectPhase::startTagInput::token
html/parser.dart::InSelectPhase::startTagScript::token
html/parser.dart::InSelectPhase::startTagOther::token
html/parser.dart::InSelectInTablePhase::startTagTable::token
html/parser.dart::InSelectInTablePhase::startTagOther::token
html/parser.dart::InForeignContentPhase::adjustSVGTagNames::token
html/parser.dart::AfterBodyPhase::startTagOther::token
html/parser.dart::InFramesetPhase::startTagFrameset::token
html/parser.dart::InFramesetPhase::startTagFrame::token
html/parser.dart::InFramesetPhase::startTagNoframes::token
html/parser.dart::InFramesetPhase::startTagOther::token
html/parser.dart::AfterFramesetPhase::startTagNoframes::token
html/parser.dart::AfterFramesetPhase::startTagOther::token
html/parser.dart::AfterAfterBodyPhase::startTagOther::token
html/parser.dart::AfterAfterFramesetPhase::startTagNoFrames::token
html/parser.dart::AfterAfterFramesetPhase::startTagOther::token
html TagAttribute token.dart::StartTagToken::attributeSpans
html CommentToken html/parser.dart::Phase::processComment::token
html CharactersToken html/parser.dart::Phase::processCharacters::token
html/parser.dart::InTablePhase::insertText::token
html SpaceCharactersToken html/parser.dart::Phase::processSpaceCharacters::token
html EndTagToken html/parser.dart::Phase::processEndTag::token
html/parser.dart::Phase::popOpenElementsUntil::token
html/parser.dart::BeforeHeadPhase::endTagImplyHead::token
html/parser.dart::BeforeHeadPhase::endTagOther::token
html/parser.dart::InHeadPhase::endTagHead::token
html/parser.dart::InHeadPhase::endTagHtmlBodyBr::token
html/parser.dart::InHeadPhase::endTagOther::token
html/parser.dart::AfterHeadPhase::endTagHtmlBodyBr::token
html/parser.dart::AfterHeadPhase::endTagOther::token
html/parser.dart::InBodyPhase::endTagP::token
html/parser.dart::InBodyPhase::endTagBody::token
html/parser.dart::InBodyPhase::endTagHtml::token
html/parser.dart::InBodyPhase::endTagBlock::token
html/parser.dart::InBodyPhase::endTagForm::token
html/parser.dart::InBodyPhase::endTagListItem::token
html/parser.dart::InBodyPhase::endTagHeading::token
html/parser.dart::InBodyPhase::endTagFormatting::token
html/parser.dart::InBodyPhase::endTagAppletMarqueeObject::token
html/parser.dart::InBodyPhase::endTagBr::token
html/parser.dart::InBodyPhase::endTagOther::token
html/parser.dart::TextPhase::endTagScript::token
html/parser.dart::TextPhase::endTagOther::token
html/parser.dart::InTablePhase::endTagTable::token
html/parser.dart::InTablePhase::endTagIgnore::token
html/parser.dart::InTablePhase::endTagOther::token
html/parser.dart::InCaptionPhase::endTagCaption::token
html/parser.dart::InCaptionPhase::endTagTable::token
html/parser.dart::InCaptionPhase::endTagIgnore::token
html/parser.dart::InCaptionPhase::endTagOther::token
html/parser.dart::InColumnGroupPhase::endTagColgroup::token
html/parser.dart::InColumnGroupPhase::endTagCol::token
html/parser.dart::InColumnGroupPhase::endTagOther::token
html/parser.dart::InTableBodyPhase::endTagTableRowGroup::token
html/parser.dart::InTableBodyPhase::endTagIgnore::token
html/parser.dart::InTableBodyPhase::endTagOther::token
html/parser.dart::InRowPhase::endTagTr::token
html/parser.dart::InRowPhase::endTagTable::token
html/parser.dart::InRowPhase::endTagTableRowGroup::token
html/parser.dart::InRowPhase::endTagIgnore::token
html/parser.dart::InRowPhase::endTagOther::token
html/parser.dart::InCellPhase::endTagTableCell::token
html/parser.dart::InCellPhase::endTagIgnore::token
html/parser.dart::InCellPhase::endTagImply::token
html/parser.dart::InCellPhase::endTagOther::token
html/parser.dart::InSelectPhase::endTagOption::token
html/parser.dart::InSelectPhase::endTagOptgroup::token
html/parser.dart::InSelectPhase::endTagSelect::token
html/parser.dart::InSelectPhase::endTagOther::token
html/parser.dart::InSelectInTablePhase::endTagTable::token
html/parser.dart::InSelectInTablePhase::endTagOther::token
html/parser.dart::AfterBodyPhase::endTagOther::token
html/parser.dart::InFramesetPhase::endTagFrameset::token
html/parser.dart::InFramesetPhase::endTagOther::token
html/parser.dart::AfterFramesetPhase::endTagHtml::token
html/parser.dart::AfterFramesetPhase::endTagOther::token

This check can be disabled by tagging the PR with skip-leaking-check.

Unused Dependencies ✔️
Package Status
html ✔️ All dependencies utilized correctly.

For details on how to fix these, see dependency_validator.

This check can be disabled by tagging the PR with skip-unused-dependencies-check.

License Headers ✔️
// Copyright (c) 2026, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

Files
no missing headers

All source files should start with a license header.

Unrelated files missing license headers
Files
pkgs/html/test/differential_test.dart
pkgs/html/test/dom_parsing_test.dart
pkgs/html/test/mxss_test.dart
pkgs/html/test/noah_ark_clause_limit_test.dart

This check can be disabled by tagging the PR with skip-license-check.

Coverage ✔️
File Coverage
pkgs/html/lib/html_escape.dart 💚 100 %
pkgs/html/lib/src/html_input_stream.dart 💚 94 % ⬆️ 0 %

This check for test coverage is informational (issues shown here will not fail the PR).

This check can be disabled by tagging the PR with skip-coverage-check.

@palmoni5
palmoni5 marked this pull request as ready for review November 10, 2026 19:48
String.fromCharCodes iterates a non-builtin list from its first element on
the VM, so reading spans from String.codeUnits made parsing quadratic in the
input length. Keep the decoded String and use substring when normalization
did not copy the code units.
@palmoni5
palmoni5 force-pushed the palmoni5/html-stream-allocations-20261110 branch from 6aff048 to 1813200 Compare November 10, 2026 20:21
@palmoni5
palmoni5 merged commit 2eb177d into main Nov 10, 2026
18 checks passed
@palmoni5
palmoni5 deleted the palmoni5/html-stream-allocations-20261110 branch November 10, 2026 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant