Investigate comparison marking unchanged or excessive content as changed

[stevenobiajulu]

Problem

Recurring public reports describe document comparison output marking unchanged content—or substantially more content than was actually edited—as deleted and inserted.

This issue tracks independent reproduction against the current Safe DOCX comparison engines. It does not assume the external reports share one implementation cause.

Research scope

• Independently construct minimal fixtures for unchanged paragraphs, run-boundary-only differences, and narrow replacements that may be expanded into broad delete/insert ranges.
• Exercise both inplace and rebuild reconstruction modes where applicable.
• Check accept/reject round trips and count phantom or over-broad revisions.
• Classify each fixture as reproduced, already supported, an existing-issue duplicate, or a distinct narrow defect.
• Do not copy customer documents, external fixtures, or vendor code.

Acceptance criteria

• At least three repository-owned fixtures are created independently from the described behavior.
• Current behavior is recorded for each fixture before implementation work begins.
• Existing comparison work and issues are checked for overlap.
• Any reproduced failures are split by mechanism when one fix would not coherently address them.
• Regression coverage proves unchanged content is not emitted as a substantive revision.

Maintainer workflow reminder

The private intake database is intentionally gitignored at data/customer-signals/customer-signals.sqlite3. Resume collection and inspect local state with:

npm run customer-signals:doctor
npm run customer-signals:status
npm run customer-signals:update -- --metadata-only --max-pages 10 --max-topics 200

Do not commit the database, private research notes, downloaded documents, or copied report text.

External signals

• https://forum.aspose.com/t/330964
• https://forum.aspose.com/t/318626
• https://forum.aspose.com/t/317236
• https://forum.aspose.com/t/289368

These links are demand signals only. Safe DOCX reproduction fixtures must be created independently from the described behavior.

[stevenobiajulu]

Independent reproduction against the current engine

Contributing a reproduction toward this issue's acceptance criteria, found while
using docx-compare on real work rather than by looking for it. Classifying it,
in this issue's own terms, as a distinct narrow defect with a specific root
cause, rather than a general "marks too much as changed" symptom.

The defect

When a paragraph in the original is split into two or more paragraphs in the
revised document, the atomizer matches the original to exactly one of the
resulting paragraphs and marks the rest as wholly new. The consequence is that
text which was not edited at all appears twice in the redline — once painted
as an insertion, once painted as a deletion.

Accept-all and reject-all are both exactly correct, so the output passes every
round-trip check. The defect is only visible on the rendered page. This is why
it is worth filing separately from the round-trip invariants: no accept/reject
property can catch it.

Splitting one paragraph into lettered or numbered sub-paragraphs is one of the
most common structural edits in contract drafting, so this is close to a
worst-case for a legal redline: the reader is told that provisions which were
carried forward verbatim were struck and rewritten.

Probably within the scope of #773, but that issue is an umbrella asking for
independent reproduction; this is a specific minimal mechanism with a fixture.

Minimal reproduction

before.docx — one paragraph:

The Account shall be responsible for all trading expenses, including commissions and custody fees.  The Manager shall be responsible for its own expenses, including salaries, rent and research.

after.docx — three paragraphs. The first two are that same paragraph split in
half, with a label added to each and a five-word lead-in added to the second.
The third is genuinely new.

(a)  The Account shall be responsible for all trading expenses, including commissions and custody fees.

(b)  Except as provided in Section 5(c), the Manager shall be responsible for its own expenses, including salaries, rent and research.

(c)  Reimbursement.  Subject to the remainder of this Section 5(c), the Account shall reimburse the Manager for the actual expenses incurred by the Manager in the following categories: (A) research; (B) rent; and (C) travel.

Fixture generator (python-docx), so the fixture is reproducible rather than
attached as a binary:

import docx
from docx.shared import Inches, Pt

BEFORE = ("The Account shall be responsible for all trading expenses, including "
          "commissions and custody fees.  The Manager shall be responsible for its "
          "own expenses, including salaries, rent and research.")
AFTER = [
 "(a)  The Account shall be responsible for all trading expenses, including commissions and custody fees.",
 "(b)  Except as provided in Section 5(c), the Manager shall be responsible for its own expenses, including salaries, rent and research.",
 "(c)  Reimbursement.  Subject to the remainder of this Section 5(c), the Account shall reimburse the Manager for the actual expenses incurred by the Manager in the following categories: (A) research; (B) rent; and (C) travel.",
]

def mk(path, paras):
    d = docx.Document()
    for t in paras:
        p = d.add_paragraph()
        p.paragraph_format.left_indent = Inches(0.5)
        p.paragraph_format.space_after = Pt(12)
        p.add_run(t)
    d.save(path)

mk("before.docx", [BEFORE])
mk("after.docx", AFTER)

safedocx compare before.docx after.docx out.docx

Why the round-trip invariants do not catch it

Walking w:ins/w:del over the output:

accept-all == after.docx   PASS  (exact)
reject-all == before.docx  PASS  (exact)

The output is textually perfect. Both projections are exactly right; it is the
attribution that is wrong. A regression test for this needs to assert something
about which runs carry revision markers, not about what the two projections
say — e.g. "text present unchanged in both documents must not appear inside both
a w:ins and a w:del".

Suggested acceptance criterion

For a fixture where one original paragraph's text is the concatenation of two
revised paragraphs' texts:

• no character sequence of ≥ N words appears inside both a w:ins and a w:del
  in the same output;
• the total inserted atom count is bounded by the actual added text, not by the
  length of the paragraph that received the split.

---

Fixture is synthetic — the only prose in it is "Alpha", "Bravo", "Charlie". No
customer document, external fixture or vendor code was used or copied.

[stevenobiajulu]

Root-caused this to paragraph-boundary restructuring, with one finding that is worse than excess redline: the accept/reject projection of a move-represented merge or split is not exact. Reproductions below are against packages/docx-compare/src via compareDocuments with buildDocxFromBodyXml, no external fixtures.

Cause

computeGroupLcs (packages/docx-compare/src/baselines/atomizer/hierarchicalLcs.ts:673) matches paragraph groups strictly one-to-one across every pass — Pass 1 exact-hash LCS, Pass 2 gap-scoped TF-IDF, Pass 3 Jaccard (whose assigned/assignedRev sets at :797-806 forbid a second match per group), Pass 4 container-position fallback for table cells. Any original paragraph therefore corresponds to at most one revised paragraph, and hierarchicalCompare marks every atom of an unmatched group wholesale deleted (:1002-1009) or inserted (:1010-1017). Nothing models a 1:N split or N:1 merge.

refineFuzzyRunsWithinAlignedParagraphs (pipeline.ts:999, selectiveWordRefinement.ts:45-108), added for #717, only refines runs inside pairs the LCS already aligned, so it cannot rescue a half that was never aligned.

What actually reproduces

Fixture: original = two paragraphs, "The Company shall deliver the Goods." / "Payment is due within thirty days."; revised = the two joined into one. And the inverse for split.

1. detectMoves: false — the #773 symptom. The whole first sentence is deleted and re-inserted verbatim, where Word deletes only the paragraph mark:

stats: insertedAtoms 13, deletedAtoms 12
DELETED:  ["The Company shall deliver the Goods."]
INSERTED: ["The Company shall deliver the Goods. "]

Projections are correct here — accept gives the revised document, reject gives the original. So this is purely excess redline: noisy, but safe.

2. Default options (detectMoves on) — round-trip break. The restructuring is rendered as a moveFrom/moveTo pair with insertedAtoms 0, deletedAtoms 0, which looks clean. It is not:

MERGE 2->1 (default)
  ACCEPT -> ["", "The Company shall deliver the Goods. Payment is due within thirty days."]
  want     ["The Company shall deliver the Goods. Payment is due within thirty days."]   *** MISMATCH ***
  REJECT -> OK

SPLIT 1->2 (default)
  REJECT -> ["", "The Company shall deliver the Goods. Payment is due within thirty days."]
  want     ["The Company shall deliver the Goods. Payment is due within thirty days."]   *** MISMATCH ***
  ACCEPT -> OK

A stranded empty paragraph survives in one direction each way — accept for a merge, reject for a split. The redline does not project back to its own inputs. A one-word substitution control is exact in both directions, so this is specific to paragraph-boundary restructuring.

This may deserve its own issue: #773 is scoped to over-marking, and a projection that does not reproduce its input is a different and higher-severity class.

What does not reproduce

A paragraph split emitting the unchanged first sentence inside both w:del and w:ins. With default options a plain 1→2 split yields insertedAtoms 0, deletedAtoms 0 and a move pair; adding a genuinely new third paragraph yields only that new paragraph as inserted (insertedAtoms 14, deletedAtoms 0), which is right. Worth recording so nobody chases duplicated-text-on-split — the duplication shows up on merge, not split, and only with detectMoves: false.

Bearing on scope

Several signals behind this cluster only say the output "differs from what the word processor produces". That is not a defect statement on its own — the host application's comparison is unspecified and version-dependent, and openspec/specs/docx-comparison/spec.md commits to no parity requirement. The reproducible parts above are stated as invariants against our own inputs instead, which is checkable without adopting parity as a goal.

Note also #814/#839 are actively reshaping the reconstruction layer; an alignment change landing before that settles may conflict.

[stevenobiajulu]

Observed at origin/main = 8f84295. PR #841 replaces the comparison engine (tagged-tree default); every observation below needs re-verification against it once merged. Regression tests are staged but deliberately not committed until then.

Independent reproduction run with nine repository-owned fixture families (all invented text, built via buildSyntheticDocx/buildDocxFromParts; no external documents or vendor code), each exercised in both inplace and rebuild, with accept/reject round-trips checked through acceptAllChanges/rejectAllChanges normalized by extractRoundTripComparisonText, and Aspose (licensed; outputs verified non-watermarked) as the comparison oracle on representative cases.

Results

fixture	inplace	rebuild	verdict
B1 unchanged 3-paragraph doc	0 revisions	0 revisions	correct
B2 identical text, run-boundary-only splits	0	0 (Aspose: 0)	correct — pins the regression test requested in #743's closing note
B3 narrow one-word replacement	del coastal + ins inland only	same (Aspose identical)	correct, minimal
B4 repeated tokens, one changed	single-token del/ins	same	correct
B5 punctuation-only (comma removed)	one del ","	same	correct
B9 run-boundary changes + one-word edit	minimal single-word del/ins	same	correct
B8 insertion adjacent to a bold run	one w:ins, zero rPrChange, zero whitespace-only rPrChange runs	same	correct — #743's whitespace-rPrChange artifact did not reproduce with this shape
B6 formatting-only change (no w:rPr → <w:b/>)	stats.formatChanges=1 but no w:rPrChange emitted; bold applied untracked; reject cannot restore	same	new distinct defect → #872
B7 paragraph split, text unchanged	first sentence emitted as full delete + re-inserted copy	same, plus a no-op <w:pPrChange><w:pPr/></w:pPrChange> on the inserted paragraph	over-broad vs Word canonical → #873; the pPrChange part duplicates #817

Diagnostics: every run used its requested mode (reconstructionModeUsed = requested, fallbackReason null, inplace passUsed = inplace_word_split throughout). Accept/reject text round-trips passed for all rows.

The reproduced mechanisms (split per fix)

1. none→formatted rPrChange omission → #872. Detection counts the change, but both emitters guard on a truthy old rPr (inPlaceModifier.ts handleFormatChanged, documentReconstructor.ts ~1900), so when the original run had no w:rPr at all, no w:rPrChange is emitted and the revised formatting lands untracked in both modes. Direction-specific: italic→bold, color→color, bold→none all emit correctly. Aspose emits <w:rPrChange><w:rPr/></w:rPrChange> (1 revision).

2. paragraph-split over-representation → #873. Word/Aspose keep the text in place and record an inserted paragraph mark plus deletion of the joining space (2 minimal revisions); we re-emit the whole first sentence as delete + inserted copy. Projection-correct but exactly the "narrow edit expanded into broad delete/insert" reviewer complaint.

3. Related but discovered under #852: attribute-blind no-op del/ins suppression silently swaps footnote references (unrejectable data loss) → #871. It belongs to the same "phantom/over-broad revision machinery" family (suppressNoOpChangePairs).

Overlap check

#743 (whitespace rPrChange over-emission — did not reproduce here; its "identical-text delete+insert" note is pinned by B2), #817 (duplicated by B7's rebuild pPrChange — not re-filed), #836/#846 (tagged-tree serialization / markdoc, different mechanisms), #857 (bench note; B1/B2 agree with its roundtrip=100 finding), #582 (rebuild fidelity, different mechanism), suppression machinery (suppressNoOpChangePairs → now #871, premergeAdjacentRuns #675), PR #765 (stale #743 draft, untouched since 2026-07-29 — neither new defect is reachable from it).

Recommendation

This run satisfies the issue's acceptance criteria as far as they can be satisfied pre-#841: ≥3 independent repo-owned fixtures (9 families), behavior recorded before any fix, overlap checked, mechanisms split per-fix (#871, #872, #873). The headline external complaint — unchanged content marked as changed — does not reproduce on this main for text content: unchanged, boundary-shifted, repeated-token, and punctuation fixtures all come out minimal and Aspose-agreeing. Remaining before closure: commit the unchanged-content/minimality regression tests once #841 lands and the expectations are re-derived against the tagged-tree engine. Recommending closure to the maintainer at that point; not closing from this investigation, and an evidence report derived from the pre-#841 engine is explicitly not a closure basis on its own.