Skip to content

Serialize Google Docs documents to Markdown - #581

Open
kcarnold wants to merge 1 commit into
mainfrom
claude/google-docs-api-capabilities-xvwtjg
Open

Serialize Google Docs documents to Markdown#581
kcarnold wants to merge 1 commit into
mainfrom
claude/google-docs-api-capabilities-xvwtjg

Conversation

@kcarnold

Copy link
Copy Markdown
Contributor

The Google Docs document reaches the model as Markdown instead of bare text, so structure the writer marked in Docs survives into the prompt.

Why

The document's only consumer is an LLM prompt, but Apps Script was flattening it to characters, which lost more than it looked like:

  • Headings arrived as ordinary sentences. Paragraph.getHeading() was never called, so nothing marked structure.
  • Lists arrived as prose. Docs renders bullets and numbers from list formatting — the glyphs are not characters, and getText() on a list item returns only the words. Nesting depth was invisible too.
  • Paragraphs ran together with no separator. DocumentApp text carries no trailing newline (the paragraph boundary is the newline; only Body.getText() inserts it), and extractTextOnly concatenated paragraph text directly. Since getParagraphs() splits the result on \n, it returned the entire document as a single paragraph on this surface.

What changed

serializeBody() in Code.gs emits Markdown: headings as #######, list items as - or sequentially-numbered N., four spaces per nesting level, blank lines between blocks. Empty paragraphs are spacing rather than content and emit nothing.

Where Docs is richer than Markdown, we flatten to the nearest form rather than invent notation — a private convention is one more thing for the model to misread:

Docs Markdown Note
Heading 1–6 #######
Title # collapses with Heading 1; Markdown's top level is the title
Subtitle plain paragraph not an outline level in Docs, so it must not claim a section boundary
Round / hollow / square bullet -
Number / latin / roman 1. numbered sequentially, so "the third step" is referenceable

Tables and images are still skipped, as they were before.

Positioning had to change with it

The old code searched the flattened text for the selected string. That resolved a phrase occurring twice to its first occurrence, and it cannot work at all against prefixes — a heading's selected text does not contain the ## in front of it.

serializeBody() now returns an index of where each top-level child's text landed, and positions are looked up from the element the cursor is actually in (walking up from the Text run via getParent()/getChildIndex()). This restores the invariant every caller assumes and getDocText() is built on: beforeCursor + selectedText + afterCursor is exactly the document. A wholly-selected heading includes its ## , which is what keeps that exact.

Unmappable positions (an element outside the body — a header, a footnote) fall back to "the whole document is before the cursor" rather than throwing, so a serialization edge case can't take the sidebar down.

Testing

Unit tests (21, in googleDocsMarkdown.test.ts) run the real Code.gs — read from disk and evaluated with a stubbed DocumentApp — so they cover the file that actually ships rather than a copy that can drift from it. The alternative, extracting a shared module, would mean either a second npm package for a folder clasp pushes verbatim or a module.exports guard in code that has no module system.

This matters for the dev loop: iterating a glyph table through clasp push + sidebar reload costs minutes per attempt, and here it costs milliseconds. Every positioning test also asserts the concatenation invariant.

A Debug page (lab tier, Labs ··· menu) renders the context exactly as the model receives it, with the cursor marked inline, plus the paragraph list and character count. Nothing else in the sidebar surfaces getDocContext() — every page folds it into a prompt, where a bad serialization is indistinguishable from the model just being confused. It works on Word and the standalone editor too, which makes it the quickest way to compare how the two editors serialize the same document.

Deliberately not flag-gated: flags are set from the URL, and neither the Word task pane nor the Google Docs sidebar has an addressable one (see flags.ts), so a flag would hide it exactly where it's needed. Happy to gate it if you'd rather it not sit in the Labs menu for beta users.

npm test → 243 passed. tsc --noEmit and eslint clean.

Not in scope

Comments — filed as #580. DocumentApp exposes none, and neither does the Docs REST API; they live in the Drive API, so reading them means adding a Drive scope and enabling the Advanced Drive Service. That's an authorization-surface change, not a serialization one.

Also noted as gaps in the docs: tables/images are dropped, and inline formatting (bold, italic, links) isn't carried — Docs stores it as attributes over text ranges rather than as characters.

Docs

  • docs/google-docs-document-serialization.md — full mapping, the flattening rationale, positioning, and the three ways to check it
  • frontend/CLAUDE.md — what the model sees, and the Word/Docs divergence a new page needs to handle
  • google-docs-addon/README.md — points at both

Deployment note

Code.gs changed, so this needs a clasp push to take effect — the bundle and the add-on deploy separately. Until then the installed add-on keeps returning bare text, which the frontend still handles (it's just a document with no Markdown in it).


Generated by Claude Code

The document's only consumer is an LLM prompt, but Apps Script was
flattening it to bare text, which lost more than it appeared to:

- Headings arrived as ordinary sentences. Nothing marked structure.
- Lists arrived as prose. Docs renders bullets and numbers from list
  formatting, so the glyphs are not characters and getText() never
  returns them; nesting was invisible too.
- Paragraphs ran together with no separator, because DocumentApp text
  carries no trailing newline and the extractor concatenated it
  directly. getParagraphs() splits on \n, so it returned the whole
  document as one paragraph.

serializeBody() now emits Markdown: headings as #, list items as - or
sequentially-numbered N., four spaces per nesting level, blank lines
between blocks. Where Docs is richer than Markdown we flatten to the
nearest form rather than invent notation — Title and Heading 1 both
become #, Subtitle becomes a paragraph (it is not an outline level),
and all ordered glyphs become N.

Positioning had to change with it. The old code searched the flattened
text for the selected string, which resolved a repeated phrase to its
first occurrence and cannot work at all against prefixes, since a
heading's selected text does not contain its "## ". serializeBody now
returns an index of where each child's text landed, and positions are
looked up from the element the cursor is actually in. This restores the
invariant every caller assumes: beforeCursor + selectedText +
afterCursor is exactly the document.

Testing, in two layers:

- Unit tests run the real Code.gs, read from disk and evaluated with a
  stubbed DocumentApp, so they cover the file that ships rather than a
  copy that can drift. Iterating the glyph table no longer costs a
  clasp push and a sidebar reload.
- A Debug page (lab tier) renders the context as the model receives it,
  cursor marked inline. It is the only way to see the real round-trip,
  and it works on every surface. Not flag-gated: flags come from the
  URL, and neither the Word task pane nor the Google Docs sidebar has
  an addressable one, so a flag would hide it exactly where it is
  needed.

Comments are untouched — DocumentApp exposes none, and reading them
needs a Drive scope. Tracked separately.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015urg1xXgUsRk3U8bUzFzC3
@kcarnold

Copy link
Copy Markdown
Contributor Author

New positioning logic needs testing!

Also, we'll need to apply the analogous thing to Word; see #155

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants