diff --git a/submissions/word-document-generator-from-template/README.md b/submissions/word-document-generator-from-template/README.md new file mode 100644 index 00000000..d7c581b4 --- /dev/null +++ b/submissions/word-document-generator-from-template/README.md @@ -0,0 +1,202 @@ +# Word Document Generator from Template + +Fill a Microsoft Word template supplied at runtime — **uploaded**, or retrieved +from **SharePoint**, **OneDrive**, or another connector — using **user input**, +approved agent **knowledge sources**, and **results from prior tool or connector +calls**. Works for any document the template defines: **policy, procedure, +report, paper, briefing, SOP, statement of work**, or similar. + +The template keeps control of structure, branding, styles, tables, headers, and +footers. A deterministic OOXML engine handles split Word runs, repeating table +rows, and validation while preserving live PAGE / NUMPAGES fields. The skill +writes a **new** DOCX — it never overwrites the original. + +## When to use it + +| Document kind | Typical template | +| --- | --- | +| Policy | Corporate policy shell (purpose, scope, rules, related docs) | +| Procedure / SOP | Numbered steps, roles, inputs/outputs | +| Report / briefing | Summary, findings table, recommendations | +| Paper | Title, abstract, body headings, references | +| Status pack | Narrative plus rows from Dataverse, SharePoint, or another connector | + +Ask the agent to create, draft, or compile the document from that template. + +## Before you start + +| Input | Why it matters | +| --- | --- | +| Word template (`.docx`) — **required** | Controls layout, placeholders, and branding. May be **uploaded**, or retrieved from **SharePoint**, **OneDrive**, or another connector | +| Document type, title, and purpose | Sets what is being drafted | +| Intended audience | Tones the language | +| Requirements | Anything the template must cover | +| Approved knowledge sources | Grounded content | +| Prior tool / connector results | Records, lists, and fields already retrieved this conversation | +| Output filename | Name of the new DOCX | + +Approved sources include knowledge, uploaded files, **and** data returned by +upstream tools or connectors. If a required fact is not in those sources, the +agent writes `Not specified in approved sources` instead of inventing it. + +## Ideal Word template structure + +The same pattern works for every document type. Use **Word styles** (Heading 1, +Heading 2, Normal) and deterministic `{{placeholders}}` — not finished body +text. The engine supports: + +- scalar paths: `{{document.title}}`, `{{sections.purpose}}`; +- repeating rows: `{{findings[].finding}}`, `{{findings[].owner}}`; +- placeholders that Word splits across multiple formatting runs; +- body, table, header, and footer text. + +**Do** + +- Put branding, page numbers, and classification in the **header / footer**. +- Use **Heading 1** for every major section the finished document must keep. +- Use a **one-row sample table** for anything that repeats (steps, findings, leave types, owners), with the array name followed by `[]`. +- Name placeholders after the field: `{{document.title}}`, `{{sections.}}`. +- Keep body cells short: `{{sections.purpose}}` or `[Insert from approved sources]`. + +**Don’t** + +- Bury finished wording in the template (it is not a knowledge source). +- Use floating text boxes or images that hide placeholders. +- Skip headings and rely on bold paragraphs — the agent may miss sections. + +### Generic skeleton + +**Header:** `Organisation | Classification | {{document.title}}` + +**Title (Heading 1):** `{{document.title}}` + +**Document control (Heading 2)** + +| Field | Placeholder | +| --- | --- | +| Type | `{{document.type}}` | +| Owner | `{{document.owner}}` | +| Version | `{{document.version}}` | +| Status | `{{document.status}}` | +| Audience | `{{document.audience}}` | + +**Body** — one Heading 1 per section, placeholder underneath. Name sections +after the template, for example: + +| Kind | Typical Heading 1s | +| --- | --- | +| Policy | Purpose, Scope, Policy statements, Responsibilities, Related documents | +| Procedure | Purpose, Scope, Roles, Procedure steps, Exceptions | +| Report | Executive summary, Findings, Analysis, Recommendations | +| Paper | Abstract, Introduction, Discussion, Conclusion, References | + +`{{sections.purpose}}`, `{{sections.scope}}`, `{{sections.findings}}`, and so on. + +**Repeating table** — keep the header row; leave **one sample data row** to clone: + +| Column A | Column B | Column C | +| --- | --- | --- | +| `{{items[].col_a}}` | `{{items[].col_b}}` | `{{items[].col_c}}` | + +Rename columns to match the document (`Step` / `Owner` / `System`, or +`Finding` / `Impact` / `Action`, or `Leave type` / `Entitlement` / `Owner`). +Use one array per sample row. + +**Footer:** `{{document.version}} | Page X of Y | {{document.status}}` + +Insert Page X of Y with Word's live PAGE and NUMPAGES fields, not typed numbers. +The engine changes only the placeholders and verifies those field instructions +remain intact. + +The agent fills placeholders from approved sources, **repeats the sample row** +for each JSON array item, and leaves gaps as `Not specified in approved +sources`. Styles, header, footer, and table formatting stay as in the template. + +### Example mapping — Leave Policy + +A leave policy is only one use of the same skeleton: Heading 1s become Purpose, +Scope, Leave types, Responsibilities; the repeating table columns become +`{{leave_types[].leave_type}}`, `{{leave_types[].entitlement}}`, +`{{leave_types[].owner}}`, `{{leave_types[].evidence}}`. + +## How it works + +1. Finds the Word template at runtime — from the upload, SharePoint, OneDrive, or the named connector. +2. Runs deterministic inspection to discover exact placeholders, repeating arrays, and Word fields. +3. Pulls facts from approved knowledge, user-supplied files, and prior tool or connector results. +4. Builds JSON that matches **this** template's fields. +5. Fills a new DOCX with split-run and repeating-row support. +6. Validates package integrity, unresolved placeholders, and live Word fields. +7. Returns the new DOCX and a machine-generated summary. + +## Quick test + +The bundled report template intentionally contains split-run placeholders, a +repeating findings row, branding, two sections, and live PAGE / NUMPAGES fields. + +```bash +# 1. Discover the template contract +python scripts/docx_template.py inspect assets/sample-template.docx \ + --output sample-manifest.json + +# 2. Fill a new file +python scripts/docx_template.py fill \ + assets/sample-template.docx assets/sample-data.json sample-output.docx \ + --summary sample-summary.json + +# 3. Verify there are no raw tokens and live fields survived +python scripts/docx_template.py validate sample-output.docx \ + --template assets/sample-template.docx --output sample-validation.json +``` + +Expected: each command exits `0`, the findings table has three data rows, no +`{{...}}` remains, and validation reports +`"field_signature_preserved": true`. + +For the full grammar and limits, see +[`references/placeholder-contract.md`](references/placeholder-contract.md). + +### Bundled files + +| File | Purpose | +| --- | --- | +| `scripts/docx_template.py` | Production inspect / fill / validate engine | +| `assets/sample-template.docx` | Realistic report template with split runs and live fields | +| `assets/sample-data.json` | Template-shaped example data | +| `assets/sample-template.manifest.json` | Expected inspection result | +| `references/placeholder-contract.md` | Exact grammar, supported scope, and limits | +| `scripts/test_docx_template.py` | Automated regression suite | +| `scripts/build_sample_template.py` | Rebuild the sample template | + +## Example requests + +> Use the Leave Policy template in SharePoint (`Policies/Templates/Leave-Policy.docx`). +> Draft version 0.1 for internal staff from approved HR knowledge. +> Save as `Leave-Policy-v0.1.docx`. + +> Fill the incident-response **procedure** template in OneDrive. +> Use the approved ops playbook for the steps. Save as `IR-Procedure-v2.docx`. + +> Get this quarter's accounts from Dataverse, then fill the **status report** +> template. Connector rows go in the findings table; knowledge base for the narrative. +> Save as `Q3-Account-Status.docx`. + +The agent returns the completed Word file plus a summary of what was filled, +what was missing, and which sources were used — including connector names. + +## Good to know + +- Output is a **draft** until a human reviews and approves it. +- Connector and tool results from earlier in the conversation are valid sources; the agent should not re-fetch them unless they are missing. +- Unsupported statements are marked for review, not presented as fact. +- The template is a prerequisite. Attach it, or point the agent at SharePoint, OneDrive, or another connector that can fetch the `.docx`. +- The template is not treated as a knowledge source unless you say so. +- Sections are not added or removed unless you explicitly ask. +- The original template in SharePoint, OneDrive, or the upload is never overwritten. +- Supported replacement content is plain text (including line breaks). Rich + HTML/Markdown, nested repeating arrays, and placeholders spanning paragraphs + are intentionally rejected or out of scope. +- Filling fails loudly on malformed tokens, invalid JSON shapes, remaining + placeholders, corrupt DOCX packages, or changed Word field instructions. +- If no `.docx` template is available, generation stops with: + `The required Word template was not supplied or could not be accessed.` diff --git a/submissions/word-document-generator-from-template/SKILL.md b/submissions/word-document-generator-from-template/SKILL.md new file mode 100644 index 00000000..aeb420ca --- /dev/null +++ b/submissions/word-document-generator-from-template/SKILL.md @@ -0,0 +1,188 @@ +--- +name: word-document-generator-from-template +description: Generates a complete Word document from a Word template supplied at runtime (uploaded, or retrieved from SharePoint, OneDrive, or another connector) plus user input, approved knowledge sources, and prior tool or connector results. Use when a user asks to create, draft, or compile any document from a template — policy, procedure, report, paper, briefing, SOP, or similar. +--- +# Word Document Generator from Template + +## Purpose + +Generate a complete Microsoft Word document of **any type the template defines** +(policy, procedure, report, paper, briefing, SOP, statement of work, or similar) using: + +- a Word template supplied at runtime (uploaded by the user, or retrieved from SharePoint, OneDrive, or another connector); +- information provided by the user; +- approved agent knowledge sources; +- files supplied with the request; and +- information retrieved from prior tool or connector calls in the same conversation (for example Dataverse, SharePoint, CRM, or any Copilot Studio action). + +The runtime template controls document type, structure, formatting, headings, +tables, headers, footers, and branding. Adapt JSON keys to **that** template — +do not assume a fixed outline. Use the bundled deterministic engine for DOCX +inspection, filling, and validation; do not implement ad-hoc run replacement. + +## Required inputs + +Before generating the document, identify: + +- the Word template to use, and where it comes from (upload, SharePoint, OneDrive, or another location); +- the document type, title, and purpose; +- the intended audience; +- any user-provided requirements; +- the approved knowledge sources to use; +- any relevant results from prior tool or connector calls; and +- the required output filename. + +If required information is unavailable, use: + +`Not specified in approved sources` + +Do not invent facts, dates, owners, approvals, obligations, or organizational information. + +## Instructions + +1. Locate the runtime `.docx` template. Use the uploaded file, or retrieve the + user-identified SharePoint / OneDrive / connector item into the working + directory. Stop with the message under **Template handling** if unavailable. +2. Inspect it before writing content: + + ```bash + python scripts/docx_template.py inspect template.docx --output manifest.json + ``` + + Read the manifest's exact scalar placeholders, repeating arrays, parts, and + live Word fields. If inspection rejects the template, report the error; do + not guess at its schema. +3. Retrieve relevant information from approved knowledge, user files, and prior + tool/connector results already in the conversation. Prefer connector-returned + records, dates, owners, and IDs over restating them from memory. +4. Generate long documents section by section. Build a JSON object whose paths + exactly match the manifest. Use arrays for repeating table rows. Use + `Not specified in approved sources` for unsupported facts. +5. Validate the JSON conceptually: all required template fields are represented, + claims are grounded, and each array item supplies the expected row fields. +6. Fill a **new** file with the deterministic engine: + + ```bash + python scripts/docx_template.py fill template.docx data.json output.docx \ + --summary fill-summary.json + ``` + + Never set `output.docx` to the template path. +7. Validate package integrity, unresolved tokens, and live Word fields: + + ```bash + python scripts/docx_template.py validate output.docx \ + --template template.docx --output validation.json + ``` + + Do not return a DOCX unless both commands succeed. +8. Return the completed DOCX plus a short generation summary: output filename, + document type, sources used, filled/defaulted fields, repeated-row counts, + and validation status. + +## Generation rules + +- Follow the runtime template's outline. JSON keys must match the inspection + manifest, not a hard-coded schema. +- Use only information from approved knowledge sources, user-supplied files, or prior tool/connector results. Do not invent facts that those sources do not contain. +- Treat prior tool and connector outputs as approved sources. Record the tool or connector name in `source_ids` (for example `Dataverse:accounts`, `SharePoint:policy-library`). +- Generate long documents section by section rather than in one response. +- Keep the structured JSON as the intermediate source of truth. +- Use clear, professional, organization-appropriate language for the stated audience. +- Preserve mandatory wording found in approved sources. +- Do not treat the template file as a knowledge source unless instructed. +- Do not add new sections unless required to complete the template. +- Do not remove sections from the template without an explicit instruction. +- Record the sources used for each major section when source information is available. +- Do not replace runs manually or clear footer/header paragraphs. The script + handles split-run tokens and preserves PAGE, NUMPAGES, TOC, REF, and other + live Word fields. + +## Template handling + +A Word template (`.docx`) is a **prerequisite**. It is supplied at runtime from one of: + +- a file **uploaded** with the request; +- **SharePoint** (document library, folder, or site); +- **OneDrive**; or +- another connector or prior tool call that returns a Word file. + +Resolve the template in this order: + +1. Use the template the user named (filename, SharePoint/OneDrive path, or library item). +2. If it is already in the runtime working directory, use that `.docx`. +3. If it is not local, retrieve it from SharePoint, OneDrive, or the identified connector. +4. If more than one Word file is available, select the one identified in the user request. + +Never overwrite the original template in SharePoint, OneDrive, or local storage. Always save a **new** DOCX. + +If the required template cannot be found or retrieved, stop document generation and report: + +'The required Word template was not supplied or could not be accessed.' + +## Template contract + +Use `{{path.to.value}}` for scalar text and `{{items[].field}}` in one sample +table row for repetition. Tokens may be split across Word runs; the engine +matches their visible paragraph text. It fills the main document, tables, +headers, and footers while preserving live Word fields. + +Read [`references/placeholder-contract.md`](references/placeholder-contract.md) +for the exact grammar, supported scope, limits, and troubleshooting. + +## Structured JSON + +Create JSON that reflects the **runtime template**. Use: + +- `document` — title, type, owner, version, status, audience, and any other metadata fields on the cover or control table; +- `sections` — one object per Heading 1 / Heading 2, keyed by a slug of that heading; +- `items` (or a name taken from the table, e.g. `leave_types`, `findings`, `steps`) — arrays for repeating tables or content blocks; +- `sources` — identifiers for knowledge, files, and connectors. + +Example shape (field names change to match the template): + +```json +{ + "document": { + "title": "Quarterly Operations Report", + "type": "Report", + "owner": "Operations", + "version": "1.0", + "status": "Draft", + "audience": "Leadership team" + }, + "sections": { + "executive_summary": "Generated section content", + "purpose": "Generated section content" + }, + "findings": [ + { + "finding": "Generated finding", + "impact": "Generated impact", + "owner": "Action owner" + } + ], + "sources": [ + { + "source_id": "SRC-001", + "title": "Approved source", + "type": "knowledge | file | connector" + } + ] +} +``` + +A Leave Policy template might use `leave_types`; a procedure might use `steps`; +a report might use `findings` or connector rows. Always use the array names +reported by template inspection. + +## Requirements + +The engine uses Python's standard library plus `lxml`, preinstalled in the +Copilot Studio sandbox. The sample-template builder and tests additionally use +preinstalled `python-docx` and Pillow. No network service or `pip install` is +needed in Copilot Studio. + +## Quality and safety + +The generated document is a draft until reviewed and approved. If a statement cannot be supported by approved knowledge, a user-supplied file, or a prior tool/connector result, do not present it as fact. Mark it for human review. diff --git a/submissions/word-document-generator-from-template/assets/sample-data.json b/submissions/word-document-generator-from-template/assets/sample-data.json new file mode 100644 index 00000000..9238c139 --- /dev/null +++ b/submissions/word-document-generator-from-template/assets/sample-data.json @@ -0,0 +1,41 @@ +{ + "document": { + "title": "Quarterly Operations Report", + "type": "Report", + "owner": "Operations", + "version": "1.0", + "status": "Draft", + "audience": "Leadership team" + }, + "sections": { + "executive_summary": "Operations remained stable during the quarter.\nThree priority findings require follow-up.", + "purpose": "Summarize quarterly operating performance using approved records and identify actions for the next reporting period.", + "scope": "Covers service delivery, incident response, and supplier operations for the reporting quarter.", + "recommendations": "Confirm action owners and due dates.\nReview progress at the next monthly operations meeting.", + "appendix": "Source extracts and detailed calculations are retained with the approved reporting evidence." + }, + "findings": [ + { + "finding": "Service response targets were met.", + "impact": "Customer commitments remained on track.", + "owner": "Service Delivery" + }, + { + "finding": "Two supplier actions remain open.", + "impact": "Minor schedule risk next quarter.", + "owner": "Vendor Management" + }, + { + "finding": "Incident review completion improved.", + "impact": "Faster closure and stronger traceability.", + "owner": "Operations Assurance" + } + ], + "sources": [ + { + "source_id": "SRC-001", + "title": "Approved quarterly operations dataset", + "type": "connector" + } + ] +} diff --git a/submissions/word-document-generator-from-template/assets/sample-template.docx b/submissions/word-document-generator-from-template/assets/sample-template.docx new file mode 100644 index 00000000..fb9aebe0 Binary files /dev/null and b/submissions/word-document-generator-from-template/assets/sample-template.docx differ diff --git a/submissions/word-document-generator-from-template/assets/sample-template.manifest.json b/submissions/word-document-generator-from-template/assets/sample-template.manifest.json new file mode 100644 index 00000000..c421869f --- /dev/null +++ b/submissions/word-document-generator-from-template/assets/sample-template.manifest.json @@ -0,0 +1,74 @@ +{ + "template": "sample-template.docx", + "scalar_placeholders": [ + "document.audience", + "document.owner", + "document.status", + "document.title", + "document.type", + "document.version", + "sections.appendix", + "sections.executive_summary", + "sections.purpose", + "sections.recommendations", + "sections.scope" + ], + "repeating_arrays": { + "findings": [ + "finding", + "impact", + "owner" + ] + }, + "parts": { + "word/document.xml": { + "scalar_placeholders": [ + "document.audience", + "document.owner", + "document.status", + "document.title", + "document.type", + "document.version", + "sections.appendix", + "sections.executive_summary", + "sections.purpose", + "sections.recommendations", + "sections.scope" + ], + "repeating_arrays": { + "findings": [ + "finding", + "impact", + "owner" + ] + } + }, + "word/footer1.xml": { + "scalar_placeholders": [ + "document.status", + "document.version" + ], + "repeating_arrays": {} + }, + "word/header1.xml": { + "scalar_placeholders": [ + "document.title" + ], + "repeating_arrays": {} + } + }, + "word_fields": { + "word/footer1.xml": { + "instructions": [ + "PAGE", + "NUMPAGES" + ], + "simple_fields": [], + "field_chars": { + "begin": 2, + "end": 2, + "separate": 2 + } + } + } +} diff --git a/submissions/word-document-generator-from-template/metadata.json b/submissions/word-document-generator-from-template/metadata.json new file mode 100644 index 00000000..87121862 --- /dev/null +++ b/submissions/word-document-generator-from-template/metadata.json @@ -0,0 +1,11 @@ +{ + "name": "Word Document Generator from Template", + "description": "Deterministically fill a runtime Word template for any document type using approved knowledge, user files, and prior tool results. Handles split placeholders and repeating tables, preserves branding and live fields, validates output, and returns a new DOCX.", + "platforms": ["Copilot Studio"], + "tags": ["word", "documents", "templates", "docx", "ooxml", "policy", "procedure", "reports", "sharepoint", "onedrive", "knowledge", "connectors"], + "author": "Nazish Qasim", + "authorUrl": "https://github.com/nazishqassim", + "version": "1.0.0", + "createdAt": "2026-08-14", + "updatedAt": "2026-08-14" +} diff --git a/submissions/word-document-generator-from-template/references/placeholder-contract.md b/submissions/word-document-generator-from-template/references/placeholder-contract.md new file mode 100644 index 00000000..39457e33 --- /dev/null +++ b/submissions/word-document-generator-from-template/references/placeholder-contract.md @@ -0,0 +1,132 @@ +# Deterministic DOCX placeholder contract + +`scripts/docx_template.py` fills plain-text placeholders in the main document, +tables, headers, and footers without flattening the surrounding Word runs. + +## Scalar values + +Use dotted JSON paths: + +```text +{{document.title}} +{{sections.executive_summary}} +{{metadata.approval.owner}} +``` + +JSON: + +```json +{ + "document": {"title": "Quarterly Operations Report"}, + "sections": {"executive_summary": "First line.\nSecond line."}, + "metadata": {"approval": {"owner": "Chief Operating Officer"}} +} +``` + +Strings, numbers, booleans, and `null` are accepted. Newlines become Word line +breaks. Objects and arrays cannot fill scalar placeholders. + +Missing scalar paths are filled with `Not specified in approved sources` and +listed in the fill summary under `defaulted_fields`. + +## Repeating table rows + +Put one sample row in the Word table and append `[]` to the array path: + +| Finding | Impact | Owner | +| --- | --- | --- | +| `{{findings[].finding}}` | `{{findings[].impact}}` | `{{findings[].owner}}` | + +JSON: + +```json +{ + "findings": [ + {"finding": "Finding A", "impact": "Low", "owner": "Team A"}, + {"finding": "Finding B", "impact": "High", "owner": "Team B"} + ] +} +``` + +The sample row is cloned twice and removed. An empty or missing array removes +the sample row and retains the table header. A row may reference exactly one +array path. Nested repeating arrays are not supported. + +Nested array paths are allowed: + +```text +{{report.findings[].title}} +{{report.findings[].rating}} +``` + +## Split Word runs + +Word may store a visible token across several runs: + +```xml +{{ +document.title +}} +``` + +The engine joins visible text while matching, then edits only the affected +`w:t` nodes. The replacement inherits the first token run's formatting. + +## Word fields + +The engine never writes to: + +- `w:instrText` +- `w:fldChar` +- `w:fldSimple` + +Before writing, it compares field instructions and field-character counts with +the template. If PAGE, NUMPAGES, TOC, REF, or another field changes, filling +fails and no output is written. + +It is safe to put a token beside live fields: + +```text +{{document.version}} | Page { PAGE } of { NUMPAGES } | {{document.status}} +``` + +Do not use a Word field itself as a placeholder. + +## Supported scope + +- `.docx` input and output +- Main body, tables, `header*.xml`, and `footer*.xml` +- Scalar plain text, numbers, booleans, `null`, and line breaks +- One repeating array per sample table row +- Split tokens contained within one paragraph or one table cell paragraph + +## Deliberate limits + +- No placeholder may span multiple paragraphs or table cells. +- No nested repeating arrays. +- No rich-text HTML/Markdown conversion inside a placeholder. +- No placeholders inside field instructions. +- Content controls and legacy MERGEFIELD values are preserved, not used as the + template syntax. +- Text embedded in unsupported package parts is not filled. + +Use Word styles, table formatting, and surrounding fixed text in the template +to achieve the desired visual design. + +## Commands + +```bash +# Discover exact fields before writing JSON +python scripts/docx_template.py inspect template.docx --output manifest.json + +# Fill a new document +python scripts/docx_template.py fill template.docx data.json output.docx \ + --summary fill-summary.json + +# Verify package integrity, unresolved tokens, and live fields +python scripts/docx_template.py validate output.docx \ + --template template.docx --output validation.json +``` + +All failures return exit code `2` and print a specific error. The original +template is never overwritten. diff --git a/submissions/word-document-generator-from-template/scripts/build_sample_template.py b/submissions/word-document-generator-from-template/scripts/build_sample_template.py new file mode 100644 index 00000000..309eeb8a --- /dev/null +++ b/submissions/word-document-generator-from-template/scripts/build_sample_template.py @@ -0,0 +1,183 @@ +#!/usr/bin/env python3 +"""Regenerate the bundled sample DOCX used by docs and tests.""" + +from __future__ import annotations + +import argparse +import tempfile +from pathlib import Path + +from docx import Document +from docx.enum.section import WD_SECTION +from docx.enum.table import WD_CELL_VERTICAL_ALIGNMENT +from docx.enum.text import WD_ALIGN_PARAGRAPH +from docx.oxml import OxmlElement +from docx.oxml.ns import qn +from docx.shared import Inches, Pt, RGBColor +from PIL import Image, ImageDraw + + +def _split_token(paragraph, token: str, pieces: tuple[str, ...] | None = None): + """Add a token as several Word runs to test run-safe replacement.""" + parts = pieces or (token[:2], token[2:-2], token[-2:]) + for index, part in enumerate(parts): + run = paragraph.add_run(part) + if index == 1: + run.bold = True + + +def _field(paragraph, instruction: str) -> None: + run = paragraph.add_run() + begin = OxmlElement("w:fldChar") + begin.set(qn("w:fldCharType"), "begin") + instr = OxmlElement("w:instrText") + instr.set(qn("xml:space"), "preserve") + instr.text = f" {instruction} " + separate = OxmlElement("w:fldChar") + separate.set(qn("w:fldCharType"), "separate") + result = OxmlElement("w:t") + result.text = "1" + end = OxmlElement("w:fldChar") + end.set(qn("w:fldCharType"), "end") + run._r.extend((begin, instr, separate, result, end)) + + +def _style_document(document: Document) -> None: + styles = document.styles + styles["Normal"].font.name = "Aptos" + styles["Normal"].font.size = Pt(10) + for name, size, color in ( + ("Title", 26, RGBColor(31, 78, 121)), + ("Heading 1", 16, RGBColor(31, 78, 121)), + ("Heading 2", 12, RGBColor(68, 68, 68)), + ): + styles[name].font.name = "Aptos Display" + styles[name].font.size = Pt(size) + styles[name].font.color.rgb = color + + +def _make_logo(path: Path) -> None: + image = Image.new("RGB", (600, 140), "white") + draw = ImageDraw.Draw(image) + draw.rounded_rectangle((5, 5, 595, 135), radius=24, fill=(31, 78, 121)) + draw.text((35, 45), "CONTOSO | DOCUMENT TEMPLATE", fill="white") + image.save(path, "PNG") + + +def build_sample(output: Path) -> None: + output.parent.mkdir(parents=True, exist_ok=True) + document = Document() + _style_document(document) + + section = document.sections[0] + section.top_margin = Inches(0.75) + section.bottom_margin = Inches(0.7) + section.left_margin = Inches(0.8) + section.right_margin = Inches(0.8) + + with tempfile.TemporaryDirectory() as temp_dir: + logo = Path(temp_dir) / "logo.png" + _make_logo(logo) + header = section.header + header_p = header.paragraphs[0] + header_p.alignment = WD_ALIGN_PARAGRAPH.CENTER + header_p.add_run().add_picture(str(logo), width=Inches(4.5)) + header_p.add_run("\nInternal | ") + _split_token(header_p, "{{document.title}}") + + title = document.add_paragraph(style="Title") + _split_token(title, "{{document.title}}", ("{{document.", "title", "}}")) + subtitle = document.add_paragraph() + subtitle.add_run("Generated deterministically from approved sources").italic = True + + document.add_heading("Document control", level=1) + control = document.add_table(rows=5, cols=2) + control.style = "Table Grid" + metadata = [ + ("Type", "{{document.type}}"), + ("Owner", "{{document.owner}}"), + ("Version", "{{document.version}}"), + ("Status", "{{document.status}}"), + ("Audience", "{{document.audience}}"), + ] + for row, (label, token) in zip(control.rows, metadata): + row.cells[0].text = label + row.cells[0].paragraphs[0].runs[0].bold = True + row.cells[1].text = "" + _split_token(row.cells[1].paragraphs[0], token) + for cell in row.cells: + cell.vertical_alignment = WD_CELL_VERTICAL_ALIGNMENT.CENTER + + for heading, token in ( + ("Executive summary", "{{sections.executive_summary}}"), + ("Purpose", "{{sections.purpose}}"), + ("Scope", "{{sections.scope}}"), + ): + document.add_heading(heading, level=1) + paragraph = document.add_paragraph() + _split_token(paragraph, token) + + document.add_heading("Findings", level=1) + findings = document.add_table(rows=2, cols=3) + findings.style = "Table Grid" + for cell, label in zip( + findings.rows[0].cells, ("Finding", "Impact", "Owner") + ): + cell.text = label + cell.paragraphs[0].runs[0].bold = True + row = findings.rows[1] + for cell, token in zip( + row.cells, + ( + "{{findings[].finding}}", + "{{findings[].impact}}", + "{{findings[].owner}}", + ), + ): + cell.text = "" + _split_token(cell.paragraphs[0], token) + + document.add_heading("Recommendations", level=1) + recommendations = document.add_paragraph() + _split_token(recommendations, "{{sections.recommendations}}") + + # A second section demonstrates that section properties and linked + # header/footer relationships survive the fill. + document.add_section(WD_SECTION.NEW_PAGE) + document.add_heading("Appendix", level=1) + appendix = document.add_paragraph() + _split_token(appendix, "{{sections.appendix}}") + + for sec in document.sections: + footer = sec.footer + footer.is_linked_to_previous = True + footer_p = section.footer.paragraphs[0] + footer_p.alignment = WD_ALIGN_PARAGRAPH.CENTER + _split_token(footer_p, "{{document.version}}") + footer_p.add_run(" | Page ") + _field(footer_p, "PAGE") + footer_p.add_run(" of ") + _field(footer_p, "NUMPAGES") + footer_p.add_run(" | ") + _split_token(footer_p, "{{document.status}}") + + document.save(output) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "output", + nargs="?", + default=str( + Path(__file__).resolve().parents[1] / "assets" / "sample-template.docx" + ), + ) + args = parser.parse_args() + output = Path(args.output).resolve() + build_sample(output) + print(output) + + +if __name__ == "__main__": + main() diff --git a/submissions/word-document-generator-from-template/scripts/docx_template.py b/submissions/word-document-generator-from-template/scripts/docx_template.py new file mode 100644 index 00000000..8a3e9068 --- /dev/null +++ b/submissions/word-document-generator-from-template/scripts/docx_template.py @@ -0,0 +1,705 @@ +#!/usr/bin/env python3 +"""Deterministic, field-safe DOCX template inspection and filling. + +The engine edits only visible WordprocessingML text in the main document, +headers, and footers. It preserves all other package parts and never rewrites +Word field instructions (PAGE, NUMPAGES, TOC, cross-references, and similar). + +Template contract: + Scalars: {{document.title}} or {{sections.purpose}} + Repeating rows: {{items[].name}} (one array path per template table row) + +Usage: + python docx_template.py inspect template.docx --output manifest.json + python docx_template.py fill template.docx data.json output.docx + python docx_template.py validate output.docx --template template.docx +""" + +from __future__ import annotations + +import argparse +import copy +import hashlib +import json +import os +import re +import sys +import zipfile +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, Callable, Iterable, Mapping + +from lxml import etree + + +W = "http://schemas.openxmlformats.org/wordprocessingml/2006/main" +XML = "http://www.w3.org/XML/1998/namespace" +NS = {"w": W} +XML_SPACE = f"{{{XML}}}space" + +REQUIRED_PARTS = {"[Content_Types].xml", "word/document.xml"} +SUPPORTED_PART_RE = re.compile(r"^word/(?:document|header\d+|footer\d+)\.xml$") +TOKEN_RE = re.compile( + r"\{\{\s*" + r"([A-Za-z_][A-Za-z0-9_-]*(?:\[\])?" + r"(?:\.[A-Za-z_][A-Za-z0-9_-]*(?:\[\])?)*)" + r"\s*\}\}" +) +TOKEN_CANDIDATE_RE = re.compile(r"\{\{.*?\}\}", re.DOTALL) +MAX_PACKAGE_FILES = 10_000 +MAX_UNCOMPRESSED_BYTES = 200 * 1024 * 1024 +DEFAULT_MISSING = "Not specified in approved sources" + + +class TemplateError(RuntimeError): + """Raised when the template contract or DOCX package is invalid.""" + + +@dataclass +class FillReport: + template: str + output: str + replaced_fields: set[str] = field(default_factory=set) + defaulted_fields: set[str] = field(default_factory=set) + repeated_rows: dict[str, int] = field(default_factory=dict) + modified_parts: set[str] = field(default_factory=set) + field_signature_preserved: bool = True + + def as_dict(self) -> dict[str, Any]: + return { + "template": self.template, + "output": self.output, + "replaced_fields": sorted(self.replaced_fields), + "defaulted_fields": sorted(self.defaulted_fields), + "repeated_rows": dict(sorted(self.repeated_rows.items())), + "modified_parts": sorted(self.modified_parts), + "field_signature_preserved": self.field_signature_preserved, + } + + +def _w(tag: str) -> str: + return f"{{{W}}}{tag}" + + +def _safe_parser() -> etree.XMLParser: + return etree.XMLParser( + resolve_entities=False, + no_network=True, + remove_blank_text=False, + recover=False, + huge_tree=False, + ) + + +def _parse_xml(content: bytes, part_name: str) -> etree._Element: + try: + return etree.fromstring(content, parser=_safe_parser()) + except (etree.XMLSyntaxError, ValueError) as exc: + raise TemplateError(f"Invalid XML in {part_name}: {exc}") from exc + + +def _serialize_xml(root: etree._Element) -> bytes: + return etree.tostring( + root, + encoding="UTF-8", + xml_declaration=True, + standalone=True, + ) + + +def _validate_zip_entries(infos: list[zipfile.ZipInfo]) -> None: + if len(infos) > MAX_PACKAGE_FILES: + raise TemplateError( + f"DOCX contains too many package entries ({len(infos)} > " + f"{MAX_PACKAGE_FILES})." + ) + total = sum(info.file_size for info in infos) + if total > MAX_UNCOMPRESSED_BYTES: + raise TemplateError( + f"DOCX uncompressed size is too large ({total} bytes > " + f"{MAX_UNCOMPRESSED_BYTES})." + ) + names = [info.filename for info in infos] + if len(names) != len(set(names)): + raise TemplateError("DOCX contains duplicate package entry names.") + missing = REQUIRED_PARTS - set(names) + if missing: + raise TemplateError( + "Not a valid DOCX package; missing: " + ", ".join(sorted(missing)) + ) + + +def _read_package(path: str | os.PathLike[str]) -> tuple[ + dict[str, bytes], dict[str, zipfile.ZipInfo] +]: + source = Path(path) + if not source.is_file(): + raise TemplateError(f"DOCX file not found: {source}") + if source.suffix.lower() != ".docx": + raise TemplateError(f"Expected a .docx file: {source}") + try: + with zipfile.ZipFile(source, "r") as archive: + infos = archive.infolist() + _validate_zip_entries(infos) + content = {info.filename: archive.read(info.filename) for info in infos} + metadata = {info.filename: info for info in infos} + except (zipfile.BadZipFile, OSError) as exc: + raise TemplateError(f"Cannot read DOCX package {source}: {exc}") from exc + return content, metadata + + +def _supported_parts(package: Mapping[str, bytes]) -> list[str]: + return sorted(name for name in package if SUPPORTED_PART_RE.match(name)) + + +def _text_nodes(paragraph: etree._Element) -> list[etree._Element]: + """Visible text nodes, excluding simple and complex Word-field contents.""" + nodes: list[etree._Element] = [] + field_depth = 0 + for node in paragraph.iter(): + if node.tag == _w("fldChar"): + kind = node.get(_w("fldCharType"), "") + if kind == "begin": + field_depth += 1 + elif kind == "end": + field_depth = max(0, field_depth - 1) + continue + if node.tag != _w("t") or field_depth: + continue + if any(ancestor.tag == _w("fldSimple") for ancestor in node.iterancestors()): + continue + nodes.append(node) + return nodes + + +def _paragraph_text(paragraph: etree._Element) -> str: + return "".join(node.text or "" for node in _text_nodes(paragraph)) + + +def _row_text(row: etree._Element) -> str: + return "".join(_paragraph_text(p) for p in row.iter(_w("p"))) + + +def _set_text(node: etree._Element, text: str) -> None: + node.text = text + if text[:1].isspace() or text[-1:].isspace(): + node.set(XML_SPACE, "preserve") + elif XML_SPACE in node.attrib: + del node.attrib[XML_SPACE] + + +def _find_node_offset( + spans: list[tuple[int, int, etree._Element]], position: int, *, end: bool = False +) -> tuple[int, etree._Element, int]: + for index, (start, stop, node) in enumerate(spans): + if start <= position < stop or (end and position == stop and stop > start): + return index, node, position - start + if spans and position == spans[-1][1]: + start, _, node = spans[-1] + return len(spans) - 1, node, position - start + raise TemplateError("Internal placeholder offset could not be mapped to a run.") + + +def _replace_in_paragraph( + paragraph: etree._Element, + resolver: Callable[[str], str | None], + *, + allow_array_tokens: bool, +) -> set[str]: + """Replace tokens even when Word split them across multiple runs.""" + nodes = _text_nodes(paragraph) + if not nodes: + return set() + text = "".join(node.text or "" for node in nodes) + matches = list(TOKEN_RE.finditer(text)) + replaced: set[str] = set() + if not matches: + return replaced + + spans: list[tuple[int, int, etree._Element]] = [] + cursor = 0 + for node in nodes: + value = node.text or "" + spans.append((cursor, cursor + len(value), node)) + cursor += len(value) + + # Reverse order keeps original offsets valid when two tokens share a run. + for match in reversed(matches): + key = match.group(1) + is_array = "[]" in key + if is_array and not allow_array_tokens: + continue + replacement = resolver(key) + if replacement is None: + continue + + first_i, first, first_offset = _find_node_offset(spans, match.start()) + last_i, last, last_offset = _find_node_offset( + spans, match.end(), end=True + ) + first_text = first.text or "" + last_text = last.text or "" + + if first is last: + _set_text( + first, + first_text[:first_offset] + replacement + first_text[last_offset:], + ) + else: + _set_text(first, first_text[:first_offset] + replacement) + for node_i in range(first_i + 1, last_i): + _set_text(spans[node_i][2], "") + _set_text(last, last_text[last_offset:]) + replaced.add(key) + return replaced + + +def _convert_newlines(root: etree._Element) -> None: + """Turn replacement newlines into Word line-break elements.""" + for text_node in list(root.iter(_w("t"))): + value = text_node.text or "" + if "\n" not in value: + continue + parent = text_node.getparent() + if parent is None or parent.tag != _w("r"): + continue + index = parent.index(text_node) + parent.remove(text_node) + lines = value.replace("\r\n", "\n").replace("\r", "\n").split("\n") + for line_i, line in enumerate(lines): + new_text = etree.Element(_w("t")) + _set_text(new_text, line) + parent.insert(index, new_text) + index += 1 + if line_i != len(lines) - 1: + parent.insert(index, etree.Element(_w("br"))) + index += 1 + + +def _tokens_in_element(element: etree._Element) -> list[str]: + tokens: list[str] = [] + for paragraph in element.iter(_w("p")): + tokens.extend(match.group(1) for match in TOKEN_RE.finditer(_paragraph_text(paragraph))) + return tokens + + +def _array_token(key: str) -> tuple[str, str] | None: + segments = key.split(".") + marked = [i for i, segment in enumerate(segments) if segment.endswith("[]")] + if not marked: + return None + if len(marked) > 1: + raise TemplateError(f"Nested repeating arrays are not supported: {key}") + index = marked[0] + segments[index] = segments[index][:-2] + array_path = ".".join(segments[: index + 1]) + item_path = ".".join(segments[index + 1 :]) + if not item_path: + raise TemplateError(f"Repeating token must name an item field: {key}") + return array_path, item_path + + +_MISSING = object() + + +def _lookup(data: Any, path: str) -> Any: + current = data + if not path: + return current + for segment in path.split("."): + if isinstance(current, Mapping) and segment in current: + current = current[segment] + else: + return _MISSING + return current + + +def _as_text(value: Any, path: str) -> str: + if value is None: + return "" + if isinstance(value, bool): + return "true" if value else "false" + if isinstance(value, (str, int, float)): + return str(value) + raise TemplateError( + f"Placeholder {path!r} requires a scalar value, got " + f"{type(value).__name__}." + ) + + +def _expand_repeating_rows( + root: etree._Element, + data: Mapping[str, Any], + missing_value: str, + report: FillReport, +) -> bool: + changed = False + for row in list(root.iter(_w("tr"))): + array_tokens = [ + key for key in _tokens_in_element(row) if _array_token(key) is not None + ] + if not array_tokens: + continue + roots = {_array_token(key)[0] for key in array_tokens} # type: ignore[index] + if len(roots) != 1: + raise TemplateError( + "A repeating table row may reference only one array; found: " + + ", ".join(sorted(roots)) + ) + array_path = next(iter(roots)) + items = _lookup(data, array_path) + if items is _MISSING: + items = [] + report.defaulted_fields.add(array_path) + if not isinstance(items, list): + raise TemplateError( + f"Repeating row {array_path!r} requires a JSON array." + ) + parent = row.getparent() + if parent is None: + raise TemplateError("Repeating table row has no parent table.") + insert_at = parent.index(row) + for item_index, item in enumerate(items): + if not isinstance(item, Mapping): + raise TemplateError( + f"{array_path}[{item_index}] must be a JSON object." + ) + clone = copy.deepcopy(row) + + def resolve_array(key: str) -> str | None: + parsed = _array_token(key) + if parsed is None or parsed[0] != array_path: + return None + item_path = parsed[1] + value = _lookup(item, item_path) + if value is _MISSING: + report.defaulted_fields.add( + f"{array_path}[{item_index}].{item_path}" + ) + return missing_value + report.replaced_fields.add( + f"{array_path}[{item_index}].{item_path}" + ) + return _as_text(value, f"{array_path}[].{item_path}") + + for paragraph in clone.iter(_w("p")): + _replace_in_paragraph( + paragraph, resolve_array, allow_array_tokens=True + ) + parent.insert(insert_at, clone) + insert_at += 1 + parent.remove(row) + report.repeated_rows[array_path] = report.repeated_rows.get(array_path, 0) + len(items) + changed = True + return changed + + +def _replace_scalars( + root: etree._Element, + data: Mapping[str, Any], + missing_value: str, + report: FillReport, +) -> bool: + changed = False + + def resolve_scalar(key: str) -> str | None: + if "[]" in key: + return None + value = _lookup(data, key) + if value is _MISSING: + report.defaulted_fields.add(key) + return missing_value + report.replaced_fields.add(key) + return _as_text(value, key) + + for paragraph in root.iter(_w("p")): + replaced = _replace_in_paragraph( + paragraph, resolve_scalar, allow_array_tokens=False + ) + changed = changed or bool(replaced) + return changed + + +def _field_signature(root: etree._Element) -> dict[str, Any]: + instructions = [ + re.sub(r"\s+", " ", (node.text or "").strip()) + for node in root.iter(_w("instrText")) + ] + simple = [ + re.sub(r"\s+", " ", (node.get(_w("instr")) or "").strip()) + for node in root.iter(_w("fldSimple")) + ] + fld_chars: dict[str, int] = {} + for node in root.iter(_w("fldChar")): + kind = node.get(_w("fldCharType"), "") + fld_chars[kind] = fld_chars.get(kind, 0) + 1 + return { + "instructions": instructions, + "simple_fields": simple, + "field_chars": dict(sorted(fld_chars.items())), + } + + +def _package_field_signature(package: Mapping[str, bytes]) -> dict[str, Any]: + signature: dict[str, Any] = {} + for part in _supported_parts(package): + root = _parse_xml(package[part], part) + part_sig = _field_signature(root) + if ( + part_sig["instructions"] + or part_sig["simple_fields"] + or part_sig["field_chars"] + ): + signature[part] = part_sig + return signature + + +def _scan_part( + root: etree._Element, +) -> tuple[set[str], dict[str, set[str]], list[str]]: + scalar: set[str] = set() + arrays: dict[str, set[str]] = {} + malformed: list[str] = [] + for paragraph in root.iter(_w("p")): + text = _paragraph_text(paragraph) + valid_spans = {match.span() for match in TOKEN_RE.finditer(text)} + for candidate in TOKEN_CANDIDATE_RE.finditer(text): + if candidate.span() not in valid_spans: + malformed.append(candidate.group(0)) + without_valid_tokens = TOKEN_RE.sub("", text) + if "{{" in without_valid_tokens or "}}" in without_valid_tokens: + excerpt = without_valid_tokens.strip() + malformed.append(excerpt[:120] or "unmatched placeholder braces") + for match in TOKEN_RE.finditer(text): + key = match.group(1) + parsed = _array_token(key) + if parsed is None: + scalar.add(key) + else: + arrays.setdefault(parsed[0], set()).add(parsed[1]) + return scalar, arrays, malformed + + +def inspect_template(path: str | os.PathLike[str]) -> dict[str, Any]: + package, _ = _read_package(path) + scalars: set[str] = set() + arrays: dict[str, set[str]] = {} + malformed: list[str] = [] + by_part: dict[str, dict[str, Any]] = {} + for part in _supported_parts(package): + root = _parse_xml(package[part], part) + part_scalars, part_arrays, part_malformed = _scan_part(root) + scalars.update(part_scalars) + malformed.extend(part_malformed) + for array_path, fields in part_arrays.items(): + arrays.setdefault(array_path, set()).update(fields) + by_part[part] = { + "scalar_placeholders": sorted(part_scalars), + "repeating_arrays": { + key: sorted(value) for key, value in sorted(part_arrays.items()) + }, + } + if malformed: + raise TemplateError( + "Malformed placeholder(s): " + ", ".join(sorted(set(malformed))) + ) + return { + "template": Path(path).name, + "scalar_placeholders": sorted(scalars), + "repeating_arrays": { + key: sorted(value) for key, value in sorted(arrays.items()) + }, + "parts": by_part, + "word_fields": _package_field_signature(package), + } + + +def _unresolved_tokens(package: Mapping[str, bytes]) -> list[dict[str, str]]: + unresolved: list[dict[str, str]] = [] + for part in _supported_parts(package): + root = _parse_xml(package[part], part) + for paragraph in root.iter(_w("p")): + text = _paragraph_text(paragraph) + for match in TOKEN_CANDIDATE_RE.finditer(text): + unresolved.append({"part": part, "token": match.group(0)}) + without_candidates = TOKEN_CANDIDATE_RE.sub("", text) + if "{{" in without_candidates or "}}" in without_candidates: + unresolved.append( + {"part": part, "token": "unmatched placeholder braces"} + ) + return unresolved + + +def _write_package( + output_path: Path, + package: Mapping[str, bytes], + metadata: Mapping[str, zipfile.ZipInfo], +) -> None: + output_path.parent.mkdir(parents=True, exist_ok=True) + with zipfile.ZipFile(output_path, "w") as archive: + for name, content in package.items(): + info = metadata[name] + archive.writestr(info, content) + + +def fill_template( + template_path: str | os.PathLike[str], + data: Mapping[str, Any], + output_path: str | os.PathLike[str], + *, + missing_value: str = DEFAULT_MISSING, +) -> dict[str, Any]: + source = Path(template_path).resolve() + output = Path(output_path).resolve() + if source == output: + raise TemplateError("Output path must differ from the template path.") + if output.suffix.lower() != ".docx": + raise TemplateError("Output path must end with .docx.") + if not isinstance(data, Mapping): + raise TemplateError("Fill data must be a JSON object.") + + package, metadata = _read_package(source) + original_fields = _package_field_signature(package) + report = FillReport(template=str(source), output=str(output)) + + for part in _supported_parts(package): + root = _parse_xml(package[part], part) + changed = _expand_repeating_rows(root, data, missing_value, report) + changed = _replace_scalars(root, data, missing_value, report) or changed + if changed: + _convert_newlines(root) + package[part] = _serialize_xml(root) + report.modified_parts.add(part) + + unresolved = _unresolved_tokens(package) + if unresolved: + details = ", ".join( + f"{item['token']} in {item['part']}" for item in unresolved + ) + raise TemplateError(f"Unresolved placeholder(s) remain: {details}") + + output_fields = _package_field_signature(package) + report.field_signature_preserved = output_fields == original_fields + if not report.field_signature_preserved: + raise TemplateError( + "Word field instructions changed during filling; output was not written." + ) + + _write_package(output, package, metadata) + validation = validate_docx(output, template_path=source) + result = report.as_dict() + result["validation"] = validation + return result + + +def validate_docx( + path: str | os.PathLike[str], + *, + template_path: str | os.PathLike[str] | None = None, +) -> dict[str, Any]: + package, _ = _read_package(path) + # Parse every XML part so corrupt output fails loudly. + for name, content in package.items(): + if name.endswith(".xml") or name.endswith(".rels"): + _parse_xml(content, name) + unresolved = _unresolved_tokens(package) + if unresolved: + details = ", ".join( + f"{item['token']} in {item['part']}" for item in unresolved + ) + raise TemplateError(f"Unresolved placeholder(s): {details}") + + fields = _package_field_signature(package) + fields_preserved: bool | None = None + if template_path is not None: + template, _ = _read_package(template_path) + fields_preserved = fields == _package_field_signature(template) + if not fields_preserved: + raise TemplateError("Word field signature differs from the template.") + + return { + "document": str(Path(path).resolve()), + "valid_docx": True, + "unresolved_placeholders": [], + "field_signature_preserved": fields_preserved, + "word_fields": fields, + "sha256": hashlib.sha256(Path(path).read_bytes()).hexdigest(), + } + + +def _load_json(path: str | os.PathLike[str]) -> Any: + try: + with open(path, "r", encoding="utf-8-sig") as handle: + return json.load(handle) + except (OSError, json.JSONDecodeError) as exc: + raise TemplateError(f"Cannot read JSON {path}: {exc}") from exc + + +def _write_json(data: Mapping[str, Any], destination: str | None) -> None: + text = json.dumps(data, indent=2, ensure_ascii=False) + if destination: + path = Path(destination) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text + "\n", encoding="utf-8") + print(text) + + +def _build_cli() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + description="Inspect, fill, and validate deterministic Word templates." + ) + sub = parser.add_subparsers(dest="command", required=True) + + inspect_p = sub.add_parser("inspect", help="Discover placeholders and fields") + inspect_p.add_argument("template", help="Input .docx template") + inspect_p.add_argument("--output", help="Optional manifest JSON path") + + fill_p = sub.add_parser("fill", help="Fill a template from JSON") + fill_p.add_argument("template", help="Input .docx template") + fill_p.add_argument("data", help="Template-shaped JSON object") + fill_p.add_argument("output", help="New output .docx path") + fill_p.add_argument( + "--missing", + default=DEFAULT_MISSING, + help=f"Fallback for absent scalar values (default: {DEFAULT_MISSING!r})", + ) + fill_p.add_argument("--summary", help="Optional fill-summary JSON path") + + validate_p = sub.add_parser("validate", help="Validate an output DOCX") + validate_p.add_argument("document", help="DOCX to validate") + validate_p.add_argument( + "--template", help="Original template for Word-field comparison" + ) + validate_p.add_argument("--output", help="Optional validation JSON path") + return parser + + +def main(argv: Iterable[str] | None = None) -> int: + if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8", errors="replace") + args = _build_cli().parse_args(list(argv) if argv is not None else None) + try: + if args.command == "inspect": + result = inspect_template(args.template) + _write_json(result, args.output) + elif args.command == "fill": + result = fill_template( + args.template, + _load_json(args.data), + args.output, + missing_value=args.missing, + ) + _write_json(result, args.summary) + else: + result = validate_docx( + args.document, template_path=args.template + ) + _write_json(result, args.output) + return 0 + except TemplateError as exc: + print(f"Error: {exc}", file=sys.stderr) + return 2 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/submissions/word-document-generator-from-template/scripts/test_docx_template.py b/submissions/word-document-generator-from-template/scripts/test_docx_template.py new file mode 100644 index 00000000..51d93d5a --- /dev/null +++ b/submissions/word-document-generator-from-template/scripts/test_docx_template.py @@ -0,0 +1,279 @@ +#!/usr/bin/env python3 +"""Regression tests for deterministic DOCX template filling.""" + +from __future__ import annotations + +import copy +import hashlib +import json +import subprocess +import sys +import tempfile +import unittest +import zipfile +from pathlib import Path + +from docx import Document +from lxml import etree + +from build_sample_template import build_sample +from docx_template import ( + TemplateError, + fill_template, + inspect_template, + validate_docx, +) + + +HERE = Path(__file__).resolve().parent +ASSETS = HERE.parent / "assets" +W = "http://schemas.openxmlformats.org/wordprocessingml/2006/main" +NS = {"w": W} + + +def _read_zip(path: Path) -> dict[str, bytes]: + with zipfile.ZipFile(path, "r") as archive: + return {name: archive.read(name) for name in archive.namelist()} + + +def _write_zip(path: Path, parts: dict[str, bytes]) -> None: + with zipfile.ZipFile(path, "w", zipfile.ZIP_DEFLATED) as archive: + for name, value in parts.items(): + archive.writestr(name, value) + + +def _visible_text(xml: bytes) -> str: + root = etree.fromstring(xml) + return "".join(root.xpath(".//w:t/text()", namespaces=NS)) + + +class DocxTemplateTests(unittest.TestCase): + def setUp(self) -> None: + self.temp = tempfile.TemporaryDirectory() + self.root = Path(self.temp.name) + self.template = self.root / "template.docx" + build_sample(self.template) + self.data = json.loads( + (ASSETS / "sample-data.json").read_text(encoding="utf-8") + ) + + def tearDown(self) -> None: + self.temp.cleanup() + + def test_inspect_finds_split_tokens_arrays_and_fields(self) -> None: + manifest = inspect_template(self.template) + self.assertIn("document.title", manifest["scalar_placeholders"]) + self.assertIn("sections.executive_summary", manifest["scalar_placeholders"]) + self.assertEqual( + manifest["repeating_arrays"], + {"findings": ["finding", "impact", "owner"]}, + ) + field_json = json.dumps(manifest["word_fields"]) + self.assertIn("PAGE", field_json) + self.assertIn("NUMPAGES", field_json) + self.assertIn("word/header1.xml", manifest["parts"]) + self.assertIn("word/footer1.xml", manifest["parts"]) + + def test_fill_preserves_fields_package_parts_and_original(self) -> None: + original_hash = hashlib.sha256(self.template.read_bytes()).hexdigest() + original_parts = _read_zip(self.template) + output = self.root / "nested" / "filled.docx" + + report = fill_template(self.template, self.data, output) + + self.assertTrue(output.is_file()) + self.assertEqual( + hashlib.sha256(self.template.read_bytes()).hexdigest(), original_hash + ) + self.assertTrue(report["field_signature_preserved"]) + self.assertEqual(report["repeated_rows"], {"findings": 3}) + self.assertEqual(report["defaulted_fields"], []) + self.assertTrue(report["validation"]["valid_docx"]) + + result_parts = _read_zip(output) + for name, content in original_parts.items(): + if name not in { + "word/document.xml", + "word/header1.xml", + "word/footer1.xml", + }: + self.assertEqual( + result_parts[name], + content, + f"Unmodified package part changed: {name}", + ) + + text = " ".join( + _visible_text(result_parts[name]) + for name in ("word/document.xml", "word/header1.xml", "word/footer1.xml") + ) + self.assertIn("Quarterly Operations Report", text) + self.assertIn("Service response targets were met.", text) + self.assertNotIn("{{", text) + + document = Document(output) + findings_table = next( + table for table in document.tables if table.rows[0].cells[0].text == "Finding" + ) + self.assertEqual(len(findings_table.rows), 4) # header + 3 items + + def test_newlines_become_word_line_breaks(self) -> None: + output = self.root / "line-breaks.docx" + fill_template(self.template, self.data, output) + root = etree.fromstring(_read_zip(output)["word/document.xml"]) + self.assertGreaterEqual(len(root.xpath(".//w:br", namespaces=NS)), 2) + + def test_missing_scalar_uses_fallback_and_reports_it(self) -> None: + data = copy.deepcopy(self.data) + del data["document"]["audience"] + output = self.root / "missing.docx" + report = fill_template(self.template, data, output) + self.assertIn("document.audience", report["defaulted_fields"]) + text = _visible_text(_read_zip(output)["word/document.xml"]) + self.assertIn("Not specified in approved sources", text) + + def test_empty_array_removes_sample_row(self) -> None: + data = copy.deepcopy(self.data) + data["findings"] = [] + output = self.root / "empty.docx" + report = fill_template(self.template, data, output) + self.assertEqual(report["repeated_rows"], {"findings": 0}) + document = Document(output) + findings_table = next( + table for table in document.tables if table.rows[0].cells[0].text == "Finding" + ) + self.assertEqual(len(findings_table.rows), 1) + + def test_wrong_array_type_fails_without_output(self) -> None: + data = copy.deepcopy(self.data) + data["findings"] = {"finding": "not an array"} + output = self.root / "bad-array.docx" + with self.assertRaisesRegex(TemplateError, "requires a JSON array"): + fill_template(self.template, data, output) + self.assertFalse(output.exists()) + + def test_complex_scalar_fails_without_output(self) -> None: + data = copy.deepcopy(self.data) + data["document"]["title"] = {"nested": "not supported"} + output = self.root / "bad-scalar.docx" + with self.assertRaisesRegex(TemplateError, "requires a scalar"): + fill_template(self.template, data, output) + self.assertFalse(output.exists()) + + def test_input_output_collision_is_rejected(self) -> None: + with self.assertRaisesRegex(TemplateError, "must differ"): + fill_template(self.template, self.data, self.template) + + def test_malformed_docx_is_rejected(self) -> None: + bad = self.root / "bad.docx" + bad.write_text("not a zip", encoding="utf-8") + with self.assertRaisesRegex(TemplateError, "Cannot read DOCX"): + inspect_template(bad) + + def test_malformed_placeholder_is_rejected(self) -> None: + parts = _read_zip(self.template) + root = etree.fromstring(parts["word/document.xml"]) + target = next( + node + for node in root.xpath(".//w:t", namespaces=NS) + if "sections.executive_summary" in (node.text or "") + ) + target.text = "sections executive_summary" + parts["word/document.xml"] = etree.tostring( + root, xml_declaration=True, encoding="UTF-8" + ) + malformed = self.root / "malformed-token.docx" + _write_zip(malformed, parts) + with self.assertRaisesRegex(TemplateError, "Malformed placeholder"): + inspect_template(malformed) + + def test_unmatched_placeholder_braces_are_rejected(self) -> None: + parts = _read_zip(self.template) + root = etree.fromstring(parts["word/document.xml"]) + target = next( + node + for node in root.xpath(".//w:t", namespaces=NS) + if "sections.executive_summary" in (node.text or "") + ) + target.text = "sections.executive_summary" + closing = target.getparent().getnext() + if closing is not None: + closing_text = closing.find(f"{{{W}}}t") + if closing_text is not None: + closing_text.text = "" + parts["word/document.xml"] = etree.tostring( + root, xml_declaration=True, encoding="UTF-8" + ) + malformed = self.root / "unmatched-token.docx" + _write_zip(malformed, parts) + with self.assertRaisesRegex(TemplateError, "Malformed placeholder"): + inspect_template(malformed) + + def test_validate_detects_removed_live_field(self) -> None: + output = self.root / "filled.docx" + fill_template(self.template, self.data, output) + parts = _read_zip(output) + footer = etree.fromstring(parts["word/footer1.xml"]) + instr = footer.xpath(".//w:instrText", namespaces=NS)[0] + instr.getparent().remove(instr) + parts["word/footer1.xml"] = etree.tostring( + footer, xml_declaration=True, encoding="UTF-8" + ) + damaged = self.root / "damaged.docx" + _write_zip(damaged, parts) + with self.assertRaisesRegex(TemplateError, "field signature"): + validate_docx(damaged, template_path=self.template) + + def test_cli_inspect_fill_validate(self) -> None: + data_path = self.root / "data.json" + data_path.write_text(json.dumps(self.data), encoding="utf-8") + manifest = self.root / "manifest.json" + output = self.root / "cli-output.docx" + summary = self.root / "summary.json" + validation = self.root / "validation.json" + script = HERE / "docx_template.py" + + commands = [ + [ + sys.executable, + str(script), + "inspect", + str(self.template), + "--output", + str(manifest), + ], + [ + sys.executable, + str(script), + "fill", + str(self.template), + str(data_path), + str(output), + "--summary", + str(summary), + ], + [ + sys.executable, + str(script), + "validate", + str(output), + "--template", + str(self.template), + "--output", + str(validation), + ], + ] + for command in commands: + result = subprocess.run( + command, capture_output=True, text=True, encoding="utf-8" + ) + self.assertEqual(result.returncode, 0, result.stderr) + for path in (manifest, output, summary, validation): + self.assertTrue(path.exists(), path) + self.assertTrue( + json.loads(validation.read_text(encoding="utf-8"))["valid_docx"] + ) + + +if __name__ == "__main__": + unittest.main(verbosity=2)