POC: First-class Documents in CALM Hub

[LeighFinegold]

Interested in being involved in this PoC? Add a comment on this issue.
Are you a maintainer who wants to help guide development? Please chip in by commenting on this issue too.

Executor of POC: @101Steeps

Aim

Prove, with a thin end-to-end proof of concept, that a CALM document (an OKF Markdown + structured-frontmatter artefact such as a Solution Architecture Document) can be authored, validated, pushed to CALM Hub via the calm workspace workflow, versioned, and retrieved as a first-class artefact rather than a Decorator, delivering the PoC agreed in #2866 and building on the #2791 decision (confirmed in Office Hours 2026-07-30, #2905) that document support is a first-class schema concept, not something implemented through decorators.

Objectives

1. Expose document endpoints in CALM Hub: a first-class, namespace-scoped Document resource that stores and serves documents (Markdown + structured frontmatter source e.g. OKF, never rendered binaries).
2. Make documents versioned first-class artefacts, mirroring the versioning other CALM Hub artefacts already receive.
3. Publish through the existing calm workspace → Hub workflow, so drift detection (check) and versioning (bump) come "for free" and the document is managed alongside its architecture.
4. Validate that an uploaded document conforms to the OKF frontmatter / basic-metadata contract.
5. Produce evidence to answer the decorator-versioning question raised in Design Discussion: Introduce a Knowledge Decorator for Architecture Documentation #2866: when a document is linked, do decorators need to become versioned documents, or do we mint a new decorator on each version bump?
6. Demonstrate the full value cycle end-to-end (author → validate → publish → retrieve) against a local CALM Hub.

Background & decision record

• Design Discussion: Documents vs Decorators #2791 Documents vs Decorators (decided). Resolved in Office Hours 2026-07-30 (Office Hours - 2026-07-30 #2905): documents are a first-class schema concept, not decorators. Decorators attach post-fact outcomes (reviews, pen-test reports, deployment logs, ARB minutes); a SAD is intent and context, a document.
• Design Discussion: Introduce a Knowledge Decorator for Architecture Documentation #2866 Open Knowledge Framework (OKF). Defines the document shape: Markdown with structured, validatable YAML frontmatter, human- and AI-readable, versioned. Records the agreement that CALM Hub supports OKF docs, validated via calm validate, pushed via calm workspace, with decorators used only to link narrative documents (the thin knowledge anchor). This epic is the PoC Design Discussion: Introduce a Knowledge Decorator for Architecture Documentation #2866
  asks for.
• Office Hours - 2026-07-30 #2905 Office Hours 2026-07-30. Confirmed the decision and set the PoC process: raise PoC work as issues, state the intended outcome and scope clearly, and apply a POC label.

Value cycle

Scope

In scope (minimum viable slice):

• One document type end-to-end (e.g. a SAD-style document or knowledge-intent document).
• Store and serve source (Markdown + frontmatter).
• Basic frontmatter/metadata validation on write.
• calm workspace publish and retrieval demo.

Out of scope (follow-ups):

• Storing rendered binaries (PDF/PNG/SVG); explicitly excluded.
• Full JSON-Schema/semantic validation of document content (tracked by Validate uploaded documents in CalmHub via CALM server sidecar #2716).
• A generic "any document" store; keep documents architectural, and avoid CALM Hub becoming a
  general-purpose file store.
• Views as an architectural concept; separate PoC track (Add First-Class Views to CALM #2344).

Constraints & guard-rails

• No rendered binaries in CALM Hub: store the OKF source (Markdown + frontmatter), not PDF/PNG/SVG.
• Don't let CALM Hub become a "SharePoint": keep documents architectural and purpose-built.
• Decorators remain the linkage mechanism for narrative (the knowledge anchor in Design Discussion: Introduce a Knowledge Decorator for Architecture Documentation #2866), never the
  store for the narrative itself.

Definition of done

• The child issues are delivered (endpoint, validation, workspace publish) and the render/register step is folded into CALM-Hub Decorator Registry and Views #2792.
• A user can calm workspace push a document and retrieve it from CALM Hub.
• The decorator-versioning question has evidence to inform a decision.

Related issues

• Design Discussion: Documents vs Decorators #2791 Documents vs Decorators (decision)
• Design Discussion: Introduce a Knowledge Decorator for Architecture Documentation #2866 Knowledge Decorator / Open Knowledge Framework (OKF)
• Office Hours - 2026-07-30 #2905 Office Hours 2026-07-30 (decision confirmed + PoC process)
• CALM-Hub Decorator Registry and Views #2792 CALM-Hub Decorator Registry and Views (render/register)
• Design Discussion: Decorator Association Scalability Concerns #2793 Decorator Association Scalability
• Validate uploaded documents in CalmHub via CALM server sidecar #2716 Validate uploaded documents in CalmHub via CALM server sidecar
• Standardize CALM document type list in calm-models #2770 Standardize CALM document type list in calm-models
• CALM Workspace vs CALM Hub CLI Command Comparison #2539 CALM Workspace vs CALM Hub CLI Command Comparison
• feat(cli): add push/pull/list/create commands for publishing and managing CALM Hub content #2377 CLI push/pull/list/create for CALM Hub content
• Referencing documents in CalmHub #2301 Referencing documents in CalmHub
• #2621 First-class behaviour document + behaviour decorator (precedent pattern)
• Add First-Class Views to CALM #2344 Add First-Class Views to CALM (separate track)
• Allow flows array to be a reference to a document referencing a flow calm-schema#20 flows remain first-class, detached from core.json

[101Steeps]

Excited to work on this!

[101Steeps]

Local POC evidence: first-class versioned narrative document

Completed local end-to-end POC for a first-class SAD-style Markdown document. It combines the Calm Hub API implementation in #3002 with the workspace publishing implementation in #3015.

What was demonstrated

A local standalone Calm Hub and a disposable Docker-backed smoke environment both completed this sequence:

Step	Demonstrated result
Author	A Markdown SAD with YAML frontmatter (title, optional description) was added to a CALM workspace.
Validate	Invalid/missing frontmatter is rejected by the workspace flow. The basic metadata contract is also validated by the Calm Hub write API.
Publish 1.0.0	calm workspace push created the architecture and the first-class sad document. Calm Hub returned a versioned Location and the workspace stored the numeric document id and location in its manifest.
Retrieve	GET /api/calm/namespaces/{namespace}/documents/sad/{id}/versions/1.0.0 returned the original stored Markdown.
Detect drift	After editing the Markdown on disk, calm workspace check correctly failed with “Run `calm workspace bump`”.
Version	calm workspace bump --minor changed the document manifest version from 1.0.0 to 1.1.0 without rewriting the Markdown source.
Publish 1.1.0	calm workspace push created an immutable 1.1.0 version; the unchanged architecture was skipped.
Retrieve again	The 1.1.0 endpoint returned the updated Markdown. A final workspace check reported the workspace up to date and the document valid.

Automated evidence

The combined #2981 smoke suite ran the same workflow against a disposable Calm Hub Docker stack:

✓ smoke/workspace-documents.smoke.spec.ts
Test Files  7 passed (7)
Tests       28 passed (28)

Local reproduction guide combining #2982 and #2983

issue-2981-maintainer-reproduction-guide.md

[VedPrakashJha-FMR]

Looking at the payment-sad.md file in the demo session:
a) my understanding is that it is an OKF markdown and I believe a non-empty "type" field is required in these OKF md files
b) Regarding "validate": are we validating that these MD files are following an OKF standard?