Add a first-class Document endpoint to CALM Hub

[LeighFinegold]

Feature Proposal

Target Project:

calm-hub Schema-side definition of the document type / structured frontmatter may sit in finos/calm-schema

Description of Feature:

Add a first-class Document resource to CALM Hub so a document (Markdown + structured frontmatter) can be created, retrieved, and updated by document type. This is the storage/API foundation for the Documents PoC and implements the decision in #2791 (documents are first-class, not decorators) using the document shape agreed in #2866. Because documents must be versioned, model it on the existing versioned artefact resources (Architecture, Pattern, Standard), which already provide namespace and version semantics. The Decorator resource is deliberately not the reference here, because decorators are unversioned (#2791).

CALM Hub stores the document source (Markdown + frontmatter) - it does not store rendered binaries (no PDF/PNG/SVG).

User Stories:

• As a CALM Hub client, I want to publish a document of a given type so that it is registered, versioned, and retrievable from the Hub.
• As a CALM Hub consumer, I want to retrieve a document by type so that I can read the latest published version without regenerating it.

Current Limitations:

• CALM Hub has no Document resource. The only way to attach a narrative artefact today is to overload the Decorator resource, which is metadata attached to other elements - not a standalone, versioned document (the problem Design Discussion: Documents vs Decorators #2791 resolves).

Proposed Implementation:

Follow the standard calm-hub resource layering (resource → service → store interface → MongoDB and NitriteDB implementations → CDI producer selecting the implementation via calm.database.mode). Take the versioning shape from a versioned artefact resource such as Architecture or Standard ({namespace}/.../{id}/versions/{version}), not from the unversioned Decorator resource.

API (new versioned Document resource), following the Architecture/Standard version pattern:

• Namespace-scoped, consistent with the rest of CALM Hub:
  • GET {namespace}/documents/{document-type} - list document ids of a type
  • POST {namespace}/documents/{document-type} - create a document; 201 Created + Location (new id)
  • GET {namespace}/documents/{document-type}/{id}/versions - list versions
  • GET {namespace}/documents/{document-type}/{id}/versions/{version} - retrieve a specific version
  • POST {namespace}/documents/{document-type}/{id}/versions/{version} - add a version (immutable once written, like other artefacts)
• Guard reads with READ and writes with WRITE scopes; validate {namespace}, {document-type} and {version} against the standard patterns/allow-lists; declare a JSON producer on every method and sanitize any echoed user input (the Hub's standard XSS guard).

Data model:

• New Document domain model + store interface with Mongo + Nitrite implementations and a CDI producer, plus a DocumentNotFoundException and matching error responses.
• Versioned like other Hub artefacts: store versions immutably as Architecture/Standard do. This is the main reason the model is a versioned resource, not the unversioned Decorator (Design Discussion: Documents vs Decorators #2791).
• {document-type} allow-list should reuse the canonical list being standardised in Standardize CALM document type list in calm-models #2770 rather than duplicating it; confirm whether a knowledge-style narrative type (e.g. SAD) is added.

Registration/audit wiring:

• Register the new resource type for audit logging and add a DOCUMENT entity type.
• Optionally include documents in the pre-seeded read-only image flow (not required for the PoC).

Alternatives Considered:

• Reuse the Decorator resource - rejected per Design Discussion: Documents vs Decorators #2791; decorators are unversioned, and documents have their own lifecycle, versioning, and validation.
• Store rendered output - rejected; Hub stores document source only (Design Discussion: Introduce a Knowledge Decorator for Architecture Documentation #2866).

Testing Strategy:

• Resource unit tests (mocked store): 400 validation for bad namespace/type, 404 for missing namespace/document, 201 on create, 200 on get.
• Mongo + Nitrite store tests at parity with a versioned resource's store tests (e.g. Architecture or Standard), covering version create, list, and retrieve.
• Integration test with containers.
• Maintain the Hub's per-class coverage threshold.

Documentation Requirements:

• OpenAPI/Swagger annotations so the endpoint is discoverable.
• CALM Hub README endpoint list.

Implementation Checklist:

• Design reviewed and approved
• Implementation completed
• Tests written and passing
• Documentation updated
• Relevant workflows updated (if needed)
• Performance impact assessed

Additional Context:

• Gated by the metadata-validation rules in the sibling validation issue (relates to Validate uploaded documents in CalmHub via CALM server sidecar #2716) - the endpoint should reject writes that fail basic structured frontmatter/metadata validation.
• The register/render mechanism (registry + views) is designed in CALM-Hub Decorator Registry and Views #2792; addressing/resolution of documents is covered by Referencing documents in CalmHub #2301.

[101Steeps]

I'll work on this first

[101Steeps]

Implementation and POC verification

The first-class, namespace-scoped narrative-document API is implemented in #3002 and has been exercised by the combined local #2981 POC.

The POC successfully used the API to:

• Create a SAD document at version 1.0.0.
• Retrieve the stored Markdown through the version endpoint.
• Create and retrieve immutable version 1.1.0 after a workspace-driven minor bump.

Verification completed in the combined POC:

• ../mvnw verify — 2,772 tests passed; JaCoCo checks passed.
• ../mvnw -P integration verify — 537 tests passed; JaCoCo checks passed.
• CLI + Calm Hub smoke suite — 7 test files / 28 tests passed.

Swagger confirms the document route set: list/create document, list versions, retrieve a version, and create a version.

Related CLI integration: #3015.