Calm Hub: Introduce NodeStore Interface

[byrash]

Feature Proposal

Target Project:

calm-hub — a new first-class store, served through the existing namespace-scoped REST API. A downstream consumption path (CLI / calm-models / VS Code plugin resolving nodes from CALM Hub) is in scope as a follow-on but is not the primary deliverable.

Description of Feature:

Introduce a NodeStore in CALM Hub alongside the existing Controls, ADRs, Standards, Flows, Patterns, Decorators, and Interfaces stores. It provides a namespace-scoped, versioned store for reusable custom node definitions, so an organization can publish a private, concrete catalog of node types and distribute it to all consumers through CALM Hub.

Today, a genuinely new node type must be bundled into the consuming tooling (for example, the VS Code plugin). That means every catalog change forces a tooling release and redistribution. Serving nodes centrally removes that coupling: consumers pull the approved catalog from CALM Hub, and node changes do not require continuous plugin updates.

Like the other stores, the NodeStore supports all three CALM Hub backends — MongoDB, NitriteDB, and the read-only GitHub backend. For GitHub-backed namespaces, node definitions are sourced straight from a Git repository by scanning predefined folder names recursively, so a team can manage its node catalog as files in Git and have CALM Hub serve them without a separate publish step.

User Stories:

• As a Enterprise Architecture team, I want to publish a private set of custom node definitions to CALM Hub so that all consumers use an approved catalog without waiting for a tooling release.
• As an architect (consumer), I want to retrieve organization-specific node definitions from CALM Hub so that I can compose architectures from approved building blocks.
• As a CALM Hub admin, I want node definitions to be namespace-scoped and permission-controlled so that catalogs stay private to the right teams.
• As a tooling maintainer, I want nodes served centrally so that I do not ship a plugin update every time a node is added or changed.
• As a team using a GitHub-backed namespace, I want CALM Hub to source node definitions from predefined folders in our repository, scanned recursively, so that our catalog is managed as code in Git and served without a separate publish step.

Current Limitations:

• Standards extend core CALM definitions through allOf composition (for example, requiring costCenter and owner on all nodes). They add constraints and required properties to existing nodes; they cannot introduce entirely new node types.
• Pattern + Standard layered governance covers "what nodes must exist" + "what properties they must have," but both assume the node types already exist in core CALM or in bundled tooling.
• Custom node definitions must therefore be distributed inside consuming tools, so the catalog cannot change without a release and redistribution.
• CALM Hub already centralizes Architectures, Patterns, Controls, ADRs, Standards, Flows, Decorators, Interfaces, and Domains — but has no equivalent store for reusable node definitions.
• The GitHub-backed backend discovers resources by folder-name convention (CalmContentDetector — architectures/, standards/, controls/, …) classified through CalmResourceType. Neither a NODE type nor a node folder convention exists today, so node files in a repository are ignored.

Proposed Implementation:

Mirror the existing store types (StandardStore / ControlStore / AdrStore) so the NodeStore is consistent with the current architecture. New components:

Layer	Component	Notes
Interface	NodeStore (org.finos.calm.store)	Namespace-scoped, versioned CRUD, matching the Standard/Control shape
Storage	MongoNodeStore, NitriteNodeStore, GitHubNodeStore	Implementations under store/mongo/, store/nitrite/, store/github/
Selection	NodeStoreProducer	CDI producer selecting mongo / standalone / github on calm.database.mode
REST	NodeResource	/calm/namespaces/{namespace}/nodes, guarded by @PermissionsAllowed(READ/WRITE)
Domain	Node, CreateNodeRequest, node exception types	Node exceptions mirror the Standard/ADR set

• Technical design: Namespace-scoped and versioned (list nodes, create node, list versions, get version, create version, delete), following the ID + version model already used by Standards.
• API changes: New GET/POST endpoints only; no change to existing routes. All endpoints declare @Produces(APPLICATION_JSON) and sanitize any echoed user input with STRICT_SANITIZATION_POLICY per the repo XSS rules.
• Data model changes: New collection/repository keyed by namespace + nodeId + version holding the node JSON definition in MongoDB and NitriteDB (including read-only mode); the GitHub backend uses git SHAs as versions.
• Dependencies on other components: Optional follow-ons — Search integration so nodes are discoverable, an MCP tool for node lookup, and a consumer-side resolver so a namespace-qualified node reference can point at a CALM Hub node.

GitHub-backed sourcing (recursive folders):

• Add NODE to CalmResourceType and register nodes/ as a known folder in CalmContentDetector (alongside the existing architectures/, standards/, … conventions). The detector already walks every path segment (getParentDirectoryName), so node files are discovered recursively at any depth under a nodes/ folder.
• GitHubNodeStore reads from the per-namespace clone, indexes node files via InMemoryRegistryService, and versions by git SHA through GitHubVersionService.
• Writes are unsupported in GitHub mode (GitHubWriteNotSupportedException), matching the other GitHub stores; the repository stays the source of truth.

Alternatives Considered:

Alternative	Why rejected
Standards (allOf composition)	Adds required properties to existing nodes; cannot define new node types
Patterns	Describe required node instances/topology, not a reusable catalog of node types
Bundling nodes in the VS Code plugin / tooling	Requires continuous releases and redistribution; no central governance or per-namespace privacy
Detecting nodes in Git by JSON structure only (no folder convention)	Rejected — a folder-name convention reuses CalmContentDetector and gives predictable, recursive discovery

Testing Strategy:

• Unit (JUnit 5): NodeStore implementations and NodeResource, covering success and error cases (>80% on new code per repo policy).
• Integration (TestContainers): MongoDB path; NitriteDB standalone and read-only paths (writes rejected in read-only).
• Security: per-namespace READ/WRITE enforcement and cross-namespace isolation.
• XSS: error responses that echo user input are sanitized.
• Smoke tests: extend smoke-test.sh (readonly + readwrite) to cover the new endpoints.
• GitHub backend: recursive discovery under the predefined node folders, registry indexing, git-SHA version listing/fetch, and write rejection in GitHub mode.

Documentation Requirements:

• CALM Hub developer guide "Store Interfaces" section (https://calm.finos.org/calm-hub/calm-hub-developer-guide/#the-store-interfaces).
• OpenAPI/Swagger annotations for the new endpoints.
• calm-hub/AGENTS.md store and resource lists.
• calm-hub/PERMISSIONS.md if any scope/grant changes are needed.
• Consumer-facing guidance on publishing and consuming nodes from CALM Hub.
• GitHub backend setup: enabling calm.database.mode=github, the nodes/ folder convention, and recursive discovery behavior.

Implementation Checklist:

• Design reviewed and approved
• Implementation completed
• Tests written and passing
• Documentation updated
• Relevant workflows updated (if needed)
• Performance impact assessed

Additional Context:

Raised in office hours with @jpgough-ms and @rocketstack-matt. The concrete node set is private to our organization, so this proposal asks for the distribution mechanism, not the specific nodes.

flowchart LR
    Repo["GitHub repo — nodes/ folders"] -->|recursive sync| Hub[(CALM Hub NodeStore)]
    Team["Enterprise Architecture team"] -->|publish versioned nodes| Hub
    Hub -->|resolve on demand| CLI[CALM CLI]
    Hub -->|resolve on demand| Plugin[VS Code Plugin]
    Hub -->|resolve on demand| Consumers[Other Consumers]

Loading