Composite catalog item schema definition

[jenniferubah]

This enhancement defines catalog items on a single spec.resources[] model, so single- and multi-resource offerings share the same shape and validation path instead of separate primitive/composite types.

[gabriel-farache]

some comment and IMO we should only have 1 shape and get rid of the useless difference between primitive and composite in the format as the primitive is just a special case of composite where there is only 1 resource

[croadfeldt]

Really like this — the composite catalog item shape is clean. Worth flagging for reuse: this model lines up almost 1:1 with two existing designs, which I think is a strong signal to converge on a shared substrate rather than maintain parallel ones.

• UDLM's Composite Service Composition Model: your resources[] / serviceType / requiresResources map directly onto its constituents[] / resource_type / depends_on DAG, and your CEL ${ordersDb.connectionString} wiring is its dependency binding-fields.
• OSAC's public Fulfillment API (github.com/osac-project): same spec/status shape, same JSON Schema 2020-12 validation, and a Template → CatalogItem → Instance catalog whose FieldDefinition{path, editable, default, validation_schema} is exactly your fields[].

Three independent designs converging on the same model is a good problem to have. Concretely, I'd suggest framing the catalog schema as the authoring/ordering projection over UDLM Resource Type Specifications (the vendor-neutral type contract):

• fields[] (editable / default / validationSchema) = a UDLM Constraint Profile over a base type — narrows, never widens, the type's contract;
• resource spec + your CEL outputs = the type's typed realized-state outputs, so cross-resource bindings are contract-checked rather than string-interpolated (this also closes your "standard outputs is follow-up" item);
• requiresResources = UDLM typed dependency edges (which DCM already validates as a DAG);
• catalog resolution → effective graph = the Intent → Requested assembly.

That keeps the catalog as the UX/curation layer and UDLM as the substrate underneath — so a CatalogItemInstance is just a UDLM Intent and its outputs are Discovered state, flowing straight into DCM's lifecycle / audit / sovereignty without a parallel model.

Happy to share the Resource Type Spec meta-schema + a few base entities we've been drafting (VM / Database / Container / Cluster) — Compute.Cluster in particular maps right onto OSAC's cluster type. Nice work 🙂

[jenniferubah]

Really like this — the composite catalog item shape is clean. Worth flagging for reuse: this model lines up almost 1:1 with two existing designs, which I think is a strong signal to converge on a shared substrate rather than maintain parallel ones.

• UDLM's Composite Service Composition Model: your resources[] / serviceType / requiresResources map directly onto its constituents[] / resource_type / depends_on DAG, and your CEL ${ordersDb.connectionString} wiring is its dependency binding-fields.
• OSAC's public Fulfillment API (github.com/osac-project): same spec/status shape, same JSON Schema 2020-12 validation, and a Template → CatalogItem → Instance catalog whose FieldDefinition{path, editable, default, validation_schema} is exactly your fields[].

Three independent designs converging on the same model is a good problem to have. Concretely, I'd suggest framing the catalog schema as the authoring/ordering projection over UDLM Resource Type Specifications (the vendor-neutral type contract):

• fields[] (editable / default / validationSchema) = a UDLM Constraint Profile over a base type — narrows, never widens, the type's contract;
• resource spec + your CEL outputs = the type's typed realized-state outputs, so cross-resource bindings are contract-checked rather than string-interpolated (this also closes your "standard outputs is follow-up" item);
• requiresResources = UDLM typed dependency edges (which DCM already validates as a DAG);
• catalog resolution → effective graph = the Intent → Requested assembly.

That keeps the catalog as the UX/curation layer and UDLM as the substrate underneath — so a CatalogItemInstance is just a UDLM Intent and its outputs are Discovered state, flowing straight into DCM's lifecycle / audit / sovereignty without a parallel model.

Happy to share the Resource Type Spec meta-schema + a few base entities we've been drafting (VM / Database / Container / Cluster) — Compute.Cluster in particular maps right onto OSAC's cluster type. Nice work 🙂

My understanding of UDLM is scarce, so not sure exactly what it is. Is UDLM going to be the standard for all schema definitions or is it something completely different? Also, could you share the link to the resource type schemas for UDLM? From what you mentioned, I'm assuming the goal is one standard that works across DCM and OSAC, is that right?

[croadfeldt]

Following up on the above — we took the convergence and built out the shared substrate, with the full reasoning written down in case it's useful here.

The substrate + why it's shaped this way — croadfeldt/udlm#1:

• The Resource Type Registry (meta-schema + base entities, including a Compute.Cluster derived from OSAC's cluster type): registry/
• Where to see the why: design-principles/core-tenets.md (the Data⇄Policy responsibility boundary — UDLM holds the data/lifecycle, DCM applies policy), design-principles/cross-cutting-requirements.md (audit / observability / dependency-graph / sovereignty as first-class principles), and docs/resource-type-registry-design-notes.md (the full decision trail + every source pulled in, plus a per-capability domain-assignment table).
• OSAC "better together" write-up: docs/osac-better-together.md.

The DCM side of the boundary — croadfeldt/dcm#5 (architecture/data-policy-boundary.md).

Net for this PR: your fields[] / CEL / requiresResources map cleanly onto Constraint Profiles + typed bindings over the dependency graph. The one design line we landed on — transformation/expressions stay in DCM policy, the portable data stays declarative — is what keeps audit reproducible and sovereignty enforceable. Happy to walk through any of it.

[croadfeldt]

Sorry, following up again and one clarification on CEL.

One substrate across DCM + OSAC is good, but more succintly I want to extend UDLM where it makes sense to. I would like to leverage the value they have that can inform UDLM and there was some enhancements to UDLM that would cover the spec that OSAC used and that had some genuine benefit overall.

UDLM is the type/contract standard, the catalog is the authoring projection over it — and the one concrete ask back is to drop CEL from the catalog spec in favor of declarative typed bindings (transformation → policy). The reason is that CEL leaves the door open to violating Data Sovereignty constraints. If the data spec allows for outreach directly, there is a potential for bypassing policies. Since the intent of CEL can be applied using the same policy mechanism, we can get the same effect while conforming.

Open to feedback on this.

[chadcrum]

The link points to #composite-catalog-item, but that anchor doesn't exist in catalog-item-schema.md. The closest heading is "Catalog item blueprint" which generates #catalog-item-blueprint.

Can you either update the link to match the actual heading, or add a "Composite catalog item" heading to catalog-item-schema.md if that's what this should reference?

[jenniferubah]

This is not needed anymore after the unification. Removed it now, see here: 784102c

[chadcrum]

The requiresResources field in the CatalogItem blueprint gets renamed to requirements in the placement payload, but there's no documentation explaining this transformation. Implementors will hit this mismatch and have to reverse-engineer it.

Either use the same field name in both places or add a note in the catalog resolution section explaining the rename and why it happens. Either way, the rename shouldn't be a surprise.

[jenniferubah]

Good catch, this is my (and my collaborative ai partner) mistake. It's supposed to be requiresResources. Updated it now, see: 2163a25

[chadcrum]

nit: The placement payload JSON example mixes naming conventions — serviceType is camelCase but the spec object uses snake_case (service_type, container_port). The blueprint YAML examples and service-type-definitions both use camelCase throughout. Is this intentional or an oversight?

[jenniferubah]

Good catch, we usually use camelCase for the examples in the enhacement, updated it now. See: 2163a25

[jenniferubah]

the one concrete ask back is to drop CEL from the catalog spec in favor of declarative typed bindings (transformation → policy). The reason is that CEL leaves the door open to violating Data Sovereignty constraints. If the data spec allows for outreach directly, there is a potential for bypassing policies. Since the intent of CEL can be applied using the same policy mechanism, we can get the same effect while conforming.

I believe we can achieve the same with CEL by:

• Using restricted CEL, meaning that every CEL expression must follow the pattern resourceName.output. So no general expressions.
• Ensuring the catalog spec with the expressions are validated during creation to verify it follows the standard schema
• During catalog instance creation, we also ensure the fields including the expressions are validated during mergeand transformation into the service type instance.
• Pertaining to policy bypass, this is a valid point. Having graph level policy would ensure sovereignty and avoid placing resources in different regions (cross region placement). Additionally, we can re-run policy after every CEL resolution within Placement. So for example, if ordersDb becomes ready with it's output, we re-run policy on app resource after the CEL expression (with ordersDb output) has been resolved.

Utlimately, I still think CEL can achieve what explicit bindings does but I may be missing something. What do you think?
Also, @machacekondra , @gciavarrini , @gabriel-farache, any thoughts or suggestions on this?

[gabriel-farache]

the one concrete ask back is to drop CEL from the catalog spec in favor of declarative typed bindings (transformation → policy). The reason is that CEL leaves the door open to violating Data Sovereignty constraints. If the data spec allows for outreach directly, there is a potential for bypassing policies. Since the intent of CEL can be applied using the same policy mechanism, we can get the same effect while conforming.

I believe we can achieve the same with CEL by:

• Using restricted CEL, meaning that every CEL expression must follow the pattern resourceName.output. So no general expressions.
• Ensuring the catalog spec with the expressions are validated during creation to verify it follows the standard schema
• During catalog instance creation, we also ensure the fields including the expressions are validated during mergeand transformation into the service type instance.
• Pertaining to policy bypass, this is a valid point. Having graph level policy would ensure sovereignty and avoid placing resources in different regions (cross region placement). Additionally, we can re-run policy after every CEL resolution within Placement. So for example, if ordersDb becomes ready with it's output, we re-run policy on app resource after the CEL expression (with ordersDb output) has been resolved.

Utlimately, I still think CEL can achieve what explicit bindings does but I may be missing something. What do you think? Also, @machacekondra , @gciavarrini , @gabriel-farache, any thoughts or suggestions on this?

I am not sure I understand your concern @croadfeldt .
In my head, the process is:

• provision parent resource(s) (the whole process, I will not detail it here)
• evaluate CEL expression in the children resource(s) so the actual value is set and used
• evaluate policies for the children (so with the real values, not the CEL expression)
• place the resource
• ...

So I do not see where and how the CEL could bypass the policies as their evaluation will come only after the CEL has replaced the expressions/placeholders with their real values. Thus, the payload that is evaluated is similar to a payload that would come straight from a user (like the parent resource is).
Could you give some more details about what is concerning to you about bypassing the policies?

[machacekondra]

OSAC's public Fulfillment API (github.com/osac-project): same spec/status shape, same JSON Schema 2020-12 validation, and a Template → CatalogItem → Instance catalog whose FieldDefinition{path, editable, default, validation_schema} is exactly your fields[].

Just FYI, fullfilment service has no notion of composite types (currently), this enhacement is about type composition. The catalog schema is already defined in different enahcement.