diff --git a/docs/README.md b/docs/README.md index 9998f992..2e388d2e 100644 --- a/docs/README.md +++ b/docs/README.md @@ -18,6 +18,7 @@ Solve a specific problem: - [Repair a validation finding](how-to/repair-validation-findings.md) - [Replan after a stale refusal](how-to/replan-after-a-stale-refusal.md) - [Ship a breaking proto change](how-to/ship-a-breaking-proto-change.md) +- [Start an event model from scratch](how-to/start-a-model-from-scratch.md) - [Track a version migration](how-to/track-a-migration.md) ## Reference diff --git a/docs/how-to/start-a-model-from-scratch.md b/docs/how-to/start-a-model-from-scratch.md new file mode 100644 index 00000000..5cff578a --- /dev/null +++ b/docs/how-to/start-a-model-from-scratch.md @@ -0,0 +1,180 @@ +# How to start an event model from scratch + +Build the smallest event model that validates with no errors or warnings: +one human-issued command, one event, and one read model, tied together by +one storyboard. The +[`acme-wishlist`](../../examples/manifests/acme-wishlist/) namespace is +the finished result; each step below names the file it adds. + +Create the entities in the order below. Each entity refers only to +entities from earlier steps, so every check runs against a complete set. + +## Check each step + +Keep the files for one namespace in one directory. After each step, check +everything written so far in one batch: + +```sh +trogon-atlas apply -f acme-wishlist/ --dry-run +``` + +`--dry-run` runs request-time validation only. It checks each manifest, +and it rejects a slice whose scenario refers to an entity that is not in +the store or in the same batch (`SCENARIO_DANGLING_REF`). It does not run +model validation, so the last step checks the whole model. + +## 1. Describe the problem space + +| File | Kind | Fields the rules check | +| --- | --- | --- | +| `project.yaml` | `Project` | slug equal to the namespace | +| `domain.yaml` | `Domain` | `project` | +| `subdomain.yaml` | `Subdomain` | `domain`, `classification` | +| `bounded-context.yaml` | `BoundedContext` | slug equal to the namespace, `realizes` | + +- A Project's slug must equal its namespace (`PROJECT_SLUG_MISMATCH`, + Error). +- A Domain without `project` warns (`DOMAIN_MISSING_PROJECT`). The server + rejects a Domain whose `project` is in another namespace. +- The rule catalog requires a Subdomain to name its Domain in the same + namespace (`SUBDOMAIN_MISSING_DOMAIN`, `SUBDOMAIN_FOREIGN_DOMAIN`) and to + carry a `classification` (`SUBDOMAIN_UNCLASSIFIED`). +- A namespace is a bounded context, and the BoundedContext slug equals the + namespace (`CONTEXT_SLUG_MISMATCH`). +- A BoundedContext that realizes no Subdomain is reported + (`BC_REALIZES_NOTHING`, Info). + +`validate_project` with a `project_id` scope finds an EventModel only +through this chain: the model's namespace, then the BoundedContext in that +namespace, its first `realizes` Subdomain, that Subdomain's Domain, and the +Domain's `project`. Without the chain, the project report is empty. + +## 2. Add the stream + +| File | Kind | Fields the rules check | +| --- | --- | --- | +| `swimlane.yaml` | `Swimlane` | `streamId` | + +Each `{placeholder}` in `streamId` must be a plain name +(`STREAM_ID_MALFORMED_PLACEHOLDER`, Error) that matches a field on this +lane's events or commands (`STREAM_ID_UNRESOLVED_PLACEHOLDER`, Warning). +`wishlist-{shopper_id}` uses the `shopper_id` field that the next step +adds. + +## 3. Add the event and the command + +| File | Kind | Fields the rules check | +| --- | --- | --- | +| `event.yaml` | `Event` | `swimlane`, `schema` | +| `command.yaml` | `Command` | `swimlane`, `schema` | + +- Every event needs a `swimlane` that exists (`EVENT_NO_SWIMLANE`, Error). +- Declare fields in `schema`, an Any with `'@type': + type.googleapis.com/trogonatlas.eventmodel.v1alpha1.Schema` + (`SCHEMA_TYPE_URL_EMPTY`, Error). +- Every command field must appear on an emitted event or in the lane's + `state` (`COMMAND_FIELD_NEVER_RECORDED`, Warning). +- A command slice's emitted events must sit on the command's swimlane + (`COMMAND_MULTI_STREAM_WRITE`, Error). + +## 4. Add who issues the command + +| File | Kind | Fields the rules check | +| --- | --- | --- | +| `persona.yaml` | `Persona` | none | +| `ui.yaml` | `Ui` | none | + +Every command needs a trigger: a CommandSlice with both `persona` and `ui`, +or an AutomationSlice that fires it (`COMMAND_HAS_NO_TRIGGER`, Error). A +human trigger is smaller, because an automation also needs a Processor and +an AutomationSlice. Every UI needs a persona beside it on a slice or a +storyboard entry (`UI_NO_PERSONA`, Error). + +## 5. Add the command slice + +| File | Kind | Fields the rules check | +| --- | --- | --- | +| `command-slice.yaml` | `CommandSlice` | `command`, `emittedEvents`, `persona`, `ui`, `scenarios` | + +- A command without a slice that emits events warns + (`COMMAND_NO_EMITTED_EVENTS`). +- Every field on an emitted event must come from the command, the lane, or + a `DerivedFieldAnnotation` (`EVENT_FIELD_UNSOURCED`, Error). The command + marks `added_at` as derived, since the system sets it. +- A slice with no scenario is reported (`SLICE_NO_SCENARIOS`, Info). Each + scenario needs an outcome (`SCENARIO_MISSING_THEN`, Error). + +## 6. Add the read model and its slice + +| File | Kind | Fields the rules check | +| --- | --- | --- | +| `read-model.yaml` | `ReadModel` | `sourceEvents`, `schema`, `key` | +| `read-model-slice.yaml` | `ReadModelSlice` | `readModel`, `sourceEvents`, `projectionRole` | + +- Every event must be sourced by a read model (`EVENT_UNOBSERVED`, + Warning). +- Every read model field must come from a source event or its lane's + `state` (`READ_MODEL_FIELD_UNSOURCED`, Error). The read model must also + carry every field of a projected event + (`RM_SOURCE_EVENT_FIELD_NOT_PROJECTED`, Error). +- Every `key` name must be a field (`RM_KEY_NOT_A_FIELD`, Error), and + every projected event must carry all key fields + (`RM_SLICE_KEY_UNSOURCED`, Error). +- A slice may only project events its read model declares + (`RM_SLICE_EVENT_NOT_DECLARED`, Error). Each declared event needs a slice + in a storyboard that projects it (`RM_EVENT_NOT_PROJECTED`, Error). +- A read model serves one consumer. Two automation slices that observe + one read model are an error (`READ_MODEL_MULTIPLE_AUTOMATIONS`). + +## 7. Add the storyboard + +| File | Kind | Fields the rules check | +| --- | --- | --- | +| `storyboard.yaml` | `Storyboard` | `entry`, `outcome`, `slices` | + +- Declare the `entry`: the read model observed, and who observes it + (`human` with persona and UI, or `automation` with a processor) + (`STORYBOARD_MISSING_ENTRY`, Warning). +- Declare the terminal events in `outcome` (`STORYBOARD_MISSING_OUTCOME`, + Warning). +- Put every slice in a storyboard (`SLICE_NOT_IN_STORYBOARD`, Warning). + Place each projection after the slice that emits its event and before + the next command or automation slice (`PROJECTION_BEFORE_EMITTER`, + `PROJECTION_NOT_ADJACENT`, Errors). + +## 8. Add the event model + +| File | Kind | Fields the rules check | +| --- | --- | --- | +| `event-model.yaml` | `EventModel` | `members` | + +The validator only checks members of the EventModel. List every entity +from steps 1 to 7 except the Project, with its `kind`. Every member must +exist (`MEMBER_NOT_FOUND`, Error), and every slice in a storyboard must be +a member (`STORYBOARD_SLICE_NOT_MEMBER`, Warning). Every event, command, +read model, persona, and UI must be wired to a slice or the storyboard +(`ORPHAN_NOT_FLAGGED`, Warning). + +## Apply and validate + +Apply the namespace: + +```sh +trogon-atlas apply -f acme-wishlist/ +``` + +Run model validation through the `validate_event_model` MCP tool: + +```json +{ "namespace": "acme-wishlist", "slug": "acme-wishlist-model", "version": 1 } +``` + +Then run `validate_project` with `project_id` set to the Project. The +finished model reports no errors or warnings, and two Info findings: +`CROSS_MODEL_REF` for the Domain's reference to the Project, which is not a +member, and `SLICE_NO_SCENARIOS` for the read model slice. + +As the model grows, run `validate_event_model` after every batch and fix +all errors. To find what a code means, call `list_validation_rules`; to +repair one, follow +[repair a validation finding](repair-validation-findings.md). diff --git a/examples/manifests/README.md b/examples/manifests/README.md index 7ce2e915..a4304cfb 100644 --- a/examples/manifests/README.md +++ b/examples/manifests/README.md @@ -1,14 +1,15 @@ # Example manifests -Complete event models for a fictional retailer, Acme, written as declarative +Event models for a fictional retailer, Acme, written as declarative manifests. Each directory is one namespace, with one multi-document YAML file per entity kind. -| Namespace | Models | -|-------------------------------|------------------------------------------------------------------| -| `acme-storefront` | Cart, checkout, payment capture, and carrier fulfillment | -| `acme-reviews` | Review eligibility and review submission for delivered orders | -| `acme-customer-data-deletion` | A customer data deletion request, across every downstream vendor | +| Namespace | Models | +|-------------------------------|-------------------------------------------------------------------| +| `acme-storefront` | Cart, checkout, payment capture, and carrier fulfillment | +| `acme-reviews` | Review eligibility and review submission for delivered orders | +| `acme-customer-data-deletion` | A customer data deletion request, across every downstream vendor | +| `acme-wishlist` | The smallest valid model: a shopper saves a product to a wishlist | Apply a namespace in one atomic batch, check it without writing, or detect drift between the files and a running server: @@ -20,11 +21,15 @@ trogon-atlas diff -f examples/manifests/acme-storefront/ # exit 1 on ``` `acme-reviews` references entities in `acme-storefront`, so apply them -together or apply the storefront first. +together or apply the storefront first. To build `acme-wishlist` step by +step, follow +[start an event model from scratch](../../docs/how-to/start-a-model-from-scratch.md). Every file here is checked against the generated manifest schema by the `trogon-atlas-client` tests, and the server's account deletion fixture test -applies `acme-customer-data-deletion` end to end. +applies `acme-customer-data-deletion` end to end. The server's +`example_manifests_validate` test checks that each namespace validates +with no errors or warnings. ## Legacy `fields` lists diff --git a/examples/manifests/acme-wishlist/bounded-context.yaml b/examples/manifests/acme-wishlist/bounded-context.yaml new file mode 100644 index 00000000..26d13397 --- /dev/null +++ b/examples/manifests/acme-wishlist/bounded-context.yaml @@ -0,0 +1,13 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: BoundedContext +metadata: + name: acme-wishlist + namespace: acme-wishlist +spec: + doc: Owns each shopper's wishlist. + realizes: + - id: + namespace: acme-wishlist + slug: wishlists + version: '1' + title: Acme Wishlist diff --git a/examples/manifests/acme-wishlist/command-slice.yaml b/examples/manifests/acme-wishlist/command-slice.yaml new file mode 100644 index 00000000..1c0851c3 --- /dev/null +++ b/examples/manifests/acme-wishlist/command-slice.yaml @@ -0,0 +1,49 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: CommandSlice +metadata: + name: add-item-to-wishlist-slice + namespace: acme-wishlist +spec: + command: + command: + id: + namespace: acme-wishlist + slug: add-item-to-wishlist + version: '1' + doc: The shopper saves a product from the wishlist page. + emittedEvents: + - event: + id: + namespace: acme-wishlist + slug: wishlist-item.added + version: '1' + persona: + persona: + id: + namespace: acme-wishlist + slug: shopper + version: '1' + scenarios: + - doc: The shopper saves a product. + emit: + events: + - event: + id: + namespace: acme-wishlist + slug: wishlist-item.added + version: '1' + id: added + title: Item added + when: + command: + id: + namespace: acme-wishlist + slug: add-item-to-wishlist + version: '1' + title: Add item to wishlist + ui: + ui: + id: + namespace: acme-wishlist + slug: wishlist-page + version: '1' diff --git a/examples/manifests/acme-wishlist/command.yaml b/examples/manifests/acme-wishlist/command.yaml new file mode 100644 index 00000000..d0eaeae2 --- /dev/null +++ b/examples/manifests/acme-wishlist/command.yaml @@ -0,0 +1,29 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: Command +metadata: + name: add-item-to-wishlist + namespace: acme-wishlist +spec: + doc: A shopper saves a product to their wishlist. + schema: + '@type': type.googleapis.com/trogonatlas.eventmodel.v1alpha1.Schema + fields: + - name: shopper_id + type: + uuid: {} + - name: product_id + type: + uuid: {} + - metadata: + - '@type': type.googleapis.com/trogonatlas.eventmodel.v1alpha1.DerivedFieldAnnotation + doc: now() when the item is added + kind: KIND_TIMESTAMP + name: added_at + type: + timestamp: {} + swimlane: + id: + namespace: acme-wishlist + slug: wishlist + version: '1' + title: Add item to wishlist diff --git a/examples/manifests/acme-wishlist/domain.yaml b/examples/manifests/acme-wishlist/domain.yaml new file mode 100644 index 00000000..a039b55a --- /dev/null +++ b/examples/manifests/acme-wishlist/domain.yaml @@ -0,0 +1,13 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: Domain +metadata: + name: shopping + namespace: acme-wishlist +spec: + doc: Helping shoppers find and keep track of products they want. + project: + id: + namespace: acme-wishlist + slug: acme-wishlist + version: '1' + title: Shopping diff --git a/examples/manifests/acme-wishlist/event-model.yaml b/examples/manifests/acme-wishlist/event-model.yaml new file mode 100644 index 00000000..d87305ea --- /dev/null +++ b/examples/manifests/acme-wishlist/event-model.yaml @@ -0,0 +1,69 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: EventModel +metadata: + name: acme-wishlist-model + namespace: acme-wishlist +spec: + doc: A shopper saves products to a wishlist. + members: + - id: + namespace: acme-wishlist + slug: shopping + version: '1' + kind: ENTITY_KIND_DOMAIN + - id: + namespace: acme-wishlist + slug: wishlists + version: '1' + kind: ENTITY_KIND_SUBDOMAIN + - id: + namespace: acme-wishlist + slug: acme-wishlist + version: '1' + kind: ENTITY_KIND_BOUNDED_CONTEXT + - id: + namespace: acme-wishlist + slug: shopper + version: '1' + kind: ENTITY_KIND_PERSONA + - id: + namespace: acme-wishlist + slug: wishlist + version: '1' + kind: ENTITY_KIND_SWIMLANE + - id: + namespace: acme-wishlist + slug: wishlist-item.added + version: '1' + kind: ENTITY_KIND_EVENT + - id: + namespace: acme-wishlist + slug: add-item-to-wishlist + version: '1' + kind: ENTITY_KIND_COMMAND + - id: + namespace: acme-wishlist + slug: wishlist-items + version: '1' + kind: ENTITY_KIND_READ_MODEL + - id: + namespace: acme-wishlist + slug: wishlist-page + version: '1' + kind: ENTITY_KIND_UI + - id: + namespace: acme-wishlist + slug: add-item-to-wishlist-slice + version: '1' + kind: ENTITY_KIND_COMMAND_SLICE + - id: + namespace: acme-wishlist + slug: wishlist-items-upsert-rms + version: '1' + kind: ENTITY_KIND_READ_MODEL_SLICE + - id: + namespace: acme-wishlist + slug: add-to-wishlist + version: '1' + kind: ENTITY_KIND_STORYBOARD + title: Acme Wishlist diff --git a/examples/manifests/acme-wishlist/event.yaml b/examples/manifests/acme-wishlist/event.yaml new file mode 100644 index 00000000..d4631a32 --- /dev/null +++ b/examples/manifests/acme-wishlist/event.yaml @@ -0,0 +1,25 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: Event +metadata: + name: wishlist-item.added + namespace: acme-wishlist +spec: + doc: A shopper added a product to their wishlist. + schema: + '@type': type.googleapis.com/trogonatlas.eventmodel.v1alpha1.Schema + fields: + - name: shopper_id + type: + uuid: {} + - name: product_id + type: + uuid: {} + - name: added_at + type: + timestamp: {} + swimlane: + id: + namespace: acme-wishlist + slug: wishlist + version: '1' + title: Wishlist item added diff --git a/examples/manifests/acme-wishlist/persona.yaml b/examples/manifests/acme-wishlist/persona.yaml new file mode 100644 index 00000000..551b7f2b --- /dev/null +++ b/examples/manifests/acme-wishlist/persona.yaml @@ -0,0 +1,9 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: Persona +metadata: + name: shopper + namespace: acme-wishlist +spec: + doc: A signed-in customer browsing the store. + role: customer + title: Shopper diff --git a/examples/manifests/acme-wishlist/project.yaml b/examples/manifests/acme-wishlist/project.yaml new file mode 100644 index 00000000..a61f855e --- /dev/null +++ b/examples/manifests/acme-wishlist/project.yaml @@ -0,0 +1,8 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: Project +metadata: + name: acme-wishlist + namespace: acme-wishlist +spec: + doc: The smallest valid Acme event model, a shopper's wishlist. + title: Acme Wishlist diff --git a/examples/manifests/acme-wishlist/read-model-slice.yaml b/examples/manifests/acme-wishlist/read-model-slice.yaml new file mode 100644 index 00000000..9dd14d5c --- /dev/null +++ b/examples/manifests/acme-wishlist/read-model-slice.yaml @@ -0,0 +1,21 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: ReadModelSlice +metadata: + name: wishlist-items-upsert-rms + namespace: acme-wishlist +spec: + doc: An added item appears on the wishlist. + projectionRole: PROJECTION_ROLE_UPSERT + readModel: + readModel: + id: + namespace: acme-wishlist + slug: wishlist-items + version: '1' + sourceEvents: + - event: + id: + namespace: acme-wishlist + slug: wishlist-item.added + version: '1' + title: 'Projection: wishlist items' diff --git a/examples/manifests/acme-wishlist/read-model.yaml b/examples/manifests/acme-wishlist/read-model.yaml new file mode 100644 index 00000000..22d985e7 --- /dev/null +++ b/examples/manifests/acme-wishlist/read-model.yaml @@ -0,0 +1,28 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: ReadModel +metadata: + name: wishlist-items + namespace: acme-wishlist +spec: + doc: The products on one shopper's wishlist, for the wishlist page. + key: + - shopper_id + - product_id + schema: + '@type': type.googleapis.com/trogonatlas.eventmodel.v1alpha1.Schema + fields: + - name: shopper_id + type: + uuid: {} + - name: product_id + type: + uuid: {} + - name: added_at + type: + timestamp: {} + sourceEvents: + - id: + namespace: acme-wishlist + slug: wishlist-item.added + version: '1' + title: Wishlist items diff --git a/examples/manifests/acme-wishlist/storyboard.yaml b/examples/manifests/acme-wishlist/storyboard.yaml new file mode 100644 index 00000000..e2f192d7 --- /dev/null +++ b/examples/manifests/acme-wishlist/storyboard.yaml @@ -0,0 +1,44 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: Storyboard +metadata: + name: add-to-wishlist + namespace: acme-wishlist +spec: + doc: The shopper saves a product and sees it on the wishlist. + entry: + human: + persona: + persona: + id: + namespace: acme-wishlist + slug: shopper + version: '1' + ui: + ui: + id: + namespace: acme-wishlist + slug: wishlist-page + version: '1' + readModel: + readModel: + id: + namespace: acme-wishlist + slug: wishlist-items + version: '1' + outcome: + events: + - event: + id: + namespace: acme-wishlist + slug: wishlist-item.added + version: '1' + slices: + - id: + namespace: acme-wishlist + slug: add-item-to-wishlist-slice + version: '1' + - id: + namespace: acme-wishlist + slug: wishlist-items-upsert-rms + version: '1' + title: Add to wishlist diff --git a/examples/manifests/acme-wishlist/subdomain.yaml b/examples/manifests/acme-wishlist/subdomain.yaml new file mode 100644 index 00000000..1c749e21 --- /dev/null +++ b/examples/manifests/acme-wishlist/subdomain.yaml @@ -0,0 +1,14 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: Subdomain +metadata: + name: wishlists + namespace: acme-wishlist +spec: + classification: CLASSIFICATION_CORE + doc: Saving products a shopper wants to buy later. + domain: + id: + namespace: acme-wishlist + slug: shopping + version: '1' + title: Wishlists diff --git a/examples/manifests/acme-wishlist/swimlane.yaml b/examples/manifests/acme-wishlist/swimlane.yaml new file mode 100644 index 00000000..eebd2c2e --- /dev/null +++ b/examples/manifests/acme-wishlist/swimlane.yaml @@ -0,0 +1,10 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: Swimlane +metadata: + name: wishlist + namespace: acme-wishlist +spec: + doc: One shopper's wishlist. + order: 1 + streamId: wishlist-{shopper_id} + title: Wishlist diff --git a/examples/manifests/acme-wishlist/ui.yaml b/examples/manifests/acme-wishlist/ui.yaml new file mode 100644 index 00000000..9f51f006 --- /dev/null +++ b/examples/manifests/acme-wishlist/ui.yaml @@ -0,0 +1,8 @@ +apiVersion: eventmodel.atlas.trogonstack.com/v1alpha1 +kind: Ui +metadata: + name: wishlist-page + namespace: acme-wishlist +spec: + doc: 'Shopper-facing: lists saved products and takes a new one.' + title: Wishlist page diff --git a/rsworkspace/crates/trogon-atlas-server/tests/example_manifests_validate.rs b/rsworkspace/crates/trogon-atlas-server/tests/example_manifests_validate.rs index a4732371..8d319490 100644 --- a/rsworkspace/crates/trogon-atlas-server/tests/example_manifests_validate.rs +++ b/rsworkspace/crates/trogon-atlas-server/tests/example_manifests_validate.rs @@ -7,13 +7,14 @@ use trogon_atlas_proto as pb; use trogon_atlas_proto::event_model_service_server::EventModelService; use trogon_atlas_server::{metrics::ServerMetrics, service::EventModelServiceImpl}; -const NAMESPACES: [(&str, &str); 3] = [ +const NAMESPACES: [(&str, &str); 4] = [ ("acme-storefront", "acme-storefront-model"), ("acme-reviews", "acme-reviews-model"), ( "acme-customer-data-deletion", "acme-customer-data-deletion-pipeline", ), + ("acme-wishlist", "acme-wishlist-model"), ]; fn manifests_dir() -> PathBuf {