diff --git a/openwiki/.last-update.json b/openwiki/.last-update.json index 900245d..40005a0 100644 --- a/openwiki/.last-update.json +++ b/openwiki/.last-update.json @@ -1,7 +1,7 @@ { - "updatedAt": "2026-09-05T12:23:10.048Z", + "updatedAt": "2026-09-06T12:40:17.952Z", "command": "update", - "gitHead": "d39ecfb983cf472121fc2641d283a8da7b288073", + "gitHead": "aba939a097febf62559dbbff687cfdfe25a4fd93", "model": "google/gemini-3.7-flash", "status": "complete", "language": "en" diff --git a/openwiki/architecture/adapters-and-runtime.md b/openwiki/architecture/adapters-and-runtime.md index 925db64..c811256 100644 --- a/openwiki/architecture/adapters-and-runtime.md +++ b/openwiki/architecture/adapters-and-runtime.md @@ -28,7 +28,10 @@ Adapters implement the application-layer repository ports against external syste ### Structured model: OpenRouter -The provider-neutral structured-model port is implemented with LangChain's `ChatOpenRouter`. The operator selects the model through `STORYRAIL_EVIDENCE_PREPARATION_MODEL`; final prepared documents are validated by StoryRail before an immutable attempt is persisted. +The provider-neutral structured-model port is implemented with LangChain's `ChatOpenRouter` in `src/adapters/model/openrouter-structured-model.ts`: +- Accepts `resolveApiKey`, `model`, optional custom `baseUrl` (configured globally via `STORYRAIL_OPENROUTER_BASE_URL` in `src/runtime/openrouter-configuration.ts`), and timeout parameters. +- Normalizes system prompts, input data, and strict Zod output schemas (`name: "storyrail_structured_response", strict: true`). +- Validates model output against strict schema and failure mappers before returning structured results. ### Credential storage @@ -70,6 +73,8 @@ All PostgreSQL adapters share a defensive pattern: they serialize the domain obj | `postgres-site-settings-persistence.ts` | `storyrail.site_settings` | `SiteSettingsRepository` | | `postgres-policy-run-repository.ts` | `storyrail.policy_runs` | `PolicyRunRepository` | | `postgres-story-delivery-repository.ts` | `storyrail.story_deliveries` | `StoryDeliveryRepository` | +| `postgres-legacy-delivery-mapping-resolution-repository.ts` | `storyrail.legacy_delivery_mapping_resolutions` | `LegacyDeliveryMappingResolutionRepository` | +| `postgres-story-delivery-reconciliation-repository.ts` | `storyrail.story_delivery_reconciliations` | `StoryDeliveryReconciliationRepository` | ### Model catalog: OpenRouter diff --git a/openwiki/architecture/application-workflows.md b/openwiki/architecture/application-workflows.md index a145dcd..43b72af 100644 --- a/openwiki/architecture/application-workflows.md +++ b/openwiki/architecture/application-workflows.md @@ -157,12 +157,33 @@ Rejection is terminal and does not contact a model. It preserves all existing wo 3. Finds the latest Article Revision (`STORY_HAS_NO_ARTICLE`). 4. Resolves the destination directory (`DeliveryDestinationDirectory`) to construct the destination instance with site settings and credentials (including its `instanceId`). 5. Derives the slug via `storyDeliverySlug(revision.headline)`. -6. Checks previous deliveries for that Story and destination instance (`findLatestSucceeded({ storyId, destinationInstanceId })`). - - If no prior delivery for the current instance is found, checks for unbound legacy deliveries (`findLatestLegacySucceeded({ storyId, destination })`). If a legacy mapping exists, it fails closed with `DESTINATION_MAPPING_REQUIRES_REVIEW` so operators can review before writing to a new instance. - - If a prior delivery for the instance exists, uses its `remoteId` to decide whether this is a `create` (first delivery, `remoteId: null`) or `update` (patching an existing remote page using its prior `remoteId`). -7. **Durability first**: Records the delivery row as `outcome: "running"` with `StoryDeliveryRepository.append` before making the external HTTP call. -8. Invokes `destination.deliver(...)`. -9. Updates the delivery record with `StoryDeliveryRepository.complete` to `succeeded` with `remoteId` (parsed from the provider response) or `failed` with failure details. Failed deliveries are never retried silently. +6. **Reconciliation check**: Checks for unresolved deliveries for that Story and destination instance (`findLatestUnresolved({ storyId, destinationInstanceId })`). If an unresolved delivery (`running` or `unknown`) exists, it verifies whether a matching `StoryDeliveryReconciliation` exists. If unreconciled, the workflow fails closed with `DESTINATION_RECONCILIATION_REQUIRED` to avoid duplicate page creation or overwriting the wrong post. +7. **Legacy mapping check**: Checks previous deliveries for that Story and destination instance (`findLatestSucceeded({ storyId, destinationInstanceId })`). + - If no prior delivery for the current instance is found, checks for unbound legacy deliveries (`findLatestLegacySucceeded({ storyId, destination })`). If a legacy mapping exists, it checks for an existing `LegacyDeliveryMappingResolution`. If unreviewed, it fails closed with `DESTINATION_MAPPING_REQUIRES_REVIEW` so operators can review before writing to a new instance. + - Determines the effective `remoteId`: prior successful delivery ID, confirmed legacy resolution ID, or reconciled ID. If present, the operation is `update`; otherwise `create` (`remoteId: null`). +8. **Durability first**: Records the delivery row as `outcome: "running"` with `StoryDeliveryRepository.append` before making the external HTTP call. +9. Invokes `destination.deliver(...)`. +10. Updates the delivery record with `StoryDeliveryRepository.complete` to: + - `succeeded` with `remoteId` and result when the remote system confirms acceptance. + - `failed` with failure details when the remote system explicitly rejects the request. + - `unknown` with `uncertainty` details when the connection drops or the remote body is unparseable, immediately gating future deliveries until reconciled. + +## Legacy delivery mapping resolution workflow + +`src/application/story-deliveries/resolve-legacy-delivery-mapping.ts` — `createResolveLegacyDeliveryMapping` records an operator's decision (`confirm` or `dismiss`) on an ambiguous legacy delivery: +1. Validates that the Story exists and resolves the current configured destination. +2. Finds the succeeded legacy delivery row, ensuring it is unbound (`destinationInstanceId: null`) and has a valid `remoteId`. +3. Verifies that the legacy mapping is the latest for that destination (`LEGACY_DELIVERY_MAPPING_STALE`). +4. Constructs and persists a `LegacyDeliveryMappingResolution` record via `LegacyDeliveryMappingResolutionRepository.append`. + +## Story delivery reconciliation workflow + +`src/application/story-deliveries/reconcile-story-delivery.ts` — `createReconcileStoryDelivery` records an operator's decision (`delivered` or `not_delivered`) on an ambiguous or uncertain delivery outcome: +1. Validates that the Story exists and resolves the current configured destination. +2. Finds the unresolved delivery row (`findUnresolvedById` and `findLatestUnresolved`), ensuring it matches the current destination and instance ID (`STORY_DELIVERY_RECONCILIATION_NOT_FOUND`). +3. Verifies that no reconciliation has already been recorded (`STORY_DELIVERY_ALREADY_RECONCILED`). +4. Validates decision consistency: `delivered` on an `update` operation must retain the exact original `remoteId`. +5. Constructs and persists a `StoryDeliveryReconciliation` record via `StoryDeliveryReconciliationRepository.append`. ## Model catalog workflow @@ -210,6 +231,8 @@ Persistence contracts are expressed as interfaces in the application layer and i | `ReviewDecisionPersistence` | `src/application/review-decisions/review-decision-persistence.ts` | `src/adapters/review-persistence/postgres-review-decision-persistence.ts` | | `StoryRejectionPersistence` | `src/application/story-rejections/story-rejection-persistence.ts` | `src/adapters/story-rejection-persistence/postgres-story-rejection-persistence.ts` | | `StoryDeliveryRepository` | `src/application/story-deliveries/story-delivery-repository.ts` | `src/adapters/story-delivery-persistence/postgres-story-delivery-repository.ts` | +| `LegacyDeliveryMappingResolutionRepository` | `src/application/story-deliveries/legacy-delivery-mapping-resolution-repository.ts` | `src/adapters/legacy-delivery-mapping-resolution-persistence/postgres-legacy-delivery-mapping-resolution-repository.ts` | +| `StoryDeliveryReconciliationRepository` | `src/application/story-deliveries/story-delivery-reconciliation-repository.ts` | `src/adapters/story-delivery-reconciliation-persistence/postgres-story-delivery-reconciliation-repository.ts` | | `SiteSettingsRepository` | `src/application/site-settings/site-settings-repository.ts` | `src/adapters/site-settings-persistence/postgres-site-settings-repository.ts` | The `*.contract.ts` files alongside several ports (`source-repositories.contract.ts`, `story-inspection-repository.contract.ts`, `agent-run-repository.contract.ts`, etc.) are shared harnesses that verify any repository implementation satisfies the same behavior contract. The PostgreSQL adapter tests run these contracts against real PostgreSQL in the integration suite. diff --git a/openwiki/architecture/database-schema.md b/openwiki/architecture/database-schema.md index 744682e..d6a69d7 100644 --- a/openwiki/architecture/database-schema.md +++ b/openwiki/architecture/database-schema.md @@ -309,6 +309,33 @@ It alters the `storyrail.agent_tool_calls` table to: - `story_deliveries_legacy_story_destination_idx` on `(story_id, destination, started_at DESC) WHERE destination_instance_id IS NULL` - Updates trigger `storyrail.story_delivery_completes_once()` to ensure `destination_instance_id` cannot be changed during completion. +## Migration 0077 — legacy delivery mapping resolutions + +`database/migrations/0077-legacy-delivery-mapping-resolutions.sql` captures operator resolutions to ambiguous legacy deliveries as immutable audit facts: +- Creates `storyrail.legacy_delivery_mapping_resolutions` table with columns `resolution_id`, `insertion_position` (identity), `story_id`, `legacy_delivery_id`, `destination`, `destination_instance_id`, `remote_id`, `decision` (`confirm` | `dismiss`), `decided_at`, and `payload` JSONB. +- Enforces payload shape check validating exact key schema and operator actor attribution (`decidedBy.type = 'operator'`). +- Adds partial index `legacy_delivery_mapping_resolutions_latest_idx` on `(story_id, legacy_delivery_id, destination_instance_id, insertion_position DESC)`. +- Adds trigger function `storyrail.legacy_delivery_mapping_resolution_is_valid()` to verify foreign key and prove that the referenced legacy delivery is an immutable, succeeded delivery with `destination_instance_id IS NULL` and matching `remote_id`. +- Adds trigger `storyrail.legacy_delivery_mapping_resolutions_are_immutable()` ensuring resolutions cannot be updated or deleted. + +## Migration 0078 — ambiguous delivery reconciliation + +`database/migrations/0078-ambiguous-delivery-reconciliation.sql` enables handling and operator reconciliation of uncertain delivery outcomes: +- Alters `storyrail.story_deliveries` outcome check to allow `outcome IN ('running', 'succeeded', 'failed', 'unknown')`. +- Updates `story_deliveries_payload_shape_check` to validate `unknown` outcomes containing an `uncertainty` object with failure codes `DESTINATION_REQUEST_OUTCOME_UNKNOWN` or `DESTINATION_ACCEPTED_RESPONSE_UNVERIFIABLE`. +- Enforces `story_deliveries_unknown_remote_id_check`: `create` operations have `remote_id IS NULL`, while `update` operations retain their non-null `remote_id`. +- Adds partial index `story_deliveries_unresolved_story_instance_idx` on `(story_id, destination_instance_id, started_at DESC, delivery_id DESC) WHERE outcome IN ('running', 'unknown') AND destination_instance_id IS NOT NULL`. +- Creates `storyrail.story_delivery_reconciliations` table (`reconciliation_id`, `insertion_position`, `story_id`, `delivery_id`, `destination`, `destination_instance_id`, `operation`, `slug`, `decision`, `remote_id`, `decided_at`, `payload`). +- Enforces trigger `storyrail.story_delivery_reconciliation_is_valid()` verifying that the snapshot matches the exact unresolved delivery and validates decision consistency (`delivered` requires `remoteId`, `not_delivered` requires `remoteId IS NULL`). +- Adds trigger `storyrail.story_delivery_reconciliations_are_immutable()` ensuring reconciliations cannot be updated or deleted. + +## Migration 0079 — agent run recovery + +`database/migrations/0079-agent-run-recovery.sql` adds recovery infrastructure for manual agent runs that were interrupted by process termination: +- Adds `recorded_at timestamptz NOT NULL DEFAULT CURRENT_TIMESTAMP` to `storyrail.agent_runs`. Existing running rows receive the migration instant, giving them a fresh recovery window. +- Adds partial index `agent_runs_stale_running_idx` on `(recorded_at ASC, append_position ASC) WHERE outcome = 'running'`. +- Updates trigger `storyrail.agent_run_completion_is_one_way()` ensuring that `recorded_at` cannot be changed during completion and that runs only complete to terminal outcomes. + ## Integration test lifecycle -The PostgreSQL integration tests (`src/adapters/source-persistence/postgres-source-repositories.test.ts` and the Story/attachment/assignment/run/article/review/writer-revision/story-rejection suites) connect via `STORYRAIL_TEST_DATABASE_URL`, verify the database name is exactly `storyrail_test`, drop and recreate the `storyrail` schema, apply migrations `0012`, `0017`, `0018`, `0024`, `0025`, `0027`, `0028`, `0030`, `0031`, `0038`, `0041`, `0049`, `0053`, `0054`, `0055`, `0056`, `0057`, `0058`, `0059`, `0060`, `0061`, `0062`, `0063`, `0064`, `0065`, `0066`, `0067`, `0068`, `0069`, `0070`, `0071`, `0072`, `0073`, `0074`, `0075`, and `0076` in order, and truncate the editorial tables (plus delete non-built-in Agent Profiles) between cases. The suite never creates or drops a database. \ No newline at end of file +The PostgreSQL integration tests (`src/adapters/source-persistence/postgres-source-repositories.test.ts` and the Story/attachment/assignment/run/article/review/writer-revision/story-rejection suites) connect via `STORYRAIL_TEST_DATABASE_URL`, verify the database name is exactly `storyrail_test`, drop and recreate the `storyrail` schema, apply migrations `0012`, `0017`, `0018`, `0024`, `0025`, `0027`, `0028`, `0030`, `0031`, `0038`, `0041`, `0049`, `0053`, `0054`, `0055`, `0056`, `0057`, `0058`, `0059`, `0060`, `0061`, `0062`, `0063`, `0064`, `0065`, `0066`, `0067`, `0068`, `0069`, `0070`, `0071`, `0072`, `0073`, `0074`, `0075`, `0076`, `0077`, `0078`, and `0079` in order, and truncate the editorial tables (plus delete non-built-in Agent Profiles) between cases. The suite never creates or drops a database. \ No newline at end of file diff --git a/openwiki/architecture/domain-model.md b/openwiki/architecture/domain-model.md index 691106a..539f995 100644 --- a/openwiki/architecture/domain-model.md +++ b/openwiki/architecture/domain-model.md @@ -176,8 +176,9 @@ Migration `0063` creates the `storyrail.newsroom_standards` table and seeds an i - `verifyArticleGrounding` inspects cited article blocks against raw and prepared source evidence documents. -- Evidence and quote comparison performs markdown-neutral text normalization (`normalizeEvidenceForComparison`): Markdown syntax (bold `**`, italics `*`, code backticks, markdown links `[text](url)` retaining link text, and images `![alt](url)` stripped completely) is normalized away on both sides before quotation matching. This prevents a Writer from being refused for quoting text from a markdown-rendered source. +- Evidence and quote comparison performs markdown-neutral text normalization (`visibleInlineMarkdown` / `comparable`): Recognized balanced inline Markdown syntax (bold `**`, italics `*`, underscores `__`/`_`, markdown links `[text](url)` retaining link text, and images `![alt](url)` stripped completely) and presentation differences (smart quotes, line breaks, over-escaped characters) are normalized away before quotation matching. Intraword punctuation, unmatched markers, and code blocks (`...`) are preserved as literal text. - Model failure codes: When an ungrounded claim or quote is found, the run is rejected with `MODEL_OUTPUT_UNGROUNDED` or `MODEL_CORRECTION_OUT_OF_SCOPE` accompanied by specific grounding findings (`CITATION_QUOTE_UNSUPPORTED`, etc.). Both failure codes are permitted to carry grounding findings in domain validation and schema. +- Grounding metrics (`measureArticleGrounding`): Computes `groundedShare` (share of cited prose characters over total prose characters) and `derivedShare` (share of 8-word sequences appearing verbatim in source evidence, identifying restatements vs. original synthesis). ## Assignments @@ -246,16 +247,34 @@ Revision 1 is created by the [Writer draft workflow](application-workflows.md#wr `story-delivery-types.ts` and `story-delivery.ts` model the delivery of a published Story's Article Revision to an external destination (e.g., StudioCMS or WordPress). Delivery is an outbound write and is tracked as an explicit audit record: -- A delivery record describes what was sent (`StoryDeliveryRequest`: `operation` ("create" | "update"), `slug`, `draft` boolean, `bodyCharacters`), the destination software name (`destination`), the destination installation identity (`destinationInstanceId`), and what came back (`outcome`: "running" | "succeeded" | "failed"), never storing the Article body itself. +- A delivery record describes what was sent (`StoryDeliveryRequest`: `operation` ("create" | "update"), `slug`, `draft` boolean, `bodyCharacters`), the destination software name (`destination`), the destination installation identity (`destinationInstanceId`), and what came back (`outcome`: "running" | "succeeded" | "failed" | "unknown"), never storing the Article body itself. - Destination instance identity: `siteDestinationInstanceId(destinationSettings)` deterministically identifies the specific remote installation as `${kind}:${baseUrl.replace(/\/+$/, "")}` (typed as `DestinationInstanceId`). This ensures a remote page identifier (`remoteId`) recovered from one installation is never mistakenly used to overwrite content if the site's configured URL or connector changes. - Legacy mapping protection: Deliveries migrated from earlier schemas carry `destinationInstanceId: null`. If a delivery is attempted without an existing confirmed mapping for the current installation, the presence of an unbound legacy mapping triggers a fail-closed `DESTINATION_MAPPING_REQUIRES_REVIEW` error to prevent unintended remote mutations until reviewed. +- Ambiguous outcomes & reconciliation: Network disconnects or unparseable 2xx responses leave an attempt in `outcome: "unknown"` with an `uncertainty` record (`DESTINATION_REQUEST_OUTCOME_UNKNOWN` or `DESTINATION_ACCEPTED_RESPONSE_UNVERIFIABLE`). Subsequent delivery attempts for that Story and destination instance fail closed with `DESTINATION_RECONCILIATION_REQUIRED` until an operator records a `StoryDeliveryReconciliation`. - Destinations support StudioCMS (`studiocms`) and WordPress (`wordpress`). - WordPress delivers block-by-block serialized Gutenberg markup (``, ``), with requested vs. assigned slug tracking when WordPress uniquifies colliding slugs. - `DELIVERY_FAILURE_CODES`: `DESTINATION_UNREACHABLE`, `DESTINATION_REJECTED`, `DESTINATION_UNAUTHORIZED`, `DESTINATION_RESPONSE_INVALID`. Note: HTTP 500 status responses are classified as `DESTINATION_REJECTED` (since the server answered and rejected), whereas timeouts (408) and rate limits (429) remain `DESTINATION_UNREACHABLE`. -- Durability pattern: Following `agent_tool_calls`, a delivery row is written as `running` before the HTTP request leaves the newsroom process, ensuring no write to the outside world occurs unrecorded. When the response arrives, the row is updated in place to `succeeded` or `failed`. +- Durability pattern: Following `agent_tool_calls`, a delivery row is written as `running` before the HTTP request leaves the newsroom process, ensuring no write to the outside world occurs unrecorded. When the response arrives, the row is updated in place to `succeeded`, `failed`, or `unknown`. - Slug generation: `storyDeliverySlug(headline)` derives a URL-safe slug (max 96 chars) deterministically from the headline. - Remote ID tracking: For `create`, destinations return the created page or post ID, which StoryRail stores in `remoteId`. Subsequent deliveries of newer revisions to the same destination instance update the existing remote resource via `PATCH`/`POST` using this `remoteId`. +## Legacy Delivery Mapping Resolutions + +`legacy-delivery-mapping-resolution-types.ts` and `legacy-delivery-mapping-resolution.ts` model the operator resolution of an ambiguous pre-instance legacy delivery mapping: +- `LegacyDeliveryMappingResolution`: snapshots `id`, `storyId`, `legacyDeliveryId`, `destination`, `destinationInstanceId`, `remoteId`, `decision` (`"confirm"` | `"dismiss"`), `decidedBy` (`OperatorActor`), and `decidedAt`. +- Confirmation (`confirm`): adopts the legacy `remoteId` for the current destination instance, allowing future deliveries to perform an `update` against that existing remote post. +- Dismissal (`dismiss`): disassociates the legacy `remoteId`, allowing subsequent deliveries to perform a fresh `create` without conflict. +- Validation (`recordLegacyDeliveryMappingResolution`): enforces non-empty identifiers, operator attribution, and valid decisions. + +## Story Delivery Reconciliations + +`story-delivery-reconciliation-types.ts` and `story-delivery-reconciliation.ts` model the operator resolution of an uncertain delivery attempt: +- `StoryDeliveryReconciliation`: snapshots `id`, `storyId`, `deliveryId`, `destination`, `destinationInstanceId`, `operation` (`"create"` | `"update"`), `slug`, `decision` (`"delivered"` | `"not_delivered"`), `remoteId` (`string | null`), `decidedBy` (`OperatorActor`), and `decidedAt`. +- Decision rules: + - `delivered`: requires a non-empty `remoteId` (for `update` operations, it must match the exact `remoteId` the update addressed). Subsequent deliveries will treat this remote post as established and update it. + - `not_delivered`: requires `remoteId: null`. If the uncertain attempt was a `create`, subsequent deliveries can execute a clean `create`. If the uncertain attempt was an `update`, the prior verified `remoteId` is retained. +- Validation (`recordStoryDeliveryReconciliation`): verifies matching operations, decision consistency, non-empty fields, and operator attribution. + ## Shared strict record schemas To eliminate schema drift between PostgreSQL persistence decoders and browser client readers, the domain defines strict Zod schemas and primitive parsers in `src/domain/editorial/`: @@ -275,4 +294,4 @@ To eliminate schema drift between PostgreSQL persistence decoders and browser cl ## Re-export barrel -`src/domain/editorial/index.ts` re-exports every module in the domain — Source intake/extraction/triage/preparation, Story creation and attachment, the state machine, Agent Profiles, Assignments, Assignment Proposals, AgentRuns, Director review, ReviewDecisions, Articles, Story Deliveries, Policy Runs, and strict record schemas — and `src/application/index.ts` re-exports the application layer's domain-facing types so callers import from a single barrel, including the writer-revisions, story-deliveries, and model-catalog modules. +`src/domain/editorial/index.ts` re-exports every module in the domain — Source intake/extraction/triage/preparation, Story creation and attachment, the state machine, Agent Profiles, Assignments, Assignment Proposals, AgentRuns, Director review, ReviewDecisions, Articles, Story Deliveries, Legacy Delivery Mapping Resolutions, Story Delivery Reconciliations, Policy Runs, and strict record schemas — and `src/application/index.ts` re-exports the application layer's domain-facing types so callers import from a single barrel, including the writer-revisions, story-deliveries, and model-catalog modules. diff --git a/openwiki/architecture/http-api.md b/openwiki/architecture/http-api.md index 697f9bf..fce17ec 100644 --- a/openwiki/architecture/http-api.md +++ b/openwiki/architecture/http-api.md @@ -323,13 +323,49 @@ Always returns 200 on success or 500 on internal failure. | Status | Condition | | ------ | --------- | -| 200 | Delivery attempt finished (`succeeded` or `failed` outcome recorded) | +| 200 | Delivery attempt finished (`succeeded`, `failed`, or `unknown` outcome recorded) | | 404 | `STORY_NOT_FOUND` | -| 409 | `STORY_NOT_PUBLISHED`, `STORY_HAS_NO_ARTICLE`, `DESTINATION_MAPPING_REQUIRES_REVIEW`, `STORY_DELIVERY_NOT_RECORDED` | +| 409 | `STORY_NOT_PUBLISHED`, `STORY_HAS_NO_ARTICLE`, `DESTINATION_MAPPING_REQUIRES_REVIEW`, `DESTINATION_RECONCILIATION_REQUIRED`, `STORY_DELIVERY_NOT_RECORDED` | | 422 | `DESTINATION_NOT_CONFIGURED`, `CREDENTIAL_UNAVAILABLE` | | 415/400| Media type / JSON / shape errors | | 500 | Internal error | +## POST /api/sites/[siteId]/stories/[storyId]/deliveries/legacy-mapping-resolution — resolve legacy destination mapping + +- Route: `src/app/api/sites/[siteId]/stories/[storyId]/deliveries/legacy-mapping-resolution/route.ts` +- Handler: `src/interfaces/http/resolve-legacy-delivery-mapping-handler.ts` +- Provider: `storyRuntimeProvider` +- Body: `{ "legacyDeliveryId": string, "decision": "confirm" | "dismiss" }` (exactly two properties). `STORYRAIL_OPERATOR_ID` must be configured. +- Workflow: `resolveLegacyDeliveryMapping`. Validates the legacy delivery mapping exists and is latest for the configured destination, then appends an immutable `LegacyDeliveryMappingResolution` row. + +| Status | Condition | +| ------ | --------- | +| 201 | Resolution recorded | +| 400 | `LEGACY_DELIVERY_MAPPING_RESOLUTION_INVALID` / shape error | +| 404 | `STORY_NOT_FOUND`, `LEGACY_DELIVERY_MAPPING_NOT_FOUND` | +| 409 | `LEGACY_DELIVERY_MAPPING_STALE`, `LEGACY_DELIVERY_MAPPING_DESTINATION_MISMATCH`, `LEGACY_DELIVERY_MAPPING_RESOLUTION_ID_CONFLICT` | +| 415 | Unsupported media type | +| 500 | `LEGACY_DELIVERY_MAPPING_RESOLUTION_NOT_RECORDED` / internal error | +| 503 | Missing `STORYRAIL_OPERATOR_ID` or `DESTINATION_NOT_CONFIGURED` / credential errors | + +## POST /api/sites/[siteId]/stories/[storyId]/deliveries/reconciliation — reconcile uncertain delivery outcome + +- Route: `src/app/api/sites/[siteId]/stories/[storyId]/deliveries/reconciliation/route.ts` +- Handler: `src/interfaces/http/reconcile-story-delivery-handler.ts` +- Provider: `storyRuntimeProvider` +- Body: `{ "deliveryId": string, "decision": "delivered" | "not_delivered", "remoteId": string | null }` (exactly three properties; `delivered` requires non-empty `remoteId`, `not_delivered` requires `remoteId: null`). `STORYRAIL_OPERATOR_ID` must be configured. +- Workflow: `reconcileStoryDelivery`. Validates the unresolved delivery (`outcome IN ('running', 'unknown')`) exists and is latest for the destination instance, verifies that for an update operation `remoteId` matches the original addressed page, and appends an immutable `StoryDeliveryReconciliation` row. + +| Status | Condition | +| ------ | --------- | +| 201 | Reconciliation recorded | +| 400 | `STORY_DELIVERY_RECONCILIATION_INVALID` / shape error | +| 404 | `STORY_NOT_FOUND`, `STORY_DELIVERY_RECONCILIATION_NOT_FOUND` | +| 409 | `STORY_DELIVERY_ALREADY_RECONCILED` | +| 415 | Unsupported media type | +| 500 | `STORY_DELIVERY_RECONCILIATION_NOT_RECORDED` / internal error | +| 503 | Missing `STORYRAIL_OPERATOR_ID` or `DESTINATION_NOT_CONFIGURED` / credential errors | + ## GET /api/sites/[siteId]/newsroom-standards — read newsroom standards - Route: `src/app/api/sites/[siteId]/newsroom-standards/route.ts` diff --git a/openwiki/architecture/newsroom-ui.md b/openwiki/architecture/newsroom-ui.md index 1b1cb42..b6d7306 100644 --- a/openwiki/architecture/newsroom-ui.md +++ b/openwiki/architecture/newsroom-ui.md @@ -57,15 +57,17 @@ It fetches Stories via `storyClient`, pending Sources via `sourceInboxRequests`, ## Publishing and delivery controls -`story-workspace.tsx` and `delivery-outcome.ts` provide operator-facing delivery controls and status reporting: -- **Delivery Trigger**: Operators can deliver a published Story's latest Article Revision to the configured destination (StudioCMS or WordPress). -- **Delivery Inspection**: Inspects and renders delivery records (`StoryDeliveryInspection`). -- **Clear Status Messaging**: Differentiates between: - - Deliveries that were never attempted ("Nothing was sent.") when credentials or destinations are unconfigured. - - Deliveries in progress ("Sending..."). - - Succeeded deliveries (showing remote ID and any modified slug). - - Refused or failed deliveries (displaying specific failure codes and reasons). -- **Re-delivery**: Allows re-delivering to update an existing remote post after a new revision is published. +`story-workspace.tsx` and `delivery-outcome.ts` provide operator-facing delivery controls, reconciliation reviews, and status reporting: +- **Delivery Trigger**: Operators can deliver a published Story's latest Article Revision to the configured destination (StudioCMS or WordPress). Every delivery requires explicit confirmation when the destination may publish immediately. +- **Delivery Inspection & Standing**: Inspects and renders delivery records (`StoryDeliveryInspection`): + - `never-delivered`: Story has not been sent anywhere. + - `in-flight`: Delivery currently executing. + - `delivered`: Succeeded delivery showing remote ID and any modified slug. + - `unknown`: Outcome cannot be confirmed; flags the delivery for operator reconciliation rather than allowing blind retries. + - `failed`: Refused or failed deliveries displaying specific failure codes and reasons. +- **Legacy Mapping Review**: When a Story was delivered under pre-migration schemas without instance identity, the UI presents an explicit review panel displaying the legacy remote post ID. Operators can confirm the post belongs to the current destination instance (enabling future update deliveries) or dismiss it (allowing a fresh create). +- **Delivery Reconciliation**: When a delivery ends in an `unknown` outcome (e.g. timeout or unreadable response), the UI presents a reconciliation form requiring the operator to inspect the remote installation and declare whether the post was `delivered` (with the remote post ID) or `not_delivered`. +- **Race-Safe Selection**: `NewsroomShell` manages `storySelectionGeneration` refs so slow inspection requests from previously selected Stories never overwrite the currently chosen Story. ## Workspaces and clients diff --git a/openwiki/architecture/source-map.md b/openwiki/architecture/source-map.md index 568cf76..53ee038 100644 --- a/openwiki/architecture/source-map.md +++ b/openwiki/architecture/source-map.md @@ -18,6 +18,7 @@ StoryRail is a single-package Next.js application (`package.json`: `name: storyr | `tsconfig.json` | strict TS, `ES2022`, `moduleResolution: Bundler`, path alias `@/*` → `./src/*`, JSX `react-jsx` | | `next.config.ts` | empty `NextConfig` | | `vitest.config.ts` | jsdom environment, `@` alias, `src/**/*.test.{ts,tsx}`, setup `./src/test/setup.ts` | +| `playwright.config.ts` | Playwright browser acceptance test suite configuration (Chromium, disposable test DB, mock services) | | `eslint.config.mjs` | next core-web-vitals + TypeScript configs; ignores build/cache output | | `.prettierrc.json`, `.prettierignore`, `.editorconfig` | formatting | | `.nvmrc` | Node 24.18.0 | @@ -33,7 +34,9 @@ StoryRail is a single-package Next.js application (`package.json`: `name: storyr - `site-types.ts`, `site-domain.ts` — `Site`, `SiteDomain`, `canonicalizeSiteDomain` - `built-in-agent-profiles.ts` — `builtInAgentProfilesForSite`, `findBuiltInAgentProfile` - `newsroom-standards-types.ts`, `newsroom-standards.ts` — `recordNewsroomStandards`, `withNewsroomStandards` -- `article-grounding.ts` — markdown-agnostic quote normalization and fact-checking checks +- `article-grounding.ts` — markdown-agnostic quote normalization, visible inline extraction, and fact-checking checks +- `legacy-delivery-mapping-resolution-types.ts`, `legacy-delivery-mapping-resolution.ts` — `LegacyDeliveryMappingResolution` and validator +- `story-delivery-reconciliation-types.ts`, `story-delivery-reconciliation.ts` — `StoryDeliveryReconciliation` and validator - `state-machine.ts` — `PERMITTED_STORY_TRANSITIONS`, `MAX_REVISION_CYCLES`, `transitionStory` - `source-types.ts` — `UrlSource`, `CanonicalSourceUrl`, intake/error types - `source-url.ts` — `canonicalizeSourceUrl` @@ -82,7 +85,7 @@ StoryRail is a single-package Next.js application (`package.json`: `name: storyr - `director-reviews/` — `run-director-review.ts` - `review-decisions/` — `record-story-review-decision.ts`, `review-decision-persistence.ts` - `story-rejections/` — `reject-story.ts`, `story-rejection-persistence.ts` -- `story-deliveries/` — `deliver-story.ts`, `delivery-destination.ts`, `story-delivery-repository.ts` +- `story-deliveries/` — `deliver-story.ts`, `delivery-destination.ts`, `story-delivery-repository.ts`, `resolve-legacy-delivery-mapping.ts`, `legacy-delivery-mapping-resolution-repository.ts`, `reconcile-story-delivery.ts`, `story-delivery-reconciliation-repository.ts` - `site-settings/` — `update-site-settings.ts`, `site-settings-repository.ts` - `index.ts` — barrel re-export @@ -107,6 +110,8 @@ StoryRail is a single-package Next.js application (`package.json`: `name: storyr - `story-listing/` — `postgres-story-listing-repository.ts`, `.test.ts` - `story-rejection-persistence/` — `postgres-story-rejection-persistence.ts`, `.test.ts` - `story-delivery-persistence/` — `postgres-story-delivery-repository.ts`, `postgres-story-delivery-decoder.ts` +- `legacy-delivery-mapping-resolution-persistence/` — `postgres-legacy-delivery-mapping-resolution-repository.ts`, `postgres-legacy-delivery-mapping-resolution-decoder.ts` +- `story-delivery-reconciliation-persistence/` — `postgres-story-delivery-reconciliation-repository.ts`, `postgres-story-delivery-reconciliation-decoder.ts` - `story-delivery/` — `studiocms-destination.ts`, `wordpress-destination.ts`, `gutenberg-blocks.ts`, `site-delivery-destination-directory.ts` - `site-settings-persistence/` — `postgres-site-settings-repository.ts` - `site-credential-persistence/` — `postgres-site-credential-repository.ts` @@ -123,6 +128,7 @@ StoryRail is a single-package Next.js application (`package.json`: `name: storyr - `assignment-editor-configuration.ts`, `assignment-editor-runtime.ts` — supervised Assignment Editor proposal runtime - `writer-configuration.ts`, `writer-runtime.ts` — supervised Writer draft and revision runtime and model resolution - `director-configuration.ts`, `director-runtime.ts` — supervised advisory Director review runtime and model resolution +- `openrouter-configuration.ts` — `resolveOpenRouterBaseUrl` for custom provider base URL routing - `index.ts` — barrel re-export ### `src/server` — lazy runtime providers @@ -157,7 +163,7 @@ StoryRail is a single-package Next.js application (`package.json`: `name: storyr - `run-director-review-handler.ts` - `record-story-review-decision-handler.ts` - `reject-story-handler.ts` -- `deliver-story-handler.ts` +- `deliver-story-handler.ts`, `resolve-legacy-delivery-mapping-handler.ts`, `reconcile-story-delivery-handler.ts` - `model-catalog-handlers.ts` - `site-settings-handlers.ts` @@ -182,6 +188,8 @@ StoryRail is a single-package Next.js application (`package.json`: `name: storyr - `api/sites/[siteId]/stories/[storyId]/review-decisions/route.ts` (POST) - `api/sites/[siteId]/stories/[storyId]/rejections/route.ts` (POST) - `api/sites/[siteId]/stories/[storyId]/deliveries/route.ts` (POST) +- `api/sites/[siteId]/stories/[storyId]/deliveries/legacy-mapping-resolution/route.ts` (POST) +- `api/sites/[siteId]/stories/[storyId]/deliveries/reconciliation/route.ts` (POST) - `api/sites/[siteId]/agent-profiles/route.ts` (GET, POST) - `api/sites/[siteId]/model-catalog/route.ts` (GET) - `api/sites/[siteId]/site-settings/route.ts` (GET, PUT) @@ -203,6 +211,13 @@ StoryRail is a single-package Next.js application (`package.json`: `name: storyr - `agent-profiles-workspace.tsx`, `agent-profile-client.ts` - `story-client.ts` +### `e2e` + +- `newsroom-smoke.spec.ts` — browser acceptance test verifying newsroom desk, queues, staff, and inbox rendering +- `supervised-editorial-journey.spec.ts` — complete supervised editorial journey from Source intake through Director review, approval, and WordPress delivery +- `support/external-services.mjs` — lightweight mock HTTP server for OpenRouter completions and WordPress post publishing during acceptance tests +- `support/reset-test-database.mjs` — database reset script verifying `storyrail_test` and dropping schema/ledger before test migrations + ### `src/test` - `setup.ts` — vitest setup (jest-dom matchers) @@ -244,6 +259,9 @@ StoryRail is a single-package Next.js application (`package.json`: `name: storyr - `0074-policy-run-source-roots.sql` — Source-rooted policy runs and reconcilable pre-Story workflows - `0075-policy-run-attempts.sql` — bounded Writer retry attempt tracking (up to 3 attempts) in policy payloads - `0076-story-delivery-instance-identity.sql` — destination installation instance identity and legacy mapping safety +- `0077-legacy-delivery-mapping-resolutions.sql` — immutable legacy delivery mapping resolution audit facts +- `0078-ambiguous-delivery-reconciliation.sql` — uncertain delivery outcome state (`unknown`) and reconciliation tracking +- `0079-agent-run-recovery.sql` — database-owned recovery timestamps and stale running AgentRun recovery ## Documentation (`docs/`) diff --git a/openwiki/quickstart.md b/openwiki/quickstart.md index 7efb7db..5e94ce2 100644 --- a/openwiki/quickstart.md +++ b/openwiki/quickstart.md @@ -71,7 +71,7 @@ pnpm dev ## PostgreSQL integration tests -Source-evidence, Story, attachment, triage, preparation, Agent Profile, Assignment, AgentRun, Article, Article Revision, Writer revision, review submission, review decision, Director review, Story rejection, tool calls, newsroom standards, archive search, site credentials, site settings, policy runs, web search, and story delivery persistence integration tests run against real PostgreSQL 18.4 (no mocks, testcontainers, or embedded databases). Provide the test-only connection through `STORYRAIL_TEST_DATABASE_URL`. The configured database name **must** be exactly `storyrail_test`. The suite never creates or drops a database, but it does drop and recreate the `storyrail` schema, applies migrations `0012`, `0017`, `0018`, `0024`, `0025`, `0027`, `0028`, `0030`, `0031`, `0038`, `0041`, `0049`, `0053`, `0054`, `0055`, `0056`, `0057`, `0058`, `0059`, `0060`, `0061`, `0062`, `0063`, `0064`, `0065`, `0066`, `0067`, `0068`, `0069`, `0070`, `0071`, `0072`, `0073`, `0074`, `0075`, and `0076` in order, truncates the editorial tables (and deletes non-built-in Agent Profiles) between cases. +Source-evidence, Story, attachment, triage, preparation, Agent Profile, Assignment, AgentRun, Article, Article Revision, Writer revision, review submission, review decision, Director review, Story rejection, tool calls, newsroom standards, archive search, site credentials, site settings, policy runs, web search, story delivery, legacy delivery mapping resolution, and story delivery reconciliation persistence integration tests run against real PostgreSQL 18.4 (no mocks, testcontainers, or embedded databases). Provide the test-only connection through `STORYRAIL_TEST_DATABASE_URL`. The configured database name **must** be exactly `storyrail_test`. The suite never creates or drops a database, but it does drop and recreate the `storyrail` schema, applies migrations `0012`, `0017`, `0018`, `0024`, `0025`, `0027`, `0028`, `0030`, `0031`, `0038`, `0041`, `0049`, `0053`, `0054`, `0055`, `0056`, `0057`, `0058`, `0059`, `0060`, `0061`, `0062`, `0063`, `0064`, `0065`, `0066`, `0067`, `0068`, `0069`, `0070`, `0071`, `0072`, `0073`, `0074`, `0075`, `0076`, `0077`, `0078`, and `0079` in order, truncates the editorial tables (and deletes non-built-in Agent Profiles) between cases. ```bash STORYRAIL_TEST_DATABASE_URL='postgresql://postgres:postgres@127.0.0.1:5432/storyrail_test' \ @@ -98,6 +98,7 @@ When `STORYRAIL_TEST_DATABASE_URL` is absent, `pnpm test` skips the PostgreSQL s | Writer revisions | [Application workflows](architecture/application-workflows.md), [Adapters and runtime](architecture/adapters-and-runtime.md) | `src/application/writer-revisions/create-writer-revision.ts`, `src/runtime/writer-runtime.ts` | `createWriterRevision`, `resolveWriterModel` | `create-writer-revision.test.ts` | `pnpm test writer-revision` | | Review submission / Director | [Application workflows](architecture/application-workflows.md), [Adapters and runtime](architecture/adapters-and-runtime.md) | `src/application/review-submissions/submit-story-review.ts`, `src/application/director-reviews/run-director-review.ts`, `src/runtime/director-runtime.ts` | `createSubmitStoryReview`, `createRunDirectorReview`, `createDirectorRuntime` | `submit-story-review.test.ts`, `run-director-review.test.ts` | `pnpm test review` | | Review decisions / Rejection | [Application workflows](architecture/application-workflows.md), [Adapters and runtime](architecture/adapters-and-runtime.md) | `src/application/review-decisions/record-story-review-decision.ts`, `src/application/story-rejections/reject-story.ts` | `createRecordStoryReviewDecision`, `createRejectStory` | `record-story-review-decision.test.ts`, `reject-story.test.ts` | `pnpm test review-decision` | +| Story delivery / Reconciliation | [Application workflows](architecture/application-workflows.md), [Adapters and runtime](architecture/adapters-and-runtime.md) | `src/application/story-deliveries/deliver-story.ts`, `src/application/story-deliveries/reconcile-story-delivery.ts`, `src/application/story-deliveries/resolve-legacy-delivery-mapping.ts` | `createDeliverStory`, `createReconcileStoryDelivery`, `createResolveLegacyDeliveryMapping` | `deliver-story.test.ts`, `reconcile-story-delivery.test.ts`, `resolve-legacy-delivery-mapping.test.ts` | `pnpm test deliver-story` | | Site credentials & settings | [Adapters and runtime](architecture/adapters-and-runtime.md), [HTTP API](architecture/http-api.md) | `src/adapters/credential-cipher/aes-gcm-credential-cipher.ts`, `src/adapters/site-credential-persistence/postgres-site-credential-repository.ts`, `src/application/site-settings/update-site-settings.ts` | `createAesGcmCredentialCipher`, `createPostgresSiteCredentialRepository`, `createUpdateSiteSettings` | `aes-gcm-credential-cipher.test.ts`, `site-settings-client.test.ts` | `pnpm test site-settings` | | Model catalog discovery | [Adapters and runtime](architecture/adapters-and-runtime.md), [HTTP API](architecture/http-api.md) | `src/adapters/model-catalog/openrouter-model-catalog.ts`, `src/application/model-catalog/model-catalog.ts` | `createOpenRouterModelCatalog`, `modelCatalogProvider` | `openrouter-model-catalog.test.ts` | `pnpm test model-catalog` | | Story delivery (StudioCMS/WordPress)| [Adapters and runtime](architecture/adapters-and-runtime.md), [Application workflows](architecture/application-workflows.md), [Newsroom UI shell](architecture/newsroom-ui.md) | `src/adapters/story-delivery/wordpress-destination.ts`, `src/adapters/story-delivery/studiocms-destination.ts`, `src/application/story-deliveries/deliver-story.ts`, `src/features/newsroom/delivery-outcome.ts` | `createWordPressDestination`, `createStudioCmsDestination`, `createDeliverStory`, `deliveryOutcome` | `wordpress-destination.test.ts`, `studiocms-destination.test.ts`, `deliver-story.test.ts`, `story-delivery-workspace.test.tsx` | `pnpm test delivery` | @@ -119,8 +120,8 @@ When `STORYRAIL_TEST_DATABASE_URL` is absent, `pnpm test` skips the PostgreSQL s - [Application workflows](architecture/application-workflows.md) — use-case orchestration and repository ports. - [Adapters and runtime composition](architecture/adapters-and-runtime.md) — PostgreSQL, Firecrawl, and OpenRouter adapters composed into focused runtimes including the Director runtime. - [HTTP API endpoints](architecture/http-api.md) — Next.js route handlers and status code maps. -- [PostgreSQL schema and migrations](architecture/database-schema.md) — the `storyrail` schema and migrations `0012`–`0076`. -- [Newsroom UI shell](architecture/newsroom-ui.md) — resizable desk, staff sidebar, Story workspace with assignment, writing, Writer revision, review submission, Director review, operator decision, and operator Story rejection, and SafeMarkdown. +- [PostgreSQL schema and migrations](architecture/database-schema.md) — the `storyrail` schema and migrations `0012`–`0079`. +- [Newsroom UI shell](architecture/newsroom-ui.md) — resizable desk, staff sidebar, Story workspace with assignment, writing, Writer revision, review submission, Director review, operator decision, operator Story rejection, delivery reconciliation, and SafeMarkdown. - [Repository source map](architecture/source-map.md) — canonical file and directory locations. - [Engineering workflow and testing](engineering-workflow.md) — branch/batch discipline, verification ownership, CI contract, and the testing strategy. diff --git a/openwiki/update-summary.md b/openwiki/update-summary.md index a0d157f..09a5a98 100644 --- a/openwiki/update-summary.md +++ b/openwiki/update-summary.md @@ -3,22 +3,27 @@ type: Reference title: OpenWiki Update Summary description: Summary of changes made to the StoryRail OpenWiki documentation during this update cycle. --- -The StoryRail OpenWiki documentation is now current with the repository state at commit d39ecfb983cf472121fc2641d283a8da7b288073. +The StoryRail OpenWiki documentation is now current with the repository state at commit aba939a097febf62559dbbff687cfdfe25a4fd93. Updated files: - /openwiki/quickstart.md - /openwiki/architecture/domain-model.md - /openwiki/architecture/database-schema.md +- /openwiki/architecture/adapters-and-runtime.md - /openwiki/architecture/application-workflows.md - /openwiki/architecture/http-api.md +- /openwiki/architecture/newsroom-ui.md - /openwiki/architecture/source-map.md - /openwiki/update-summary.md The documentation reflects all changes including: -- Destination instance identity binding in `storyrail.story_deliveries` (`destination_instance_id`), migration `0076-story-delivery-instance-identity.sql`, and legacy mapping safety (`DESTINATION_MAPPING_REQUIRES_REVIEW`) (`d39ecfb`) -- Shared strict Zod schemas across domain, persistence decoders, and browser clients, ending drift between PostgreSQL and UI readers (`e0c46f8`, `a71865c`) -- Bounded Writer retries (up to 3 attempts with fresh run identities) and migration `0075-policy-run-attempts.sql` (`18d219b`) -- Pre-Story reconcilable policy runs with Source rooting and migration `0074-policy-run-source-roots.sql` (`9936048`) +- Supervised editorial browser acceptance tests in Playwright (`e2e/newsroom-smoke.spec.ts`, `e2e/supervised-editorial-journey.spec.ts`, `playwright.config.ts`), mock services server (`e2e/support/external-services.mjs`), and database reset automation (`e2e/support/reset-test-database.mjs`) (`43b3079`, `aba939a`) +- Configurable provider base URL (`STORYRAIL_OPENROUTER_BASE_URL`) with runtime validation in `src/runtime/openrouter-configuration.ts` (`aba939a`) +- Interrupted manual run recovery (`0cff50b`), migration `0079-agent-run-recovery.sql` with database-owned `recorded_at` timestamps and partial indexing (`agent_runs_stale_running_idx`), and stale running AgentRun recovery in `reconcile-abandoned-work.ts` +- Ambiguous delivery outcome tracking (`unknown` outcome), operator delivery reconciliation workflow (`reconcile-story-delivery.ts`), PostgreSQL reconciliation persistence, and migration `0078-ambiguous-delivery-reconciliation.sql` (`18e954c`) +- Legacy delivery mapping resolutions (`00b4167`), migration `0077-legacy-delivery-mapping-resolutions.sql`, snapshot validation, and operator review workflow (`resolve-legacy-delivery-mapping.ts`) +- Preserving literal Markdown text during article quote grounding normalization (`visibleInlineMarkdown`) (`b326185`) +- Prevention of stale story inspection selections in `NewsroomShell` via generational request tracking (`b896886`) - End-to-end URL-to-delivered post Autopilot sequence with research budget settings and migrations `0072-policy-runs-from-a-url.sql` and `0073-research-budget-settings.sql` (`3f93984`) - Interactive Story Rail navigation, pinned compact rail on manuscript scroll, and streamlined one-action editorial operations (`be6c968`) - Real-time tool activity stream and budget metrics in inspection and newsroom workspace (`1ba59c6`)