From 2a61da81f776a434e0af5573a5ac26e09c05c431 Mon Sep 17 00:00:00 2001 From: Managed by Ansible Date: Wed, 17 Jun 2026 00:26:05 +0000 Subject: [PATCH 1/9] =?UTF-8?q?docs:=20log=20decision=20=E2=80=94=20don't?= =?UTF-8?q?=20use=20FDB=20spool=20instanceId=20for=20cross-ref=20(keep=20l?= =?UTF-8?q?abel)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Record the considered-and-declined design (mirroring the Spoolman spool ID into FDB's user-editable instanceId via a prefix) and the rationale: instanceId is FDB's physical-tag match key; linkage is already handled by the label field + SpoolMapping. Includes the FDB 1.43-1.49 review notes (no breaking changes; MIN_FDB stays 1.33.0). --- docs/decisions.md | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/docs/decisions.md b/docs/decisions.md index e1c69f7..44524be 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -1,5 +1,52 @@ # Decision record +## 2026-06-17 — FDB spool `instanceId` is NOT used for cross-reference; keep `label` + +**Context.** Filament DB 1.48.0 moved the 5-byte hex roll ID (`instanceId`) from the +filament onto each **spool**, and 1.49.0 made it **user-editable**. Format (verified in +FDB `src/models/Filament.ts`): the auto-generated default is +`crypto.randomBytes(5).toString("hex")` → **10 lowercase hex chars** ("Prusament +format"); a user-set value is validated as `[A-Za-z0-9._-]{1,128}` and must be unique +(partial-unique index, scoped to non-deleted documents). It lives on both the filament +and the spool subdoc during the transition; the spool one is canonical going forward. + +**Considered.** Optionally (Settings toggle) mirror the Spoolman spool ID into the FDB +spool's `instanceId` using a user-chosen numeric prefix to stay 10 chars and globally +unique — e.g. prefix `1` + zero-padded id → `1000000042`. This would give a single +human-readable roll number derivable in both systems. + +**Decision: declined the write path. The bridge does NOT write `instanceId`. The +Spoolman spool ID stays in the FDB spool `label` field (`FILAMENTDB_SPOOLMAN_ID_FIELD`, +default `label`), backed by the `SpoolMapping` table.** + +**Rationale.** +- `instanceId` is FDB's **physical-tag match key** — desktop and mobile scanners resolve + QR/NFC against it (1.48), and 1.49 made it editable precisely so users can store their + *real* Prusament roll IDs / NFC tag IDs. Overwriting it with a synthetic cross-ref + number would break scan-by-tag, stomp user-entered values, and strand already-printed + labels / written tags (FDB does not re-write tags when the ID changes). +- Linkage is **already solved**: the Spoolman spool ID is stored in the FDB spool `label` + field *and* in the bridge's SQLite `SpoolMapping`. Encoding it into `instanceId` is a + redundant third copy. +- It's **one-directional decoration**: Spoolman cannot read FDB's `instanceId`, so it adds + no new sync capability on either side. +- The proposed `1000000xxx` encoding only holds 3 digits of ID (≤999 spools) before + overflow; a safe scheme would need `prefix + 9-digit zero-pad`, reinforcing that this is + fiddly for little gain. + +**Future-looking (not implemented).** If `instanceId` is interesting to the bridge at all, +the valuable direction is the opposite — **read** it (and the new `Filament.openprinttagSnapshot` +field added in 1.47.2) so the bridge can *surface* the user's real roll ID / OpenPrintTag +identity (e.g. mirror into a Spoolman extra field), adding information rather than +destroying it. Needs the `openprinttagSnapshot` schema from `/api-docs` before any work. + +**Upstream review context.** Reviewed FDB releases 1.43.0 → 1.49.0 (latest): no breaking +changes for the bridge. We address spools by the stable subdoc `_id` and send minimal PUT +payloads (`{label}` / `{totalWeight}`), so the per-spool/editable `instanceId` is never +clobbered; `instanceId` is also in `_strip_computed` for the filament path. 1.44.0 +(hide out-of-stock) is a UI-only filter — `GET /api/filaments` still returns all records. +`MIN_FDB` stays `1.33.0`. README "latest tested" should be bumped 1.42.0 → 1.49.0. + ## 2026-06-13 — Debug: added POST /api/debug/clear-spoolman-opentag-ids Added a fourth standalone debug endpoint that blanks the three OpenPrintTag identity From 4c17f72445937fe7c6108b881a154b0ec7c0f0dd Mon Sep 17 00:00:00 2001 From: Managed by Ansible Date: Wed, 17 Jun 2026 00:29:10 +0000 Subject: [PATCH 2/9] docs: bump latest-tested Filament DB to 1.49.0 Reviewed FDB 1.43.0-1.49.0; no breaking changes for the bridge (MIN_FDB stays 1.33.0). Spoolman unchanged at 0.23.1. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 5bcc2b6..be916b6 100644 --- a/README.md +++ b/README.md @@ -140,7 +140,7 @@ per-system warning explaining why sync is off — so you can see and fix it. An version does *not* block sync (that is treated as a connectivity issue, surfaced as `degraded` health, not as "too old"). -Latest tested upstreams: **Filament DB 1.42.0** and **Spoolman 0.23.1**. +Latest tested upstreams: **Filament DB 1.49.0** and **Spoolman 0.23.1**. - **Filament DB** — the bridge gates version-specific features automatically. - **Spoolman** — the bridge creates its required extra fields (`filamentdb_id`, `filamentdb_spool_id`, etc.) automatically on startup if they are missing. From f9fd645fdbc1a93be68de1fd5acf499f5931d93f Mon Sep 17 00:00:00 2001 From: Managed by Ansible Date: Wed, 17 Jun 2026 02:47:57 +0000 Subject: [PATCH 3/9] feat: mirror archive/retire lifecycle bidirectionally for synced spools (FR-21) --- CHANGELOG.md | 17 + CLAUDE.md | 17 +- backend/app/api/config.py | 16 + backend/app/api/conflicts.py | 27 +- backend/app/core/conflict_apply.py | 62 ++++ backend/app/core/differ.py | 24 ++ backend/app/core/engine.py | 197 ++++++++++- backend/app/models/config.py | 6 + backend/app/schemas/api.py | 7 + backend/tests/test_api.py | 11 +- backend/tests/test_conflict_apply.py | 81 +++++ backend/tests/test_differ.py | 74 ++++ backend/tests/test_engine.py | 322 +++++++++++++++++- docs/configuration.md | 20 +- docs/decisions.md | 61 +++- docs/prd.md | 9 +- docs/spoolman-writes.md | 31 +- docs/sync-model.md | 29 +- frontend/src/api/types.ts | 4 + frontend/src/pages/Settings.test.tsx | 2 + frontend/src/pages/Settings.tsx | 40 ++- ...06-17-archive-retire-bidirectional-sync.md | 219 ++++++++++++ 22 files changed, 1231 insertions(+), 45 deletions(-) create mode 100644 prompts/done/2026-06-17-archive-retire-bidirectional-sync.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 4fdb57b..b1734a0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,23 @@ GitHub release. ## [Unreleased] +### Added + +- **Bidirectional archive/retire lifecycle sync (FR-21)** — a mapped spool's lifecycle + state now mirrors between Spoolman (`archived`) and Filament DB (`retired`) in both + directions: archiving/retiring one side flips the other, and un-archiving/un-retiring + mirrors back too (re-enabling weight sync). A new `archive_sync` policy category + (`archive_sync_direction`, default `two_way`; `archive_conflict_policy`, default `manual`) + governs it from Settings → Archive / retire sync; `newest_wins` is rejected (422) since + the state is a boolean with no timestamp. The wizard import gate is preserved — *unmapped* + archived spools are still never auto-imported; only *mapped-pair* diffing includes archived + spools. The lifecycle pass runs **after** the weight pass, so a depleted-and-archived spool + settles its final decrement and FDB usage-log entry (and refreshes both snapshots) before + the archive bit mirrors. A one-sided flip is a clean push; only a both-sides-diverge-to- + opposite-states case queues a `cross_system` lifecycle conflict, whose human resolution + writes the chosen state to both systems. The "Never import empties" setting was relabeled + "Skip empty & archived spools on import" to clarify it is import-only (config key unchanged). + ## [0.2.0] — 2026-06-15 ### Added diff --git a/CLAUDE.md b/CLAUDE.md index 9184959..d11efc6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -271,7 +271,9 @@ Several settings can be changed at runtime via the Settings UI (stored in SQLite | `new_filament_policy` | `manual_review` | What the engine does when it detects an unmapped filament: `manual_review` queues an actionable `new_filament` conflict; `auto_import` creates it immediately using the wizard code path. Falls back to `manual_review` if `variant_parent_mode` is `unset` and the filament looks like a variant-cluster member. | | `new_spool_policy` | `manual_review` | What the engine does when an unmapped spool appears on an already-mapped filament: `manual_review` queues a `new_spool` conflict; `auto_import` creates it immediately. A spool is always held when its filament is unmapped, regardless of this setting. | | `debug_mode` | `false` | Enables `POST /api/debug/clear-spoolman-fdb-refs`, `POST /api/debug/reset-bridge-state`, and `POST /api/debug/full-reset` (403 when off) | -| `never_import_empties` | `false` | Wizard skips spools with zero remaining weight at preview/execute | +| `never_import_empties` | `false` | **Import-only** (UI label: "Skip empty & archived spools on import"). Wizard + ongoing new-spool import skip spools with zero remaining weight or that are archived, at preview/execute. Does NOT affect ongoing archive/retire lifecycle mirroring for already-mapped pairs (that runs regardless). Config key unchanged. | +| `archive_sync_direction` | `two_way` | Direction for the archive/retire lifecycle category (`two_way` / `spoolman_to_filamentdb` / `filamentdb_to_spoolman`). Mirrors SM `archived` ↔ FDB `retired` for already-mapped spool pairs. | +| `archive_conflict_policy` | `manual` | Conflict policy for the archive/retire category (consulted only under `two_way` when both sides diverge to opposite states): `manual` / `spoolman_wins` / `filamentdb_wins`. `newest_wins` is rejected at the API (422) — the state is a boolean with no timestamp. | | `sync_log_retention_days` | `30` | Sync log entries older than this are pruned automatically | | `variant_parent_mode` | `unset` | Wizard variant hierarchy mode: `unset` (must choose), `promote_color` (original behavior), or `generic_container` (colorless container parent for every cluster). See `docs/variant-parent-mode.md`. | | `api_token_enabled` | `false` | When `true`, requests may authenticate via `Authorization: Bearer ` or `X-API-Key`. Toggle in Settings → Security. | @@ -297,6 +299,19 @@ Spoolman `remaining_weight` is net filament. Filament DB `totalWeight` is gross agreed values** (SM `remaining_weight` and FDB `totalWeight`), or the propagated change is re-detected as a fresh change on the other side next cycle → ping-pong. +### Archive/retire lifecycle sync +Archive/retire lifecycle state mirrors **bidirectionally for already-mapped spool pairs**: +SM `archived` ↔ FDB `retired`. A dedicated lifecycle pass runs **after** the weight pass on +purpose — a spool is usually archived right as it hits ~0 g, so the final weight decrement +(and its FDB usage-log entry) must settle and both snapshots refresh *before* the archive +bit mirrors, or the far side lands retired/archived with a stale weight and a missing usage +entry. One-sided flips (either direction, archive or un-archive) are clean pushes; only a +both-sides-flip-to-opposite-states divergence queues a `cross_system` conflict +(`field_name="lifecycle"`). Governed by the `archive_sync` category (`archive_sync_direction` +/ `archive_conflict_policy`). The wizard import gate (`never_import_empties`) still keeps +*unmapped* archived spools out of auto-import — only mapped pairs are mirrored. After any +lifecycle push, refresh BOTH snapshots (same anti-ping-pong rule as weight). + ### Filament DB API endpoints the bridge uses - `GET /api/filaments` — list all filaments with embedded spools - `GET /api/filaments/:id` — single filament with full detail diff --git a/backend/app/api/config.py b/backend/app/api/config.py index bea90f5..a59e4ce 100644 --- a/backend/app/api/config.py +++ b/backend/app/api/config.py @@ -132,6 +132,8 @@ def _config_response(db: Session) -> ConfigResponse: weight_conflict_policy=cfg.get("weight_conflict_policy", "manual"), material_properties_sync_direction=cfg.get("material_properties_sync_direction", "filamentdb_to_spoolman"), material_properties_conflict_policy=cfg.get("material_properties_conflict_policy", "manual"), + archive_sync_direction=cfg.get("archive_sync_direction", "two_way"), + archive_conflict_policy=cfg.get("archive_conflict_policy", "manual"), new_spool_sync_direction=cfg.get("new_spool_sync_direction", "two_way"), new_filament_policy=cfg.get("new_filament_policy", "manual_review") or "manual_review", new_spool_policy=cfg.get("new_spool_policy", "manual_review") or "manual_review", @@ -175,6 +177,20 @@ def update_config( }, ) + # newest_wins is also meaningless for archive/retire — the state is a boolean, + # not a timestamped value, so there is no "newer" side to pick. + if payload.archive_conflict_policy == "newest_wins": + raise HTTPException( + status_code=422, + detail={ + "code": "invalid_conflict_policy", + "message": ( + "newest_wins is not supported for archive_sync — archive/retire " + "is a boolean state with no comparable timestamp." + ), + }, + ) + updates = payload.model_dump(exclude_unset=True) # Extract sync_interval_seconds before persisting so we can reschedule. diff --git a/backend/app/api/conflicts.py b/backend/app/api/conflicts.py index fcf14e1..83efee7 100644 --- a/backend/app/api/conflicts.py +++ b/backend/app/api/conflicts.py @@ -301,10 +301,35 @@ async def resolve_conflict( db.refresh(c) return _to_response(c, db) - # All other conflict types: record-only resolution (no upstream writes). if payload.resolution == "manual" and payload.value is None: raise api_error(422, "manual_value_required", "A manual resolution requires a value") + # Lifecycle (archive/retire) conflicts converge by WRITING the chosen boolean state + # to both systems (human-approved, not silent auto-apply), then refreshing snapshots + # so the conflict does not re-queue. All other cross_system conflicts stay record-only. + if c.field_name == "lifecycle": + from app.core.conflict_apply import apply_lifecycle_conflict + # Pre-record the manual boolean so the apply helper can read it. + if payload.resolution == "manual": + c.resolved_value = json.dumps(bool(payload.value)) + spoolman = request.app.state.spoolman + filamentdb = request.app.state.filamentdb + try: + await apply_lifecycle_conflict(c, payload.resolution, db, spoolman, filamentdb) + except Exception as exc: + import logging + logging.getLogger(__name__).error( + "apply_lifecycle_conflict failed for conflict %d: %s", conflict_id, _scrub(exc) + ) + raise api_error( + 502, "upstream_write_failed", + f"Upstream write failed; conflict not resolved. Detail: {exc}" + ) + db.commit() + db.refresh(c) + return _to_response(c, db) + + # All other conflict types: record-only resolution (no upstream writes). c.resolution = payload.resolution c.resolved_value = json.dumps(_resolved_value(c, payload.resolution, payload.value)) c.resolved_at = datetime.datetime.now(datetime.timezone.utc) diff --git a/backend/app/core/conflict_apply.py b/backend/app/core/conflict_apply.py index e58cf0a..4f9e2eb 100644 --- a/backend/app/core/conflict_apply.py +++ b/backend/app/core/conflict_apply.py @@ -170,6 +170,68 @@ def _auto_resolve_siblings( ) +async def apply_lifecycle_conflict( + conflict: Conflict, + resolution: str, + db: Session, + spoolman: SpoolmanClient, + filamentdb: FilamentDBClient, +) -> None: + """Apply a human-chosen resolution for a boolean archive/retire lifecycle conflict. + + Lifecycle conflicts are ``cross_system`` conflicts with ``field_name == "lifecycle"``. + Unlike other ``cross_system`` conflicts (which resolve record-only), the user's choice + here is a concrete boolean state that we converge by writing it to BOTH systems, then + refreshing both spool snapshots so the engine does not re-queue the conflict next cycle. + + ``resolution``: + - "spoolman" → adopt the Spoolman archived state on both sides. + - "filamentdb" → adopt the Filament DB retired state on both sides. + - "manual" → adopt the explicit boolean stored in ``conflict.resolved_value`` + (already set by the caller before this is invoked). + + Raises on upstream failure (caller leaves the conflict open). + """ + spoolman_archived = bool(_decode(conflict.spoolman_value)) + fdb_retired = bool(_decode(conflict.filamentdb_value)) + + if resolution == "spoolman": + target = spoolman_archived + elif resolution == "filamentdb": + target = fdb_retired + else: # manual — explicit boolean recorded on the row + target = bool(_decode(conflict.resolved_value)) + + sm_spool_id: int = conflict.spoolman_id # type: ignore[assignment] + fdb_filament_id: str = conflict.filamentdb_filament_id # type: ignore[assignment] + fdb_spool_id: str = conflict.filamentdb_spool_id # type: ignore[assignment] + + cycle_id = f"conflict-apply-{conflict.id}-{uuid.uuid4().hex[:8]}" + + # Write the converged state to BOTH systems. Either may already match `target`; + # the writes are idempotent. + await spoolman.update_spool(sm_spool_id, {"archived": target}) + await filamentdb.update_spool(fdb_filament_id, fdb_spool_id, {"retired": target}) + + # Refresh both snapshot lifecycle bits to the converged value (anti-ping-pong). + _merge_snapshot(db, "spoolman", "spool", str(sm_spool_id), {"archived": target}) + _merge_snapshot(db, "filamentdb", "spool", fdb_spool_id, {"retired": target}) + + _log( + db, cycle_id, "conflict_apply", "update", "spool", + spoolman_id=sm_spool_id, + fdb_filament_id=fdb_filament_id, + fdb_spool_id=fdb_spool_id, + field_name="lifecycle", + old_value="diverged", + new_value=("retired/archived" if target else "live/active"), + ) + + conflict.resolution = resolution + conflict.resolved_value = json.dumps(target) + conflict.resolved_at = datetime.datetime.now(datetime.timezone.utc) + + async def apply_master_divergence( conflict: Conflict, action: str, diff --git a/backend/app/core/differ.py b/backend/app/core/differ.py index 4d3a49b..ca16177 100644 --- a/backend/app/core/differ.py +++ b/backend/app/core/differ.py @@ -45,6 +45,11 @@ class SpoolPairChangeset: fdb_field_changes: list[FieldChange] = field(default_factory=list) field_conflicts: list[str] = field(default_factory=list) # fdb_path names + # Lifecycle (archive/retire) — boolean diff, no threshold + sm_archive_change: FieldChange | None = None # SM spool.archived flipped + fdb_retire_change: FieldChange | None = None # FDB spool.retired flipped + archive_conflict: bool = False # both sides flipped + def diff_spool_pair( sm_spool: "SpoolmanSpool", @@ -88,6 +93,25 @@ def diff_spool_pair( if sm_wc and fdb_wc: cs.weight_conflict = True + # ---- Lifecycle (archive/retire) diff ---- + # Boolean state, so a plain inequality against the snapshot is the change test. + # When a snapshot lacks the key (legacy rows), default to the current value so a + # missing baseline is NOT mistaken for a flip (no spurious push on first sight). + sm_arch_now = bool(sm_spool.archived) + sm_arch_snap = bool(sm_snapshot.get("archived", sm_arch_now)) + fdb_ret_now = bool(fdb_spool.retired) + fdb_ret_snap = bool(fdb_snapshot.get("retired", fdb_ret_now)) + + sm_ac = sm_arch_snap != sm_arch_now + fdb_rc = fdb_ret_snap != fdb_ret_now + + if sm_ac: + cs.sm_archive_change = FieldChange("archived", sm_arch_snap, sm_arch_now) + if fdb_rc: + cs.fdb_retire_change = FieldChange("retired", fdb_ret_snap, fdb_ret_now) + if sm_ac and fdb_rc: + cs.archive_conflict = True + # ---- Field mapping diff ---- if field_maps and sm_extra_decoded is not None and fdb_field_values is not None: sm_extra_snap: dict = sm_snapshot.get("_extra_decoded", {}) diff --git a/backend/app/core/engine.py b/backend/app/core/engine.py index 886977b..2679265 100644 --- a/backend/app/core/engine.py +++ b/backend/app/core/engine.py @@ -437,6 +437,19 @@ def _sm_snapshot_dict( return d +def _refresh_lifecycle_snapshots( + db: Session, sm_spool_id: int, fdb_spool_id: str, sm_archived: bool, fdb_retired: bool +) -> None: + """Converge BOTH lifecycle bits to the agreed value (anti-ping-pong). + + Merges (not full upsert) so the weight / field baselines the weight + field passes + already wrote in this cycle are preserved. Same both-sides-refresh rule the weight + pass uses — see the 2026-06-10 weight ping-pong decision in docs/decisions.md. + """ + _merge_snapshot(db, "spoolman", "spool", str(sm_spool_id), {"archived": sm_archived}) + _merge_snapshot(db, "filamentdb", "spool", fdb_spool_id, {"retired": fdb_retired}) + + def _fdb_snapshot_dict(spool: FDBSpool, filament_detail=None, field_maps: list[FieldMapping] | None = None) -> dict: """Build the snapshot dict for a FDB spool. If filament_detail and field_maps are provided the mapped FDB field values are embedded under _field_values so the @@ -2451,6 +2464,8 @@ async def run_sync_cycle( weight_policy: str = config.get("weight_conflict_policy", "manual") matprop_direction: str = config.get("material_properties_sync_direction", "filamentdb_to_spoolman") matprop_policy: str = config.get("material_properties_conflict_policy", "manual") + archive_direction: str = config.get("archive_sync_direction", "two_way") + archive_policy: str = config.get("archive_conflict_policy", "manual") new_spool_direction: str = config.get("new_spool_sync_direction", "two_way") # New-record handling policies. new_filament_policy: str = config.get("new_filament_policy", "manual_review") or "manual_review" @@ -2502,7 +2517,12 @@ async def run_sync_cycle( result.errors += 1 return result + # Active-only set: used for NEW-spool detection so an unmapped archived spool is + # never auto-imported during ongoing sync (preserves the wizard import gate). sm_spools: dict[int, SpoolmanSpool] = {s.id: s for s in sm_spools_all if not s.archived} + # Active + archived set: used for MAPPED-pair diffing so a mapped spool that flips + # to archived still reaches the differ and its lifecycle state can be mirrored. + sm_spools_with_archived: dict[int, SpoolmanSpool] = {s.id: s for s in sm_spools_all} sm_all_ids: set[int] = {s.id for s in sm_spools_all} # includes archived sm_filaments: dict[int, Any] = {f.id: f for f in sm_filaments_all} fdb_filaments: dict[str, FDBFilament] = {f.id: f for f in fdb_filaments_all} @@ -2597,7 +2617,10 @@ async def run_sync_cycle( # ---- Process mapped spool pairs ---- for mapping in spool_mappings: - sm_spool = sm_spools.get(mapping.spoolman_spool_id) + # Mapped pairs look up against active + archived so a mapped spool that flips + # to archived still reaches the diff loop (its archive bit gets mirrored to FDB + # in the lifecycle pass). Unmapped archived spools never enter this loop. + sm_spool = sm_spools_with_archived.get(mapping.spoolman_spool_id) fdb_entry = fdb_spool_index.get(mapping.filamentdb_spool_id) if sm_spool is None: @@ -2649,7 +2672,11 @@ async def run_sync_cycle( _queue_deletion_conflict(db, cycle_id, mapping, deleted_side="spoolman") result.conflicts += 1 else: - # Present but archived — keep existing skip behavior. + # Defensive: the SM spool id is in sm_all_ids but missing from the + # combined (active + archived) lookup — should be impossible since the + # combined dict is keyed by every fetched spool. Treat as a benign skip + # rather than a deletion. (Mapped archived spools now reach the diff loop + # and are mirrored by the lifecycle pass; they no longer land here.) if dry_run: fdb_fil = fdb_filaments.get(mapping.filamentdb_filament_id) result.preview.append({ @@ -2659,7 +2686,7 @@ async def run_sync_cycle( "label": _preview_label(fdb_filament=fdb_fil), "field": None, "old": None, "new": None, - "reason": "Spoolman spool archived or not in active set", + "reason": "Spoolman spool not in active set", "spoolman_id": mapping.spoolman_spool_id, "fdb_filament_id": mapping.filamentdb_filament_id, "fdb_spool_id": mapping.filamentdb_spool_id, @@ -2670,7 +2697,7 @@ async def run_sync_cycle( spoolman_id=mapping.spoolman_spool_id, fdb_filament_id=mapping.filamentdb_filament_id, fdb_spool_id=mapping.filamentdb_spool_id, - error_message="SM spool not in active set (archived?)", + error_message="SM spool not in current fetch set", ) result.skipped += 1 continue @@ -2968,6 +2995,168 @@ async def run_sync_cycle( _upsert_snapshot(db, "spoolman", "spool", str(sm_spool.id), _sm_snapshot_dict(sm_spool, field_maps)) _upsert_snapshot(db, "filamentdb", "spool", fdb_spool.id, _fdb_snapshot_dict(fdb_spool)) + # ---- Lifecycle (archive/retire) sync ---- + # Runs AFTER the weight pass on purpose. A spool is usually archived/retired + # right as it hits ~0 g, so the final weight decrement (and its FDB usage-log + # audit entry) must settle first — otherwise the far side lands retired/archived + # carrying a stale weight and missing its final usage entry. + # + # The flags come from the changeset computed at the top of the pair (against the + # pre-cycle snapshot), so the weight pass refreshing the snapshot dicts above + # (which rebuild ``archived``/``retired`` from the live spool) does NOT clobber + # the detection — we already captured the flip in ``cs``. + lifecycle_sm_changed = cs.sm_archive_change is not None + lifecycle_fdb_changed = cs.fdb_retire_change is not None + + # Target converged boolean for whichever side we push to. + sm_archived_now = bool(sm_spool.archived) + fdb_retired_now = bool(fdb_spool.retired) + + # When BOTH sides changed but they landed on the SAME state (e.g. both archived + # in this cycle), there is no real divergence — converge silently. Only a both- + # changed-to-OPPOSITE-states case is a genuine conflict. The resolver only sees + # booleans, so collapse the agreeing-both-changed case to a NOOP up front. + both_changed_converged = ( + lifecycle_sm_changed and lifecycle_fdb_changed + and sm_archived_now == fdb_retired_now + ) + + if (lifecycle_sm_changed or lifecycle_fdb_changed) and not both_changed_converged: + # booleans → never timestamp-eligible (sm_ts/fdb_ts stay None). + lifecycle_action = resolve_sync_action( + sm_changed=lifecycle_sm_changed, + fdb_changed=lifecycle_fdb_changed, + direction=archive_direction, + policy=archive_policy, + ) + + if lifecycle_action == SyncAction.QUEUE_CONFLICT: + if not dry_run: + if not _has_open_conflict( + db, "spool", "lifecycle", + spoolman_id=sm_spool.id, + fdb_spool_id=fdb_spool.id, + conflict_type="cross_system", + ): + _queue_conflict( + db, cycle_id, "spool", "lifecycle", + spoolman_id=sm_spool.id, + fdb_filament_id=fdb_filament_id, + fdb_spool_id=fdb_spool.id, + spoolman_value=sm_archived_now, + filamentdb_value=fdb_retired_now, + conflict_type="cross_system", + ) + result.conflicts += 1 + else: + result.preview.append({ + "action": "conflict", + "entity_type": "spool", + "direction": None, + "label": _preview_label(sm_spool=sm_spool, fdb_filament=fdb_filaments.get(fdb_filament_id)), + "field": "lifecycle", + "old": sm_archived_now, + "new": fdb_retired_now, + "reason": "both sides changed archive/retire (old=SM archived, new=FDB retired)", + "spoolman_id": sm_spool.id, + "fdb_filament_id": fdb_filament_id, + "fdb_spool_id": fdb_spool.id, + }) + result.conflicts += 1 + + elif lifecycle_action == SyncAction.PUSH_SM_TO_FDB: + # SM archived state is authoritative → mirror to FDB.retired. + target = sm_archived_now + try: + if not dry_run: + await filamentdb.update_spool(fdb_filament_id, fdb_spool.id, {"retired": target}) + _refresh_lifecycle_snapshots(db, sm_spool.id, fdb_spool.id, target, target) + _log( + db, cycle_id, "spoolman_to_filamentdb", "update", "spool", + spoolman_id=sm_spool.id, fdb_filament_id=fdb_filament_id, + fdb_spool_id=fdb_spool.id, field_name="lifecycle", + old_value="retired" if fdb_retired_now else "live", + new_value=( + "retired in FDB (archived in Spoolman)" if target + else "live in FDB (un-archived in Spoolman)" + ), + ) + else: + result.preview.append({ + "action": "update", + "entity_type": "spool", + "direction": "spoolman_to_filamentdb", + "label": _preview_label(sm_spool=sm_spool, fdb_filament=fdb_filaments.get(fdb_filament_id)), + "field": "lifecycle", + "old": fdb_retired_now, "new": target, + "reason": "retired in FDB" if target else "un-retired in FDB", + "spoolman_id": sm_spool.id, + "fdb_filament_id": fdb_filament_id, + "fdb_spool_id": fdb_spool.id, + }) + result.updated += 1 + except Exception as exc: + logger.error("Cycle %s: SM→FDB lifecycle sync failed spool %s: %s", cycle_id, sm_spool.id, exc) + if not dry_run: + _log( + db, cycle_id, "spoolman_to_filamentdb", "error", "spool", + spoolman_id=sm_spool.id, fdb_filament_id=fdb_filament_id, + fdb_spool_id=fdb_spool.id, error_message=str(exc), + ) + result.errors += 1 + + elif lifecycle_action == SyncAction.PUSH_FDB_TO_SM: + # FDB retired state is authoritative → mirror to SM.archived. + target = fdb_retired_now + try: + if not dry_run: + await spoolman.update_spool(sm_spool.id, {"archived": target}) + _refresh_lifecycle_snapshots(db, sm_spool.id, fdb_spool.id, target, target) + _log( + db, cycle_id, "filamentdb_to_spoolman", "update", "spool", + spoolman_id=sm_spool.id, fdb_filament_id=fdb_filament_id, + fdb_spool_id=fdb_spool.id, field_name="lifecycle", + old_value="archived" if sm_archived_now else "active", + new_value=( + "archived in Spoolman (retired in FDB)" if target + else "active in Spoolman (un-retired in FDB)" + ), + ) + else: + result.preview.append({ + "action": "update", + "entity_type": "spool", + "direction": "filamentdb_to_spoolman", + "label": _preview_label(sm_spool=sm_spool, fdb_filament=fdb_filaments.get(fdb_filament_id)), + "field": "lifecycle", + "old": sm_archived_now, "new": target, + "reason": "archived in Spoolman" if target else "un-archived in Spoolman", + "spoolman_id": sm_spool.id, + "fdb_filament_id": fdb_filament_id, + "fdb_spool_id": fdb_spool.id, + }) + result.updated += 1 + except Exception as exc: + logger.error("Cycle %s: FDB→SM lifecycle sync failed spool %s: %s", cycle_id, sm_spool.id, exc) + if not dry_run: + _log( + db, cycle_id, "filamentdb_to_spoolman", "error", "spool", + spoolman_id=sm_spool.id, fdb_filament_id=fdb_filament_id, + fdb_spool_id=fdb_spool.id, error_message=str(exc), + ) + result.errors += 1 + + else: + # NOOP (e.g. a locked one-way destination drifted) — converge both + # snapshot lifecycle bits so the change is not re-detected next cycle. + if not dry_run: + _refresh_lifecycle_snapshots(db, sm_spool.id, fdb_spool.id, sm_archived_now, fdb_retired_now) + + elif both_changed_converged and not dry_run: + # Both sides flipped to the same state in one cycle — no write needed, just + # converge the snapshot lifecycle bits so it doesn't re-fire next cycle. + _refresh_lifecycle_snapshots(db, sm_spool.id, fdb_spool.id, sm_archived_now, fdb_retired_now) + # ---- Field mapping sync (FR-11) ---- if field_maps: fm_for_spool = filament_mappings_by_sm.get(sm_spool.filament.id) diff --git a/backend/app/models/config.py b/backend/app/models/config.py index 137c6c8..653751e 100644 --- a/backend/app/models/config.py +++ b/backend/app/models/config.py @@ -18,6 +18,12 @@ "weight_conflict_policy": '"manual"', "material_properties_sync_direction": '"filamentdb_to_spoolman"', "material_properties_conflict_policy": '"manual"', + # Archive/retire lifecycle sync for already-mapped spool pairs. + # two_way mirrors a one-sided archive/retire flip to the other system; a genuine + # both-sides-diverge flip queues a manual cross_system conflict. newest_wins is + # NOT applicable (booleans aren't timestamp-eligible) and is rejected at the API. + "archive_sync_direction": '"two_way"', + "archive_conflict_policy": '"manual"', # New spool creation direction: two_way = bidirectional (= today's behavior). "new_spool_sync_direction": '"two_way"', # New-record handling policies (ongoing sync, not wizard). diff --git a/backend/app/schemas/api.py b/backend/app/schemas/api.py index 35e3e85..99a04cd 100644 --- a/backend/app/schemas/api.py +++ b/backend/app/schemas/api.py @@ -290,6 +290,10 @@ class ConfigResponse(BaseModel): weight_conflict_policy: ConflictPolicy = "manual" material_properties_sync_direction: SyncDirection2 = "filamentdb_to_spoolman" material_properties_conflict_policy: ConflictPolicy = "manual" + # Archive/retire lifecycle sync (mirrors SM archived ↔ FDB retired for mapped pairs). + # newest_wins is rejected (booleans aren't timestamp-eligible) — same as material_properties. + archive_sync_direction: SyncDirection2 = "two_way" + archive_conflict_policy: ConflictPolicy = "manual" new_spool_sync_direction: SyncDirection2 = "two_way" # New-record handling policies # manual_review (default) → queue an actionable conflict; auto_import → create immediately. @@ -327,6 +331,9 @@ class ConfigUpdateRequest(BaseModel): weight_conflict_policy: ConflictPolicy | None = None material_properties_sync_direction: SyncDirection2 | None = None material_properties_conflict_policy: ConflictPolicy | None = None + # Archive/retire lifecycle sync. + archive_sync_direction: SyncDirection2 | None = None + archive_conflict_policy: ConflictPolicy | None = None new_spool_sync_direction: SyncDirection2 | None = None # New-record handling policies new_filament_policy: NewRecordPolicy | None = None diff --git a/backend/tests/test_api.py b/backend/tests/test_api.py index eda562d..0cc10a5 100644 --- a/backend/tests/test_api.py +++ b/backend/tests/test_api.py @@ -161,7 +161,8 @@ def test_dry_run_returns_preview_and_applies_nothing(db): # Pair 2: no snapshots (first baseline) → skip entry db.add(SpoolMapping(spoolman_spool_id=2, filamentdb_filament_id="fil-2", filamentdb_spool_id="spool-2")) - # Pair 3: SM spool not in active set (archived) → skip entry + # Pair 3: archived SM spool whose mapped FDB spool no longer exists and whose + # cross-ref is cleared → stale-connection skip entry (no live link to protect). db.add(SpoolMapping(spoolman_spool_id=99, filamentdb_filament_id="fil-1", filamentdb_spool_id="spool-99")) # Pair 4: both weights changed → conflict entry @@ -185,7 +186,7 @@ def test_dry_run_returns_preview_and_applies_nothing(db): _sm_spool(1, 795.0), _sm_spool(2, 500.0), _sm_spool(3, 850.0), # changed from 900 snapshot → weight conflict - archived, # sm_id=99 archived (in sm_all_ids but not active) → skip + archived, # sm_id=99 archived; mapped FDB spool gone, cross-ref empty → stale skip ]) filamentdb = _fake_filamentdb(filaments=[ _fdb_filament("fil-1", "spool-1", 1000.0), @@ -199,7 +200,7 @@ def test_dry_run_returns_preview_and_applies_nothing(db): body = resp.json() assert body["dry_run"] is True assert body["updated"] == 1 - assert body["skipped"] == 2 # archived (pair 3) + first-baseline (pair 2) + assert body["skipped"] == 2 # stale-connection (pair 3) + first-baseline (pair 2) assert body["conflicts"] == 1 # weight conflict (pair 4) assert len(body["preview"]) >= 4 @@ -209,10 +210,10 @@ def test_dry_run_returns_preview_and_applies_nothing(db): assert entry["action"] in valid_actions, f"unexpected action: {entry}" assert entry.get("label"), f"missing label in {entry}" - # Skip entry for archived spool (sm_id=99 not in active set). + # Skip entry for the archived spool whose FDB counterpart is gone (stale connection). archived_skips = [p for p in body["preview"] if p["action"] == "skip" and p.get("spoolman_id") == 99] assert len(archived_skips) == 1 - assert "archived" in archived_skips[0]["reason"].lower() + assert "stale" in archived_skips[0]["reason"].lower() # Skip entry for first-baseline pair (sm_id=2, no prior snapshot). baseline_skips = [p for p in body["preview"] if p["action"] == "skip" and p.get("spoolman_id") == 2] diff --git a/backend/tests/test_conflict_apply.py b/backend/tests/test_conflict_apply.py index cc09a89..523c2de 100644 --- a/backend/tests/test_conflict_apply.py +++ b/backend/tests/test_conflict_apply.py @@ -770,3 +770,84 @@ def test_api_divergence_context_returns_correct_shape(): assert v["fdb_id"] == VARIANT_FDB_ID assert v["spoolman_filament_id"] == SM_FIL_ID assert v["inherited"] is True + + +# --------------------------------------------------------------------------- +# Lifecycle (archive/retire) cross_system conflict resolution +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_apply_lifecycle_conflict_writes_both_sides_and_converges(): + """Resolving a lifecycle cross_system conflict writes the chosen boolean to BOTH + systems and refreshes both snapshots so it does not re-queue.""" + from app.core.conflict_apply import apply_lifecycle_conflict + + db = _make_db() + # Diverged: SM archived=true, FDB retired=false. + c = Conflict( + entity_type="spool", + field_name="lifecycle", + conflict_type="cross_system", + spoolman_id=7, + filamentdb_filament_id="fil-7", + filamentdb_spool_id="spool-7", + spoolman_value=json.dumps(True), + filamentdb_value=json.dumps(False), + ) + db.add(c) + # Seed snapshots in the diverged state. + db.add(Snapshot(source="spoolman", entity_type="spool", entity_id="7", + data=json.dumps({"remaining_weight": 0.0, "archived": True}))) + db.add(Snapshot(source="filamentdb", entity_type="spool", entity_id="spool-7", + data=json.dumps({"totalWeight": 200.0, "retired": False}))) + db.commit() + + spoolman = AsyncMock() + spoolman.update_spool = AsyncMock(return_value=MagicMock()) + filamentdb = AsyncMock() + filamentdb.update_spool = AsyncMock(return_value={}) + + # Resolution "spoolman" → adopt SM archived (True) on both sides. + await apply_lifecycle_conflict(c, "spoolman", db, spoolman, filamentdb) + db.commit() + + spoolman.update_spool.assert_awaited_once_with(7, {"archived": True}) + filamentdb.update_spool.assert_awaited_once_with("fil-7", "spool-7", {"retired": True}) + assert c.resolved_at is not None + assert json.loads(c.resolved_value) is True + sm_snap = db.query(Snapshot).filter_by(source="spoolman", entity_type="spool", entity_id="7").first() + fdb_snap = db.query(Snapshot).filter_by(source="filamentdb", entity_type="spool", entity_id="spool-7").first() + assert json.loads(sm_snap.data)["archived"] is True + assert json.loads(fdb_snap.data)["retired"] is True + + +@pytest.mark.asyncio +async def test_apply_lifecycle_conflict_filamentdb_resolution(): + """Resolution 'filamentdb' adopts the FDB retired state on both sides.""" + from app.core.conflict_apply import apply_lifecycle_conflict + + db = _make_db() + c = Conflict( + entity_type="spool", field_name="lifecycle", conflict_type="cross_system", + spoolman_id=8, filamentdb_filament_id="fil-8", filamentdb_spool_id="spool-8", + spoolman_value=json.dumps(True), filamentdb_value=json.dumps(False), + ) + db.add(c) + db.add(Snapshot(source="spoolman", entity_type="spool", entity_id="8", + data=json.dumps({"archived": True}))) + db.add(Snapshot(source="filamentdb", entity_type="spool", entity_id="spool-8", + data=json.dumps({"retired": False}))) + db.commit() + + spoolman = AsyncMock() + spoolman.update_spool = AsyncMock(return_value=MagicMock()) + filamentdb = AsyncMock() + filamentdb.update_spool = AsyncMock(return_value={}) + + await apply_lifecycle_conflict(c, "filamentdb", db, spoolman, filamentdb) + db.commit() + + spoolman.update_spool.assert_awaited_once_with(8, {"archived": False}) + filamentdb.update_spool.assert_awaited_once_with("fil-8", "spool-8", {"retired": False}) + assert json.loads(c.resolved_value) is False diff --git a/backend/tests/test_differ.py b/backend/tests/test_differ.py index 5e00441..dea2bec 100644 --- a/backend/tests/test_differ.py +++ b/backend/tests/test_differ.py @@ -258,3 +258,77 @@ def test_snapshot_pla_silk_vs_current_pla_silk_and_fdb_pla_no_flap(self): assert not cs.sm_field_changes assert not cs.fdb_field_changes assert not cs.field_conflicts + + +def _sm_spool_arch(spool_id: int, remaining: float, archived: bool) -> SpoolmanSpool: + return SpoolmanSpool( + id=spool_id, + filament=SpoolmanFilament(id=1, name="PLA"), + remaining_weight=remaining, + archived=archived, + ) + + +def _fdb_spool_ret(spool_id: str, total: float, retired: bool) -> FDBSpool: + return FDBSpool(**{"_id": spool_id, "totalWeight": total, "retired": retired}) + + +class TestLifecycleDiff: + def test_no_lifecycle_change(self): + cs = diff_spool_pair( + _sm_spool_arch(1, 800.0, False), _fdb_spool_ret("a", 1000.0, False), "fdb-fil-1", + sm_snapshot={"remaining_weight": 800.0, "archived": False}, + fdb_snapshot={"totalWeight": 1000.0, "retired": False}, + threshold=THRESHOLD, + ) + assert cs.sm_archive_change is None + assert cs.fdb_retire_change is None + assert not cs.archive_conflict + + def test_sm_archive_flip_detected(self): + cs = diff_spool_pair( + _sm_spool_arch(1, 800.0, True), _fdb_spool_ret("a", 1000.0, False), "fdb-fil-1", + sm_snapshot={"remaining_weight": 800.0, "archived": False}, + fdb_snapshot={"totalWeight": 1000.0, "retired": False}, + threshold=THRESHOLD, + ) + assert cs.sm_archive_change is not None + assert cs.sm_archive_change.old_value is False + assert cs.sm_archive_change.new_value is True + assert cs.fdb_retire_change is None + assert not cs.archive_conflict + + def test_fdb_retire_flip_detected(self): + cs = diff_spool_pair( + _sm_spool_arch(1, 800.0, False), _fdb_spool_ret("a", 1000.0, True), "fdb-fil-1", + sm_snapshot={"remaining_weight": 800.0, "archived": False}, + fdb_snapshot={"totalWeight": 1000.0, "retired": False}, + threshold=THRESHOLD, + ) + assert cs.fdb_retire_change is not None + assert cs.fdb_retire_change.new_value is True + assert cs.sm_archive_change is None + assert not cs.archive_conflict + + def test_both_flip_is_archive_conflict(self): + cs = diff_spool_pair( + _sm_spool_arch(1, 800.0, True), _fdb_spool_ret("a", 1000.0, False), "fdb-fil-1", + sm_snapshot={"remaining_weight": 800.0, "archived": False}, + fdb_snapshot={"totalWeight": 1000.0, "retired": True}, + threshold=THRESHOLD, + ) + assert cs.archive_conflict is True + assert cs.sm_archive_change is not None + assert cs.fdb_retire_change is not None + + def test_missing_baseline_not_treated_as_flip(self): + # Legacy snapshot with no archived/retired keys → defaults to current → no flip. + cs = diff_spool_pair( + _sm_spool_arch(1, 800.0, True), _fdb_spool_ret("a", 1000.0, True), "fdb-fil-1", + sm_snapshot={"remaining_weight": 800.0}, + fdb_snapshot={"totalWeight": 1000.0}, + threshold=THRESHOLD, + ) + assert cs.sm_archive_change is None + assert cs.fdb_retire_change is None + assert not cs.archive_conflict diff --git a/backend/tests/test_engine.py b/backend/tests/test_engine.py index fa51c6c..6ad688f 100644 --- a/backend/tests/test_engine.py +++ b/backend/tests/test_engine.py @@ -110,6 +110,34 @@ def _seed_matprop_config(db, direction: str = "filamentdb_to_spoolman", policy: db.commit() +def _seed_archive_config(db, direction: str = "two_way", policy: str = "manual"): + """Set archive/retire lifecycle sync direction and conflict policy in BridgeConfig.""" + from app.models.config import BridgeConfig + db.merge(BridgeConfig(key="archive_sync_direction", value=json.dumps(direction))) + db.merge(BridgeConfig(key="archive_conflict_policy", value=json.dumps(policy))) + db.commit() + + +def _sm_spool_arch(spool_id: int, remaining: float, archived: bool, extra: dict | None = None) -> SpoolmanSpool: + return SpoolmanSpool( + id=spool_id, + filament=SpoolmanFilament(id=10, name="PLA", vendor=SpoolmanVendor(id=1, name="ELEGOO")), + remaining_weight=remaining, + archived=archived, + extra=extra or {}, + ) + + +def _fdb_filament_ret(fid: str, spool_id: str, total_weight: float, retired: bool, tare: float = 200.0) -> FDBFilament: + return FDBFilament.model_validate({ + "_id": fid, + "name": "PLA", + "vendor": "elegoo", + "spoolWeight": tare, + "spools": [{"_id": spool_id, "totalWeight": total_weight, "retired": retired}], + }) + + # --------------------------------------------------------------------------- # Tests # --------------------------------------------------------------------------- @@ -238,9 +266,10 @@ async def test_archived_imported_spool_no_pingpong(db): """An imported archived→retired spool must not be re-animated or bounced. Mirrors the wizard importing SM spool #65 (archived, used up) as a RETIRED FDB - spool with a SpoolMapping. The mapped-pair loop must hit the archived-skip branch - (engine.py ~2380) every cycle: no weight diff, no usage log, no FDB→SM decrement, - no conflict, no duplicate-spool creation on either side — across multiple cycles. + spool with a SpoolMapping. Both sides are already in the dead state, so the + mapped-pair loop must NOOP every cycle: no weight diff, no usage log, no FDB→SM + decrement, no lifecycle conflict, no lifecycle mirror, no duplicate-spool creation + on either side — across multiple cycles. """ sm_spool = SpoolmanSpool( id=65, @@ -270,12 +299,11 @@ async def test_archived_imported_spool_no_pingpong(db): r1 = await run_sync_cycle(db, spoolman, fdb_client, dry_run=False, cycle_id="arch-c1") r2 = await run_sync_cycle(db, spoolman, fdb_client, dry_run=False, cycle_id="arch-c2") - # No spurious work on either cycle — the archived pair is skipped, not synced. + # No spurious work on either cycle — both sides already dead → NOOP, not synced. for r in (r1, r2): assert r.updated == 0, f"archived pair must not update; got {r.updated}" assert r.conflicts == 0, f"archived pair must not conflict; got {r.conflicts}" assert r.errors == 0 - assert r.skipped >= 1, "archived pair must hit the skip branch" # No writes to either system for the archived/retired spool. fdb_client.log_usage.assert_not_called() fdb_client.update_spool.assert_not_called() @@ -738,8 +766,10 @@ async def test_sm_deletion_queues_conflict(db): @pytest.mark.asyncio -async def test_archived_sm_spool_logs_skip_no_conflict(db): - """Archived Spoolman spool is in sm_all_ids → skip logged, no deletion conflict.""" +async def test_archived_sm_spool_with_unknown_baseline_is_noop(db): + """Archived mapped spool whose snapshot lacks an 'archived' baseline must NOT be + treated as a fresh flip — a missing baseline defaults to the current value so the + pair is a clean NOOP (no conflict, no spurious mirror, no skip-as-deletion).""" archived_spool = SpoolmanSpool( id=1, filament=SpoolmanFilament(id=10, name="PLA", vendor=SpoolmanVendor(id=1, name="ELEGOO")), @@ -749,19 +779,25 @@ async def test_archived_sm_spool_logs_skip_no_conflict(db): ) fdb_fil = _fdb_filament("fil-1", "spool-1", 1000.0) _add_spool_mapping(db, 1, "fil-1", "spool-1") + # Legacy snapshots: no 'archived'/'retired' keys, weights already in agreement. _store_snapshot(db, "spoolman", "spool", "1", {"remaining_weight": 800.0}) _store_snapshot(db, "filamentdb", "spool", "spool-1", {"totalWeight": 1000.0}) + _seed_weight_config(db, direction="two_way", policy="manual") spoolman = _fake_spoolman(spools=[archived_spool]) fdb_client = _fake_filamentdb(filaments=[fdb_fil]) - result = await run_sync_cycle(db, spoolman, fdb_client, dry_run=False, cycle_id=CYCLE_ID) + with patch("app.core.engine._settings") as ms: + _patch_settings(ms) + result = await run_sync_cycle(db, spoolman, fdb_client, dry_run=False, cycle_id=CYCLE_ID) assert result.conflicts == 0 - assert result.skipped == 1 + assert result.updated == 0 + assert result.errors == 0 assert db.query(Conflict).count() == 0 - skip_log = db.query(SyncLog).filter_by(action="skip").first() - assert skip_log is not None + # No upstream writes for either system. + fdb_client.update_spool.assert_not_called() + spoolman.update_spool.assert_not_called() # --------------------------------------------------------------------------- @@ -2926,3 +2962,267 @@ async def test_variant_member_unset_mode_held_for_review(db): fdb_client.create_filament.assert_not_called() c = db.query(Conflict).filter_by(entity_type="filament", field_name="new_filament", spoolman_id=70).first() assert c is not None, "Expected new_filament conflict when variant_parent_mode=unset" + + +# --------------------------------------------------------------------------- +# Archive / retire lifecycle sync (FR-21 symmetric) +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_lifecycle_sm_archive_pushes_fdb_retire(db): + """Mapped spool flips archived=true in SM (FDB unchanged) → engine sets FDB + retired=true, both snapshots converge, no ping-pong next cycle, no conflict.""" + sm = _sm_spool_arch(1, 800.0, archived=True) + fdb = _fdb_filament_ret("fil-1", "spool-1", 1000.0, retired=False) + _add_spool_mapping(db, 1, "fil-1", "spool-1") + _store_snapshot(db, "spoolman", "spool", "1", {"remaining_weight": 800.0, "archived": False}) + _store_snapshot(db, "filamentdb", "spool", "spool-1", {"totalWeight": 1000.0, "retired": False}) + _seed_weight_config(db, direction="two_way", policy="manual") + _seed_archive_config(db, direction="two_way", policy="manual") + + spoolman = _fake_spoolman(spools=[sm]) + fdb_client = _fake_filamentdb(filaments=[fdb]) + + with patch("app.core.engine._settings") as ms: + _patch_settings(ms) + r1 = await run_sync_cycle(db, spoolman, fdb_client, dry_run=False, cycle_id="lc-1") + + assert r1.conflicts == 0 + assert r1.updated == 1 + assert fdb_client.update_spool.call_count == 1 + assert fdb_client.update_spool.call_args.args[2] == {"retired": True} + # Snapshots converged. + assert _snap_value(db, "spoolman", "spool", "1", "archived") is True + assert _snap_value(db, "filamentdb", "spool", "spool-1", "retired") is True + + # No ping-pong: FDB now retired, SM still archived, snapshots converged. + fdb_after = _fdb_filament_ret("fil-1", "spool-1", 1000.0, retired=True) + fdb_client2 = _fake_filamentdb(filaments=[fdb_after]) + with patch("app.core.engine._settings") as ms: + _patch_settings(ms) + r2 = await run_sync_cycle(db, spoolman, fdb_client2, dry_run=False, cycle_id="lc-1b") + assert r2.updated == 0 + assert r2.conflicts == 0 + fdb_client2.update_spool.assert_not_called() + spoolman.update_spool.assert_not_called() + + +@pytest.mark.asyncio +async def test_lifecycle_fdb_retire_pushes_sm_archive(db): + """Mapped spool flips retired=true in FDB (SM unchanged) → engine sets SM + archived=true, converges, no ping-pong.""" + sm = _sm_spool_arch(1, 800.0, archived=False) + fdb = _fdb_filament_ret("fil-1", "spool-1", 1000.0, retired=True) + _add_spool_mapping(db, 1, "fil-1", "spool-1") + _store_snapshot(db, "spoolman", "spool", "1", {"remaining_weight": 800.0, "archived": False}) + _store_snapshot(db, "filamentdb", "spool", "spool-1", {"totalWeight": 1000.0, "retired": False}) + _seed_weight_config(db, direction="two_way", policy="manual") + _seed_archive_config(db, direction="two_way", policy="manual") + + spoolman = _fake_spoolman(spools=[sm]) + fdb_client = _fake_filamentdb(filaments=[fdb]) + + with patch("app.core.engine._settings") as ms: + _patch_settings(ms) + r1 = await run_sync_cycle(db, spoolman, fdb_client, dry_run=False, cycle_id="lc-2") + + assert r1.conflicts == 0 + assert r1.updated == 1 + assert spoolman.update_spool.call_count == 1 + assert spoolman.update_spool.call_args.args[1] == {"archived": True} + assert _snap_value(db, "spoolman", "spool", "1", "archived") is True + assert _snap_value(db, "filamentdb", "spool", "spool-1", "retired") is True + + # No ping-pong next cycle. + sm_after = _sm_spool_arch(1, 800.0, archived=True) + spoolman2 = _fake_spoolman(spools=[sm_after]) + with patch("app.core.engine._settings") as ms: + _patch_settings(ms) + r2 = await run_sync_cycle(db, spoolman2, fdb_client, dry_run=False, cycle_id="lc-2b") + assert r2.updated == 0 + assert r2.conflicts == 0 + spoolman2.update_spool.assert_not_called() + fdb_client.update_spool.assert_not_called() + + +@pytest.mark.asyncio +async def test_lifecycle_depletion_and_archive_same_cycle_weight_first(db): + """THE ORDERING GUARANTEE: a mapped spool's remaining drops to ~0 g AND archived + flips true in the same cycle → the engine logs the final usage decrement in FDB + FIRST (correct post-decrement totalWeight + usage entry source=spoolman), THEN sets + retired=true. FDB ends retired with the decremented weight and the usage entry.""" + # SM dropped 800 → 0 (used 800) and archived in the same cycle. + sm = _sm_spool_arch(1, 0.0, archived=True) + # FDB still shows the pre-decrement gross (1000 = 800 net + 200 tare), not retired. + fdb = _fdb_filament_ret("fil-1", "spool-1", 1000.0, retired=False, tare=200.0) + _add_spool_mapping(db, 1, "fil-1", "spool-1") + _store_snapshot(db, "spoolman", "spool", "1", {"remaining_weight": 800.0, "archived": False}) + _store_snapshot(db, "filamentdb", "spool", "spool-1", {"totalWeight": 1000.0, "retired": False}) + _seed_weight_config(db, direction="two_way", policy="manual") + _seed_archive_config(db, direction="two_way", policy="manual") + + spoolman = _fake_spoolman(spools=[sm]) + fdb_client = _fake_filamentdb(filaments=[fdb]) + + with patch("app.core.engine._settings") as ms: + _patch_settings(ms) + r = await run_sync_cycle(db, spoolman, fdb_client, dry_run=False, cycle_id="lc-dep") + + assert r.conflicts == 0 + assert r.errors == 0 + # Weight pass logged the final usage decrement of 800 g with source=spoolman. + assert fdb_client.log_usage.call_count == 1 + usage_args, usage_kwargs = fdb_client.log_usage.call_args + # positional: (filament_id, spool_id, grams) + assert usage_args[2] == pytest.approx(800.0) + assert usage_kwargs.get("source") == "spoolman" + # Lifecycle pass set retired=true (separate update_spool call). + retired_calls = [c for c in fdb_client.update_spool.call_args_list if c.args[2] == {"retired": True}] + assert len(retired_calls) == 1, "lifecycle must set retired=true after weight settles" + # FDB snapshot ends with the decremented gross weight AND retired=true. + assert _snap_value(db, "filamentdb", "spool", "spool-1", "totalWeight") == pytest.approx(200.0) + assert _snap_value(db, "filamentdb", "spool", "spool-1", "retired") is True + assert _snap_value(db, "spoolman", "spool", "1", "archived") is True + # SM snapshot weight converged too (no stale weight that would re-fire next cycle). + assert _snap_value(db, "spoolman", "spool", "1", "remaining_weight") == pytest.approx(0.0) + + +@pytest.mark.asyncio +async def test_lifecycle_unarchive_mirrors_back(db): + """Un-archive (true→false) in SM mirrors to FDB retired=false and re-enables sync.""" + sm = _sm_spool_arch(1, 800.0, archived=False) + fdb = _fdb_filament_ret("fil-1", "spool-1", 1000.0, retired=True) + _add_spool_mapping(db, 1, "fil-1", "spool-1") + _store_snapshot(db, "spoolman", "spool", "1", {"remaining_weight": 800.0, "archived": True}) + _store_snapshot(db, "filamentdb", "spool", "spool-1", {"totalWeight": 1000.0, "retired": True}) + _seed_weight_config(db, direction="two_way", policy="manual") + _seed_archive_config(db, direction="two_way", policy="manual") + + spoolman = _fake_spoolman(spools=[sm]) + fdb_client = _fake_filamentdb(filaments=[fdb]) + + with patch("app.core.engine._settings") as ms: + _patch_settings(ms) + r = await run_sync_cycle(db, spoolman, fdb_client, dry_run=False, cycle_id="lc-un") + + assert r.conflicts == 0 + assert fdb_client.update_spool.call_args.args[2] == {"retired": False} + assert _snap_value(db, "filamentdb", "spool", "spool-1", "retired") is False + assert _snap_value(db, "spoolman", "spool", "1", "archived") is False + + +@pytest.mark.asyncio +async def test_lifecycle_both_flip_same_state_noop(db): + """Both sides flip to the SAME dead state in one cycle → NOOP, snapshots converge, + no conflict, no writes.""" + sm = _sm_spool_arch(1, 800.0, archived=True) + fdb = _fdb_filament_ret("fil-1", "spool-1", 1000.0, retired=True) + _add_spool_mapping(db, 1, "fil-1", "spool-1") + _store_snapshot(db, "spoolman", "spool", "1", {"remaining_weight": 800.0, "archived": False}) + _store_snapshot(db, "filamentdb", "spool", "spool-1", {"totalWeight": 1000.0, "retired": False}) + _seed_weight_config(db, direction="two_way", policy="manual") + _seed_archive_config(db, direction="two_way", policy="manual") + + spoolman = _fake_spoolman(spools=[sm]) + fdb_client = _fake_filamentdb(filaments=[fdb]) + + with patch("app.core.engine._settings") as ms: + _patch_settings(ms) + r = await run_sync_cycle(db, spoolman, fdb_client, dry_run=False, cycle_id="lc-both") + + assert r.conflicts == 0 + assert r.errors == 0 + fdb_client.update_spool.assert_not_called() + spoolman.update_spool.assert_not_called() + assert _snap_value(db, "spoolman", "spool", "1", "archived") is True + assert _snap_value(db, "filamentdb", "spool", "spool-1", "retired") is True + + +@pytest.mark.asyncio +async def test_lifecycle_divergence_queues_cross_system_conflict(db): + """Genuine divergence (SM archives, FDB un-retires) with policy=manual → one + cross_system lifecycle conflict queued; no writes; no re-queue next cycle.""" + sm = _sm_spool_arch(1, 800.0, archived=True) # flipped false→true + fdb = _fdb_filament_ret("fil-1", "spool-1", 1000.0, retired=False) # flipped true→false + _add_spool_mapping(db, 1, "fil-1", "spool-1") + _store_snapshot(db, "spoolman", "spool", "1", {"remaining_weight": 800.0, "archived": False}) + _store_snapshot(db, "filamentdb", "spool", "spool-1", {"totalWeight": 1000.0, "retired": True}) + _seed_weight_config(db, direction="two_way", policy="manual") + _seed_archive_config(db, direction="two_way", policy="manual") + + spoolman = _fake_spoolman(spools=[sm]) + fdb_client = _fake_filamentdb(filaments=[fdb]) + + with patch("app.core.engine._settings") as ms: + _patch_settings(ms) + r1 = await run_sync_cycle(db, spoolman, fdb_client, dry_run=False, cycle_id="lc-div") + + assert r1.conflicts == 1 + fdb_client.update_spool.assert_not_called() + spoolman.update_spool.assert_not_called() + conflict = db.query(Conflict).filter_by(entity_type="spool", field_name="lifecycle").first() + assert conflict is not None + assert conflict.conflict_type == "cross_system" + + # No re-queue next cycle while the conflict is open. + with patch("app.core.engine._settings") as ms: + _patch_settings(ms) + r2 = await run_sync_cycle(db, spoolman, fdb_client, dry_run=False, cycle_id="lc-div2") + assert r2.conflicts == 0 + assert db.query(Conflict).filter_by(entity_type="spool", field_name="lifecycle").count() == 1 + + +@pytest.mark.asyncio +async def test_lifecycle_one_way_sm_to_fdb_ignores_fdb_flip(db): + """direction=spoolman_to_filamentdb → an FDB-side retire flip does NOT write SM.""" + sm = _sm_spool_arch(1, 800.0, archived=False) + fdb = _fdb_filament_ret("fil-1", "spool-1", 1000.0, retired=True) # FDB flipped + _add_spool_mapping(db, 1, "fil-1", "spool-1") + _store_snapshot(db, "spoolman", "spool", "1", {"remaining_weight": 800.0, "archived": False}) + _store_snapshot(db, "filamentdb", "spool", "spool-1", {"totalWeight": 1000.0, "retired": False}) + _seed_weight_config(db, direction="two_way", policy="manual") + _seed_archive_config(db, direction="spoolman_to_filamentdb", policy="manual") + + spoolman = _fake_spoolman(spools=[sm]) + fdb_client = _fake_filamentdb(filaments=[fdb]) + + with patch("app.core.engine._settings") as ms: + _patch_settings(ms) + r = await run_sync_cycle(db, spoolman, fdb_client, dry_run=False, cycle_id="lc-1way") + + assert r.conflicts == 0 + spoolman.update_spool.assert_not_called() # SM never written under SM→FDB lock + # FDB-side drift snapshot converges (locked destination) so it doesn't re-fire. + assert _snap_value(db, "filamentdb", "spool", "spool-1", "retired") is True + + +@pytest.mark.asyncio +async def test_lifecycle_unmapped_archived_spool_not_imported(db): + """An UNMAPPED archived spool on an already-mapped filament during ongoing sync is + still NOT imported, even with new_spool_policy=auto_import (import gate preserved).""" + # SM filament 10 is already mapped to FDB fil-9; spool 500 on it is unmapped + archived. + # fil-9 has no spools so there is no FDB→SM new-spool noise to confound the assertion. + archived = _sm_spool_arch(500, 0.0, archived=True) + fdb = FDBFilament.model_validate({ + "_id": "fil-9", "name": "PLA", "vendor": "elegoo", "spoolWeight": 200.0, "spools": [], + }) + db.add(FilamentMapping(spoolman_filament_id=10, filamentdb_id="fil-9")) + db.flush() + _seed_weight_config(db, direction="two_way", policy="manual") + _seed_archive_config(db, direction="two_way", policy="manual") + from app.api.config import set_config_value + set_config_value(db, "new_spool_policy", "auto_import") + db.commit() + + spoolman = _fake_spoolman(spools=[archived]) + fdb_client = _fake_filamentdb(filaments=[fdb]) + + with patch("app.core.engine._settings") as ms: + _patch_settings(ms) + r = await run_sync_cycle(db, spoolman, fdb_client, dry_run=False, cycle_id="lc-unmapped") + + # The archived unmapped spool must not be created in FDB (still excluded from + # the active-only new-spool detection set). + fdb_client.create_spool.assert_not_called() + assert r.created == 0 diff --git a/docs/configuration.md b/docs/configuration.md index ac3d8f0..ca37f06 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -45,7 +45,7 @@ privileges via `gosu`. No manual `chown` is ever needed — pre-existing root-ow ### Two-axis sync model Each data category is configured independently on two axes (Settings → Weight sync / -Material properties sync / New spools): +Material properties sync / Archive / retire sync / New spools): - **Sync direction** — which side may write to the other: - `filamentdb_to_spoolman` — Filament DB is authoritative; changes flow to Spoolman only @@ -61,11 +61,19 @@ Material properties sync / New spools): winner is indeterminate, the conflict falls back to `manual`. Defaults: weight syncs `spoolman_to_filamentdb`; material properties sync -`filamentdb_to_spoolman`; new-spool creation is `two_way` (direction only — there is no -conflict policy for creation). +`filamentdb_to_spoolman`; archive/retire syncs `two_way`; new-spool creation is `two_way` +(direction only — there is no conflict policy for creation). Under a one-way direction, drift on the locked side is ignored (NOOP), not reverted. +The **archive / retire** category (`archive_sync_direction` / `archive_conflict_policy`) +mirrors a *mapped* spool's lifecycle state between Spoolman (`archived`) and Filament DB +(`retired`) — see [sync-model.md](sync-model.md) for the pass. Its state is a boolean, so +`newest_wins` is **rejected** (422) for `archive_conflict_policy` — there is no comparable +timestamp; use `manual` (default), `spoolman_wins`, or `filamentdb_wins`. A one-sided flip +is a clean push; only a both-sides-diverge-to-opposite-states case consults the policy. +This is independent of `never_import_empties` (below), which only governs *import*. + ## Cross-reference fields These control which fields the bridge uses to store cross-reference IDs. Change them only @@ -132,12 +140,14 @@ Stored in SQLite (`BridgeConfig`); changes take effect without a restart. | Auto-sync enabled | `false` | Sync | Master switch for scheduled sync. Enabling requires a completed wizard and is gated behind the backup dialog. | | `sync_interval_seconds` | env (`120`) | Sync | Auto-sync interval; rescheduled immediately on save (min 30 s). | | `sync_log_retention_days` | `30` | Sync | Sync-log rows older than this are pruned at the start of each auto-sync tick. `0` = keep forever. | -| Weight / material-properties / new-record direction + policy | see above | Sync → category cards | The two-axis model. | +| Weight / material-properties / archive-retire / new-record direction + policy | see above | Sync → category cards | The two-axis model. | +| `archive_sync_direction` | `two_way` | Sync → Archive / retire sync | Which side's archive/retire flip is mirrored: `two_way` (default), `spoolman_to_filamentdb`, or `filamentdb_to_spoolman`. Applies to mapped pairs only. | +| `archive_conflict_policy` | `manual` | Sync → Archive / retire sync | Consulted only under `two_way` when both sides diverge to opposite states: `manual` (default — queue a `cross_system` lifecycle conflict), `spoolman_wins`, or `filamentdb_wins`. `newest_wins` is **rejected** (422) — the state is a boolean with no timestamp. | | `sync_weight_threshold_grams` | `2.0` | Sync → Weight sync | Weight changes smaller than this are ignored (suppresses net/gross rounding churn). | | `weight_precision_decimals` | `2` | Sync → Weight sync | Decimal places used when comparing/writing weights. | | `new_filament_policy` | `manual_review` | Sync → New records | What the engine does when it detects an unmapped filament: `manual_review` queues a `new_filament` conflict (actionable — the Conflicts page "Add" button imports it); `auto_import` creates the filament automatically and writes the cross-reference. Defaults to `manual_review` for both fresh and existing installs. When `variant_parent_mode` is `unset` and the filament looks like a variant-cluster member, auto-import falls back to `manual_review` regardless of this setting (can't group variants without a mode). | | `new_spool_policy` | `manual_review` | Sync → New records | What the engine does when an unmapped spool appears whose filament **is already mapped**: `manual_review` queues a `new_spool` conflict; `auto_import` creates the spool immediately. A spool is always held when its filament is unmapped, regardless of this setting — the filament tier must resolve first. | -| `never_import_empties` | `false` | Sync → New records | Controls both empty and archived spools. When `false` (default): all spools import, including depleted (`remaining ≤ 0`) and archived ones. Archived spools import as **retired** FDB spools (spool only — the filament stays live). When `true`: spools with `remaining ≤ 0` are skipped (whether active or archived); archived spools with positive remaining weight still import as retired. The filament definition always imports regardless. | +| `never_import_empties` | `false` | Sync → New records | **Import-time only** (UI label: "Skip empty & archived spools on import") — governs which spools the wizard and ongoing new-spool import create; it does **not** affect archive/retire mirroring for already-paired spools (that runs regardless — see `archive_sync_direction` above). When `false` (default): all spools import, including depleted (`remaining ≤ 0`) and archived ones. Archived spools import as **retired** FDB spools (spool only — the filament stays live). When `true`: spools with `remaining ≤ 0` are skipped (whether active or archived); archived spools with positive remaining weight still import as retired. The filament definition always imports regardless. | | `variant_parent_mode` | `unset` | Import & matching | **Required before the wizard runs** (Spoolman→FDB direction): `promote_color` or `generic_container`. See [variant-parent-mode.md](variant-parent-mode.md). | | `container_parent_marker` | env (`(Master)`) | Import & matching | Marker on generic-container names; checkbox + text field, visible in `generic_container` mode. | | `variant_line_keywords` | env seed | Import & matching | See `VARIANT_LINE_KEYWORDS`. | diff --git a/docs/decisions.md b/docs/decisions.md index 44524be..da50a71 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -1,6 +1,65 @@ # Decision record -## 2026-06-17 — FDB spool `instanceId` is NOT used for cross-reference; keep `label` +## 2026-06-17 — Archive/retire to sync bidirectionally for already-synced spools (FR-21 symmetric, design agreed) + +**Context.** Archive/retire lifecycle state is import-only and one-directional today +(FR-21 "Partial"). A mapped spool archived in Spoolman drops out of the active set +(`engine.py:2505` filters `if not s.archived`) and silently stops syncing while staying +live in FDB; a spool retired in FDB never propagates to Spoolman (the differ never reads +`retired`). Users archive/retire depleted spools routinely, so paired spools drift out of +sync. Design agreed with the user; implementation tracked in +`prompts/2026-06-17-archive-retire-bidirectional-sync.md`. + +**Decisions.** +- **Keep the bulk-import gate, add post-sync mirroring** — intentionally asymmetric. The + wizard's `never_import_empties` gate (which also skips archived spools at import) stays: + unmapped archived spools are NOT auto-imported during ongoing sync. But a spool that + flips archived/retired *after* it's mapped IS mirrored to the other side. Import is about + not cluttering FDB with already-dead inventory; mirroring is about keeping already-paired + spools honest. +- **Split the archived-spool filter by purpose.** New-spool detection keeps the active-only + set (preserves the import gate); mapped-pair diffing uses active + archived so the + archive transition reaches the differ. The data is already fetched (the Spoolman client + returns active + archived). +- **Mirror both directions of the boolean flip** — archive→retire and un-archive→un-retire. + Same mechanism; un-archiving re-enables weight sync for the pair. +- **New independent policy category `archive_sync`** (two axes: `archive_sync_direction` + default `two_way`, `archive_conflict_policy` default `manual`), resolved by the existing + `resolve_sync_action`. No timestamps → `newest_wins` is not applicable to this category. +- **No new conflict type.** Reuse `cross_system`. The common one-sided flip is a clean push, + not a conflict; only genuine divergence (both sides flipped to opposite states since the + last snapshot) queues a conflict — consistent with the "never auto-resolve conflicts" rule. +- **Weight settles before the archive bit — ordering is a correctness requirement.** The + lifecycle pass runs AFTER the weight pass, not before. A spool is usually archived/retired + right as it hits ~0 g; the final weight decrement and its FDB usage-log audit entry + (`POST …/usage`, `source:"spoolman"`) must be propagated and both snapshots refreshed + *before* the archive/retire bit is mirrored. Mirror first and the far side would land + retired/archived carrying a stale (too-high) weight with its final usage entry lost. Do not + skip the weight pass for an about-to-be-archived spool; a spool already dead on both sides + from a prior cycle NOOPs the weight pass naturally (snapshots converged). +- **Anti-ping-pong is mandatory.** After any lifecycle push, refresh BOTH snapshots to the + converged value (same rule as the 2026-06-10 weight fix below). +- **Reword, don't rename.** The `never_import_empties` config key stays; only its + human-facing label/help text is reworded to make clear it skips empty AND archived spools + at import and that ongoing sync mirrors lifecycle state regardless of the setting. + +**As built.** Implemented as designed above, with two deviations the implementer flagged: + +- **Lifecycle conflict resolution writes upstream** via a scoped + `core/conflict_apply.py:apply_lifecycle_conflict` rather than the generic record-only + `cross_system` resolve path. A normal `cross_system` resolution only records the chosen + value (no upstream write), but a lifecycle conflict's choice is a concrete boolean state, + so resolving it (`spoolman`/`filamentdb`/`manual`) writes that boolean to BOTH systems + (`spoolman.update_spool({"archived": …})` + `filamentdb.update_spool({"retired": …})`), + refreshes both snapshots, and logs under a `conflict_apply` cycle id. The conflicts API + routes `field_name == "lifecycle"` to this helper before the generic record-only branch; + an upstream failure leaves the conflict open (`502 upstream_write_failed`). +- **Both-changed-to-same-state collapses to a converge before the resolver is called.** The + engine detects `both_changed_converged` (both sides flipped this cycle AND landed on the + same state, e.g. both archived) up front and treats it as a NOOP-with-snapshot-refresh, + so `resolve_sync_action` / `QUEUE_CONFLICT` is only ever reached for a genuine + opposite-state divergence. This keeps the common "user archived in both systems" + case from spuriously queuing a conflict. **Context.** Filament DB 1.48.0 moved the 5-byte hex roll ID (`instanceId`) from the filament onto each **spool**, and 1.49.0 made it **user-editable**. Format (verified in diff --git a/docs/prd.md b/docs/prd.md index 132aa9b..57621ca 100644 --- a/docs/prd.md +++ b/docs/prd.md @@ -412,9 +412,12 @@ Field names are configurable via environment variables. - On sync error: post with error details and retry status *(planned)* - Optional: daily summary of sync activity *(planned)* -#### FR-21: Spoolman archive/retire sync *(Partial — v0.2.0)* -- Archived Spoolman spools are detected and excluded from sync cycles -- Symmetric propagation (`retired: true` in FDB ↔ `archived` in Spoolman) is not yet implemented +#### FR-21: Spoolman archive/retire sync +- Lifecycle state mirrors **bidirectionally for already-mapped spool pairs**: archiving a spool in Spoolman (`archived`) retires it in Filament DB (`retired: true`), retiring it in FDB archives it in Spoolman, and both un-flips mirror back too (un-archive re-enables weight sync) +- A dedicated `archive_sync` policy category governs it: `archive_sync_direction` (default `two_way`) and `archive_conflict_policy` (default `manual`). `newest_wins` is rejected (the state is a boolean with no comparable timestamp) +- **Import gate preserved (intentionally asymmetric):** *unmapped* archived spools are still NOT auto-imported during ongoing sync — only the *mapped-pair* diffing set includes archived spools so a post-mapping flip is mirrored. Import is about not cluttering FDB with already-dead inventory; mirroring keeps already-paired spools honest +- **Weight settles before the archive bit:** the lifecycle pass runs after the weight pass, so a depleted-and-archived spool propagates its final decrement and FDB usage-log entry (and refreshes both snapshots) before the archive/retire bit mirrors — never retired/archived with a stale weight or missing its final usage entry +- A one-sided flip is a clean push (not a conflict). Only genuine divergence (both sides flipped to opposite states since the last snapshot) queues a `cross_system` conflict with `field_name="lifecycle"`; resolving it writes the chosen boolean to both systems and refreshes both snapshots. Both sides flipping to the same state converges silently #### FR-22: Print history enrichment *(Not implemented — v0.2.0)* - Planned: when a weight decrement is synced from Spoolman, optionally create a `POST /api/print-history` record in Filament DB diff --git a/docs/spoolman-writes.md b/docs/spoolman-writes.md index cb83c22..43eb078 100644 --- a/docs/spoolman-writes.md +++ b/docs/spoolman-writes.md @@ -53,6 +53,7 @@ lone change, one-way FDB→SM, or an FDB-winning conflict policy). See | Filament | `weight` | Native-scalar sync resolves FDB→SM (from FDB `netFilamentWeight`) | | Filament | `extra.filamentdb_material_tags` | Finish-tag sync resolves FDB→SM (Filament DB ≥ 1.33.0); CSV of OpenPrintTag IDs from FDB `optTags` | | Spool | `extra.{mapped field}` | Generic field-mapping sync (FR-11) resolves FDB→SM; arbitrary mapped FDB fields stored as spool extras | +| Spool | `archived` (bool) | Lifecycle sync resolves FDB→SM — a *mapped* spool retired in Filament DB (`retired`) flips Spoolman `archived` to match; un-retire mirrors back (`archived: false`). Runs **after** the weight pass so a depleted spool's final decrement settles first. Governed by `archive_sync_direction` / `archive_conflict_policy`, not by `never_import_empties`. See [sync-model.md](sync-model.md). | New-spool creation during a cycle (gated by `new_spool_sync_direction`): @@ -82,20 +83,34 @@ New-spool creation during a cycle (gated by `new_spool_sync_direction`): For FDB spool creates: when the source SM spool is **archived**, the bridge sets `retired: true` on the FDB spool payload so the archived state is preserved at import. Only the spool is marked retired — the filament record is always created as a normal, non-retired filament. -`archived`/`retired` is set once at import and is NOT a synced field in ongoing auto-sync. +Beyond this import-time stamp, `archived`/`retired` IS a synced field in ongoing auto-sync +for already-mapped pairs (lifecycle sync, see the auto-sync table above and +[sync-model.md](sync-model.md)). ## Conflict-resolution writes (on-demand, human-approved) -Resolving a **master_divergence** conflict (`POST /api/conflicts/{id}/resolve` with an -`action`) is the one conflict type that writes upstream — the chosen action is the -authorisation (see `docs/conflicts.md`): +Two conflict types write upstream on resolution (`POST /api/conflicts/{id}/resolve`) — the +chosen action/resolution is the authorisation (see `docs/conflicts.md`): + +**`master_divergence`** (resolved with an `action`): | Action | Entity | Op | Field(s) | |---|---|---|---| | `apply_all` | Filament | update | The diverged native field (`material`, `density`, `diameter`, `spool_weight`, `weight`, `settings_bed_temp`, `settings_extruder_temp`) on **every mapped Spoolman filament in the variant line** | | `variant_override` / `ignore` | — | — | No Spoolman writes (FDB-only / no-op respectively) | -All other conflict types are record-only — resolving them performs no Spoolman writes. +**Lifecycle** — a `cross_system` conflict with `field_name="lifecycle"` (resolved with +`resolution` = `spoolman`/`filamentdb`/`manual`). Unlike all other `cross_system` conflicts +(which are record-only), this one converges by writing the chosen boolean to **both** +systems and refreshing both snapshots (via the scoped `apply_lifecycle_conflict` path, not +the generic record-only resolver): + +| Entity | Op | Field(s) | +|---|---|---| +| Spool | update | `archived` (chosen boolean) | +| Spool (FDB) | update | `retired` (same chosen boolean) | + +All other conflict types are record-only — resolving them performs no upstream writes. ## OpenTag cleanup tool writes (on-demand, on Apply) @@ -116,8 +131,10 @@ bag (scoped exception — see `docs/decisions.md`). ## What the bridge never writes to Spoolman -`location`, `lot_nr`, `archived`, `comment`, and **per-spool `price`** (cost write-back -targets the filament price only). The bridge never deletes Spoolman records. +`location`, `lot_nr`, `comment`, and **per-spool `price`** (cost write-back targets the +filament price only). The bridge never deletes Spoolman records. (`archived` *is* written — +by the lifecycle sync pass for mapped pairs and by lifecycle conflict resolution — see +above.) **The bridge never sets both `color_hex` and `multi_color_hexes` on the same Spoolman filament in a single PATCH.** Spoolman returns 422 if both are present simultaneously. diff --git a/docs/sync-model.md b/docs/sync-model.md index 9cea89b..0df26d4 100644 --- a/docs/sync-model.md +++ b/docs/sync-model.md @@ -13,10 +13,15 @@ Every cycle (scheduled, or via **Sync now**): the version check, no writes. An unknown version does not block (that's a connectivity concern, surfaced as `degraded` health). 2. **Snapshot fetch.** All Spoolman spools + filaments and all Filament DB filaments - (with embedded spools) are fetched. Archived Spoolman spools are excluded from sync. + (with embedded spools) are fetched. The archived-spool filter is **split by purpose**: + *new-spool detection* uses the active-only set (so an unmapped archived spool is never + auto-imported — the wizard import gate is preserved), while *mapped-pair diffing* uses + the active + archived set (so a mapped spool that flips to archived still reaches the + differ and its lifecycle state can be mirrored). 3. **Mapped-pair processing.** For every `SpoolMapping`, the pair's current values are diffed against the last stored snapshots. First sight of a pair just stores a baseline - (no writes). Then the weight pass and the field-mapping pass run per pair. + (no writes). Then the weight pass, the **lifecycle (archive/retire) pass** (after + weight — see below), and the field-mapping pass run per pair. 4. **Stale-link handling.** A mapped record missing upstream either queues a deletion conflict (a live, still-linked counterpart exists) or purges the bridge-local mapping (nothing left to protect). See [conflicts.md](conflicts.md). @@ -42,14 +47,30 @@ policy)`: | One side changed | push | NOOP (drift ignored) | push to the other side | | Both sides changed | push (source wins by definition) | — | apply the conflict policy: `manual` → queue, `spoolman_wins`/`filamentdb_wins` → push, `newest_wins` → compare timestamps (weight only; indeterminate → queue) | -Weight uses the `weight` category settings; every other pass uses `material_properties`. -New-spool creation has a direction but no policy. +Weight uses the `weight` category settings; the lifecycle pass uses the `archive_sync` +category; every other pass uses `material_properties`. New-spool creation has a direction +but no policy. + +### Why the lifecycle pass runs after the weight pass + +A spool is usually archived/retired right as it hits ~0 g, so the depletion (final weight +decrement) and the lifecycle flip often arrive in the **same** cycle. The weight pass must +settle first: it propagates the final decrement, logs the FDB usage-log audit entry +(`POST …/usage`, `source:"spoolman"`), and refreshes both snapshots — *before* the lifecycle +pass mirrors the archive/retire bit. Mirror first and the far side would land +retired/archived carrying a stale (too-high) weight with its final usage entry lost. (A +spool already dead on both sides from a prior cycle NOOPs the weight pass naturally, since +its snapshots already converged — no special skip is needed.) After any lifecycle push the +engine refreshes **both** snapshots' lifecycle bits to the converged value, the same +anti-ping-pong invariant the weight pass follows; a both-sides-flip-to-same-state case +writes nothing but still refreshes both snapshots so it doesn't re-fire next cycle. ## The passes | Pass | What syncs | Notes | |---|---|---| | **Weight** | SM `remaining_weight` ↔ FDB spool `totalWeight` | SM→FDB decrements are logged as **usage entries** (audit trail preserved); increases update `totalWeight` directly. FDB→SM writes `remaining_weight = totalWeight − tare`. Changes below `sync_weight_threshold_grams` (default 2 g) are ignored. | +| **Lifecycle (archive/retire)** | SM spool `archived` ↔ FDB spool `retired` | Boolean mirror for **mapped pairs only**, runs **after** the weight pass. A one-sided flip (either direction, archive or un-archive) is a clean push; both sides flipping to the *same* state converges silently; only a both-sides-flip-to-*opposite*-states divergence queues a `cross_system` conflict with `field_name="lifecycle"`. Uses the `archive_sync` category (`archive_sync_direction` / `archive_conflict_policy`); `newest_wins` is not applicable to a boolean. | | **Field mapping** | configured/auto-matched Spoolman *extra* fields ↔ FDB fields | `FIELD_MAPPINGS` / exact-name auto-match; inherited FDB variant fields are skipped (writing them would detach the variant from its parent). | | **Cost** | SM effective price ↔ FDB `cost` | SM side uses the first spool with a price, falling back to the filament price; FDB→SM writes the *filament* price only — never per-spool prices. | | **Temperatures** | SM `settings_bed_temp` / `settings_extruder_temp` ↔ FDB `temperatures.bed` / `.nozzle` | FDB writes read-modify-write the `temperatures` object so sibling temps survive. | diff --git a/frontend/src/api/types.ts b/frontend/src/api/types.ts index 6c86fca..d9c80f0 100644 --- a/frontend/src/api/types.ts +++ b/frontend/src/api/types.ts @@ -266,6 +266,8 @@ export interface ConfigResponse { weight_conflict_policy: ConflictPolicy material_properties_sync_direction: SyncDirection2 material_properties_conflict_policy: Exclude + archive_sync_direction: SyncDirection2 + archive_conflict_policy: Exclude new_spool_sync_direction: SyncDirection2 // New-record handling policies new_filament_policy: NewRecordPolicy @@ -297,6 +299,8 @@ export interface ConfigUpdateRequest { weight_conflict_policy?: ConflictPolicy | null material_properties_sync_direction?: SyncDirection2 | null material_properties_conflict_policy?: Exclude | null + archive_sync_direction?: SyncDirection2 | null + archive_conflict_policy?: Exclude | null new_spool_sync_direction?: SyncDirection2 | null // New-record handling policies new_filament_policy?: NewRecordPolicy | null diff --git a/frontend/src/pages/Settings.test.tsx b/frontend/src/pages/Settings.test.tsx index 01f45fd..8f73177 100644 --- a/frontend/src/pages/Settings.test.tsx +++ b/frontend/src/pages/Settings.test.tsx @@ -91,6 +91,8 @@ function makeConfig(overrides?: Partial): ConfigResponse { weight_conflict_policy: 'manual', material_properties_sync_direction: 'filamentdb_to_spoolman', material_properties_conflict_policy: 'manual', + archive_sync_direction: 'two_way', + archive_conflict_policy: 'manual', new_spool_sync_direction: 'two_way', new_filament_policy: 'manual_review', new_spool_policy: 'manual_review', diff --git a/frontend/src/pages/Settings.tsx b/frontend/src/pages/Settings.tsx index 5069b5b..c03ff16 100644 --- a/frontend/src/pages/Settings.tsx +++ b/frontend/src/pages/Settings.tsx @@ -193,6 +193,8 @@ export default function Settings() { const [weightPolicy, setWeightPolicy] = useState(null) const [matDir, setMatDir] = useState(null) const [matPolicy, setMatPolicy] = useState(null) + const [archiveDir, setArchiveDir] = useState(null) + const [archivePolicy, setArchivePolicy] = useState(null) const [newSpoolDir, setNewSpoolDir] = useState(null) // New-record handling policies @@ -258,6 +260,8 @@ export default function Settings() { (weightPolicy != null && weightPolicy !== data.weight_conflict_policy) || (matDir != null && matDir !== data.material_properties_sync_direction) || (matPolicy != null && matPolicy !== data.material_properties_conflict_policy) || + (archiveDir != null && archiveDir !== data.archive_sync_direction) || + (archivePolicy != null && archivePolicy !== data.archive_conflict_policy) || (newSpoolDir != null && newSpoolDir !== data.new_spool_sync_direction) || (newFilamentPolicy != null && newFilamentPolicy !== data.new_filament_policy) || (newSpoolPolicy != null && newSpoolPolicy !== data.new_spool_policy) || @@ -310,6 +314,8 @@ export default function Settings() { const wPol = weightPolicy ?? data.weight_conflict_policy const mDir = matDir ?? data.material_properties_sync_direction const mPol = (matPolicy ?? data.material_properties_conflict_policy) as MatConflictPolicy + const aDir = archiveDir ?? data.archive_sync_direction + const aPol = (archivePolicy ?? data.archive_conflict_policy) as MatConflictPolicy const nsDir = newSpoolDir ?? data.new_spool_sync_direction const nfPol = newFilamentPolicy ?? data.new_filament_policy const nsPol = newSpoolPolicy ?? data.new_spool_policy @@ -514,6 +520,8 @@ export default function Settings() { weight_conflict_policy: wPol, material_properties_sync_direction: mDir, material_properties_conflict_policy: mPol, + archive_sync_direction: aDir, + archive_conflict_policy: aPol, new_spool_sync_direction: nsDir, new_filament_policy: newFilamentPolicy ?? undefined, new_spool_policy: newSpoolPolicy ?? undefined, @@ -817,6 +825,31 @@ export default function Settings() { + {/* Archive / retire sync card — full width */} +
+

Archive / retire sync

+

+ Keeps a spool's archived (Spoolman) / retired (Filament DB) state in sync for + already-paired spools. Archiving or retiring one side mirrors to the other; un-archiving + mirrors back too. Unmapped archived spools are still never imported during sync. +

+ { + setArchiveDir(v) + if (v !== 'two_way') setArchivePolicy('manual') + }} + tip="Which side's archive/retire flips get mirrored to the other. Two-way mirrors both directions and queues a conflict only when both sides diverge to opposite states." + tipHref="/docs/sync-model" + /> + setArchivePolicy(v)} + /> +
+ {/* New records card — full width under the 2-column pair */}

New records

@@ -880,11 +913,12 @@ export default function Settings() {
- Never import empties - + Skip empty & archived spools on import +

- Empty/depleted spools are skipped on import; the filament definition is still imported. + Empty/depleted and archived spools are skipped on import; the filament definition is + still imported. Already-synced spools keep mirroring their archive/retire state.