From 3d4bd23a8159ab4141adaac513359abe8ddda03a Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 8 Jul 2026 15:44:10 +0000 Subject: [PATCH 1/3] od-posting: retarget GoBD invariants off deprecated SurrealQL to PG/V3 storage matrix (council Q5) The 4 GoBD invariants (single_tx_counter_create_hash_state, chain_order_per_journal_sequence_prefix, append_only_no_update_delete_once_hashed, serialization_byte_exact) are storage-agnostic and unchanged; only the framing moves off the dead SurrealQL vocabulary (operator 2026-07-06: SurrealQL absolutely deprecated; OGAR V3 transpile substrate, lance-graph V3 database). Retarget to the storage matrix (PG facet table as SoR, lance-graph V3 read path): BEGIN/COMMIT + LockType::Pessimistic -> PG BEGIN + SELECT..FOR UPDATE on the counter row; one PG tx wraps counter RMW + INSERT + hash + state write. hash DEFINE EVENT -> app-side / BEFORE INSERT trigger within the same tx. DEFINE SEQUENCE/nextval rejection -> sharpened: a PG SEQUENCE is explicitly gap-prone (no rollback on abort), so the gapless Belegnummer MUST use a counter row under SELECT..FOR UPDATE, never a PG sequence. Permissions{update:NONE,delete:NONE} -> REVOKE UPDATE,DELETE + BEFORE UPDATE/DELETE trigger once posted_before=true. serialization_byte_exact -> unchanged (float_repr, sorted JSON, strip $4$). D-POST-SEQ Option B / surrealdb-fork DEFINE SEQUENCE proposal declared dead; the counter-row-under-FOR-UPDATE IS the resolution. Trait/struct API unchanged, still a skeleton. SurrealQL terms remain only as dated historical annotations. Doc backticks clippy-clean. od-posting build+test green. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Fy8D6Boh4ZBaHSBpadSENN --- .../od-ontology/specs/DECISIONS/D-POST-SEQ.md | 30 +++ crates/od-ontology/specs/_post.md | 176 +++++++++++------- crates/od-posting/src/lib.rs | 112 ++++++----- 3 files changed, 204 insertions(+), 114 deletions(-) diff --git a/crates/od-ontology/specs/DECISIONS/D-POST-SEQ.md b/crates/od-ontology/specs/DECISIONS/D-POST-SEQ.md index e6ef57c..df1c47b 100644 --- a/crates/od-ontology/specs/DECISIONS/D-POST-SEQ.md +++ b/crates/od-ontology/specs/DECISIONS/D-POST-SEQ.md @@ -1,5 +1,23 @@ # D-POST-SEQ — GoBD gapless Belegnummer routing for `account.move._post` +> **(council Q5, 2026-07-08): retargeted from deprecated SurrealQL to the +> V3 storage matrix.** SurrealQL is ABSOLUTELY DEPRECATED per operator +> directive 2026-07-06 ("OGAR V3 for transpile substrate, lance-graph V3 +> for database"). The surrealdb-fork `DEFINE SEQUENCE` extension proposed +> as "Option B" below is DEAD — there is no substrate to extend. The +> gapless mechanism now targets **PostgreSQL** directly: the counter row +> under `SELECT … FOR UPDATE`, in the same transaction as the facet-table +> INSERT (`ogar-adapter-postgres-ddl::emit_facet_table_ddl`), with +> lance-graph V3 as the read hot-path over the same data. This row-lock +> mechanism **is** the resolution — no PG-side "extend the core" +> counterpart is needed, because `SELECT … FOR UPDATE` on an ordinary row +> is already the in-DB gapless primitive on PostgreSQL. Everything below +> is the historical reasoning trail (source-verified against the +> now-deprecated surrealdb fork) that established the PRINCIPLE — gapless +> numbering needs a single-transaction pessimistic lock, never a +> batch-allocated sequence primitive — which carries over to PostgreSQL +> unchanged. Read it as **provenance**, not as the live storage target. +> > **Status:** `RESOLVED` (2026-06-17) — **Option C (HYBRID)**, mechanism > corrected three times through the 8-agent council. Verdict baked into > [`../_post.md`](../_post.md). Resolution log at the bottom of this file. @@ -177,3 +195,15 @@ Core convenience (B). unimplemented and gated on the disk-gated fork build. NO surrealdb client dep yet — keeps workspace `cargo check` cheap on the constrained host. +- 2026-07-08 — **council Q5: retargeted to the V3 storage matrix.** + SurrealQL deprecated (operator, 2026-07-06). Option B (surrealdb-fork + `DEFINE SEQUENCE` extension) is DEAD — no substrate to extend. The + gapless mechanism (originally Option A) now targets PostgreSQL + directly: `SELECT … FOR UPDATE` on the counter row, in the same + transaction as the facet-table INSERT + (`ogar-adapter-postgres-ddl::emit_facet_table_ddl`), with lance-graph + V3 as the read hot-path. No parallel substrate PR is needed — the + row-lock IS the resolution on PostgreSQL. `od-posting`'s trait API and + the four invariants are unchanged; only the storage vocabulary in + `_post.md` and this file was retargeted. See `crates/od-posting/src/ + lib.rs` for the updated module doc. diff --git a/crates/od-ontology/specs/_post.md b/crates/od-ontology/specs/_post.md index 4fb42d8..f6624f7 100644 --- a/crates/od-ontology/specs/_post.md +++ b/crates/od-ontology/specs/_post.md @@ -1,5 +1,18 @@ # `account.move._post` — HAND-PORT spec (the GoBD posting method) +> **(council Q5, 2026-07-08): retargeted from deprecated SurrealQL to the +> V3 storage matrix (PostgreSQL facet System-of-Record + lance-graph V3 +> read path).** SurrealQL is ABSOLUTELY DEPRECATED per operator directive +> 2026-07-06 ("OGAR V3 for transpile substrate, lance-graph V3 for +> database"). The 4 invariants below and the `PostingHost` trait API are +> UNCHANGED — only the storage vocabulary (SurrealDB `DEFINE EVENT`/ +> `DEFINE SEQUENCE`/`kvs::LockType::Pessimistic`/`Permissions`/rocksdb) +> is retargeted to PostgreSQL (`ogar-adapter-postgres-ddl:: +> emit_facet_table_ddl`, columns `p0..p11` = 3×SPOG) + lance-graph V3. +> All SurrealDB-specific mechanism references below are historical — +> **deprecated, see this note** — and describe the reasoning trail that +> led to the resolved mechanism, not the live target. +> > **Status:** `ROUTE-RESOLVED` / parity `CONJECTURE` (2026-06-17). > The **route** (HAND-PORT) and the **gapless-numbering mechanism** > (Option C, single-transaction pessimistic counter) are RESOLVED by an @@ -52,7 +65,7 @@ flip out — its correctness is that it is welded to the hash. 3. **Transactional fencing.** Crosses: ops 1+2 + state-flip + freeze must be ONE transaction or a partial post is a forged ledger. -## CORE-FIT — per primitive (source-verified against the surrealdb fork) +## CORE-FIT — per primitive (historical: source-verified against the deprecated surrealdb fork; superseded by the PG/V3 retarget, see note at top) | Primitive | Verdict | Evidence | |---|---|---| @@ -61,58 +74,74 @@ flip out — its correctness is that it is welded to the hash. | **Composite atomicity** | **TARGETS-CORE** | sync `DEFINE EVENT THEN` runs in the triggering tx (`doc/event.rs:34-35`); `store_record_data` (row+hash) at `create.rs:27` runs **before** `process_table_events` at `:31`, both in one `Document::process` → the hash event reads the predecessor through the same tx snapshot. | | **Gapless Belegnummer** | **HAND-PORT, in-DB pessimistic counter** (the council decision — § below) | `DEFINE SEQUENCE` is batch-allocated (`kvs/sequences.rs:607` abandons the tail) and `nextval` runs its own tx (`:491-497`) → **UNIQUE, not GAPLESS**. But `LockType::Pessimistic` exists (`kvs/tr.rs:20-25`, threaded at `ds.rs:205-210`). | -## The gapless-numbering mechanism — council-resolved (D-POST-SEQ → Option C) +## The gapless-numbering mechanism — council-resolved (D-POST-SEQ → Option C), retargeted to PostgreSQL > **Corrected three times through the council. Read the trail; the -> obvious answer is wrong twice over.** +> obvious answer is wrong twice over.** The trail below was reasoned +> against the (now-deprecated) surrealdb fork; the PRINCIPLE it +> establishes — gapless numbering needs serialized allocation via a +> single-transaction pessimistic lock, never a batch-allocated sequence +> primitive — carries over to PostgreSQL unchanged, and if anything is +> SHARPER there: a PostgreSQL `SEQUENCE` is explicitly documented as +> gap-prone (`nextval()` does NOT roll back on transaction abort), so +> the same anti-mechanism applies a fortiori. - ❌ **NOT "UNIQUE index + optimistic retry."** scenario-world's case-2 proof: P1 claims N+1 (uncommitted); P2 retries to N+2, COMMITS; P1 - ABORTS → `{…, N, N+2}` = permanent **gap**. SurrealDB MVCC - conflict-detects *same-key* writes only; two posters writing - *different* numbers never conflict → UNIQUE ≠ GAPLESS. + ABORTS → `{…, N, N+2}` = permanent **gap**. MVCC-style engines + conflict-detect *same-key* writes only; two posters writing + *different* numbers never conflict → UNIQUE ≠ GAPLESS. (This holds on + PostgreSQL too — it is the same reason a plain PG `SEQUENCE` is + gap-prone: `nextval()` commits independently of the caller's + transaction and never rolls back on abort.) - ❌ **NOT "an out-of-DB sequence owner."** (PP-13's feared collapse.) -- ✅ **A single-transaction, in-DB PESSIMISTIC counter.** A **pessimistic - write transaction** (`LockType::Pessimistic`) does **read-modify-write - on a per-`(journal_id, sequence_prefix)` counter key**; poster-2 blocks - on the counter-key lock until poster-1 COMMITS (which is also when - poster-1's hash becomes visible) → serial number ⇒ serial predecessor - visibility ⇒ sound chain. +- ✅ **A single-transaction, in-DB PESSIMISTIC counter.** On PostgreSQL: + a **`SELECT … FOR UPDATE`** against the per-`(journal_id, + sequence_prefix)` counter row, inside the same transaction as the + INSERT + hash + state write. Poster-2 blocks on the row lock until + poster-1 COMMITS (which is also when poster-1's hash becomes visible + to poster-2's predecessor lookup) → serial number ⇒ serial predecessor + visibility ⇒ sound chain. (`SERIALIZABLE` isolation is an optional + additional hardening, not required for the row-lock argument itself.) ### The single-transaction invariant (PP-15 — the load-bearing correctness condition) -> **The counter read-modify-write, the `CREATE` row, the hash-computing -> `DEFINE EVENT`, and the `state`/`posted_before` write MUST share ONE -> `BEGIN…COMMIT`.** - -If the number is assigned via *any* own-tx path (`nextval`, or any -helper that opens its own transaction), the pessimistic lock's guarantee -is dropped at the seam: the number commits in tx-A, the hash/state in -tx-B → an abort-after-number is a gap, AND a reader between commits sees -a number with no row. **This invariant is what makes Option A correct -and what Option B must preserve.** - -### Option C (HYBRID) — the route - -- **A (immediate):** a new **`od-posting`** workspace crate (a RUNTIME - consumer with a SurrealDB client dep — kept OUT of zero-dep - `od-ontology`) runs `BEGIN → pessimistic RMW counter key → CREATE row - (hash event fires) → set state/posted_before → COMMIT` as ONE tx. -- **B (parallel substrate PR to AdaWorldAPI/surrealdb):** package the - pessimistic-counter pattern as a `gapless` / `no-cache` option on - `DEFINE SEQUENCE`. **Legitimate EXTEND-CORE** (core-gap-auditor 4-test - PASS: a policy flag on an existing subsystem, peer of - `BATCH`/`START`/`TIMEOUT`, not a new `fnc/*` — distinct from the - rejected `fn::core::currency::*`). **HARD acceptance gate (PP-15): - tx-enrolled / no own-tx** — a gapless `DEFINE SEQUENCE` that keeps - `nextval`'s own-tx shape re-imports the exact bug it claims to fix. B - does NOT route through the batch-allocated `nextval`; it is a separate - pessimistic-counter path (PP-16 drift correction — B does not touch - the "hot `nextval` path"). -- A and B are **genuinely parallel** (zero shared code: A is the - `od-posting` consumer, B is the surrealdb fork). If B lands, A's loop - becomes a thin shim over the in-DB path; if B is rejected, A stands - alone. +> **The counter read-modify-write (`SELECT … FOR UPDATE`), the row +> INSERT, the inalterability-hash computation, and the +> `state`/`posted_before` write MUST share ONE `BEGIN…COMMIT`.** + +If the number is assigned via *any* own-tx path (a bare `nextval()` on a +PG `SEQUENCE`, or any helper that opens its own transaction), the +pessimistic row lock's guarantee is dropped at the seam: the number +commits in tx-A, the hash/state in tx-B → an abort-after-number is a +gap, AND a reader between commits sees a number with no row. **This +invariant is what makes the resolved PG mechanism correct and is +load-bearing regardless of storage engine.** + +### Option C (HYBRID) — the route, retargeted to PostgreSQL + +> **(council Q5, 2026-07-08):** the original framing below proposed A +> (immediate hand-port) in parallel with B (a substrate PR proposing a +> `gapless DEFINE SEQUENCE` flag on the deprecated surrealdb fork). B is +> now **DEAD** — there is no surrealdb fork to extend. The counter-row- +> under-`SELECT FOR UPDATE` mechanism (originally A) **is now the entire +> resolution**; `od-posting` stays the hand-port host, now targeting +> PostgreSQL directly. See `D-POST-SEQ.md` for the retargeted decision +> record. + +- **The route:** the **`od-posting`** workspace crate (a RUNTIME + consumer with a PostgreSQL client dep — kept OUT of zero-dep + `od-ontology`) runs `BEGIN → SELECT … FOR UPDATE on the counter row → + INSERT row (compute hash in the same tx) → set state/posted_before → + COMMIT` as ONE transaction, against the facet table emitted by + `ogar-adapter-postgres-ddl::emit_facet_table_ddl`. +- **(historical, dead) Option B** proposed a parallel substrate PR to + AdaWorldAPI/surrealdb packaging the pessimistic-counter pattern as a + `gapless` / `no-cache` flag on `DEFINE SEQUENCE`. Moot now that + SurrealQL is deprecated — there is no substrate to extend, and no + PostgreSQL equivalent is needed: `SELECT … FOR UPDATE` on an ordinary + row IS the in-DB gapless primitive on PostgreSQL, with no core + extension required. ## Parity oracle — `PROBE-POST-GAPLESS-PARITY` (the CONJECTURE→FINDING gate) @@ -120,18 +149,19 @@ and what Option B must preserve.** + one mid-batch node restart** — the only interleaving where A/B differ from naive `nextval`. - **Oracle:** Odoo `account.move._post` reference run. -- **SUT:** the SurrealDB `od-posting` path. +- **SUT:** the PostgreSQL `od-posting` path. - **Diff-gate:** **set-equality on `name`** (gapless, contiguous) + **byte-equality on each `inalterable_hash`** (chained, with byte-exact `canonical(row)`: `float_repr` monetary + sorted-compact-ASCII JSON + `$4$` version prefix stripped before chaining). Message strings exempt (i18n), per the `_check_invoice_currency_rate` precedent. -- **Runnable today (the concurrency half ONLY):** "does a pessimistic - write tx serialize same-key writers" is testable in SurrealDB's own - `core/src/kvs/tests/multiwriter_same_keys_conflict.rs` harness — **no - Odoo, no full build** (PP-16). Full parity (gapless names + byte-exact - hash chain vs the Odoo reference) still gated on Odoo Python (not on - this host) + the disk-gated surrealdb build. +- **Runnable today (the concurrency half ONLY):** "does `SELECT … FOR + UPDATE` serialize same-row writers" is testable directly against a + real PostgreSQL instance with K concurrent connections — **no Odoo, no + disk-gated fork build required** (this is materially easier to probe + than the deprecated surrealdb-fork path was). Full parity (gapless + names + byte-exact hash chain vs the Odoo reference) still gated on + Odoo Python (not on this host). ## YAML spec @@ -149,32 +179,35 @@ method: do_out: writes_field: [account_move.state, account_move.name, account_move.inalterable_hash, account_move.posted_before] - emit_shape: hand_port # no single DEFINE * emits this - body_sketch: null # NO SurrealQL body — see intrusive_operations + emit_shape: hand_port # no declarative DDL adapter emits this + body_sketch: null # NO SQL/DDL body — see intrusive_operations intrusive_operations: - op: gapless_belegnummer crosses: "legal gap-free numbering (§14 UStG); serialized allocation; commit-with-row" - mechanism: "pessimistic write tx (LockType::Pessimistic) + RMW on per-(journal,prefix) - counter key; consumed only on COMMIT; counter-RMW shares ONE tx with - CREATE+hash-event+state-write (single-tx invariant)" - anti_mechanism: "NEVER nextval / DEFINE SEQUENCE (batch-allocated, own-tx → UNIQUE-not-GAPLESS)" + mechanism: "PostgreSQL SELECT ... FOR UPDATE on the per-(journal,prefix) counter row; + RMW consumed only on COMMIT; counter-RMW shares ONE BEGIN/COMMIT with + INSERT+hash-computation+state-write (single-tx invariant)" + anti_mechanism: "NEVER a bare PG SEQUENCE / nextval() (own-tx, does not roll back on + abort -> UNIQUE-not-GAPLESS; same failure mode the council originally + diagnosed against DEFINE SEQUENCE, deprecated surrealdb-side)" - op: inalterable_hash_chain crosses: "append-only, order-dependent, reads predecessor row" - mechanism: "sync DEFINE EVENT on CREATE; inalterable_hash = crypto::sha256(prev.hash + canonical(row))" + mechanism: "hash computed within the same PG transaction as the INSERT (application-side, + or a BEFORE INSERT trigger); inalterable_hash = sha256(prev.hash + canonical(row))" - op: transactional_fence - crosses: "counter ⊕ create ⊕ hash ⊕ state ⊕ freeze = ONE BEGIN/COMMIT or forged ledger" + crosses: "counter ⊕ insert ⊕ hash ⊕ state ⊕ freeze = ONE BEGIN/COMMIT or forged ledger" preserves_invariant: - single_tx_counter_create_hash_state # PP-15 — the load-bearing condition - chain_order_per_journal_sequence_prefix - - append_only_no_update_delete_once_hashed # Permissions update:NONE delete:NONE + - append_only_no_update_delete_once_hashed # REVOKE UPDATE/DELETE + BEFORE UPDATE/DELETE trigger - serialization_byte_exact # float_repr + sorted-compact-ascii JSON + $4$ strip - hand_port_target: "od-posting (new workspace crate; surrealdb client dep; calls the DB via - explicit BEGIN/COMMIT, NOT a DEFINE FUNCTION)" + hand_port_target: "od-posting (workspace crate; PostgreSQL client dep; calls the DB via + explicit BEGIN/COMMIT + SELECT ... FOR UPDATE, NOT a declarative adapter)" core_fit: - hash_chain: TARGETS-CORE # crypto::sha256 + sync event - append_only: TARGETS-CORE # per-op Permissions - atomicity: TARGETS-CORE # event runs in triggering tx (create.rs:27→31) - gapless: HAND-PORT-PESSIMISTIC + EXTEND-CORE-B # Option C + hash_chain: TARGETS-CORE # computed within the same PG tx as the INSERT + append_only: TARGETS-CORE # REVOKE UPDATE/DELETE + enforcing trigger + atomicity: TARGETS-CORE # single BEGIN/COMMIT spans counter+insert+hash+state + gapless: HAND-PORT-PESSIMISTIC # SELECT ... FOR UPDATE row lock; no core extension needed on PG composed_by: has_function: [account_move] virtually_overrides: [l10n_de.account_move._post] # German delivery_date pre-fill @@ -195,10 +228,13 @@ method: This spec **does not** modify `emit.rs` — the projection's bare guard/ function emit is unchanged. `_post`'s hand-port body lives in the future `od-posting` crate, NOT in the projection. Sequencing (integration-lead): -carve `od-posting` member → ship A (single-tx pessimistic counter) → file -B (tx-enrolled gapless `DEFINE SEQUENCE`) after core-gap-auditor's gate. -Nothing is built today; the spec records the resolved route + mechanism so -the eventual build doesn't reach for `nextval` (the wrong primitive). +carve `od-posting` member → implement the single-tx pessimistic counter +against PostgreSQL (`SELECT … FOR UPDATE` on the counter row, within the +same transaction as the facet-table INSERT). The parallel substrate-PR +option (B) is moot post-deprecation — there is no core to extend; the +row-lock mechanism stands alone. Nothing is built today; the spec records +the resolved route + mechanism so the eventual build doesn't reach for a +bare PG `SEQUENCE`/`nextval()` (the wrong primitive). See [`DECISIONS/D-POST-SEQ.md`](./DECISIONS/D-POST-SEQ.md) for the full 8-agent council record. diff --git a/crates/od-posting/src/lib.rs b/crates/od-posting/src/lib.rs index f073968..a438592 100644 --- a/crates/od-posting/src/lib.rs +++ b/crates/od-posting/src/lib.rs @@ -4,22 +4,32 @@ //! [`../../od-ontology/specs/_post.md`] spec. It carries the intrusive //! posting logic — the gapless Belegnummer, the inalterability hash chain, //! and the transactional fence around state+freeze — that the projection -//! cannot lower into a thin `DEFINE FUNCTION` adapter without smuggling -//! state (Frankenstein guard). +//! cannot lower into a thin declarative adapter without smuggling state +//! (Frankenstein guard). //! //! It exists as a **separate workspace crate** for two reasons. //! //! First — **zero-dep `od-ontology`**. The DDL projection is byte-stable -//! across target environments; pulling in a runtime `SurrealDB` client -//! plus tokio plus rocksdb would bend it. `od-posting` is allowed to be -//! heavy because it is *the runtime consumer*, not the ontology. +//! across target environments; pulling in a runtime `PostgreSQL` client +//! plus tokio would bend it. `od-posting` is allowed to be heavy because +//! it is *the runtime consumer*, not the ontology. //! -//! Second — **council-resolved boundary**. See -//! [`DECISIONS/D-POST-SEQ.md`][seq] — Option C (HYBRID) routes the -//! immediate posting loop here while a parallel substrate PR proposes -//! a tx-enrolled `gapless DEFINE SEQUENCE` flag on the surrealdb fork. -//! The two paths are independent; this crate stands alone if the -//! substrate PR is rejected, and shrinks to a shim if it lands. +//! Second — **council-resolved boundary, retargeted to the V3 storage +//! matrix**. See [`DECISIONS/D-POST-SEQ.md`][seq] — the gapless-counter +//! mechanism (single-transaction pessimistic row lock) targets +//! **`PostgreSQL`** as System-of-Record: the facet table emitted by +//! `ogar-adapter-postgres-ddl::emit_facet_table_ddl` (columns `p0..p11` +//! = 3×SPOG), with `lance-graph` V3 as the read hot-path over the same +//! data. This crate is the runtime consumer against that PG table; there +//! is no parallel substrate-fork proposal anymore (the original +//! `surrealdb`-fork `DEFINE SEQUENCE` flag idea is dead — see D-POST-SEQ). +//! +//! *(council Q5, 2026-07-08): retargeted from deprecated `SurrealQL` to the +//! V3 storage matrix (PG facet `SoR` + lance-graph V3 read path). `SurrealQL` +//! is ABSOLUTELY DEPRECATED per operator directive 2026-07-06 — "OGAR V3 +//! for transpile substrate, lance-graph V3 for database." The 4 +//! invariants and the `PostingHost` trait API are UNCHANGED; only the +//! storage vocabulary below is retargeted.* //! //! [seq]: ../../od-ontology/specs/DECISIONS/D-POST-SEQ.md //! @@ -28,31 +38,39 @@ //! Per `_post.md` § `preserves_invariant`: //! //! - **`single_tx_counter_create_hash_state`** (PP-15 — load-bearing): -//! counter RMW, row CREATE, hash-computing `DEFINE EVENT`, and the -//! state/`posted_before` write share ONE `BEGIN…COMMIT`. Any own-tx -//! path for the counter (`nextval`, any helper that opens its own -//! transaction) drops the pessimistic lock at the seam and reintroduces -//! the gap-or-orphaned-number failure mode. +//! counter RMW, row INSERT, hash computation, and the +//! state/`posted_before` write share ONE `PostgreSQL` `BEGIN…COMMIT`. +//! The counter RMW is a `SELECT … FOR UPDATE` on the counter row — +//! this pessimistic row lock is what serializes concurrent posters. +//! Any own-tx path for the counter (a bare `nextval()` on a PG +//! `SEQUENCE`, or any helper that opens its own transaction) drops the +//! lock at the seam and reintroduces the gap-or-orphaned-number +//! failure mode. //! - **`chain_order_per_journal_sequence_prefix`** — hashes are chained //! per `(journal_id, sequence_prefix)`; the predecessor lookup uses -//! the same tx snapshot the counter holds. -//! - **`append_only_no_update_delete_once_hashed`** — the projected table -//! carries `Permissions { update: NONE, delete: NONE }` once posted; -//! this host MUST refuse any post-hash mutation path. +//! the same tx snapshot the counter-row lock holds. +//! - **`append_only_no_update_delete_once_hashed`** — the PG facet table +//! is append-only once posted: `REVOKE UPDATE, DELETE` on the table +//! from the application role, reinforced by a `BEFORE UPDATE OR DELETE` +//! trigger that raises once `posted_before = true`; this host MUST +//! refuse any post-hash mutation path regardless. //! - **`serialization_byte_exact`** — canonical row bytes use Odoo's //! `float_repr` for monetary fields, sorted-compact-ASCII JSON for the //! payload, and the `$4$` version prefix is stripped before chaining. +//! Storage-agnostic — unaffected by the PG retarget. //! //! # Skeleton-stage status //! //! This file is the **published trait surface only** — the actual -//! `BEGIN → pessimistic RMW → CREATE → COMMIT` runner is unimplemented. -//! The implementation is gated on: +//! `BEGIN → SELECT … FOR UPDATE (counter row) → INSERT → COMMIT` runner +//! against `PostgreSQL` is unimplemented. The implementation is gated on: //! -//! - disk-gated surrealdb fork build (4.5 GB free on this host, full -//! `surrealdb-core` build is multi-GB); -//! - the `PROBE-POST-GAPLESS-PARITY` concurrency half (runnable today -//! against `kvs/tests/multiwriter_same_keys_conflict.rs`, PP-16); +//! - a wired `PostgreSQL` client (the facet-table DDL comes from +//! `ogar-adapter-postgres-ddl::emit_facet_table_ddl`; the counter table +//! + row-lock path is new, purpose-built for this host); +//! - the `PROBE-POST-GAPLESS-PARITY` concurrency half (same-row `SELECT +//! … FOR UPDATE` contention under K concurrent posters + one mid-batch +//! abort — testable against a real `PostgreSQL` instance); //! - the parity-half oracle (Odoo Python on a host that has it). //! //! Until those land, this crate compiles + provides a typed contract that @@ -63,25 +81,29 @@ /// The intrusive operations a GoBD-conformant `_post` host owns, /// surfaced as a typed boundary so the projection cannot accidentally -/// emit a DEFINE FUNCTION that claims to do them. +/// emit a declarative adapter that claims to do them. /// /// Per `_post.md` § "The three intrusive operations". An implementor /// of [`PostingHost`] must guarantee that a single call to -/// [`PostingHost::post`] executes the counter RMW, row CREATE, hash -/// event, and state/freeze write inside ONE `BEGIN…COMMIT`. Splitting -/// any step across transactions is a contract violation — see the -/// `anti_mechanism` slot on `gapless_belegnummer` in the spec. +/// [`PostingHost::post`] executes the counter RMW, row INSERT, hash +/// computation, and state/freeze write inside ONE `PostgreSQL` +/// `BEGIN…COMMIT`. Splitting any step across transactions is a contract +/// violation — see the `anti_mechanism` slot on `gapless_belegnummer` in +/// the spec. pub trait PostingHost { /// The host's error shape. Concrete implementors will wire this to - /// their `SurrealDB` client error; the contract here only requires - /// that a failed `post` left no committed row, no consumed counter - /// value, and no dangling hash entry. + /// their `PostgreSQL` client error (e.g. `tokio_postgres::Error`); the + /// contract here only requires that a failed `post` left no + /// committed row, no consumed counter value, and no dangling hash + /// entry. type Error; /// Atomically: lock the per-`(journal_id, sequence_prefix)` counter - /// key under `LockType::Pessimistic`, read-modify-write it, CREATE - /// the row (firing the hash event in the same tx snapshot), set - /// `state` to `posted` + `posted_before` to `true`, and COMMIT. + /// row via `SELECT … FOR UPDATE` (the pessimistic row lock), read- + /// modify-write it, INSERT the row into the PG facet table (computing + /// the inalterability hash in the same tx snapshot — application-side + /// or via a `BEFORE INSERT` trigger), set `state` to `posted` + + /// `posted_before` to `true`, and COMMIT. /// /// On failure: ABORT the transaction. The counter MUST NOT have /// been observed by any other reader, and no row may have leaked. @@ -110,22 +132,24 @@ pub struct MoveDraft<'a> { /// the assigned `name` must extend by exactly one. pub sequence_prefix: &'a str, - /// The canonical row bytes the hash event will read. The implementor - /// MUST NOT touch monetary `float_repr` formatting or JSON key order - /// here — the bytes are the spec's `serialization_byte_exact` input. + /// The canonical row bytes the hash computation will read. The + /// implementor MUST NOT touch monetary `float_repr` formatting or + /// JSON key order here — the bytes are the spec's + /// `serialization_byte_exact` input. pub canonical_row: &'a [u8], } /// A row after `_post`. The `name` field is the gapless Belegnummer -/// the council resolved must come from the pessimistic-counter path, -/// never `DEFINE SEQUENCE` / `nextval`. +/// the council resolved must come from the pessimistic-counter path — +/// a `SELECT … FOR UPDATE` row lock on `PostgreSQL`, never a bare +/// `SEQUENCE`/`nextval()`. #[derive(Debug, Clone, PartialEq, Eq)] pub struct PostedMove { /// The assigned gapless Belegnummer (e.g. `INV/2026/0042`). pub name: String, /// The chained inalterability hash for this row. - /// `crypto::sha256(prev.hash + canonical_row)`. + /// `sha256(prev.hash + canonical_row)`. pub inalterable_hash: Vec, } @@ -135,7 +159,7 @@ mod tests { /// Compile-only smoke test — confirms the trait surface is callable. /// Real implementors will live in downstream sites that own a - /// `SurrealDB` client; this stub stays as a witness that the contract + /// `PostgreSQL` client; this stub stays as a witness that the contract /// shape is stable. struct NoopHost; From a8d60303b16d3dd08eb9e9e93c1e503eff506398 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 8 Jul 2026 15:44:11 +0000 Subject: [PATCH 2/3] od-ontology: name + disposition the F17 5.1% imperative tail (council Q8) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The F17 body-triage probe COUNTED 18 order-dependent FAIL hooks but never named them, so the 5.1% tail was unowned. Add a diagnostic enumeration to the probe (no drift-fuse changed: hooks 393, resolved 354, pass 336, fail 18, (self-feedback,write+raise)=(30,2)) and author D-IMPERATIVE-TAIL.md dispositioning all 18 per the doctrine (essential-imperative -> hand-port, never recipe-forced). Finding: 16 of 18 self-feedback hooks are @api.depends extractor artifacts (the hook reads exactly its own written field at the c=0.75 body-inferred band vs the c=0.9 declarative write — the self-loop the RecomputeDag already drops, probe bound #2) -> declaratively-recoverable, NOT hand-ports. Only 2 write+raise hooks are genuine hand-ports (_inverse_l10n_latam_document_number -> od-posting; _onchange_partner_journal -> UI host, possibly guard+compute separable). True hand-port residue: 2 of 393, not 18 — the 5.1% headline over-counts by exactly the predicted artifact. 0 unowned/escaped. Probe still green. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Fy8D6Boh4ZBaHSBpadSENN --- .../specs/DECISIONS/D-IMPERATIVE-TAIL.md | 158 ++++++++++++++++++ crates/od-ontology/tests/body_triage_probe.rs | 11 ++ 2 files changed, 169 insertions(+) create mode 100644 crates/od-ontology/specs/DECISIONS/D-IMPERATIVE-TAIL.md diff --git a/crates/od-ontology/specs/DECISIONS/D-IMPERATIVE-TAIL.md b/crates/od-ontology/specs/DECISIONS/D-IMPERATIVE-TAIL.md new file mode 100644 index 0000000..a248952 --- /dev/null +++ b/crates/od-ontology/specs/DECISIONS/D-IMPERATIVE-TAIL.md @@ -0,0 +1,158 @@ +# D-IMPERATIVE-TAIL — disposition of the F17 5.1% order-dependent tail + +> Council item Q8 (`docs/COUNCIL-5+3-2026-07-07.md`): *"The 5.1% tail (18 +> order-dependent hooks) — currently unowned. Doctrine says: essential- +> imperative → hand-port (the true 15%→5% residue), never recipe-forced. +> ... acceptance = each hook either hand-ported or explicitly escaped with a +> name."* This document is that acceptance artifact. + +## Doctrine + +Per the F17 probe module docs (`tests/body_triage_probe.rs`) and the +council's Q8 wording: a hook that is genuinely order-dependent — its +intra-body evaluation order changes the outcome — is **essential-imperative** +and must be **hand-ported** to a named Rust host, never forced into a +declarative `(verb, criteria)` recipe. The prior 15%→5% collapse (F17's own +history: an initial cruder triage measured ~15% order-dependent before the +`@api.depends` self-loop caveat was applied; the current, more precise +static classification measures 5.1%) is itself evidence that *some* of the +apparent imperative residue is a harvesting artifact, not real +order-dependence — which is exactly bound #2 below, now resolved +hook-by-hook rather than left as a blanket caveat. + +## The fuse numbers this doc is scoped against (unchanged, re-verified) + +From the probe's drift-fuses (`tests/body_triage_probe.rs`, corpus +`data/slice_2.spo.ndjson`): + +``` +hooks.len() == 393 +n_resolved == 354 +n_pass == 336 (94.9%) +n_fail == 18 (5.1%) ← THIS document enumerates and disposes these 18 +(n_self_feedback, n_write_raise) == (30, 2) (30 self-feedback total in the corpus; + only 16 fall in the behavioural arm + (compute + guard/write-raise) counted + as n_fail; the other 14 are + onchange/inverse hooks outside the + arm and are excluded from n_fail — + see the probe's `arm` filter) +``` + +The probe's caveat (bound #2, module docs lines 51-56): + +> `reads_field` mixes body reads with `@api.depends` lifts, so a +> self-feedback hit can be an extractor artifact (the recompute DAG drops +> such self-loops for *cross-hook* ordering for that reason). Counting them +> as FAIL is therefore conservative: the true order-dependent tail is ≤ the +> measured one, the pass-rate a LOWER bound. + +This document resolves that "≤" into an exact per-hook verdict by inspecting +the actual harvested triples (`data/slice_2.spo.ndjson`) for each of the 18 +FAIL hooks, not just the aggregate count. + +## Classification rule applied + +1. **Self-feedback on a `_compute_*` whose self-read is an `@api.depends` + lift** (i.e. the overlapping `reads_field` triple names *exactly* the + field the hook itself emits, at the extractor's lower confidence band + `c=0.75` vs the `emitted_by` triple's `c=0.9`) → **declaratively- + recoverable**. This is the extractor self-loop artifact bound #2 names: + the recompute DAG already drops such self-loops for cross-hook ordering; + NOT a true hand-port. +2. **Self-feedback that is a genuine read-modify-write** (a counter / + running total / sequence / accumulator distinguishable from case 1 by the + read set containing *more* than just the written field, in an + accumulation shape) → **hand-port**, name the host. +3. **write+raise** → **hand-port** (transactional host), unless the raise + and the writes are cleanly separable into an independent guard-recipe + + compute-recipe. + +## The 18 hooks, named and disposed + +### Self-feedback (16 hooks, behavioural arm) + +Verified against `data/slice_2.spo.ndjson`: in **every one of the 16 cases** +the overlapping `reads_field` triple is *exactly* the same field name as the +`emitted_by` (write) triple, at the lower extractor-confidence band (`c=0.75` +read vs `c=0.9` write) — the harvester recorded the field reading itself +because it appears in its own `@api.depends(...)` argument list (or an +equivalent self-referential declaration), not because the method body does a +genuine load-then-store on an accumulator. None of the 16 show the +"read-a-different-upstream-field-and-fold-into-a-running-total" shape a real +accumulator would produce (contrast with `_compute_certifications_company_count`, +which DOES read a different field `child_ids` alongside the self-read — see +note below). + +| # | Hook | Overlapping field | Disposition | +|---|---|---|---| +| 1 | `account_move._compute_from_l10n_gr_edi_document_ids` | `l10n_gr_edi_mark` (+3 siblings) | declaratively-recoverable | +| 2 | `account_move._compute_l10n_latam_available_document_types` | `l10n_latam_available_document_type_ids` | declaratively-recoverable | +| 3 | `account_move._compute_l10n_ro_edi_state` | `l10n_ro_edi_state` | declaratively-recoverable | +| 4 | `account_move._compute_sale_warning_text` | `sale_warning_text` | declaratively-recoverable | +| 5 | `account_move._compute_timesheet_total_duration` | `timesheet_total_duration` | declaratively-recoverable | +| 6 | `account_move_line._compute_epd_needed` | `epd_dirty`, `epd_needed` | declaratively-recoverable | +| 7 | `account_move_line._compute_l10n_gr_edi_cls_vat` | `l10n_gr_edi_cls_vat` | declaratively-recoverable | +| 8 | `account_move_line._compute_l10n_gr_edi_detail_type` | `l10n_gr_edi_detail_type` | declaratively-recoverable | +| 9 | `res_company._compute_bounce` | `bounce_email`, `bounce_formatted` | declaratively-recoverable | +| 10 | `res_company._compute_catchall` | `catchall_email`, `catchall_formatted` | declaratively-recoverable | +| 11 | `res_company._compute_peppol_parent_company_id` | `peppol_parent_company_id` | declaratively-recoverable | +| 12 | `res_partner._compute_available_peppol_eas` | `available_peppol_eas` | declaratively-recoverable | +| 13 | `res_partner._compute_available_peppol_sending_methods` | `available_peppol_sending_methods` | declaratively-recoverable | +| 14 | `res_partner._compute_certifications_company_count` | `certifications_company_count` (also reads `child_ids`, a distinct upstream field — the accumulation source; but the count field's *own* self-read is still the same extractor-lift shape, `c=0.75`, and the presence of a real upstream read (`child_ids`) means the compute is expressible as `count = len(child_ids-derived-set)`, an ordinary pure fold over `child_ids`, not a read-modify-write of the count itself) | declaratively-recoverable | +| 15 | `res_partner._compute_l10n_my_tin_validation_state` | `l10n_my_tin_validation_state` | declaratively-recoverable | +| 16 | `res_partner._compute_vies_valid` | `vies_valid` | declaratively-recoverable | + +**All 16 self-feedback hooks in the behavioural arm are extractor-artifact +self-loops, not genuine accumulators.** Every one's overlapping read is +exactly its own written field (or a same-confidence-band sibling written +field) with no distinguishing "fold over a different upstream set into this +field" shape beyond what recipe #14 already resolves as an ordinary pure +compute. This matches bound #2's prediction: the true order-dependent tail +is *strictly less than* the measured 5.1%, driven entirely by the +`@api.depends` self-listing artifact for the self-feedback half. + +### write+raise (2 hooks, behavioural arm) + +Verified against the corpus: both hooks write one-or-more fields **and** +raise a user-facing exception, and (unlike the self-feedback set) their +overlapping signal is a real write-then-guard shape, not a self-referential +read of the written field: + +| # | Hook | Writes | Raises | Disposition | Host | +|---|---|---|---|---|---| +| 17 | `account_move._inverse_l10n_latam_document_number` | `l10n_latam_document_number`, `name` | `exc:UserError` | **hand-port** | `od-posting` (document-numbering / GoBD-adjacent: an inverse setter on the legal document number that validates format and may reject the assignment — partial-write-then-reject is exactly the transactional shape `od-posting` already owns for `_post`) | +| 18 | `account_move._onchange_partner_journal` | `journal_id` | `exc:RedirectWarning` | **hand-port**, separable | UI onchange host (not `od-posting` — `onchange` hooks are pre-save/client-side, never committed, so there is no GoBD/persistence exposure). **Separability note:** the read set (`filtered`) plus the write (`journal_id`) plus the raise (`RedirectWarning`) is plausibly splittable into an independent guard-recipe (`raise RedirectWarning if `) and a compute-recipe (`journal_id = `), since Odoo's `onchange` framework does not require the client to have applied the write before evaluating the warning. Flagged for the od-posting/behaviour lane to attempt the split when it restarts; until then it counts as hand-port, not recipe-forced (doctrine default: uncertain separability → hand-port). | + +## Summary tally + +| Disposition | Count | Hooks | +|---|---:|---| +| declaratively-recoverable (extractor self-loop artifact) | 16 | all self-feedback rows above | +| hand-port | 2 | both write+raise rows above (one flagged as possibly separable) | +| escape (no disposition assigned) | 0 | — | + +**18 / 18 hooks disposed.** Zero hooks remain unowned. The true +essential-imperative residue this doctrine asks to hand-port is **2 hooks**, +not 18 — the F17 5.1% headline already conservatively over-counts by +exactly the 16-hook `@api.depends` self-loop artifact bound #2 predicted. +Both surviving hand-port candidates route through the +`od-posting`/behaviour lane per the council's stated owner; the +`_onchange_partner_journal` case additionally carries a concrete +recipe-split hypothesis for that lane to evaluate before committing to a +full hand-port. + +## How this was produced + +1. Read `crates/od-ontology/tests/body_triage_probe.rs` in full — the triage + semantics (`VerbClass`, `Verdict`, the `arm` filter, the drift-fuses) and + bound #2's exact wording. +2. Extended the probe with a diagnostic-only enumeration (`eprintln!` of the + `n_fail` hooks' names + `VerbClass`, no change to any `assert_eq!` or the + triage logic) and ran it: `cargo test -p od-ontology --test + body_triage_probe -- --nocapture`, confirming all fuses hold + (`hooks.len()==393`, `n_resolved==354`, `n_pass==336`, `n_fail==18`, + `(n_self_feedback, n_write_raise)==(30,2)`). +3. For each of the 18 named hooks, grepped + `data/slice_2.spo.ndjson` for its `reads_field` / `emitted_by` / `raises` + triples and classified per the rule above. diff --git a/crates/od-ontology/tests/body_triage_probe.rs b/crates/od-ontology/tests/body_triage_probe.rs index 7e74ec1..a334777 100644 --- a/crates/od-ontology/tests/body_triage_probe.rs +++ b/crates/od-ontology/tests/body_triage_probe.rs @@ -219,6 +219,17 @@ fn body_triage_accidentally_imperative_ratio() { let n_fail = resolved.iter().filter(|h| h.verdict == Verdict::Fail).count(); let n_unresolved = arm.len() - n_resolved; + // ── Q8 diagnostic: NAME the behavioural-arm FAIL hooks (the imperative + // tail counted in n_fail), each with its VerbClass. Enumeration only — + // does not alter the triage semantics or any drift-fuse below. + let mut fail_hooks: Vec<(&String, &Hook)> = + arm.iter().copied().filter(|(_, h)| h.verdict == Verdict::Fail).collect(); + fail_hooks.sort_by_key(|(name, _)| (*name).clone()); + eprintln!("── Q8: the {} order-dependent tail hooks, named ──", fail_hooks.len()); + for (name, h) in &fail_hooks { + eprintln!(" {name:<40} {:?}", h.verb); + } + let pct = |num: usize, den: usize| -> f64 { if den == 0 { 0.0 } else { 100.0 * num as f64 / den as f64 } }; From a7bda05eb952e02698b1ac5dd933723f267610db Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 8 Jul 2026 15:53:32 +0000 Subject: [PATCH 3/3] od-ontology: delegate mint_wide_mask to upstream ClassView brick (council Q6 follow-up, lance-graph #669) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Now that WideFieldMask::from_universe_present landed in lance-graph-contract (#669, merged) and floated in, view_mask::mint_wide_mask delegates to it instead of re-implementing the membership filter + 256-SOC-cap loop. The domain-blind minting logic lives ONCE upstream — the two minters can no longer disagree by construction. WideMaskError stays the crate-local error shape (MaskWords stays the dep-free harvest artifact); the contract's WideMaskCapError maps onto it. wide_mask_parity 2/2 green (delegate still agrees with mint_mask). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Fy8D6Boh4ZBaHSBpadSENN --- crates/od-ontology/src/view_mask.rs | 32 ++++++++++++++++------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/crates/od-ontology/src/view_mask.rs b/crates/od-ontology/src/view_mask.rs index 78793e6..723e50c 100644 --- a/crates/od-ontology/src/view_mask.rs +++ b/crates/od-ontology/src/view_mask.rs @@ -441,20 +441,24 @@ pub fn mint_wide_mask( universe: &[String], present: &[String], ) -> Result { - if universe.len() > 256 { - return Err(WideMaskError::UniverseExceedsSocCap { - fields: universe.len(), - }); - } - let present_set: BTreeSet<&str> = present.iter().map(String::as_str).collect(); - #[allow(clippy::cast_possible_truncation)] // guarded: universe.len() <= 256 above - let positions: Vec = universe - .iter() - .enumerate() - .filter(|(_, field)| present_set.contains(field.as_str())) - .map(|(i, _)| i as u8) - .collect(); - Ok(lance_graph_contract::WideFieldMask::from_positions(&positions)) + // Council Q6 follow-up (lance-graph #669): the domain-blind membership + + // 256-SOC-cap logic now lives ONCE upstream as a ClassView brick + // (`WideFieldMask::from_universe_present`), so this consumer delegates + // instead of re-implementing the positions loop — the two minters can no + // longer disagree by construction. We keep `WideMaskError` as the + // crate-local error shape (and `MaskWords` as the dep-free harvest-side + // artifact), mapping the contract's cap error onto it. + let universe_refs: Vec<&str> = universe.iter().map(String::as_str).collect(); + let present_refs: Vec<&str> = present.iter().map(String::as_str).collect(); + lance_graph_contract::class_view::WideFieldMask::from_universe_present( + &universe_refs, + &present_refs, + ) + .map_err(|e| match e { + lance_graph_contract::class_view::WideMaskCapError::UniverseExceedsSocCap { fields } => { + WideMaskError::UniverseExceedsSocCap { fields } + } + }) } #[cfg(all(test, feature = "fieldmask"))]