diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 6200919..323442b 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -11,3 +11,22 @@ source packet → adapt() → atom → classify() → semion.frame.v0 Router home: `yggdrasil.belief`. Mixed slang+sign hits Hyperlex first. Mixed tradition+sign hits Athanor first. No circular import of Abraxas. No `source_space` invention in this package. + +## Workflow-derived architecture — advisory target + +Read [workflows](specs/workflows.md) before implementing these boundaries. Current code is the small pipeline above, not a completed service or encoder. Proposed validation and evidence components are not shipped. + +| Component | Workflow responsibility | Contract / boundary | +|---|---|---| +| Shared input/denial validation | WF-001, WF-002; applied at every public entry, including CLI direct classify | CON-001; no adapter-only bypass | +| T0 classifier | WF-003; deterministic relation and evidence classification | CON-002; never executes interpretants | +| Export bridge | WF-004; validate then emit one inert dictionary | CON-003; external consumer owns runtime chain | +| Operator-side label/snapshot tools | WF-005, WF-006, WF-010 | CON-004/CON-005; private data, immutable revisions; not imported by T0 | +| Offline evaluation/review | WF-007, WF-008 | CON-005/CON-006; pinned features and evidence, no activation | +| Authorized handoff preparation | WF-009 | CON-006; recipient scope and bytes validated independently | + +Keep T0 dependency-light and network-free. Do not add required torch/transformers, a database service, queues or cloud infrastructure to satisfy documentation structure. CLI and library must share validated behavior. Raw source/label evidence stays outside the unchanged v0 output as sidecar references pending a separately reviewed contract change. + +Validate packaging in an isolated installed wheel: current __main__.py reads a repository-relative VERSION that may not ship. Performance SLO, external consumer compatibility, real router deployment and T1 architecture are NOT_COMPUTABLE. No model/backbone choice is made until DEC-001/DEC-006 and a separate encoder cycle. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/README.md b/README.md index 260382b..f882043 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,7 @@ # Semion +Specification review: start at [docs/START_HERE.md](docs/START_HERE.md). The complete advisory methodology is separate from runtime conformance; see [assessment](specs/assessment.md) and [open decisions](specs/decisions.md). This does not govern or activate. +

Semion hero — bronze triad of icon, index, symbol

@@ -27,21 +29,23 @@ The Validate badge may stay red when Actions billing blocks the workflow even if |-----------|-------------------| | T0 `classify` / `adapt` / chain export (`compat`) | Trained encoder weights / Hub upload | | Spec 004 encoder **gate** (name_gate **false**) | `ALLOW_TRAIN` / `ALLOW_HUB` from Boof | -| M2 rebalance gold floors **PASS** (local SoT + Aaron pack) | Full labeled SoT in git | +| Historical M2 floor **PASS report** (not independently verified here) | Full labeled SoT in git | | Hero + OG rasters | Auto Settings social-preview write | -## Current state (OBSERVED 2026-09-11 PT) +## Latest reported state (2026-09-11 PT; qualified 2026-09-13 UTC) + +Counts below are reports from the later HQ dataset note and Notion handoff, not independently inspected corpus bytes. Floors, pack integrity, source rights and E-S3 readiness are NOT_COMPUTABLE in this review. Earlier 482-row/150-weak counts are superseded reports, not the current selected subset. A 242-row preferred HQ pool cannot alone satisfy the 300-row training floor; selection and train permission remain unresolved. | Area | State | |------|-------| | Spec 000–003 | Specified + code | | Spec 004 encoder gate | Specified · `name_gate` **false** · dataset note landed | -| Local gold SoT | `~/.semion/corpus/dataset.jsonl` — **482** gold (**not in git**) | -| OBSERVED share | **14 / 482** (~3%) — wrap/seed lawful INFERRED; AMC/Foundations preferred | +| Local gold SoT | `~/.semion/corpus/dataset.jsonl` — **452 reported** gold (**not in git; not independently verified**) | +| OBSERVED share | **14 / 452 reported** (~3%) — wrap/seed lawful INFERRED; AMC/Foundations preferred | | Hunt plan | [`docs/amc-foundations-hunt-plan.md`](docs/amc-foundations-hunt-plan.md) | -| Splits | train **330** · val **50** · test **102** | -| keep_weak | **150** (demoted wrap-symbol; **not** gold) | -| Spec 004 class floors | **PASS** (icon/index/symbol ≥15%; mixed/NC ≥8%) | +| Splits | reported train **300** · val **50** · test **102** | +| keep_weak | **180 reported** (demoted wrap-symbol; **not** gold) | +| Spec 004 class floors | Historical **PASS report**; current independent verification **NOT_COMPUTABLE** | | Hub / HF | **Skipped** (Danny) | | Boof `ALLOW_TRAIN` | **false** — Aaron trains on Spark | | Aaron Notion pack | https://app.notion.com/p/3d83e8ba2f5c81608499deffc060160d | @@ -130,3 +134,5 @@ Hyperlex (form / lexical) · Athanor (tradition structure) · **Semion (sign rel ## License Code: MIT. Corpus atoms carry their own licenses — see [`LICENSE_POLICY.md`](LICENSE_POLICY.md). + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/STATUS.md b/STATUS.md index 122fcc6..bc1344f 100644 --- a/STATUS.md +++ b/STATUS.md @@ -1,5 +1,13 @@ # STATUS +## Current review qualification — 2026-09-13 UTC + +OBSERVED base: e5ed91bd90afb0429cf816504c7d5e622bbbc153. Nine existing local tests pass; reproduced gaps remain in direct route denial, epistemic attribution and NC consistency. [Advisory specification](docs/START_HERE.md) now covers the full methodology; target runtime acceptance is NOT_COMPUTABLE, not complete. + +Latest reported HQ pool is 452 (300/50/102), keep_weak 180, OBSERVED labels 14, preferred gold_hq 242. The private pack has not been independently validated. Its quality/floor PASS entries below are historical reports. Training subset, permission semantics and label-hidden evaluation remain [open decisions](specs/decisions.md). name_gate remains false; no training, Hub, router, canon or Notion mutation occurs in this patch. + +## Historical status ledger — preserved, not refreshed proof + **Lane**: SHADOW **Date**: 2026-09-11 **Version**: 0.1.0 @@ -29,3 +37,5 @@ | Pre-rebalance zip | quarantined under `/workspace/semion-gold/quarantine/` (404-row pack) | | OBSERVED share | **14 / 452** (~3%) — hunt plan: `docs/amc-foundations-hunt-plan.md` | | Main tip | `e69e5bf` — M2 HQ pack handoff #6; AMC high7 docs PR advances tip | + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/docs/START_HERE.md b/docs/START_HERE.md new file mode 100644 index 0000000..0915fde --- /dev/null +++ b/docs/START_HERE.md @@ -0,0 +1,30 @@ +# Semion specification entry point + +Status: ADVISORY_REVIEW. This does not govern or activate. + +Semion classifies signs in circulation, adapts existing source packets, and exports one frame. It remains a BELIEF submodule. It does not own minds, Hyperlex form, Athanor tradition gold, Brier calibration, or Abraxas runtime chains. + +## Read in this order + +| Stage | Artifact | +|---|---| +| 1 Constitution/doctrine | [Unchanged root constitution](../constitution.md); [authority and decisions](../specs/decisions.md) | +| 2 Domain model | [Domain](../specs/domain-model.md) | +| 3 Requirements | [Requirements](../specs/requirements.md) | +| 4 Journeys | [Journeys](../specs/journeys.md) | +| 5 Workflows | [Workflow registry and specifications](../specs/workflows.md) | +| 6 State machines | [Transitions](../specs/state-machines.md) | +| 7 Contracts | [Contract catalog](../specs/contracts.md) | +| 8 Data model | [Data and provenance](../specs/data-model.md) | +| 9 Security/privacy/governance | [Boundaries](../specs/security-privacy-governance.md) | +| 10 Architecture | [Architecture](../ARCHITECTURE.md) | +| 11 Acceptance criteria | [Acceptance](../specs/acceptance.md) | +| 12 Traceability | [Traceability](../specs/traceability.json) | +| 13 Tasks | [Ordered implementation tasks](../specs/completion-tasks.md) | +| 14 Verification | [Verification and limitations](../specs/verification.md) | + +These documents complete the advisory specification structure, not runtime conformance, dataset certification, or E-S3. Existing numbered specifications retain their identity and historical cycle notes. New REQ/WF/SM/CON/AC/TASK IDs are repository-local, stable, and never renumbered; retire with a replacement reference. + +Start with [current evidence and conflicts](../specs/assessment.md). Proposed behavioral tightening is explicitly not shipped. The eight open decisions prevent an unconditional implementation-ready claim. No new operator role or authority layer is created. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/docs/aaron-train-handoff.md b/docs/aaron-train-handoff.md index b05f133..d5059fe 100644 --- a/docs/aaron-train-handoff.md +++ b/docs/aaron-train-handoff.md @@ -1,5 +1,7 @@ # Aaron train handoff — Semion M2 HQ pack +Current advisory qualification (2026-09-13 UTC): the instructions below are preserved historical source text, not a new train/transfer authorization. The full 452-row pool and preferred 242-row HQ subset are different datasets; do not train heldout rows or claim the subset satisfies full-pool floors. Resolve [DEC-001, DEC-002 and DEC-006](../specs/decisions.md) before use. Pack contents, hash, rights and actual authorization remain NOT_COMPUTABLE in this review. No Notion page or private pack was changed. + **Audience:** Aaron Godbout (Spark) **Date:** 2026-09-11 PT **Repo pin:** `scrimshawlife-ctrl/Semion@e69e5bf` (main tip before this docs PR; docs tip advances on merge) @@ -64,3 +66,5 @@ Gold pool **452**: OBSERVED **14** · inferred wrap **~210** · inferred seed ** - Train from the quarantined pre-rebalance (404-row) zip - Train from the refuse-reshuffle pack (superseded; pre-HQ leakfix) - Grow page-dump wrap mass or pad to 600 + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/docs/index.md b/docs/index.md index eaa8080..7840779 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,6 +1,11 @@ # Semion docs +- [Start here: complete advisory specification](START_HERE.md) +- [Evidence, gaps and implementation readiness](../specs/assessment.md) + - [Quickstart](quickstart.md) - [Dual-use](dual-use.md) - Constitution: `../.specify/memory/constitution.md` - Specs: `../specs/` + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/out/audit/spec-completion.latest.json b/out/audit/spec-completion.latest.json new file mode 100644 index 0000000..7e67591 --- /dev/null +++ b/out/audit/spec-completion.latest.json @@ -0,0 +1,41 @@ +{ + "schema_version": "semion.advisory.spec_completion_receipt.v1", + "boundary": "Advisory specification reconciliation only. This does not govern or activate. No runtime, constitution, runtime schema, corpus, training, Hub, Notion or merge mutation.", + "evidence_packet": { + "base_commit": "e5ed91bd90afb0429cf816504c7d5e622bbbc153", + "repo": "https://github.com/scrimshawlife-ctrl/Semion", + "graph_project": "Semion", + "base_graph_nodes": 620, + "base_graph_edges": 801, + "latest_corpus_counts_reported_not_verified": {"gold": 452, "train": 300, "val": 50, "test": 102, "keep_weak": 180, "observed_labels": 14, "preferred_hq_pool": 242}, + "independent_corpus_quality": "NOT_COMPUTABLE", + "independent_pack_integrity": "NOT_COMPUTABLE", + "e_s3": "NOT_COMPUTABLE", + "notion_source": "https://app.notion.com/p/3d83e8ba2f5c81608499deffc060160d", + "notion_source_last_edited_at": "2026-09-12T00:58:21.253Z", + "notion_native_verification": "unverified", + "notion_writeback": "NOT_PERFORMED" + }, + "task": "Complete a repository-relevant advisory specification across all 14 methodology stages, with stable workflows and explicit unresolved decisions.", + "output_expected": "Reviewable documentation, verification script, sanitized receipt and PR; proposed behavior is not marked shipped.", + "validation": { + "status": "PASSED_LOCAL", + "methodology_stages": 14, + "requirements": 20, + "workflows": 10, + "workflow_fields_each": 16, + "state_machines": 6, + "contracts": 6, + "acceptance_bundles": 10, + "implementation_tasks": 10, + "open_decisions": 8, + "documentation_negative_controls": 11, + "existing_pytest_passed": 9, + "runtime_target_conformance": "NOT_COMPUTABLE" + }, + "residual_risks": ["Eight decisions remain open", "Runtime gaps remain unfixed", "Private data and external consumer are unverified", "No train/name/Hub authority inferred"], + "recommended_next_advisory_action": "Operator reviews DEC-001 through DEC-008; then bounded T0 validation and refusal tasks before a separate encoder cycle.", + "authorization": "Operator approved local specification reconciliation, then requested handling PRs from this chat. No merge or runtime authority inferred.", + "governance_route": "N/A: no Abraxas Loop 805 orchestration surface exists in Semion. No substitute governance created; Sprint 001 linkage NOT_COMPUTABLE.", + "provenance": "Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base)" +} diff --git a/specs/000-semion-spine/spec.md b/specs/000-semion-spine/spec.md index 77a488a..1b8ef64 100644 --- a/specs/000-semion-spine/spec.md +++ b/specs/000-semion-spine/spec.md @@ -1,5 +1,9 @@ # Spec 000 — Semion spine +Advisory completion: [methodology index](../../docs/START_HERE.md), [requirements](../requirements.md), [workflows](../workflows.md), [contracts](../contracts.md), [acceptance](../acceptance.md), [traceability](../traceability.json), and [tasks](../completion-tasks.md). Original cycle text below is historical; proposed target behavior and unresolved decisions are separate from current runtime proof. This does not govern or activate. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) + **Feature**: Specialist home for sign-relation work in the Abraxas stack **Date**: 2026-09-11 **Status**: SHADOW specify diff --git a/specs/001-semion-triad/spec.md b/specs/001-semion-triad/spec.md index 4b52240..a3cac90 100644 --- a/specs/001-semion-triad/spec.md +++ b/specs/001-semion-triad/spec.md @@ -1,5 +1,9 @@ # Spec 001 — Semion triad T0 +Advisory completion: [methodology index](../../docs/START_HERE.md), [requirements](../requirements.md), [workflows](../workflows.md), [contracts](../contracts.md), [acceptance](../acceptance.md), [traceability](../traceability.json), and [tasks](../completion-tasks.md). Original cycle text below is historical; proposed target behavior and unresolved decisions are separate from current runtime proof. This does not govern or activate. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) + **Feature**: Map symbolic-corpus atoms to `semion.frame.v0` **Date**: 2026-09-11 **Status**: SHADOW specify + T0 code in-repo diff --git a/specs/002-corpus-adapt/spec.md b/specs/002-corpus-adapt/spec.md index 5e54b17..78811c3 100644 --- a/specs/002-corpus-adapt/spec.md +++ b/specs/002-corpus-adapt/spec.md @@ -1,5 +1,9 @@ # Spec 002 — Corpus adapt +Advisory completion: [methodology index](../../docs/START_HERE.md), [requirements](../requirements.md), [workflows](../workflows.md), [contracts](../contracts.md), [acceptance](../acceptance.md), [traceability](../traceability.json), and [tasks](../completion-tasks.md). Original cycle text below is historical; proposed target behavior and unresolved decisions are separate from current runtime proof. This does not govern or activate. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) + **Feature**: Field map from AMC / Foundations / TEACH atoms onto Semion input **Date**: 2026-09-11 **Status**: SHADOW specify + T0 adapter in-repo diff --git a/specs/003-semiosis-chain/spec.md b/specs/003-semiosis-chain/spec.md index 406f4f0..aa842da 100644 --- a/specs/003-semiosis-chain/spec.md +++ b/specs/003-semiosis-chain/spec.md @@ -1,5 +1,9 @@ # Spec 003 — Semiosis chain +Advisory completion: [methodology index](../../docs/START_HERE.md), [requirements](../requirements.md), [workflows](../workflows.md), [contracts](../contracts.md), [acceptance](../acceptance.md), [traceability](../traceability.json), and [tasks](../completion-tasks.md). Original cycle text below is historical; proposed target behavior and unresolved decisions are separate from current runtime proof. This does not govern or activate. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) + **Feature**: Export `semion.frame.v0` as a SemiosisFrame-shaped dict without owning YGGDRASIL **Date**: 2026-09-11 **Status**: SHADOW specify + export compat in-repo. Rune mint closed. diff --git a/specs/004-encoder-gate/dataset.md b/specs/004-encoder-gate/dataset.md index 7e232b4..b6ea34e 100644 --- a/specs/004-encoder-gate/dataset.md +++ b/specs/004-encoder-gate/dataset.md @@ -1,5 +1,7 @@ # Dataset — Semion encoder (gated) +Review qualification (2026-09-13 UTC): this dated dataset note is retained as source evidence, not independent validation of the private corpus. The latest 452-row report is consistent with the linked Notion handoff; 242 preferred HQ rows alone cannot meet the 300-row training floor. Effective subset, permission timing and fair T0 comparison are unresolved in [DEC-001/002/006](../decisions.md). No floor is waived and no training is authorized by this patch. See [WF-005 through WF-009](../workflows.md) and [CON-004 through CON-006](../contracts.md). + **Status**: SHADOW note. Not a harvest. name_gate still false. **Date**: 2026-09-11 **Amend**: M2 rebalance doctrine (2026-09-11 PT) — wrap/seed as lawful INFERRED gold; OBSERVED AMC/Foundations still preferred. @@ -121,3 +123,5 @@ N in the gate is the **floor** table above until an operator raises it in this f ## Hunt plan pointer See [`docs/amc-foundations-hunt-plan.md`](../../docs/amc-foundations-hunt-plan.md). + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/004-encoder-gate/spec.md b/specs/004-encoder-gate/spec.md index 751fc77..1608e18 100644 --- a/specs/004-encoder-gate/spec.md +++ b/specs/004-encoder-gate/spec.md @@ -1,5 +1,9 @@ # Spec 004 — Encoder gate (closed) +Advisory completion: [methodology index](../../docs/START_HERE.md), [requirements](../requirements.md), [workflows](../workflows.md), [contracts](../contracts.md), [acceptance](../acceptance.md), [traceability](../traceability.json), and [tasks](../completion-tasks.md). Original cycle text below is historical; proposed target behavior and unresolved decisions are separate from current runtime proof. This does not govern or activate. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) + **Feature**: Conditions under which `semion-encoder-*` may exist as weights **Date**: 2026-09-11 **Status**: SHADOW specify. Train closed. name_gate false. diff --git a/specs/acceptance.md b/specs/acceptance.md new file mode 100644 index 0000000..934eb7b --- /dev/null +++ b/specs/acceptance.md @@ -0,0 +1,105 @@ +# Acceptance criteria + +All cases below are proposed runtime/data acceptance, not passed by specification lint. Each requires positive, negative, and failure/recovery controls. Test owners follow completion-tasks.md; closure records exact revision, environment, input and output hashes. + +## AC-001 + +Workflow: WF-001 + +Given each supported alias, explicit null, Unicode input and single/multiple inventory buckets, normalization is deterministic and preserves source identity. Conflicting aliases, non-object roots, oversized input, malformed inventories and invalid JSONL fail with bounded errors and row position; no partial committed batch or evidence upgrade. + +Current evidence: Existing E-C0/E-C1 fixtures cover only a subset; add parametrized boundaries. + +Target conformance: NOT_COMPUTABLE. + +## AC-002 + +Workflow: WF-002 + +Given forecast_request, settled_forecast, bare slang_atom, tradition_atom, unknown payload and restricted interpretant, every entry point returns denial/non-sign or a bounded input error. Direct classify cannot bypass it. Benign smoke/flag controls still pass; refused export has no active action and frozen false/null lanes. + +Current evidence: Existing forecast tests pass; bare slang positive counterexample reproduced. + +Target conformance: NOT_COMPUTABLE. + +## AC-003 + +Workflow: WF-003 + +Given approved label/NC precedence, smoke/flag/mixed/no-hint/missing-object inputs produce expected class and is_sign. A corpus_ref alone never becomes label evidence or OBSERVED; explicit source epistemic is not upgraded. Repeat calls yield equal JSON; invalid class types do not crash. All frame fields and cross-field implications validate. + +Current evidence: Three evidence/precedence gaps reproduced; DEC-003 and DEC-005 must close. + +Target conformance: NOT_COMPUTABLE. + +## AC-004 + +Workflow: WF-004 + +Given valid, refused, NC, inconsistent and unknown-failure frames, exporter rejects invalid frames and suppresses all failed actions. Valid export preserves triad/reference and false/null lanes with no imports, network or rune writes. Pinned external schema validates all positive controls; missing schema is NOT_COMPUTABLE. + +Current evidence: Two current bridge tests pass, external schema not inspected; DEC-004 open. + +Target conformance: NOT_COMPUTABLE. + +## AC-005 + +Workflow: WF-005 + +Given reviewed observed/inferred, weak, unknown-rights, disputed and NC-positive rows, only properly approved eligible rows enter gold; inferred remains inferred, weak stays separate, NC-positive is rejected. Every included label links exact content and source/rights evidence. + +Current evidence: Current dataset schema accepts NC-positive counterexample; private rows unverified. + +Target conformance: NOT_COMPUTABLE. + +## AC-006 + +Workflow: WF-006 + +Given selected snapshots, enforce 300/50/100 and selected-pool class floors; independently check NC train count, symbol share, group/source/sign-form overlap, duplicates and exclusion manifest. A 242-row pool cannot PASS a 300-row training floor. Changing one byte invalidates its hash; interrupted freeze cannot appear FROZEN. + +Current evidence: No private pack validation performed; DEC-002 and DEC-007 open. + +Target conformance: NOT_COMPUTABLE. + +## AC-007 + +Workflow: WF-007 + +Given approved identical label-hidden features and frozen rows, score T0 and authorized candidate with row-level predictions; recompute correct/evaluated, confusion and slice supports. Injected target labels, changed holdouts and missing predictions invalidate the benchmark; zero support or absent weights returns NOT_COMPUTABLE. Ties fail strict beat-T0 condition. + +Current evidence: No candidate or independently verified benchmark available; DEC-006 open. + +Target conformance: NOT_COMPUTABLE. + +## AC-008 + +Workflow: WF-008 + +Given evidence packets with every single required condition independently removed or failed, no review may open train/name/Hub gates. A complete synthetic packet can become REVIEW_READY only, never activation. Approval for transfer must not satisfy training or Hub. Existing E-S3 conditions remain unchanged. + +Current evidence: Document-only semantics; DEC-001 and DEC-008 open. + +Target conformance: NOT_COMPUTABLE. + +## AC-009 + +Workflow: WF-009 + +Given safe and tampered bundles, verify every relative member and byte hash; reject traversal, missing file, unknown rights or recipient, expired/mismatched scope and secret-bearing public receipts. Prepare-only stops at VERIFIED; transfer and acknowledgement remain separate and never authorize training. + +Current evidence: Notion page text read; original pack bytes and transfer not verified. + +Target conformance: NOT_COMPUTABLE. + +## AC-010 + +Workflow: WF-010 + +Given an approved correction, preserve prior row/snapshot bytes, link a new decision and snapshot, mark affected evaluations stale, and list impacted handoffs. An unapproved correction changes nothing. Notification remains pending until separately authorized and observed. + +Current evidence: No real corpus corrections or external notifications performed. + +Target conformance: NOT_COMPUTABLE. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/assessment.md b/specs/assessment.md new file mode 100644 index 0000000..cbfd366 --- /dev/null +++ b/specs/assessment.md @@ -0,0 +1,47 @@ +# Evidence-backed assessment + +## Boundary + +OBSERVED repository base: e5ed91bd90afb0429cf816504c7d5e622bbbc153. Advisory documentation only; root constitution, its mirror, runtime, runtime schemas, fixtures, training, Hub, and Notion are unchanged. One conceptual change: specification reconciliation and completion. Encoder implementation must remain a separate cycle. + +## Evidence packet + +- OBSERVED: five numbered specs exist, but workflows were four spine bullets with no stable IDs, complete exceptions, state contracts, or end-to-end traceability. +- OBSERVED: full code graph built under project Semion: 620 nodes and 801 edges at the base. Graph resolution found adapt, classify, and frame_to_semiosis. Counts describe indexing, not correctness. +- OBSERVED: nine existing local tests pass on Python 3.12. The baseline is smoke coverage, not a full contract proof. +- OBSERVED: README's 482/330/50/102 and keep_weak 150 conflict with later STATUS, dataset note, and Notion handoff's 452/300/50/102 and keep_weak 180. +- OBSERVED: the Notion handoff reports pack SHA256 af83977d1251b7f62b07a184f14fc34c52b603a4c54406609fb247f43f6156c8 and gold_hq 242. Native Notion verification was unverified. Attachment contents and hash were not independently validated. +- INFERRED: 452 is the latest reported snapshot, not a measured local dataset. The 242-row subset alone cannot meet a 300-row train floor, before even reserving holdouts. +- OBSERVED: root constitution is fuller than its abbreviated mirror; this patch references the root without amending either. +- OBSERVED: milestones and old checklists describe different dates; Abraxas router swap and consumer activation cannot be verified from this repo. + +## Reproduced runtime gaps + +| ID | Evidence at base | Consequence / next task | +|---|---|---| +| GAP-001 | classify accepts bare slang_atom with flag + symbol label as a sign | Spec 001 route rejection differs from adapter; TASK-002 | +| GAP-002 | sign_form=smoke + corpus_ref=unverified yields OBSERVED and is_sign=true without object or label | A reference is not evidence or a corpus label; TASK-003 | +| GAP-003 | explicit classification=NOT_COMPUTABLE + smoke/fire becomes index and OBSERVED | Abstention precedence needs a decision and test; TASK-003, DEC-003 | +| GAP-004 | dataset schema accepts NC with gold_is_sign=true | Cross-field rules missing; TASK-006 | +| GAP-005 | classify calls lower on caller classification without a type guard; output validation is absent | Malformed input can crash or violate frame types; TASK-001 | +| GAP-006 | compat forwards free-text interpretant, and handles only two failure strings | Closed action mapping and consumer validation not proven; TASK-004 | +| GAP-007 | frame schema does not require brier or failure; dataset source_lane is optional | Structural validation alone cannot enforce all prose constraints; TASK-001 and TASK-006 | +| GAP-008 | CLI loads VERSION outside the package; wheel installation not tested | Installed CLI behavior NOT_COMPUTABLE; TASK-009 | + +## Implementation-readiness + +T0 is an existing prototype with nine passing tests. Target contract conformance is NOT_COMPUTABLE until the acceptance cases in acceptance.md run against a separately approved implementation. Corpus quality, floor satisfaction, effective training subset, legal eligibility, operator train authorization, T1 performance, external SemiosisFrame compatibility, live routing, and release readiness are NOT_COMPUTABLE. + +No percentage-complete score is computed: artifact existence is not equivalent to coverage or acceptance. + +## Sources + +- [Pinned repository](https://github.com/scrimshawlife-ctrl/Semion/tree/e5ed91bd90afb0429cf816504c7d5e622bbbc153) +- [Semion HQ handoff](https://app.notion.com/p/3d83e8ba2f5c81608499deffc060160d), last edited 2026-09-12T00:58:21.253Z; source text only. +- Local paths: src/semion/classify.py, adapt.py, compat.py, __main__.py; schemas/; tests/; specs/004-encoder-gate/dataset.md. + +## Recommended next advisory action + +Review DEC-001 through DEC-008. Then implement TASK-001 through TASK-005 as bounded T0 work, with packet changes separate from any encoder cycle. Do not infer training permission from specification approval. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/completion-tasks.md b/specs/completion-tasks.md new file mode 100644 index 0000000..7972556 --- /dev/null +++ b/specs/completion-tasks.md @@ -0,0 +1,24 @@ +# Prioritized completion tasks + +All tasks are PROPOSED, not implemented by this documentation PR. Existing operator authority controls any corpus writes, training, handoff, consumer or publication. Owners: repository maintainer for T0, existing corpus/evaluation reviewer for data/eval, operator for consequential approval. + +| ID | Priority | Workflow | Deliverable | Concrete files/surface | Dependencies | +|---|---|---|---|---|---| +| TASK-001 | P0 | WF-001 | Typed shared input validation, alias conflicts, deterministic JSONL failure and limits | src/semion/adapt.py; shared validator; tests | DEC-007 only for later private storage; input policy can be reviewed independently | +| TASK-002 | P0 | WF-002 | Unify denial across adapter, direct classify, CLI and export; benign and hostile controls | src/semion/classify.py, adapt.py, compat.py; tests | TASK-001 | +| TASK-003 | P0 | WF-003 | Implement approved epistemic and class precedence with frame semantic validation | src/semion/classify.py; validation; tests | TASK-001, TASK-002; DEC-003, DEC-005 | +| TASK-004 | P1 | WF-004 | Pin external consumer schema and validate inert export with closed action mapping | src/semion/compat.py; contract fixtures; tests | TASK-003; DEC-004 | +| TASK-005 | P1 | WF-005 | Implement reviewed label/rights sidecars and separate eligible/weak/held outcomes | separate operator-approved corpus tooling; synthetic fixtures only in git | DEC-005, DEC-007; separate gold-settle scope | +| TASK-006 | P1 | WF-006 | Validate selected-snapshot floors, groups, provenance and immutable split hashes | separate snapshot tooling; synthetic manifests and tests | TASK-005; DEC-002, DEC-007 | +| TASK-007 | P2 | WF-007 | Freeze fair evaluation protocol and build scorer, no training in same packet/schema cycle | separate evaluation tools and fixtures; no required ML dependencies | TASK-003, TASK-006; DEC-001, DEC-006 | +| TASK-008 | P2 | WF-008 | Produce condition-by-condition E-S3 advisory report without opening any gates | evaluation receipt and existing operator review surface | TASK-007; DEC-001, DEC-008 | +| TASK-009 | P2 | WF-009 | Verify safe bundle/recipient scope and installed distribution behavior before authorized handoff | handoff validator; isolated wheel/CLI smoke tests; no corpus in repo | TASK-006, TASK-008; DEC-007, DEC-008 | +| TASK-010 | P1 | WF-010 | Implement append-only correction and stale-result impact analysis; reconcile documentation links | correction tooling; synthetic regression tests; affected spec docs | TASK-005, TASK-006; DEC-007, DEC-008 | + +## Definition of ready and done + +Ready: referenced decisions closed with evidence, workflow reviewed, contract pinned, scope approved, positive/negative/recovery fixtures specified. Done: associated AC passes at an exact commit, existing regression tests pass, no frozen-lane expansion, sanitized receipt attached, traceability status updated with real evidence. Tests that merely prove the current bug exists cannot close target acceptance. + +Sequence: T0 TASK-001 -> TASK-002 -> TASK-003 -> TASK-004. Data TASK-005 -> TASK-006 -> TASK-010 only under separate scope. Evaluation TASK-007 -> TASK-008, then TASK-009. Keep encoder implementation separate from packet/schema changes under constitution X. No new issues, milestone or Loop slice is invented by this plan; Loop 805/Sprint 001 applicability to this repo is N/A until an actual operator reference is provided. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/contracts.md b/specs/contracts.md new file mode 100644 index 0000000..c3b4bdc --- /dev/null +++ b/specs/contracts.md @@ -0,0 +1,55 @@ +# Contract catalog + +Status: PROPOSED, not runtime migration. Existing schemas remain unchanged. Contracts below specify validation behavior for a separate engineering cycle; unresolved DEC entries block affected claims. + +## CON-001 Input and normalized atom + +Entry points accept one JSON object, never an array/string/null root. Proposed limits: UTF-8 request <=64 KiB, representamen/sign_form <=512 Unicode code points, other text fields <=2048 code points, source ids <=256 code points, inventory arrays <=100 items each; reject before expensive processing. These are implementation-review targets, not measured current limits. + +Accepted payload_class is sign_atom or corpus_atom; absent payload permits legacy atom/source-shape validation. An unknown nonempty class is denied. Reject forecast_request, settled_forecast, bare slang_atom and tradition_atom consistently at every entry point. Mixed-domain routing remains upstream. + +Aliases: sign_form, representamen, raw_value, input_signal.raw_value; sign_content or input_signal.information_content; object_candidate or object; interpretant_candidate or interpretant.action_type; classification or sign_class or inventory. Presence, not truthiness, determines selection. Contradictory non-null aliases are an input conflict, not silently selected. Missing optional object/interpretant is null. Empty representamen cannot produce a sign. + +Typed sign_inventory has optional icons/indexes/symbols arrays of structural entries; one nonempty bucket gives its class, multiple buckets mixed, none NC absent a valid explicit label. Bucket population is not independent proof of label correctness. Reject invalid bucket types. Preserve original source locator and raw content hash outside the unchanged frame rather than forging schema fields. + +Batch adapter must specify UTF-8, blank-line skip, input row position, accepted/refused/invalid counts and per-row outcomes. Proposed batch default is fail whole batch on malformed JSON, retaining diagnostic position and no committed partial output; any later partial mode must be explicitly named. + +## CON-002 Frame + +Current shape is schemas/semion.frame.v0.schema.json, duplicated in contracts/semion.frame.v0.json. Proposed semantic validator additionally requires all frame fields emitted by classify, including brier=null and failure. + +is_sign=true requires nonempty representamen, computable class, failure=null, and actual object or supported label evidence. corpus_ref presence alone is insufficient. sign_class=NOT_COMPUTABLE implies is_sign=false. A refused frame has NC class/epistemic, failure=SPECIALIST_LANE_VIOLATION, no interpretant and false/null lanes. Missing evidence has a bounded NC reason and no invented reference. + +OBSERVED for a class requires the approved DEC-005 rubric and verifiable label evidence; an input label or source string alone is not proof. Heuristic labels are INFERRED. Explicit NC precedence and conflicting label semantics remain DEC-003. Raw arbitrary interpretant prose must not be promoted to executable action. + +Malformed input produces an input error envelope with code INVALID_INPUT, field/path and bounded reason, not a schema-invalid v0 frame. CLI should print structured error to stderr and nonzero exit; no traceback, raw restricted payload, or partial successful frame. Exit 0 indicates valid processing (including bounded abstention/refusal), not positive sign or authorization. These error-envelope and exit semantics are proposed, not shipped. + +## CON-003 Export + +Input must satisfy CON-002, not merely be an arbitrary dict. Preserve representamen -> input_signal.raw_value; corpus_ref -> frame_id (null allowed); is_sign/promotion_reason -> sign_status; object and sign_class; epistemic/failure -> provenance. Frozen lanes remain false/false/null. + +Refusal or NC yields is_sign=false and action_type=NOT_COMPUTABLE. Known successful action prefixes may only be mapped under DEC-004. STATE_UPDATE:alert is currently copied by tests; splitting it into action_type plus state_delta is a proposal awaiting the consumer schema, not a backwards-compatible fact. + +Omit source_space, target_space, focus_of_attention. Do not invent interpreter identity. Empty state_delta/external_effect/new_knowledge_units are inert placeholders in the current bridge, not proof of real updates. Unknown consumer schema: compatibility NOT_COMPUTABLE. A SemiosisFrame.v1 string alone proves no external schema conformance. + +## CON-004 Label and row evidence + +Existing dataset row fields remain as shipped. Proposed sidecar per row: row_id, content_sha256, source_locator, source_lane, source_run_id or explicit unavailable reason, label_decision_id/revision, reviewer_ref, operator_approval_ref when settlement occurs, label_evidence_refs, rights_evidence_refs, epistemic, eligibility, supersedes and UTC decision time. + +No unknown source identity or rights decision may silently become verified. NC implies gold_is_sign=false; positive gold_is_sign requires computable class and nonempty representamen. Weak/held/rejected membership stays outside gold. Source-lane presence is required for eligible selection even though v0 schema makes it optional. Sidecar links use exact content hashes to avoid changing v0 under this documentation patch. + +## CON-005 Snapshot and evaluation + +Manifest: schema/version, snapshot_id, parent_snapshot_ref, selected row id/hash/split/group entries, exclusion reasons, grouping/normalization version, source and class counts, epistemic counts, split hashes, quality check statuses, effective input fields, created_at and scope refs. Sort entries by row_id; hash canonical UTF-8 JSON with sorted keys and compact separators, no NaN. Hash raw artifact bytes separately; record hash kind to avoid confusing content-normalized and raw hashes. + +Existing floors: train>=300, val>=50, test>=100; icon/index/symbol >=15% each and mixed/NC >=8% each on selected gold pool. NC not in train under current HQ quality bar; train symbol share <=0.85. These coexist and can block small HQ subsets; do not waive either. Data targets are not minimum proof of correctness. Report exact row/sign_form/source_run/provenance overlap; near-duplicate pass awaits DEC-007 grouping rule. Freeze before candidate experimentation and disallow train/test group overlap. + +Benchmark output: model/code/config hashes, snapshot+feature-protocol hashes, ordered row predictions/targets, count correct and evaluated, confusion matrix, support per class and mixed/missing-object/refusal slices, failures, dual-use outcomes. Exact-match=correct/evaluated; evaluated=0 -> NOT_COMPUTABLE. Candidate must strictly exceed T0 on identical approved feature view to satisfy the existing comparison condition; a numeric tie fails. A statistical margin is not invented. Adaptation label fidelity is a separately named test, never substituted for predictive accuracy. Gold targets/classification/sign_class must not enter predictive features under the proposed DEC-006 protocol. + +## CON-006 Receipt and handoff evidence + +Proposed receipt: receipt_id, workflow_id, attempt_id, started_at/finished_at in valid UTC, status, input/config/code/output hash references, counts, bounded errors, evidence status and supersedes. No finish earlier than start. Failure requires reason. PASS requires evidence, not an empty list. Unknown measurement -> status NOT_COMPUTABLE, value null, reason required. Hash strings are lowercase SHA256, 64 hex digits. + +Handoff manifest additionally lists approved recipient/purpose, exact approval reference, each safe relative path and byte hash, snapshot reference, rights restrictions, exclusions and verification instructions. No absolute operator paths, secrets, raw private corpus text or temporary signed URLs in public receipts. Missing/invalid scope prevents transfer; a review receipt never functions as an executable permission token. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/data-model.md b/specs/data-model.md new file mode 100644 index 0000000..d7d441d --- /dev/null +++ b/specs/data-model.md @@ -0,0 +1,28 @@ +# Data model and lifecycle + +## Existing storage + +Runtime functions consume dicts and return dicts. Git contains schemas, seed fixtures and documentation, not the private corpus. Earlier spine atoms.jsonl and later dataset.jsonl name different artifacts; do not rename or import either. The latest reported gold SoT is the private dataset.jsonl plus separate keep_weak.jsonl. Its content is NOT_COMPUTABLE here. + +## Proposed logical relations + +| Relation | Key / references | Lifecycle and validation | +|---|---|---| +| SourceEvidence | source_id, raw content hash, source lane, locator, rights refs | Append observed source evidence; source existence is separate from class truth | +| AtomProjection | row_id + content hash -> SourceEvidence | Typed field projection per CON-001; cannot overwrite source | +| LabelDecision | decision id/revision -> row hash, evidence, reviewer, approval | ELIGIBLE/WEAK/HELD/REJECTED; preserve epistemic; correction links supersedes | +| Snapshot | snapshot id/hash -> selected row hashes and decisions | Immutable FROZEN membership; split/group assignment and exclusions explicit | +| Evaluation | evaluation id -> snapshot, engine, config, feature protocol | Predictions and support retained; stale after dependent correction | +| Handoff | handoff id -> snapshot/artifact hash manifest and scope | VERIFIED/DELIVERED/ACKNOWLEDGED distinct; receipt not authority | + +No database engine is mandated. A local append-only JSON/JSONL sidecar implementation is sufficient if atomic replacement of a new manifest is verified and prior snapshots remain unchanged. This is a proposal, not new storage or migration in this patch. + +## Identity and atomicity + +row_id is a stable locator; content_sha256 changes when text, label-bearing source, or normalization changes. Decisions reference exact content. Reject duplicate row_id with differing bytes within a snapshot; exact duplicate content is reviewed/grouped, not counted repeatedly to meet floors. Freeze writes a new manifest atomically only after all checks; interrupted attempts leave no FROZEN receipt. Corrections create a new revision and invalidate dependent claims. + +## Privacy and retention + +Private payloads remain in operator-controlled storage; Git receives schema/fixture and sanitized evidence only. No deletion schedule is invented: DEC-007 must establish retention and access before implementation. Rights withdrawal blocks further distribution while the operator decides lawful retention of existing evidence. No home directory is inspected, migrated or modified by these specs. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/decisions.md b/specs/decisions.md new file mode 100644 index 0000000..706fc9d --- /dev/null +++ b/specs/decisions.md @@ -0,0 +1,26 @@ +# Source precedence and unresolved decisions + +This is a projection of existing constraints, not a constitution amendment or new authority. The operator approved factual reconciliation and recording ambiguity; no training interpretation was approved. + +## Existing authority + +Root constitution.md is the full Semion constitution and says only the operator amends it. The .specify mirror is abbreviated, not expanded by this patch. External Abraxas canon and router state are dependencies, not facts this repository can certify. Later dated records can qualify historical counts, but cannot open a closed gate. + +## Decision register + +All entries are OPEN. Owner denotes an existing reviewing role, not newly granted authority. Missing decisions are NOT_COMPUTABLE, not defaults granting permission. + +| ID | Question and conflicting evidence | Safe proposed treatment | Existing reviewer / closure evidence | +|---|---|---|---| +| DEC-001 | Does E-S3 gate starting an experimental run, naming weights, or both? Constitution II requires beating T0 before trained encoder exists, while handoff says Aaron trains | No run authorized by this patch. Keep name_gate false; separate experiment permission from release proof in a future operator decision | Operator; exact action, artifact, machine, data hash, and sentence | +| DEC-002 | Is the effective training input full 300 split or preferred 242-row HQ pool? | Preserve split assignments; never train held NC/test rows. Recompute floors on actual selected subset, do not borrow full-pool counts | Operator + dataset reviewer; selected row manifest and counts | +| DEC-003 | Should explicit NC dominate heuristic hints, and how are conflicting labels handled? | Propose explicit NC abstention and contradictory labels to review, not inferred OBSERVED | Operator/domain reviewer; fixture-level adjudication | +| DEC-004 | What exact consumer action enum and SemiosisFrame.v1 schema apply? Existing tests use STATE_UPDATE:alert, dataset head uses STATE_UPDATE | Export remains compatibility-shaped only; proposed split action_type/state_delta requires verified consumer contract before change | Abraxas consumer maintainer; pinned schema and compatibility tests | +| DEC-005 | What evidence warrants OBSERVED for a class? Current code treats corpus_ref presence as enough | Propose independently referenced label review; source existence and label correctness tracked separately | Existing corpus reviewer; rubric and signed/attributed label evidence | +| DEC-006 | How should T0/T1 be compared when T0 reads classification labels? | Propose label-hidden feature-parity benchmark plus separate adapter fidelity test; no score until protocol frozen | Evaluation reviewer/operator; feature allowlist, slice hashes, metric protocol | +| DEC-007 | What retention, access, license scope and near-duplicate grouping apply to private corpus/handoff? | No export where rights/access unknown. Exact hashes required; near-duplicate threshold not invented | Operator/source steward; scoped policy reference and grouped manifest | +| DEC-008 | Which external router swap/replay, dual-use reviews and scoped approvals are current? | Report historical claims only; leave runtime/Hub closed in this task | Operator + existing HERMENEUT/ADVERSARY roles; source receipts and exact target | + +Do not relabel historical sources OBSERVED merely because these rows name a reviewer. Resolving a decision updates the decision with evidence, then its dependent requirements, workflows, acceptance cases, and tasks together. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/domain-model.md b/specs/domain-model.md new file mode 100644 index 0000000..c6c6d33 --- /dev/null +++ b/specs/domain-model.md @@ -0,0 +1,24 @@ +# Domain model + +## Entities and relationships + +- Source packet: existing AMC, Foundations, or another explicitly supported structural packet. Its source lane and content provenance are separate from inferred sign meaning. +- Atom: normalized representamen candidate, object candidate, interpretant candidate, class candidate, payload class, and source reference. Normalization does not create truth or permission. +- Sign relation: representamen refers to an object under an interpretant understood as a typed action/state delta. Missing object remains null; missing evidence remains NOT_COMPUTABLE. +- Frame: one semion.frame.v0 output. icon, index, symbol, mixed and NOT_COMPUTABLE are mutually exclusive class values. is_sign is an eligibility assertion within this structural task, never evidence of a mind. +- Label decision: target class/is_sign, reviewer, evidence, epistemic, and revision for a particular row content hash. GOLD membership is not identical to OBSERVED. +- Dataset snapshot: immutable selected rows with split membership, grouping and exclusions. A preferred subset is its own snapshot, not interchangeable with a parent pool. +- Evaluation: one pinned implementation and feature set scored on one frozen snapshot with row-level predictions. Evidence absence is not zero performance. +- Handoff: a recipient-scoped bundle of permitted artifacts with content hashes and restrictions. Receipt of a bundle is not authorization to train or publish. +- Correction: superseding decision linked to an earlier row/snapshot. Never silently rewrite a frozen test. +- Approval reference: evidence of an existing operator decision for an exact action. Merely storing it grants no authority. + +## Invariants + +Signs are not minds. Adaptation, classification, export, dataset membership, training permission, and publication permission are different predicates. A run id is a locator, not a label or license. A class may be computable while object remains null; epistemic must reflect the class evidence, not source string presence. NC is not a positive sign. Semion never sets phenomenal or forecast_eligible true, computes Brier, executes interpretants, mints runes, or mutates canon. + +## Boundaries + +Hyperlex owns lexical/form routes; Athanor owns tradition/family/correspondence gold; abx.brier owns settled calibration. Mixed packets are routed upstream first. Semion's exporter returns data only. Abraxas owns any chain lifecycle and consumer validation. This specification does not merge these repos or duplicate their governance. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/journeys.md b/specs/journeys.md new file mode 100644 index 0000000..ed91c92 --- /dev/null +++ b/specs/journeys.md @@ -0,0 +1,14 @@ +# Journeys + +| ID | Actor and goal | Sequence | Failure or handoff outcome | +|---|---|---|---| +| JRN-001 | Corpus user classifies a lawful existing sign | WF-001 then WF-003 | WF-002 returns bounded refusal; missing evidence produces NC | +| JRN-002 | Abraxas integrator obtains a structural export | WF-003 then WF-004 | Consumer incompatibility is reported, no runtime chain executed | +| JRN-003 | Corpus reviewer prepares an auditable selected dataset | WF-005 then WF-006 | Unknown rights/label evidence excluded or held; insufficient selected subset blocks readiness | +| JRN-004 | Evaluation reviewer compares an authorized candidate to T0 | WF-006 then WF-007 then WF-008 | Missing candidate, permission, or credible comparison leaves E-S3 NOT_COMPUTABLE | +| JRN-005 | Operator provides an authorized handoff | WF-008 then WF-009 | Unknown recipient scope or mismatched hash blocks transfer; receipt is not train/Hub permission | +| JRN-006 | Reviewer corrects a mislabeled or leaked record | WF-010 then WF-005 and WF-006 | Prior snapshot retained as superseded; affected results are invalidated, not edited silently | + +Workflows below requirements and journeys, above contracts and architecture, are mandatory. Each workflow specifies its own exceptions and evidence; these summaries do not substitute for them. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/requirements.md b/specs/requirements.md new file mode 100644 index 0000000..6d2598a --- /dev/null +++ b/specs/requirements.md @@ -0,0 +1,32 @@ +# Requirements + +Status: proposed acceptance targets, not a claim of current implementation. Existing constitution controls. DEC-001 through DEC-008 remain open; affected behavior may be specified as a proposal but not treated as ratified. Verification status is in traceability.json. + +| ID | Existing spec | Required target behavior | Workflow | +|---|---|---|---| +| REQ-001 | 000/002 | Accept only a JSON object with supported source shape and typed fields; malformed or oversized inputs yield a bounded input error without a traceback or partial success. | WF-001 | +| REQ-002 | 002 | Normalize supported aliases deterministically, preserve source identity and explicit nulls, and report conflicting aliases; adaptation must not upgrade epistemic evidence. | WF-001 | +| REQ-003 | 000/001/002 | Apply the same route rejection at direct classify, adapter, batch, and export entry points; forecast, settled forecast, bare slang/tradition and unknown nonempty payload classes must not become positive signs. | WF-002 | +| REQ-004 | 000/003 | Restricted-intent or malformed outputs remain non-sign with inert interpretant and hard false/null lanes; preserve a bounded reason without reproducing restricted instructions. | WF-002 | +| REQ-005 | 001 | Classify deterministically with explicit, reviewed precedence for valid labels, explicit NC, and heuristic hints; is_sign needs representamen plus object or actual supported label, not a reference alone. | WF-003 | +| REQ-006 | 000/001 | Carry label epistemic independently of source reference; missing or weak evidence cannot become OBSERVED. Every output obeys frozen false/null lanes and consistent failure/is_sign fields. | WF-003 | +| REQ-007 | 003 | Validate a frame before export; preserve refusal, triad fields and source references, and map action values only according to a pinned consumer contract. | WF-004 | +| REQ-008 | 003 | Export one data structure without importing Abraxas or writing runes, source_space, target_space, focus_of_attention, or runtime state; do not claim SemiosisFrame conformance until consumer validation exists. | WF-004 | +| REQ-009 | 004 | Bind each label to exact row content, source lane, rights evidence, epistemic, reviewer and decision revision; lawful inferred gold must remain labeled INFERRED. | WF-005 | +| REQ-010 | 004 | Exclude keep_weak from gold and exclude TEACH raw abstract/slang/tradition/forecast inputs from automatic gold; NC implies gold_is_sign=false. Report held and rejected rows separately. | WF-005 | +| REQ-011 | 004 | Freeze immutable row/split hashes and group related sources before evaluation; recompute floors on the actually selected subset and preserve holdout assignments. | WF-006 | +| REQ-012 | 004 | Measure the existing 300/50/100 split and class-floor rules separately from quality; report NC distribution, exact/near-duplicate leakage, source overlap and exclusions. Unknown checks cannot PASS. | WF-006 | +| REQ-013 | 004 | Score pinned T0 and any separately authorized candidate using identical approved feature views without target-label leakage; retain row predictions and per-class support. | WF-007 | +| REQ-014 | 004 | Report exact-match, class confusion, mixed/missing-object/refusal slices, failures and model/config/data hashes; absent candidate or frozen benchmark yields NOT_COMPUTABLE, not a fabricated score. | WF-007 | +| REQ-015 | 000/004 | Evaluate E-S3 from independent dataset, comparison, dual-use and exact operator evidence; an artifact name, delivery, or recorded count never opens the gate. | WF-008 | +| REQ-016 | 000/004 | Keep train, naming, Hub, router swap and canon decisions separate and action-scoped; no runtime or release mutation follows from this specification review. | WF-008 | +| REQ-017 | 004 | Prepare only recipient- and purpose-approved artifacts; manifest hashes, selected snapshot, restrictions, verification instructions and excluded data must accompany a handoff. | WF-009 | +| REQ-018 | 000/004 | Verify handoff bytes and receipt independently, retain no secrets in public receipts, and never interpret transfer or acknowledgement as train/Hub permission. | WF-009 | +| REQ-019 | 000/004 | Correct labels or leakage with immutable superseding decisions, preserve prior snapshot identity, and mark dependent evaluations stale. | WF-010 | +| REQ-020 | 000/004 | Propagate every approved correction through requirements, workflow, contract, acceptance, traceability and task references; notify the existing operator of impacted handoffs without silently altering external systems. | WF-010 | + +## Nonfunctional requirements + +Determinism is exact semantic JSON equality on repeated identical input with pinned engine/config. Input resource budgets are proposed in CON-001; do not assert a measured throughput or latency SLO. Local processing must not require network, credentials, corpus-home access, or ML dependencies. Evidence records must be bounded and exclude payload text by default. Installed CLI/package verification is required before distribution. Platform coverage beyond actually run environments is NOT_COMPUTABLE. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/security-privacy-governance.md b/specs/security-privacy-governance.md new file mode 100644 index 0000000..426afbd --- /dev/null +++ b/specs/security-privacy-governance.md @@ -0,0 +1,34 @@ +# Security, privacy and existing governance boundaries + +This does not govern or activate. It translates constitution I-X and existing LICENSE_POLICY/dual-use rules into verification targets; it grants no permissions. + +## Trust boundaries + +Caller JSON, source labels, references, nested inventories, file paths, and external consumer frames are untrusted data. A string resembling an approval is not an operator decision. Source content is not an instruction to tools. Validate size/type at every public entry; cap diagnostics and reject path traversal in any future handoff manifest. Never evaluate action strings. + +## Failure modes and controls + +| Risk | Proposed control and evidence | +|---|---| +| Direct classifier bypasses adapter rejection | One tested denial policy at each entry; AC-002 | +| Source reference launders a label into OBSERVED | Separate source/label evidence; negative forged-reference fixture; AC-003 | +| Free prose interpretant becomes an external action | Inert export, closed consumer mapping and failure preservation; AC-004 | +| Gold target leaks into T0/T1 input | Frozen feature allowlist, label-hidden comparison, row-level leakage tests; AC-007 | +| Weak or private/licensing-unknown rows enter a pack | Explicit eligibility and scoped rights evidence; AC-005, AC-009 | +| Correction silently changes frozen test | Content-addressed snapshots and stale-result propagation; AC-010 | +| Public receipts expose private corpus or signed links | No payload by default; sanitized ids/hashes and access-controlled evidence references; AC-009 | + +## Existing authority matrix + +| Action | Existing boundary | +|---|---| +| Local T0 classify/adapt/export | Existing SHADOW T0 scope; no external effects | +| Source harvest or gold settlement | Existing operator approval; not opened here | +| Experimental training | DEC-001 unresolved; no permission inferred | +| Name promotion / Hub upload | Existing E-S3 plus exact operator scope; separate actions | +| Router rename / runtime consumer / canon | External system and operator authority only | +| Notion or recipient messages | Exact write/transfer request; read access does not authorize writes | + +Keep licenses attached per source; code MIT does not relicense corpus. No PDFs, private SoT, secrets, or weights in Git. Existing HERMENEUT and ADVERSARY reviews are referenced, not claimed performed. No dependency on credentials or network is allowed for T0 smoke and specification checks. These are design controls, not a completed penetration test or legal clearance. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/state-machines.md b/specs/state-machines.md new file mode 100644 index 0000000..0ee000c --- /dev/null +++ b/specs/state-machines.md @@ -0,0 +1,20 @@ +# State machines + +These are proposed engineering states, not authority grants. Terminal means terminal for an attempt; a new attempt requires new input/revision and receipt. No timer or automatic retry grants missing permission. + +| ID | Scope | Allowed transitions and guards | +|---|---|---| +| SM-001 | Packet attempt | RECEIVED -> VALIDATING. Valid supported atom -> NORMALIZED. Invalid types/limits -> INVALID. Denied route/content -> REFUSED. NORMALIZED -> CLASSIFIED only with computable relation; otherwise ABSTAINED. CLASSIFIED/ABSTAINED/REFUSED -> EXPORT_VALIDATING on explicit export request; valid inert mapping -> EXPORTED, invalid mapping -> INVALID | +| SM-002 | Label review | CANDIDATE -> REVIEWING with source evidence. Approved coherent label/rights -> ELIGIBLE. Explicit weak decision -> WEAK. Unknown evidence -> HELD. Known disallowed source/rights -> REJECTED. Approved correction moves prior ELIGIBLE to SUPERSEDED and creates a new CANDIDATE | +| SM-003 | Snapshot | DRAFT -> VALIDATING. All selected-row eligibility, quality, grouping and approved floor checks satisfied -> FROZEN. Failure or unknown required check -> BLOCKED, with failed versus unknown checks distinguished. Approved revision creates new DRAFT and marks old FROZEN SUPERSEDED | +| SM-004 | Evaluation | REQUESTED -> CHECKING. Complete authorized pinned comparison -> SCORED. Missing required evidence -> NOT_COMPUTABLE. Leakage, row mismatch or invalid protocol -> INVALID. Superseding data/protocol -> STALE; never replace old score in place | +| SM-005 | Gate review | CLOSED -> REVIEWING on explicit evidence review. Every existing condition evidenced -> REVIEW_READY. Known failure -> BLOCKED. Missing evidence -> NOT_COMPUTABLE. No transition to OPEN exists here; activation remains outside this advisory specification | +| SM-006 | Handoff | DRAFT -> VERIFIED after scope/rights/hash check. Separate transfer permission and successful transfer -> DELIVERED; recipient confirmation -> ACKNOWLEDGED. Unknown permission -> BLOCKED; hash mismatch -> INVALID; interrupted transfer -> INCOMPLETE. Retry same bytes/destination only under still-valid scope; changed bytes restart DRAFT | + +## Cross-machine rules + +REFUSED and ABSTAINED are different outcomes but both export no active interpretant. HELD is not rejected forever; evidence can start a new review. ELIGIBLE does not imply OBSERVED, FROZEN does not mean trained, SCORED does not mean approved, REVIEW_READY does not open name_gate, and ACKNOWLEDGED does not imply permission to use. + +A correction propagates SUPERSEDED -> STALE across dependent snapshots/results and marks handoff impact, without remotely deleting or mutating anything. Duplicate attempts return the prior receipt only when workflow, input hash, config hash, and scope match; otherwise allocate a distinct attempt. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/traceability.json b/specs/traceability.json new file mode 100644 index 0000000..9b48f62 --- /dev/null +++ b/specs/traceability.json @@ -0,0 +1,347 @@ +{ + "status": "ADVISORY_REVIEW", + "base_commit": "e5ed91bd90afb0429cf816504c7d5e622bbbc153", + "methodology": [ + "Constitution/doctrine", + "Domain model", + "Requirements", + "Journeys", + "Workflows", + "State machines", + "Contracts", + "Data model", + "Security/privacy/governance", + "Architecture", + "Acceptance criteria", + "Traceability", + "Tasks", + "Verification" + ], + "rows": [ + { + "requirement": "REQ-001", + "workflow": "WF-001", + "state_machines": [ + "SM-001" + ], + "contracts": [ + "CON-001" + ], + "acceptance": "AC-001", + "task": "TASK-001", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Existing E-C0/E-C1 fixtures cover only a subset; add parametrized boundaries." + }, + { + "requirement": "REQ-002", + "workflow": "WF-001", + "state_machines": [ + "SM-001" + ], + "contracts": [ + "CON-001" + ], + "acceptance": "AC-001", + "task": "TASK-001", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Existing E-C0/E-C1 fixtures cover only a subset; add parametrized boundaries." + }, + { + "requirement": "REQ-003", + "workflow": "WF-002", + "state_machines": [ + "SM-001" + ], + "contracts": [ + "CON-001", + "CON-002", + "CON-003" + ], + "acceptance": "AC-002", + "task": "TASK-002", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Existing forecast tests pass; bare slang positive counterexample reproduced." + }, + { + "requirement": "REQ-004", + "workflow": "WF-002", + "state_machines": [ + "SM-001" + ], + "contracts": [ + "CON-001", + "CON-002", + "CON-003" + ], + "acceptance": "AC-002", + "task": "TASK-002", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Existing forecast tests pass; bare slang positive counterexample reproduced." + }, + { + "requirement": "REQ-005", + "workflow": "WF-003", + "state_machines": [ + "SM-001" + ], + "contracts": [ + "CON-001", + "CON-002" + ], + "acceptance": "AC-003", + "task": "TASK-003", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Three evidence/precedence gaps reproduced; DEC-003 and DEC-005 must close." + }, + { + "requirement": "REQ-006", + "workflow": "WF-003", + "state_machines": [ + "SM-001" + ], + "contracts": [ + "CON-001", + "CON-002" + ], + "acceptance": "AC-003", + "task": "TASK-003", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Three evidence/precedence gaps reproduced; DEC-003 and DEC-005 must close." + }, + { + "requirement": "REQ-007", + "workflow": "WF-004", + "state_machines": [ + "SM-001" + ], + "contracts": [ + "CON-002", + "CON-003" + ], + "acceptance": "AC-004", + "task": "TASK-004", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Two current bridge tests pass, external schema not inspected; DEC-004 open." + }, + { + "requirement": "REQ-008", + "workflow": "WF-004", + "state_machines": [ + "SM-001" + ], + "contracts": [ + "CON-002", + "CON-003" + ], + "acceptance": "AC-004", + "task": "TASK-004", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Two current bridge tests pass, external schema not inspected; DEC-004 open." + }, + { + "requirement": "REQ-009", + "workflow": "WF-005", + "state_machines": [ + "SM-002" + ], + "contracts": [ + "CON-004" + ], + "acceptance": "AC-005", + "task": "TASK-005", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Current dataset schema accepts NC-positive counterexample; private rows unverified." + }, + { + "requirement": "REQ-010", + "workflow": "WF-005", + "state_machines": [ + "SM-002" + ], + "contracts": [ + "CON-004" + ], + "acceptance": "AC-005", + "task": "TASK-005", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Current dataset schema accepts NC-positive counterexample; private rows unverified." + }, + { + "requirement": "REQ-011", + "workflow": "WF-006", + "state_machines": [ + "SM-003" + ], + "contracts": [ + "CON-004", + "CON-005" + ], + "acceptance": "AC-006", + "task": "TASK-006", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "No private pack validation performed; DEC-002 and DEC-007 open." + }, + { + "requirement": "REQ-012", + "workflow": "WF-006", + "state_machines": [ + "SM-003" + ], + "contracts": [ + "CON-004", + "CON-005" + ], + "acceptance": "AC-006", + "task": "TASK-006", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "No private pack validation performed; DEC-002 and DEC-007 open." + }, + { + "requirement": "REQ-013", + "workflow": "WF-007", + "state_machines": [ + "SM-004" + ], + "contracts": [ + "CON-005", + "CON-006" + ], + "acceptance": "AC-007", + "task": "TASK-007", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "No candidate or independently verified benchmark available; DEC-006 open." + }, + { + "requirement": "REQ-014", + "workflow": "WF-007", + "state_machines": [ + "SM-004" + ], + "contracts": [ + "CON-005", + "CON-006" + ], + "acceptance": "AC-007", + "task": "TASK-007", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "No candidate or independently verified benchmark available; DEC-006 open." + }, + { + "requirement": "REQ-015", + "workflow": "WF-008", + "state_machines": [ + "SM-005" + ], + "contracts": [ + "CON-005", + "CON-006" + ], + "acceptance": "AC-008", + "task": "TASK-008", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Document-only semantics; DEC-001 and DEC-008 open." + }, + { + "requirement": "REQ-016", + "workflow": "WF-008", + "state_machines": [ + "SM-005" + ], + "contracts": [ + "CON-005", + "CON-006" + ], + "acceptance": "AC-008", + "task": "TASK-008", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Document-only semantics; DEC-001 and DEC-008 open." + }, + { + "requirement": "REQ-017", + "workflow": "WF-009", + "state_machines": [ + "SM-006" + ], + "contracts": [ + "CON-005", + "CON-006" + ], + "acceptance": "AC-009", + "task": "TASK-009", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Notion page text read; original pack bytes and transfer not verified." + }, + { + "requirement": "REQ-018", + "workflow": "WF-009", + "state_machines": [ + "SM-006" + ], + "contracts": [ + "CON-005", + "CON-006" + ], + "acceptance": "AC-009", + "task": "TASK-009", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "Notion page text read; original pack bytes and transfer not verified." + }, + { + "requirement": "REQ-019", + "workflow": "WF-010", + "state_machines": [ + "SM-002", + "SM-003", + "SM-004" + ], + "contracts": [ + "CON-004", + "CON-005", + "CON-006" + ], + "acceptance": "AC-010", + "task": "TASK-010", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "No real corpus corrections or external notifications performed." + }, + { + "requirement": "REQ-020", + "workflow": "WF-010", + "state_machines": [ + "SM-002", + "SM-003", + "SM-004" + ], + "contracts": [ + "CON-004", + "CON-005", + "CON-006" + ], + "acceptance": "AC-010", + "task": "TASK-010", + "verification": "specs/verification.md", + "runtime_status": "NOT_COMPUTABLE", + "evidence": "No real corpus corrections or external notifications performed." + } + ], + "provenance": "Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base)" +} diff --git a/specs/verification.md b/specs/verification.md new file mode 100644 index 0000000..8ad8d63 --- /dev/null +++ b/specs/verification.md @@ -0,0 +1,29 @@ +# Verification plan and measured scope + +## Reproducible documentation checks + +Run from repository root with Python >=3.10: + +```text +python -B specs/verification.py +python -B -m pytest -q -p no:cacheprovider +git diff --check +``` + +The specification checker uses the standard library only. It checks workflow fields, unique definitions, the 14-stage index, traceability references, local Markdown destinations, fixed-lane documentation, protected runtime/constitution/schema identity against the reviewed base, and unchanged duplicate frame schemas. Eleven negative controls remove a workflow field, duplicate a requirement, break a trace reference, falsely claim PASS, and inject duplicates into each of seven definition categories before conversion to sets; each must be detected. These checks measure documentation integrity, not runtime acceptance. + +Existing pytest requires the declared development dependency. Read scripts before executing. No check needs network, home corpus, training, credentials, or external writes. Local Python 3.12 is the measured environment; 3.10/3.11 and hosted CI results must be reported only if observed separately. + +## Baseline and target evidence + +The reviewed base has nine existing tests. They exercise smoke/flag, one forecast rejection, simple field mapping and two exports. They do not prove typed input, direct slang/tradition denial, epistemic soundness, complete dual-use refusal, dataset quality, installed wheel behavior or external SemiosisFrame schema compatibility. + +All AC-001 through AC-010 target conformance remains NOT_COMPUTABLE. A later implementation must run their positive, negative, boundary and recovery cases and store exact code/config/input/output hashes. Historical test evidence cannot close a newly proposed requirement. + +## Release of evidence, not authority + +Every run distinguishes OBSERVED test outcomes from INFERRED design implications and NOT_COMPUTABLE unavailable artifacts. Review all changed files and verify only README, STATUS, ARCHITECTURE, docs, specs and sanitized out/audit receipt paths changed. Root constitution, its mirror, src, schemas, contracts, tests, fixtures, dependency metadata and CI are unchanged. + +A PR check may show the existing smoke suite, not this new specification checker unless explicitly added in a separate CI change. No auto-merge, workflow dispatch, training or Hub action follows from green checks. + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/verification.py b/specs/verification.py new file mode 100644 index 0000000..1f2fc6c --- /dev/null +++ b/specs/verification.py @@ -0,0 +1,143 @@ +"""Check advisory spec integrity, not runtime conformance. No network or data writes.""" +import copy +import json +import re +import shutil +import subprocess +from pathlib import Path +from urllib.parse import unquote + +ROOT = Path(__file__).resolve().parents[1] +BASE = "e5ed91bd90afb0429cf816504c7d5e622bbbc153" +FIELDS = ["Purpose", "Actors", "Triggers", "Preconditions", "Inputs", "Happy path", + "Alternate/failure paths", "State transitions", "Terminal states", "Side effects", + "Invariants", "Permissions", "Observability/audit", "Acceptance criteria", + "Dependencies", "Unresolved items"] +STAGES = ["Constitution/doctrine", "Domain model", "Requirements", "Journeys", "Workflows", + "State machines", "Contracts", "Data model", "Security/privacy/governance", + "Architecture", "Acceptance criteria", "Traceability", "Tasks", "Verification"] + + +def read(rel): + return (ROOT / rel).read_text(encoding="utf-8") + + +def workflow_errors(text): + errors = [] + chunks = re.split(r"(?m)^## (WF-\d{3})\s*$", text) + ids = chunks[1::2] + if len(ids) != len(set(ids)) or not ids: + errors.append("workflow ids") + for wid, body in zip(chunks[1::2], chunks[2::2]): + for field in FIELDS: + matches = re.findall(r"(?m)^\| " + re.escape(field) + r" \| (.+) \|$", body) + if len(matches) != 1 or not matches[0].strip(): + errors.append(wid + ":" + field) + return errors + + +def requirement_errors(text): + ids = re.findall(r"(?m)^\| (REQ-\d{3}) \|", text) + return not ids or len(ids) != len(set(ids)) + + +def duplicate_errors(definitions): + return [key for key, values in definitions.items() + if not values or len(values) != len(set(values))] + + +def trace_errors(trace, definitions): + errors = [] + rows = trace["rows"] + ids = [r["requirement"] for r in rows] + if set(ids) != definitions["REQ"] or len(ids) != len(set(ids)): + errors.append("requirement coverage") + for row in rows: + for key, prefix in [("requirement", "REQ"), ("workflow", "WF"), + ("acceptance", "AC"), ("task", "TASK")]: + if row[key] not in definitions[prefix]: + errors.append(key) + for key, prefix in [("state_machines", "SM"), ("contracts", "CON")]: + if not row[key] or any(v not in definitions[prefix] for v in row[key]): + errors.append(key) + if row["runtime_status"] != "NOT_COMPUTABLE" or not row["evidence"].strip(): + errors.append("unsupported conformance") + if not (ROOT / row["verification"]).is_file(): + errors.append("verification path") + for key, prefix in [("workflow", "WF"), ("acceptance", "AC"), ("task", "TASK")]: + if {r[key] for r in rows} != definitions[prefix]: + errors.append("unmapped " + key) + return errors + + +def main(): + workflows = read("specs/workflows.md") + requirements = read("specs/requirements.md") + definitions = { + "WF": re.findall(r"(?m)^## (WF-\d{3})\s*$", workflows), + "REQ": re.findall(r"(?m)^\| (REQ-\d{3}) \|", requirements), + "SM": re.findall(r"(?m)^\| (SM-\d{3}) \|", read("specs/state-machines.md")), + "CON": re.findall(r"(?m)^## (CON-\d{3}) ", read("specs/contracts.md")), + "AC": re.findall(r"(?m)^## (AC-\d{3})\s*$", read("specs/acceptance.md")), + "TASK": re.findall(r"(?m)^\| (TASK-\d{3}) \|", read("specs/completion-tasks.md")), + "DEC": re.findall(r"(?m)^\| (DEC-\d{3}) \|", read("specs/decisions.md")), + } + assert not duplicate_errors(definitions), duplicate_errors(definitions) + for key in definitions: + duplicate = copy.deepcopy(definitions) + duplicate[key].append(duplicate[key][0]) + assert duplicate_errors(duplicate) == [key], key + definitions = {key: set(values) for key, values in definitions.items()} + assert not workflow_errors(workflows) + assert not requirement_errors(requirements) + trace = json.loads(read("specs/traceability.json")) + assert trace["methodology"] == STAGES + index = read("docs/START_HERE.md") + for n, stage in enumerate(STAGES, 1): + assert f"| {n} {stage} |" in index + assert not trace_errors(trace, definitions) + assert workflow_errors(workflows.replace("| Permissions |", "| Removed |", 1)) + row = next(line for line in requirements.splitlines() if line.startswith("| REQ-")) + assert requirement_errors(requirements + "\n" + row) + broken = copy.deepcopy(trace) + broken["rows"][0]["workflow"] = "WF-999" + assert trace_errors(broken, definitions) + broken = copy.deepcopy(trace) + broken["rows"][0]["runtime_status"] = "PASS" + assert trace_errors(broken, definitions) + + git = shutil.which("git") or r"C:\Program Files\Git\cmd\git.exe" + def command(*args): + return subprocess.check_output([git, *args], cwd=ROOT, text=True, encoding="utf-8") + changed = set(command("diff", "--name-only", BASE).splitlines()) + changed.update(command("ls-files", "--others", "--exclude-standard").splitlines()) + # This exact sanitized receipt is intentionally included even if out/ is ignored. + receipt = "out/audit/spec-completion.latest.json" + if (ROOT / receipt).is_file(): + changed.add(receipt) + assert changed + allowed = {"README.md", "STATUS.md", "ARCHITECTURE.md"} + for rel in sorted(changed): + assert rel in allowed or rel.startswith(("specs/", "docs/", "out/audit/")), rel + text = read(rel) + assert "Provenance:" in text and BASE in text, rel + if rel.endswith(".md"): + for prefix, ids in definitions.items(): + for ref in re.findall(r"\b" + prefix + r"-\d{3}\b", text): + assert ref in ids, (rel, ref) + for link in re.findall(r"\]\(([^)]+)\)", text): + if ":" in link or link.startswith("#"): + continue + target = (ROOT / rel).parent / unquote(link.split("#", 1)[0]) + target = target.resolve() + assert target.is_relative_to(ROOT) and target.exists(), (rel, link) + assert json.loads(read("schemas/semion.frame.v0.schema.json")) == json.loads(read("contracts/semion.frame.v0.json")) + subprocess.run([git, "diff", "--check", BASE], cwd=ROOT, check=True) + print("SEMION_SPEC_PASS " + " ".join(f"{k}={len(v)}" for k, v in definitions.items()) + + f" stages={len(STAGES)} workflow_fields={len(FIELDS)} negative_controls={4 + len(definitions)} changed_files={len(changed)}") + + +if __name__ == "__main__": + main() + +# Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base) diff --git a/specs/workflows.md b/specs/workflows.md new file mode 100644 index 0000000..02975bf --- /dev/null +++ b/specs/workflows.md @@ -0,0 +1,230 @@ +# Workflows + +Status: proposed specification. This does not govern or activate. All 16 fields are mandatory. Existing runtime gaps are listed in assessment.md; these workflows describe target behavior, not claims of conformance. + +## Registry + +| ID | Purpose | +|---|---| +| WF-001 | Normalize existing source packet | +| WF-002 | Refuse out-of-lane and restricted requests | +| WF-003 | Classify a normalized sign atom | +| WF-004 | Export one compatible semiosis structure | +| WF-005 | Review corpus labels and eligibility | +| WF-006 | Freeze and validate selected dataset | +| WF-007 | Evaluate frozen T0 and candidate | +| WF-008 | Review existing encoder gates without activation | +| WF-009 | Prepare and verify authorized handoff | +| WF-010 | Correct labels, leaks, or stale evidence | + +## WF-001 + +| Field | Specification | +|---|---| +| Purpose | Normalize existing source packet | +| Actors | Caller; adapter; existing source steward | +| Triggers | Explicit adapt call or one batch row | +| Preconditions | Existing source packet in scope; no scrape requested | +| Inputs | JSON object; aliases and typed inventories per CON-001 | +| Happy path | Check type and limits; check route first; detect alias conflicts; normalize selected fields; preserve provenance; return accepted atom | +| Alternate/failure paths | Malformed inventory/type returns input error; alias conflict is held for review; missing optional object stays null; rejected routes go to WF-002; a failed batch row cannot disappear | +| State transitions | SM-001: RECEIVED -> VALIDATING -> NORMALIZED; invalid -> INVALID; lane denial -> REFUSED | +| Terminal states | NORMALIZED, INVALID, REFUSED | +| Side effects | Return data only; proposed caller-owned receipt, no source write | +| Invariants | Normalization does not label GOLD or upgrade evidence; repeated input gives same result | +| Permissions | Existing caller's local read scope only; source harvesting is not implied | +| Observability/audit | CON-006 row index, source hash, schema/map version, status and bounded reason; no raw source in public logs | +| Acceptance criteria | AC-001 | +| Dependencies | REQ-001, REQ-002; CON-001; existing Spec 002 | +| Unresolved items | DEC-005 evidence mapping; proposed resource limits require implementation review | + +## WF-002 + +| Field | Specification | +|---|---| +| Purpose | Refuse out-of-lane and restricted requests | +| Actors | Caller; adapter/classifier/export boundary | +| Triggers | Denied payload class, restricted action content, invalid frame, or failed upstream row | +| Preconditions | No authorization inferred from packet fields | +| Inputs | Original input classification and bounded error category, not executable instructions | +| Happy path | Identify denial before heuristics; return NC/non-sign; clear active interpretant; force forecast/phenomenal false and brier null; carry safe diagnostic | +| Alternate/failure paths | Unknown nonempty payload class refused; malformed JSON uses input error not a fake frame; benign structural discussion must remain allowed; batch refusal retained as an outcome | +| State transitions | SM-001: VALIDATING -> REFUSED; export validation failure -> INVALID | +| Terminal states | REFUSED or INVALID | +| Side effects | No routing execution or external retry; return bounded result | +| Invariants | No positive sign or actionable content in refused export; source evidence unchanged | +| Permissions | No special approval can be manufactured by a payload; existing doctrine bounds all callers | +| Observability/audit | Reason code and workflow id; restricted content excluded; counters distinguish invalid, NC and refusal | +| Acceptance criteria | AC-002 | +| Dependencies | REQ-003, REQ-004; CON-001, CON-002, CON-003; dual-use-gate.md | +| Unresolved items | DEC-004 consumer action contract; content rubric needs existing domain/security reviewer | + +## WF-003 + +| Field | Specification | +|---|---| +| Purpose | Classify a normalized sign atom | +| Actors | Corpus user; T0 classifier; label reviewer for disputed labels | +| Triggers | Accepted atom sent to classify, including direct API/CLI calls | +| Preconditions | Boundary validated; reviewed label precedence required before target behavior ships | +| Inputs | Normalized atom and optional verified label-evidence context; CON-001 | +| Happy path | Validate direct input; preserve object null; select class by approved precedence; calculate is_sign from actual label/object evidence; attach honest epistemic; validate complete frame | +| Alternate/failure paths | No hint or insufficient relation gives NC; explicit NC proposed to abstain; conflicting label evidence held/NC; bad types produce bounded error; heuristic result never becomes OBSERVED solely from corpus_ref | +| State transitions | SM-001: NORMALIZED -> CLASSIFIED or ABSTAINED; VALIDATING -> INVALID/REFUSED | +| Terminal states | CLASSIFIED, ABSTAINED, INVALID, REFUSED | +| Side effects | One frame returned; no corpus/label mutation | +| Invariants | Flags false/null; references are not labels; inferred hints remain inferred; no psyche interpretation | +| Permissions | Read/classify only under existing T0 scope; label settlement requires existing operator authority | +| Observability/audit | Engine version, input hash, evidence refs, selected rule and outcome in caller-owned receipt | +| Acceptance criteria | AC-003 | +| Dependencies | REQ-005, REQ-006; CON-002; WF-001, WF-002 | +| Unresolved items | DEC-003 precedence and DEC-005 OBSERVED evidence semantics | + +## WF-004 + +| Field | Specification | +|---|---| +| Purpose | Export one compatible semiosis structure | +| Actors | Integrator; export bridge; external Abraxas consumer maintainer | +| Triggers | Explicit export of one validated Semion frame | +| Preconditions | Frame schema and cross-field checks pass; consumer schema pin required for conformance claim | +| Inputs | CON-002 frame; CON-003 consumer mapping version | +| Happy path | Validate frame; preserve refusal; map representamen/object/class/reference and approved action representation; force frozen lanes; return dict | +| Alternate/failure paths | Unknown consumer version blocks conformance claim; invalid input rejected; any failed/NC frame suppresses action; missing corpus_ref remains null, never invented; consumer rejection returned to integrator | +| State transitions | SM-001: CLASSIFIED/ABSTAINED/REFUSED -> EXPORT_VALIDATING -> EXPORTED; invalid -> INVALID | +| Terminal states | EXPORTED or INVALID; compatibility may remain NOT_COMPUTABLE | +| Side effects | Return dict only; no import, network, runtime chain, or rune mint | +| Invariants | Do not create spaces/focus/interpreter identity; no actions executed | +| Permissions | Local export only; consumer activation is outside Semion and needs its own authority | +| Observability/audit | Consumer schema hash when available, frame hash, exporter revision, validation result | +| Acceptance criteria | AC-004 | +| Dependencies | REQ-007, REQ-008; CON-003; WF-002, WF-003 | +| Unresolved items | DEC-004 exact action mapping and external schema; DEC-008 consumer status | + +## WF-005 + +| Field | Specification | +|---|---| +| Purpose | Review corpus labels and eligibility | +| Actors | Existing corpus reviewer; operator for gold-settle | +| Triggers | Explicit review of existing candidate rows, not an automatic harvest | +| Preconditions | Source rights and exact review scope known; operator settlement permission required | +| Inputs | Candidate row content, source receipt, license evidence, proposed class and label rationale | +| Happy path | Check source and rights; distinguish label evidence from source observation; review triad coherence; record decision and epistemic; segregate gold, weak, held, refused | +| Alternate/failure paths | Unknown rights/evidence -> HELD; keep_weak not promoted automatically; NC gold_is_sign must be false; disputed label -> HELD; TEACH/raw peer rows not auto-gold | +| State transitions | SM-002: CANDIDATE -> REVIEWING -> ELIGIBLE/WEAK/HELD/REJECTED | +| Terminal states | ELIGIBLE, WEAK, HELD, REJECTED | +| Side effects | Proposed append-only private review record only after authority; no corpus write by this patch | +| Invariants | INFERRED gold stays INFERRED; no floor padding; denied sources remain denied | +| Permissions | Existing operator gold-settle gate; specification approval is not settlement | +| Observability/audit | Row content hash, reviewer/approval references, rationale, label version and rights decision; no secret body | +| Acceptance criteria | AC-005 | +| Dependencies | REQ-009, REQ-010; CON-004; LICENSE_POLICY.md | +| Unresolved items | DEC-005 evidence rubric; DEC-007 rights and retention | + +## WF-006 + +| Field | Specification | +|---|---| +| Purpose | Freeze and validate selected dataset | +| Actors | Dataset reviewer; evaluation reviewer | +| Triggers | Explicit proposal to select a dataset for evaluation | +| Preconditions | WF-005 decisions available; no unknown eligibility included | +| Inputs | Selected row list, source groups, split assignments, schema and exclusion manifest | +| Happy path | Validate row semantics; retain holdouts; group related sources; hash selected rows; compute selected counts/class floors/quality; freeze manifest before candidate run | +| Alternate/failure paths | Too small selected subset -> BLOCKED, not borrowed parent counts; duplicate/group overlap -> BLOCKED; unknown near-dup method -> NOT_COMPUTABLE; NC in train fails current HQ quality bar; changed input makes new snapshot | +| State transitions | SM-003: DRAFT -> VALIDATING -> FROZEN or BLOCKED | +| Terminal states | FROZEN or BLOCKED | +| Side effects | Private immutable manifest after scoped approval; no training, no public corpus | +| Invariants | Snapshot-specific counts; test labels never become training features; floor pass does not prove quality | +| Permissions | Existing local data access and scoped dataset operation only | +| Observability/audit | Snapshot hash, row hashes, group rule version, counts by split/class/epistemic, exclusions, leakage checks | +| Acceptance criteria | AC-006 | +| Dependencies | REQ-011, REQ-012; CON-004, CON-005; WF-005 | +| Unresolved items | DEC-002 effective subset; DEC-007 near-duplicate grouping and rights | + +## WF-007 + +| Field | Specification | +|---|---| +| Purpose | Evaluate frozen T0 and candidate | +| Actors | Evaluation reviewer; existing authorized training operator supplies candidate | +| Triggers | Explicit evaluation request | +| Preconditions | Frozen snapshot; pinned engine/config; candidate use separately authorized; approved feature protocol | +| Inputs | Selected split hashes; target labels separate from input features; T0 and candidate revisions | +| Happy path | Freeze feature allowlist; score same heldout rows; retain predictions; compute exact-match and class/slice/refusal results; record failed cases and hashes | +| Alternate/failure paths | Missing candidate or permission -> NOT_COMPUTABLE; exposed gold/classification labels -> INVALID benchmark; row mismatch/leakage -> INVALID; zero slice support -> NOT_COMPUTABLE, not perfect score; no training performed by eval | +| State transitions | SM-004: REQUESTED -> CHECKING -> SCORED; missing proof -> NOT_COMPUTABLE; invalid benchmark -> INVALID | +| Terminal states | SCORED, NOT_COMPUTABLE, INVALID | +| Side effects | Local evaluation artifacts only; no naming or Hub change | +| Invariants | Separate adapter label fidelity from label-hidden predictive accuracy; same rows/features for both engines | +| Permissions | Read-only evaluation permission does not imply training permission | +| Observability/audit | CON-006 predictions hash, protocol/config/model/data refs, numerator/denominator, confusion/support and errors | +| Acceptance criteria | AC-007 | +| Dependencies | REQ-013, REQ-014; CON-005, CON-006; WF-006 | +| Unresolved items | DEC-001 candidate availability/authority; DEC-006 comparison protocol | + +## WF-008 + +| Field | Specification | +|---|---| +| Purpose | Review existing encoder gates without activation | +| Actors | Operator; existing evaluation and dual-use reviewers | +| Triggers | Evidence packet submitted for E-S3 review | +| Preconditions | Independent evidence for all existing E-S3 conditions; no auto-approval | +| Inputs | Dataset manifest, benchmark, dual-use review, scoped operator sentence, allowed name | +| Happy path | Check each condition separately; verify references and target; report satisfied, failed, unknown; produce advisory recommendation for operator | +| Alternate/failure paths | Any missing evidence -> NOT_COMPUTABLE; known failing condition -> BLOCKED; Notion delivery not approval; ambiguous train/name timing -> DEC-001, no activation | +| State transitions | SM-005: CLOSED -> REVIEWING -> REVIEW_READY/BLOCKED/NOT_COMPUTABLE; REVIEW_READY does not mean OPEN | +| Terminal states | REVIEW_READY, BLOCKED, NOT_COMPUTABLE; gates unchanged | +| Side effects | Advisory review receipt only | +| Invariants | Train, name, Hub, router, canon permissions are distinct; name_gate remains false in this patch | +| Permissions | Existing operator alone governs approval; no new policy or authority layer | +| Observability/audit | Condition-by-condition status with evidence hashes, reviewer, exact target and unverified fields | +| Acceptance criteria | AC-008 | +| Dependencies | REQ-015, REQ-016; existing Spec 004 E-S3; WF-006, WF-007 | +| Unresolved items | DEC-001 training lifecycle; DEC-008 external reviews and operator scope | + +## WF-009 + +| Field | Specification | +|---|---| +| Purpose | Prepare and verify authorized handoff | +| Actors | Existing operator; authorized sender; named recipient | +| Triggers | Exact request to prepare or transfer an identified pack | +| Preconditions | Rights and recipient/purpose scope verified; no transfer authorized by these docs | +| Inputs | Selected immutable snapshot/artifacts, approval reference, restrictions, destination | +| Happy path | Build manifest without secrets; hash permitted files; include reproduction and exclusions; verify bytes; transfer only if separately authorized; record recipient acknowledgement separately | +| Alternate/failure paths | Unknown rights/recipient -> BLOCKED; mismatch -> INVALID and no use; interrupted transfer -> INCOMPLETE retry only same approved scope; no acknowledgement -> delivery unverified | +| State transitions | SM-006: DRAFT -> VERIFIED -> DELIVERED -> ACKNOWLEDGED; failures -> BLOCKED/INVALID/INCOMPLETE | +| Terminal states | VERIFIED if preparation only; ACKNOWLEDGED, BLOCKED, INVALID, INCOMPLETE | +| Side effects | Only authorized bundle/transfer, never Hub or train; no implicit email/Notion write | +| Invariants | Recipient receipt is not release proof; preserve original split labels; exclude keep_weak unless explicitly separate weak artifact | +| Permissions | Existing exact transfer approval, distinct from train and Hub | +| Observability/audit | Artifact sha256 manifest, snapshot ref, allowed recipient reference, transfer outcome; no signed URLs or secrets in public receipt | +| Acceptance criteria | AC-009 | +| Dependencies | REQ-017, REQ-018; CON-005, CON-006; WF-008 | +| Unresolved items | DEC-007 transfer/retention; DEC-008 actual approvals | + +## WF-010 + +| Field | Specification | +|---|---| +| Purpose | Correct labels, leaks, or stale evidence | +| Actors | Corpus/evaluation reviewer; operator; affected recipient | +| Triggers | Verified discrepancy, rights change, or label correction proposal | +| Preconditions | Affected immutable records identified; correction scope approved before mutation | +| Inputs | Old decision/snapshot hashes, correction evidence, impacted results and handoff refs | +| Happy path | Record proposed correction; obtain existing approval; append superseding decision; create new snapshot; mark affected evaluations stale; prepare scoped notification; re-run dependent checks | +| Alternate/failure paths | Unconfirmed issue stays HELD; do not overwrite test or historical receipts; unavailable recipients remain unnotified; rights withdrawal blocks further use; rollback selects a known eligible prior snapshot only | +| State transitions | SM-002 eligible -> SUPERSEDED; SM-003 FROZEN -> SUPERSEDED; SM-004 SCORED -> STALE; revised candidates restart review | +| Terminal states | SUPERSEDED/STALE with linked replacement, or HELD | +| Side effects | Only approved local correction records; external notification separately scoped | +| Invariants | No silent history rewrite; a later count does not retroactively certify old evidence | +| Permissions | Existing operator correction and communication authority, not inferred from detected defect | +| Observability/audit | Old/new hashes, reason, reviewer/approval ref, impacted evaluations, notification status | +| Acceptance criteria | AC-010 | +| Dependencies | REQ-019, REQ-020; CON-004, CON-005, CON-006; WF-005, WF-006 | +| Unresolved items | DEC-007 retention of superseded records; DEC-008 recipient authority | + +Provenance: Notion Sprint 001 Hub [not inspected; Semion handoff read 2026-09-13 UTC] + Loop 805 Slice N/A (Semion advisory) + Hash: e5ed91bd90afb0429cf816504c7d5e622bbbc153 (base)