Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). Thi
### Added
- Image-trust contracts — the Assay's measured-not-asserted discipline applied to images, for both OS and agents: `ImageTrustReport` (per-dimension trust checks — reproducibility, attestation_verify, sbom_completeness, signature_verify, measured_boot, provenance_depth — each carrying a `Measurement` via `$ref`, plus a render-time `projectedTrust` ok/sad/bad verdict; unifies OS and agent images through `subjectKind`) and `AgentImage` (agent runtime-artifact identity + provenance parallel to `OSImage`, distinct from process-level `AgentPassport`). `ImagePromotionGate` now requires a `trustReportRef` when `decision: approved` (non-breaking if/then), and the new `validate-image-trust-examples` target enforces projection soundness plus the cross-document rule that an approval must reference a report projecting `ok` — closing, for images, the self-assertable-approval gap the Assay hardening closed for verifiers. Reuses `Measurement`'s gate-eligibility invariants for free via `$ref` (a declared/assumed check cannot reach `ok`). All invariants verified biting. See `schemas/image-trust.README.md`.
- Assay fleet-tier contracts (cloud-mesh): `AssayRollup` (fleet/cohort aggregate of `ReasoningAssay` verdicts over a window — ok/sad/bad distribution, calibration-drift view of which `AssayStandard` versions are live across the fleet, unassayed-reason breakdown) and `AssayStandardRollout` (canary→widen/halt promotion of a new `AssayStandard` version across cohorts, gated on an observed `AssayRollup`; rides the release-bundle + lifecycle machinery). New `validate-assay-fleet-examples` target enforces aggregation/rollout soundness (distribution sums, drift-flag agreement, rolloutPct vs node share, decision/phase consistency, and no-promotion-by-hope), all verified biting. This is the boundary between prophet-mesh single-user-local (node self-view, no fleet dashboards, structural) and cloud-mesh (rollup + rollout + dashboards). See `schemas/assay-fleet.README.md`.
- **Six-layer fingerprint stack (SP-FPRINT-STACK-001)** — classification that can distinguish "I don't know" from "my evidence contradicts itself". `ClassificationStance` over FOUR (POS/NEG/**ZERO**/**INADMISSIBLE**) with the support-vs-plausibility reading tag in the TYPE (DR-5); `FingerprintLayerEvidence` with per-layer discriminated witnesses for L1 OntoDT / L2 OntoDQ / L3 glossary / L4 operational semantics / L5 table topic / L6 key graph; `ColumnFingerprint` (pool -> guard -> quantize, in that order); `ColumnDriftObservation` (L5-D1/L5-D2, blocking); `EstateAdmissibilityReport` (the phase-0 deliverable — which layers this estate can trust, shipped BEFORE any classification); `ClassificationEnforcementPolicy` (**DR-4 settled**: POS/NEG not knobs, INADMISSIBLE never fail-open, ZERO an attested per-resource-class knob unavailable at confidential/restricted). Recompute-don't-trust throughout: stance from evidence, pooling from admissible layers, `n_eff` from the covariance spectrum (participation ratio — Herfindahl demoted to a cheap precheck, since two perfectly correlated layers give H=0.5 while supplying one layer's worth of information), drift from the measured distance. Axiom **X1** (parthood and subtyping disjoint; closure traverses subtyping only; open-world counts are intervals with INADMISSIBLE reported separately) enforced as a CI invariant. `DataClass.classifier.kind` const **retired**: TensorFlow Lattice was archived by its owner in April 2026, so the pin now enumerates the structurally-monotone constructions with the dead name deprecated-but-accepted, and gains a `fingerprintStack` binding. 81 checks + 9 schema negative vectors + **25 gates proven to bite** by mutation harness + an M5 property test verified against a deliberately non-monotone aggregator (the #265 vacuous-constraint defect, guarded against structurally this time).
- Trained DataClass classifiers — adversarial hardening (follow-up to #264): stronger fixture where BOTH monotone features (token_magnitude, decimal_places) genuinely drive the label so the monotone constraint actually BINDS (was vacuous — decimal_places held constant); standardiser fit on the TRAIN split only (no test leakage); ModelManifest named honestly `monotone-logistic` (a realisation of the declared tf-lattice-wide-and-deep contract, not a full lattice). Validator now checks BOTH per-class heads monotone AND asserts the constraint binds (non-vacuous), and compares reproduced accuracy at the stored 4-decimal precision (fixes a latent tolerance bug the trivial 1.0-accuracy fixture had masked). 8 teeth.
- Trained DataClass classifiers v0.1 — specified→trained→registered→reproduced (closes the program's last owed thread): `tools/train_dataclass_classifiers.py` trains real per-class LOGISTIC (one-vs-rest) + per-table SOFTMAX heads, MONOTONE-constrained in `monotonicFeatures` (weight >=0 projection = TF-Lattice essence), deterministically; emits trained weights + a conformant `ModelManifest` (id=DataClass modelRef) + train/eval `RunRecord`s (ids=runRef/evalRunRef) so the previously-dangling DataClass classifier references now RESOLVE. `validate-trained-classifiers` (recompute-don't-trust): digest matches weight bytes; eval accuracy REPRODUCED from weights+fixture (inflated claim fails); model verified genuinely monotone (perturb up → score never drops); refs resolve; artifacts schema-conform. Teeth-verified (tampered weights refused). Uses numpy.
- Learned spell-correction + user dictionary v0.1 — from context, not dictionary matching (task #13): `tools/learned_dictionary.py` decides each UNKNOWN token from a count-based skip-gram word-sense predictor (PPMI + truncated SVD; SGNS≈PPMI-SVD) instead of a static wordlist. LEARN a token that recurs with a coherent word-sense (a real domain term the dictionary hadn't seen); CORRECT a rare token to the known word its SENSE matches (skip-gram cosine picks the target, not edit distance alone — a near-spelled but different-sense token is not auto-corrected); leave the rest UNKNOWN (fail-closed, human-admitted). Every decision is a proposal, never a silent rewrite. `validate-learned-dictionary` teeth: epistemiclevel learned, reciept->receipt by sense, qwzptl unknown, learned term never auto-corrected. Uses numpy.
Expand Down
9 changes: 7 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
.PHONY: validate validate-resource-contract validate-measurement validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-ingestion-pipeline-examples validate-data-acquisition-examples validate-glossary-alignment-examples validate-data-class-examples validate-table-keys validate-dag-loop validate-vocab-currency-loop vocab-currency-dogfood validate-glossary-promotion validate-agreement validate-stopword-analysis stopword-analysis-live validate-kgram-differential validate-learned-dictionary train-dataclass-classifiers validate-assay-fleet-examples validate-image-trust-examples validate-schema-references
.PHONY: validate validate-resource-contract validate-measurement validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-ingestion-pipeline-examples validate-data-acquisition-examples validate-glossary-alignment-examples validate-data-class-examples validate-table-keys validate-dag-loop validate-vocab-currency-loop vocab-currency-dogfood validate-glossary-promotion validate-agreement validate-stopword-analysis stopword-analysis-live validate-kgram-differential validate-learned-dictionary train-dataclass-classifiers validate-assay-fleet-examples validate-image-trust-examples validate-schema-references validate-fingerprint-stack

validate: validate-data-class-examples validate-glossary-alignment-examples validate-data-acquisition-examples validate-ingestion-pipeline-examples validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-resource-contract validate-table-keys validate-dag-loop validate-vocab-currency-loop validate-glossary-promotion validate-agreement validate-stopword-analysis validate-kgram-differential validate-learned-dictionary validate-trained-classifiers validate-multiverseal-twin-examples validate-assay-fleet-examples validate-image-trust-examples validate-schema-references
validate: validate-data-class-examples validate-glossary-alignment-examples validate-data-acquisition-examples validate-ingestion-pipeline-examples validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-resource-contract validate-table-keys validate-dag-loop validate-vocab-currency-loop validate-glossary-promotion validate-agreement validate-stopword-analysis validate-kgram-differential validate-learned-dictionary validate-trained-classifiers validate-multiverseal-twin-examples validate-assay-fleet-examples validate-image-trust-examples validate-schema-references validate-fingerprint-stack
@echo "OK: validate"

validate-assay-fleet-examples:
Expand All @@ -24,6 +24,11 @@ validate-ingestion-pipeline-examples:
python3 -m pip install --user jsonschema >/dev/null
python3 tools/validate_ingestion_pipeline_examples.py

validate-fingerprint-stack:
python3 -m pip install --user jsonschema >/dev/null
python3 tools/validate_fingerprint_stack.py
python3 tools/test_fingerprint_stack_teeth.py

validate-data-class-examples:
python3 -m pip install --user jsonschema >/dev/null
python3 tools/validate_data_class_examples.py
Expand Down
35 changes: 35 additions & 0 deletions examples/classification_enforcement_policy.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"id": "urn:srcos:classification-enforcement:estate-default-v1",
"type": "ClassificationEnforcementPolicy",
"specVersion": "2.0.0",
"posBehavior": "allow",
"negBehavior": "deny",
"inadmissibleBehavior": "fail-closed",
"resourceClasses": [
{
"resourceClass": "reference-lookup",
"sensitivity": "public",
"zeroBehavior": "fail-open",
"attestationRef": "urn:srcos:attestation:dr4-reference-lookup-2026-08-01",
"rationale": "Published reference tables carry no personal data and over-blocking them strands every downstream join. Serving an unclassified reference column is a known, bounded exposure that the data steward accepted on the record."
},
{
"resourceClass": "operational-analytics",
"sensitivity": "internal",
"zeroBehavior": "fail-closed",
"attestationRef": null,
"rationale": "Default. An unclassified column in an internal analytics surface is more likely to be an un-onboarded field than a deliberately open one."
},
{
"resourceClass": "customer-master",
"sensitivity": "restricted",
"zeroBehavior": "fail-closed",
"attestationRef": null,
"rationale": "Customer master is where the personal data is. A column we cannot classify is exactly the column not to serve."
}
],
"abstentionBudget": {
"maxZeroFraction": 0.35,
"onExceed": "escalate-human"
}
}
23 changes: 23 additions & 0 deletions examples/column_drift.repurposed_status.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"id": "urn:srcos:column-drift:ops-orders-status-code-2026-08",
"type": "ColumnDriftObservation",
"specVersion": "2.0.0",
"subjectColumnRef": "urn:srcos:column:ops.orders.status_code",
"baseline": {
"profileRunId": "profile-2026-02-02-ops-orders",
"schemaVersion": "ops.orders@v12",
"observedAt": "2026-02-02T03:15:00Z"
},
"current": {
"profileRunId": "profile-2026-08-03-ops-orders",
"schemaVersion": "ops.orders@v12",
"observedAt": "2026-08-03T04:40:00Z"
},
"metric": "jensen-shannon",
"distance": 0.61,
"threshold": 0.15,
"schemaChanged": false,
"drifted": true,
"verdict": "silent-repurpose",
"stewardQueueRef": "urn:srcos:steward-queue:data-governance-drift"
}
Loading
Loading