Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). Thi
## [Unreleased]

### Added
- DataClass v0.1 (OntoDT/OntoDQ data-side governance, task #14; AMG 2.0 deck): a `DataClass` binds a datatype-ontology class (OntoDT), the business `GlossaryTerm` (biz↔data), and a `ValidValues` domain (enum/range/regex), with an optional **TF-Lattice wide-and-deep** classifier that is a first-class CATALOGED model — `modelRef`→`ModelManifest`, `runRef`→`RunRecord`, `compute` on **Ray/TritFabric**, `monotonicFeatures` lattice constraint, and `labels` that are `GlossaryTerm` URNs (assigned in the glossary). `EntityField` gains an optional `dataClassRef`; the drift-guard enforces field↔class domain conformance + classifier integrity (labels are glossary terms, model is cataloged, compute is ray/tritfabric) + ontological typing. 4 negative vectors, teeth-verified. Backward-compatible. Two-level classifier: per-class LOGISTIC head (`DataClass.classifier`, one-vs-rest, required `evalRunRef` — individually testable per class/glossary term) + a per-table SOFTMAX `TableClassifier` aligning an LSA bag-of-words embedding AND a doc2vec sentence-encoder (the n-ary logit→class assignment), a cataloged model on Ray/TritFabric assigning N DataClasses. Drift-guard enforces both.
- Glossary alignment v0.1 (vocabulary as governance substrate, task #13): `GlossaryTerm` extended with `partOfSpeech` (NP/VP, Tesnière), `status`, a 3-method `alignment` (formal ontology class + NP↔VP vector link in the sovereign 768 space + estate entity/service binding), and typed `relations` over a closed RDF/RDFS/SKOS/FOAF predicate set (is-type/has-type, is-a, has-a, is-member/has-member, skos:broader/narrower/related, foaf:member, has-datatype). Fail-closed drift-guard (`validate_glossary_alignment_examples.py`): an `approved` term must be captured+vector-aligned+implemented (NP→entity, VP→action/service) or it's a governance hole; vector links must be reciprocal; relation inverse-pairs reciprocate; `skos:related` is symmetric; `is-a` subsumption is acyclic. Teeth-verified. Backward-compatible (optional). Follow-ups: LSA↔LDA currency detection + agreement test vs the blast-radius graph + neurosymbolic domain.
- Receipt **epistemicLevel** (SP-GATE-003, closes audit G3): `ReleaseReceipt` and `SyncCycleReceipt` may now carry `epistemicLevel` (`Proved`/`Speculative`/`Refuted`) + an `attestation` block — the MEET of attestation ∧ verification. Optional + backward-compatible; `tools/validate_sync_cycle_receipts.py` RECOMPUTES the meet and rejects a declared value that disagrees, so a receipt can never claim `Proved` while unsigned or after a failed check (absence of a signature returns Speculative, not Proved). Consumes the explicit `attestation.signed` from SP-GATE-002 (source-os #312) and the SyncCycleReceipt attestation from SP-GATE-001. Examples: verified+signed→Proved, verified-but-unsigned→Speculative, failed→Refuted, dry-run→Speculative. Teeth-verified.
- Data-Acquisition **citable commons** (WO-COMMONS-2): a promoted platform `CatalogEntry` may carry `promotion.doi` and the approving `DataAcquisitionRequest` records the same `mintedDoi` — a citable DOI (Zenodo/DataCite-style) minted when the governance ∧ IP/legal meet approves promotion. Invariants (schema `allOf` + validator): a DOI may appear only on a `promoted` entry, `mintedDoi` only on an `approved` DAR, and where both are present they must be equal (minted once). Grounds the governed platform tier as a citable open commons.
Expand Down
8 changes: 6 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,12 +1,16 @@
.PHONY: validate validate-resource-contract validate-measurement validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-ingestion-pipeline-examples validate-data-acquisition-examples validate-glossary-alignment-examples
.PHONY: validate validate-resource-contract validate-measurement validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-ingestion-pipeline-examples validate-data-acquisition-examples validate-glossary-alignment-examples validate-data-class-examples

validate: validate-glossary-alignment-examples validate-data-acquisition-examples validate-ingestion-pipeline-examples validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-resource-contract
validate: validate-data-class-examples validate-glossary-alignment-examples validate-data-acquisition-examples validate-ingestion-pipeline-examples validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples validate-resource-contract
@echo "OK: validate"

validate-ingestion-pipeline-examples:
python3 -m pip install --user jsonschema >/dev/null
python3 tools/validate_ingestion_pipeline_examples.py

validate-data-class-examples:
python3 -m pip install --user jsonschema >/dev/null
python3 tools/validate_data_class_examples.py

validate-glossary-alignment-examples:
python3 -m pip install --user jsonschema >/dev/null
python3 tools/validate_glossary_alignment_examples.py
Expand Down
40 changes: 40 additions & 0 deletions examples/data_class.currency.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
{
"id": "urn:srcos:data-class:currency-amount-usd",
"type": "DataClass",
"specVersion": "2.0.0",
"ontologyClassRef": "urn:srcos:ontology-class:datatype.CurrencyAmount",
"glossaryTermRef": "urn:srcos:glossary:revenue",
"domain": {
"kind": "range",
"values": [],
"min": 0,
"max": null,
"pattern": null
},
"dataQualityRules": [
"dq.nonnegative",
"dq.currency.usd.scale2"
],
"classifier": {
"kind": "tf-lattice-wide-and-deep",
"modelRef": "urn:srcos:model-manifest:dataclass-assigner-lattice-v1",
"version": "1.0.0",
"runRef": "urn:srcos:run:dataclass-assigner-train-001",
"compute": {
"platform": "ray",
"assetRef": "urn:srcos:physical-asset:ray-cluster-01",
"serviceRef": "urn:srcos:tritfabric-consumption-api:assigner"
},
"monotonicFeatures": [
"token_magnitude",
"decimal_places"
],
"labels": [
"urn:srcos:glossary:revenue",
"urn:srcos:glossary:cost"
],
"head": "logistic",
"evalRunRef": "urn:srcos:run:dataclass-currency-logistic-eval-001",
"threshold": 0.5
}
}
16 changes: 16 additions & 0 deletions examples/entity_field.revenue.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"name": "revenue_usd",
"dataType": "decimal",
"nullable": false,
"validValues": {
"kind": "range",
"values": [],
"min": 0,
"max": null,
"pattern": null
},
"glossaryTerms": [
"urn:srcos:glossary:revenue"
],
"dataClassRef": "urn:srcos:data-class:currency-amount-usd"
}
27 changes: 27 additions & 0 deletions examples/table_classifier.finance.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
{
"id": "urn:srcos:table-classifier:finance-ledger-v1",
"type": "TableClassifier",
"specVersion": "2.0.0",
"schemaRef": "urn:srcos:schema-definition:finance-ledger",
"head": "softmax",
"embeddings": [
{
"kind": "lsa-bag-of-words",
"dimension": 300
},
{
"kind": "doc2vec-sentence-encoder",
"dimension": 300
}
],
"modelRef": "urn:srcos:model-manifest:table-assigner-softmax-v1",
"version": "1.0.0",
"runRef": "urn:srcos:run:table-assigner-train-001",
"compute": {
"platform": "tritfabric",
"assetRef": "urn:srcos:physical-asset:trit-cluster-01"
},
"assignsClasses": [
"urn:srcos:data-class:currency-amount-usd"
]
}
243 changes: 243 additions & 0 deletions fixtures/data-class/conformance.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,243 @@
{
"description": "Negative vectors for the DataClass (OntoDT/OntoDQ) family v0.1. Each MUST fail on its named failValidator; validate_data_class_examples.py enforces both, plus: a DataClass must be ontologically typed + glossary-linked + domain-bounded; a classifier's labels are GlossaryTerms and its compute is Ray/TritFabric; and a bound EntityField's validValues must conform to the DataClass domain.",
"cases": [
{
"schema": "DataClass.json",
"reason": "classifier kind is pinned to tf-lattice-wide-and-deep.",
"failValidator": "const",
"document": {
"id": "urn:srcos:data-class:currency-amount-usd",
"type": "DataClass",
"specVersion": "2.0.0",
"ontologyClassRef": "urn:srcos:ontology-class:datatype.CurrencyAmount",
"glossaryTermRef": "urn:srcos:glossary:revenue",
"domain": {
"kind": "range",
"values": [],
"min": 0,
"max": null,
"pattern": null
},
"dataQualityRules": [
"dq.nonnegative",
"dq.currency.usd.scale2"
],
"classifier": {
"kind": "random-forest",
"modelRef": "urn:srcos:model-manifest:dataclass-assigner-lattice-v1",
"version": "1.0.0",
"runRef": "urn:srcos:run:dataclass-assigner-train-001",
"compute": {
"platform": "ray",
"assetRef": "urn:srcos:physical-asset:ray-cluster-01",
"serviceRef": "urn:srcos:tritfabric-consumption-api:assigner"
},
"monotonicFeatures": [
"token_magnitude",
"decimal_places"
],
"labels": [
"urn:srcos:glossary:revenue",
"urn:srcos:glossary:cost"
],
"head": "logistic",
"evalRunRef": "urn:srcos:run:dataclass-currency-logistic-eval-001",
"threshold": 0.5
}
}
},
{
"schema": "DataClass.json",
"reason": "classifier compute platform is closed (ray|tritfabric).",
"failValidator": "enum",
"document": {
"id": "urn:srcos:data-class:currency-amount-usd",
"type": "DataClass",
"specVersion": "2.0.0",
"ontologyClassRef": "urn:srcos:ontology-class:datatype.CurrencyAmount",
"glossaryTermRef": "urn:srcos:glossary:revenue",
"domain": {
"kind": "range",
"values": [],
"min": 0,
"max": null,
"pattern": null
},
"dataQualityRules": [
"dq.nonnegative",
"dq.currency.usd.scale2"
],
"classifier": {
"kind": "tf-lattice-wide-and-deep",
"modelRef": "urn:srcos:model-manifest:dataclass-assigner-lattice-v1",
"version": "1.0.0",
"runRef": "urn:srcos:run:dataclass-assigner-train-001",
"compute": {
"platform": "lambda"
},
"monotonicFeatures": [
"token_magnitude",
"decimal_places"
],
"labels": [
"urn:srcos:glossary:revenue",
"urn:srcos:glossary:cost"
],
"head": "logistic",
"evalRunRef": "urn:srcos:run:dataclass-currency-logistic-eval-001",
"threshold": 0.5
}
}
},
{
"schema": "DataClass.json",
"reason": "classifier labels must be GlossaryTerm URNs (assigned in the glossary).",
"failValidator": "pattern",
"document": {
"id": "urn:srcos:data-class:currency-amount-usd",
"type": "DataClass",
"specVersion": "2.0.0",
"ontologyClassRef": "urn:srcos:ontology-class:datatype.CurrencyAmount",
"glossaryTermRef": "urn:srcos:glossary:revenue",
"domain": {
"kind": "range",
"values": [],
"min": 0,
"max": null,
"pattern": null
},
"dataQualityRules": [
"dq.nonnegative",
"dq.currency.usd.scale2"
],
"classifier": {
"kind": "tf-lattice-wide-and-deep",
"modelRef": "urn:srcos:model-manifest:dataclass-assigner-lattice-v1",
"version": "1.0.0",
"runRef": "urn:srcos:run:dataclass-assigner-train-001",
"compute": {
"platform": "ray",
"assetRef": "urn:srcos:physical-asset:ray-cluster-01",
"serviceRef": "urn:srcos:tritfabric-consumption-api:assigner"
},
"monotonicFeatures": [
"token_magnitude",
"decimal_places"
],
"labels": [
"not-a-glossary-urn"
],
"head": "logistic",
"evalRunRef": "urn:srcos:run:dataclass-currency-logistic-eval-001",
"threshold": 0.5
}
}
},
{
"schema": "DataClass.json",
"reason": "ontologyClassRef is required — a data class must be ontologically typed.",
"failValidator": "required",
"document": {
"id": "urn:srcos:data-class:currency-amount-usd",
"type": "DataClass",
"specVersion": "2.0.0",
"glossaryTermRef": "urn:srcos:glossary:revenue",
"domain": {
"kind": "range",
"values": [],
"min": 0,
"max": null,
"pattern": null
},
"dataQualityRules": [
"dq.nonnegative",
"dq.currency.usd.scale2"
],
"classifier": {
"kind": "tf-lattice-wide-and-deep",
"modelRef": "urn:srcos:model-manifest:dataclass-assigner-lattice-v1",
"version": "1.0.0",
"runRef": "urn:srcos:run:dataclass-assigner-train-001",
"compute": {
"platform": "ray",
"assetRef": "urn:srcos:physical-asset:ray-cluster-01",
"serviceRef": "urn:srcos:tritfabric-consumption-api:assigner"
},
"monotonicFeatures": [
"token_magnitude",
"decimal_places"
],
"labels": [
"urn:srcos:glossary:revenue",
"urn:srcos:glossary:cost"
],
"head": "logistic",
"evalRunRef": "urn:srcos:run:dataclass-currency-logistic-eval-001",
"threshold": 0.5
}
}
},
{
"schema": "TableClassifier.json",
"reason": "table classifier head is pinned to softmax (n-ary).",
"failValidator": "const",
"document": {
"id": "urn:srcos:table-classifier:finance-ledger-v1",
"type": "TableClassifier",
"specVersion": "2.0.0",
"schemaRef": "urn:srcos:schema-definition:finance-ledger",
"head": "logistic",
"embeddings": [
{
"kind": "lsa-bag-of-words",
"dimension": 300
},
{
"kind": "doc2vec-sentence-encoder",
"dimension": 300
}
],
"modelRef": "urn:srcos:model-manifest:table-assigner-softmax-v1",
"version": "1.0.0",
"runRef": "urn:srcos:run:table-assigner-train-001",
"compute": {
"platform": "tritfabric",
"assetRef": "urn:srcos:physical-asset:trit-cluster-01"
},
"assignsClasses": [
"urn:srcos:data-class:currency-amount-usd"
]
}
},
{
"schema": "TableClassifier.json",
"reason": "embedding kind is closed to lsa-bag-of-words | doc2vec-sentence-encoder.",
"failValidator": "enum",
"document": {
"id": "urn:srcos:table-classifier:finance-ledger-v1",
"type": "TableClassifier",
"specVersion": "2.0.0",
"schemaRef": "urn:srcos:schema-definition:finance-ledger",
"head": "softmax",
"embeddings": [
{
"kind": "word2vec"
},
{
"kind": "doc2vec-sentence-encoder"
}
],
"modelRef": "urn:srcos:model-manifest:table-assigner-softmax-v1",
"version": "1.0.0",
"runRef": "urn:srcos:run:table-assigner-train-001",
"compute": {
"platform": "tritfabric",
"assetRef": "urn:srcos:physical-asset:trit-cluster-01"
},
"assignsClasses": [
"urn:srcos:data-class:currency-amount-usd"
]
}
}
]
}
Loading
Loading