feat(schemas): Twin World-Model & GAIA + EvalItem pack - #238
Merged
Conversation
Give a Twin a dynamical, geospatially-grounded world model beneath its governance lifecycle, and add the EvalItem contract for the provider-neutral model tournament that feeds the Intelligence-Superiority Bench (iSOTA). World-model + GAIA pack (6 schemas): WorldModel (x+ = A*x + B*(G*u)), StateVector, ImpulseGate, Impulse, Region, GaiaObservation. GAIA is the exogenous world model: a Region is grounded in GAIA cells, an exogenous_shock Impulse must cite a GaiaObservation, and a WorldModel that admits exogenous shocks must declare exogenous_source=gaia -- fail-closed, asserted by rejection. 16 rejection invariants (incl. exogenous-shock=>GAIA, closed-gate=>no-admit, state components in [0,1]). EvalItem: the atomic contract of the Provider Eval Seed Strategy -- corpus A/B/C, task family, expected traits, grading method, risk class. Ungradeable items and provider-sourced items with no provider named are rejected; the harness stays provider-neutral. 6 rejection invariants. Both validators wired into `make validate`; examples carry the `type` discriminator for the Validate-Spec example mapper.
There was a problem hiding this comment.
Pull request overview
Adds two new normative schema packs to the spec repo—(1) the Twin World-Model + GAIA grounding contracts and (2) the EvalItem contract for provider-neutral evaluation seeds—along with shipped examples and repository-native validators wired into make validate.
Changes:
- Introduces 6 World-Model/GAIA schemas plus canonical examples and a validator with rejection invariants.
- Introduces the EvalItem schema plus a canonical example and a validator with rejection invariants.
- Wires both validators into the top-level
validateMakefile target.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/validate_world_model_examples.py | Validates the world-model + GAIA schema pack against shipped examples and negative controls. |
| tools/validate_eval_item_examples.py | Validates the EvalItem schema against its example and rejection invariants. |
| schemas/WorldModel.json | Defines the Twin WorldModel contract including GAIA coupling constraints. |
| schemas/StateVector.json | Defines normalized latent state readings with provenance. |
| schemas/Region.json | Defines geospatial grounding via centroid + GAIA cells. |
| schemas/ImpulseGate.json | Defines governed admission gates with fail-closed constraints. |
| schemas/Impulse.json | Defines impulses into the state-space model, including GAIA citation rules. |
| schemas/GaiaObservation.json | Defines exogenous GAIA observations and provenance requirements. |
| schemas/EvalItem.json | Defines the EvalItem contract for provider-neutral eval corpora. |
| Makefile | Adds validate-world-model-examples and validate-eval-item-examples to make validate. |
| examples/world-model.json | Canonical WorldModel example fixture. |
| examples/state-vector.json | Canonical StateVector example fixture. |
| examples/region.json | Canonical Region example fixture. |
| examples/impulse.json | Canonical Impulse example fixture. |
| examples/impulse-gate.json | Canonical ImpulseGate example fixture. |
| examples/gaia-observation.json | Canonical GaiaObservation example fixture. |
| examples/eval-item.json | Canonical EvalItem example fixture. |
Comment on lines
+12
to
+13
| "source": { "type": "string", "enum": ["provider", "internal"] }, | ||
| "provider": { "type": "string" }, |
Comment on lines
+36
to
+37
| "region_ref": { "type": "string", "description": "Region.region_id this world model is grounded in." }, | ||
| "exogenous_source": { "type": "string", "enum": ["gaia", "none"] } |
Comment on lines
+22
to
+25
| "source_ref": { | ||
| "type": "string", | ||
| "description": "For exogenous_shock: the GaiaObservation.observation_id this impulse was derived from." | ||
| }, |
| @@ -1,12 +1,20 @@ | |||
| .PHONY: validate validate-resource-contract validate-measurement validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement | |||
| .PHONY: validate validate-resource-contract validate-measurement validate-value-type validate-source-locator validate-sourceos-repo-manifest validate-mesh-action-registry validate-control-plane-examples validate-nlboot-examples validate-lattice-data-governai-examples validate-ops-history-examples validate-runtime-observability-examples validate-interpretability-examples validate-lifecycle-boundary-examples validate-svf-contracts validate-sync-cycle-receipts validate-onboarding-examples validate-runtime-causality-examples validate-agentic-os-examples validate-triparty-examples validate-labor-market-examples validate-supply-chain-risk-examples validate-reasoning-examples validate-mpcc-event-examples validate-knowledge-nugget-examples validate-semantic-action-examples validate-epistemic-kernel-examples validate-ab-update-examples validate-device-service-examples validate-duplicate-schema-ids validate-lawful-dispatch-receipt validate-architectural-building-block validate-agent-passport-examples validate-seam-definition-examples validate-agent-system-vocabulary validate-genesis-inception-examples validate-measurement validate-world-model-examples validate-eval-item-examples | |||
Comment on lines
+59
to
+60
| must_reject("EvalItem provider source without provider name", | ||
| lambda d: {**{k: val for k, val in d.items() if k != "provider"}, "source": "provider"}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gives a Twin a dynamical, geospatially-grounded world model beneath its governance lifecycle, and adds the EvalItem contract for the provider-neutral model tournament that feeds the Intelligence-Superiority Bench (iSOTA). Follows the genesis-pack pattern: draft-2020 schemas +
typediscriminator + rejection-invariant validators wired intomake validate.World-model + GAIA pack (6 schemas)
WorldModel(x⁺ = A·x + B·(G·u)) ·StateVector·ImpulseGate·Impulse·Region·GaiaObservationGAIA is the exogenous world model. The couplings are enforced fail-closed and proven by rejection:
Regionis grounded in ≥1 GAIA cell (ungrounded region → rejected);exogenous_shockImpulsemust cite aGaiaObservation(source_ref) — no exogenous state without GAIA behind it;WorldModelwhose gates admitexogenous_shockmust declareexogenous_source: gaia(+ aregion_ref);ImpulseGateclosed ⇒gate_factor0;Impulseclosed gate ⇒admitted:false);∈ [0,1]; a model with no dims / no gates is not a model.16 rejection invariants.
EvalItem (1 schema)
The atomic contract of the Provider Eval Seed Strategy: corpus A (provider seed) / B (Sherlock task, weighted) / C (adversarial), task family,
expected_answer_traits,grading_method,risk_class. Ungradeable items (no expected traits) and provider-sourced items with noprovidernamed are rejected — the harness stays provider-neutral (provider is a label, never a winner). 6 rejection invariants.Verification (local)
make validate-world-model-examples→ OK, 16 invariantsmake validate-eval-item-examples→ OK, 6 invariants$idguard: 335 uniquetypeand validateContext
These are the real schemas behind the Prophet Cockpit · Twin Workshop World Model page (geospatial COP + ontology graph + state-space/impulse-gate dynamics). Next: the provider-neutral tournament harness → iSOTA, and the Vue cockpit port.