Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
e7d57cd
docs(intelligence): extension workspace scaffold + evidence-claim RFC…
claude Aug 12, 2026
7e51f72
feat(intelligence): ontology — canonical envelopes, validators, ID gr…
claude Aug 12, 2026
a06ad5e
feat(intelligence): store — bronze/silver/gold medallion (phase 6)
claude Aug 12, 2026
72c1c85
feat(intelligence): bitemporal knowledge graph engine (phases 5, 15)
claude Aug 12, 2026
3f14a9e
feat(intelligence): registry loader + cross-reference validation
claude Aug 12, 2026
1941380
feat(intelligence): real-time state engine (phase 14)
claude Aug 12, 2026
ee5ccd5
feat(intelligence): pact-bridge — fact proposals + competing-hypothes…
claude Aug 12, 2026
b648c33
feat(intelligence): global grid registry — 71 electrical systems (pha…
claude Aug 12, 2026
2593404
feat(intelligence): global source registry — 67 data sources (phases …
claude Aug 12, 2026
722541e
feat(intelligence): observability gap registry — 35 NEM/WEM gaps (pha…
claude Aug 12, 2026
30d7fbb
feat(intelligence): NEM structural seed — Gladstone cluster world mod…
claude Aug 12, 2026
3517fad
feat(intelligence): connector SDK + AEMO/AIS/BOM/ENTSO-E/EIA connecto…
claude Aug 12, 2026
bfe7727
feat(intelligence): derived-intelligence rules (phases 10-13, 16)
claude Aug 12, 2026
2740174
feat(intelligence): REST + SSE API + SDUI composer (phase 18)
claude Aug 12, 2026
a7f13c6
feat(intelligence): deck.gl GPU globe prototype with LOD tiers (phase…
claude Aug 12, 2026
219272a
ci(intelligence): path-filtered workflow + end-to-end smoke
claude Aug 12, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 82 additions & 0 deletions .github/workflows/intelligence.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# CI for the intelligence/ extension workspace only. Path-filtered so spec/CLI/MCP
# work never waits on it (mirrors conformance.yml's approach).
name: intelligence

on:
push:
branches: [main]
paths:
- "intelligence/**"
- ".github/workflows/intelligence.yml"
pull_request:
paths:
- "intelligence/**"
- ".github/workflows/intelligence.yml"

jobs:
build-test:
name: Build + unit tests
runs-on: ubuntu-latest
defaults:
run:
working-directory: intelligence
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
cache-dependency-path: intelligence/package-lock.json
- run: npm ci
- run: npm run build
- run: npm test

validate-data:
name: Validate registries + schemas
runs-on: ubuntu-latest
defaults:
run:
working-directory: intelligence
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
cache-dependency-path: intelligence/package-lock.json
- run: npm ci
- run: npm run build -w @pact-tailor/ontology && npm run build -w @pact-tailor/registry
- run: npm run validate:data

smoke:
name: End-to-end smoke (fixtures → API → SSE)
runs-on: ubuntu-latest
defaults:
run:
working-directory: intelligence
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
cache-dependency-path: intelligence/package-lock.json
- run: npm ci
- run: npm run build
- run: bash scripts/smoke.sh

globe-build:
name: Globe bundle builds
runs-on: ubuntu-latest
defaults:
run:
working-directory: intelligence
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
cache-dependency-path: intelligence/package-lock.json
- run: npm ci
- run: npm run build:globe
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,6 @@ dist/

# Claude Code session state (per-session worktrees, not part of repo)
.claude/

# Intelligence workspace runtime output (bronze/silver/gold data lake)
intelligence/var/
154 changes: 154 additions & 0 deletions docs/v2-prep/rfc-intelligence-fact-evidence-extension.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# RFC: `au.tailor.intel.claim` — an evidence-carrying claim payload for the PACT `fact` resource type

> **Status:** RFC — for discussion. Implementation ships in the `intelligence/`
> extension workspace; **no spec text or registry entry changes in this PR.**
> **Proposed resource type:** `au.tailor.intel.claim` (registration deferred — see §6)
> **Target PACT version:** v2.2 (§14 resource types; §24 Matters unaffected)
> **Owner:** Tailor (extension workspace); registry decision remains with the maintainer
> **Related:** `spec/v2.2/resource-types.yaml` built-in `fact` type; GOVERNANCE.md
> decision D2=B (domain-specific Tailor needs are extensions, not core); issue #34
> (KG product extraction — explicitly *not* what this workspace is, see §7)
> **Reference implementation:** `intelligence/schemas/inference.schema.json` +
> `intelligence/packages/pact-bridge/` (competing-hypotheses demo against
> `reference-server/`)

---

## 1. Summary

PACT's built-in `fact` resource type verifies knowledge claims into a knowledge graph,
but its proposal payload — `{ claim, evidence, tier, sources }` — is described in the
registry as a string, with no JSON Schema behind it. `tier` and `sources` are undefined
in spec text; `evidence` has no structure; nothing binds a claim to the observations
that justify it.

The intelligence workspace needs exactly that binding: its rule is that **derived
intelligence must always point back to its evidence and can never masquerade as direct
telemetry**. This RFC documents the formalisation the workspace implements — a
schema'd, evidence-carrying claim payload — and proposes it, eventually, as a
registered custom resource type `au.tailor.intel.claim`.

## 2. Motivation

A live intelligence system produces statements like:

> Vessel X departing Gladstone is likely carrying metallurgical coal. Confidence 0.91.

That statement is useful precisely to the degree that its support is inspectable. The
informal `fact` payload can carry the sentence; it cannot carry, in any checkable way,
*which* AIS observations, berth assignments, and structural supply edges support it,
which evidence cuts against it, what rule produced it, or what its status became after
other agents weighed in.

PACT already solves the adjacent problem for *authority* (hash-chained event log,
`authorization_proof`, mandates). This RFC applies the same discipline to *epistemics*:
claims carry their evidence, their counter-evidence, their method, and their lineage.

## 3. The payload

Normative schema: `intelligence/schemas/inference.schema.json`. Shape (informative):

```json
{
"inference_id": "infer:2026-08-12:rgt-cargo-01",
"claim": "Vessel MV Example, departing RG Tanna berth 1, is loading metallurgical coal.",
"claim_structured": {
"subject": "vessel:imo:9700000",
"predicate": "carries",
"object": "commodity:coal-metallurgical",
"qualifiers": { "loading_at": "terminal:au-qld:rg-tanna" }
},
"tier": "B",
"confidence": 0.91,
"status": "INFERRED",
"method": "rule:cargo-inference@1.0.0",
"evidence": [
{ "kind": "observation", "ref": "01J...ULID", "role": "supports" },
{ "kind": "structural", "ref": "rel:terminal-supplies-met-coal", "role": "supports" }
],
"contrary_evidence": [
{ "kind": "observation", "ref": "01J...ULID2", "role": "contradicts" }
],
"sources": ["source:ais:aisstream", "source:aemo:nemweb-dispatchis"],
"produced_at": "2026-08-12T03:15:00Z",
"event_time_range": { "from": "2026-08-12T01:00:00Z", "to": "2026-08-12T03:10:00Z" },
"lineage": { "correlation_id": "corr-...", "in_response_to": null, "prev_hash": "..." },
"pact": { "fabric_id": "...", "proposal_id": "..." }
}
```

Mapping to the built-in `fact` payload is total: `claim` → `claim`,
`evidence ∪ contrary_evidence` → `evidence`, `tier` → `tier`, `sources` → `sources` —
so an `au.tailor.intel.claim` proposal degrades losslessly to a legacy `fact` proposal
for servers that only know the built-in type. The bridge in
`intelligence/packages/pact-bridge/src/fact-mapper.ts` implements this mapping today.

## 4. Semantics

- **Evidence tiers:** `A` — the claimed quantity was directly observed; `B` — one-hop
proxy or derivation from direct observations; `C` — multi-hop inference or pattern
match. Tier is about *distance from observation*, confidence is about *strength*;
the two are orthogonal and both required.
- **Status lifecycle:** `INFERRED → CORROBORATED | CONTESTED → VERIFIED | RETRACTED`.
Transitions happen through PACT consensus (propose/object/salience on the claim
field) or through new evidence; records are append-only — a status change is a new
revision, never an edit that loses the contested history.
- **Contrary evidence is first-class.** An objection that cites evidence lands in
`contrary_evidence` and survives resolution. A claim that won consensus still shows
what cut against it.
- **Non-masquerade rule (normative for the workspace):** an inference envelope can
never be written to the observation store, and no observation field is
author-choosable to claim otherwise. Consumers can always distinguish reported from
inferred — this is the payload-level expression of PACT's evidence-backed-claim
intent in §14.5.
- **Lineage** reuses the event-log vocabulary (`correlation_id`, `in_response_to`,
`prev_hash` per §6.1/§6.4) rather than inventing a parallel provenance chain. An
inference chain is auditable with the same tooling as a PACT event chain.

## 5. Consensus flow (worked example, implemented)

The Gladstone demo (`intelligence/packages/pact-bridge/`): two agents propose competing
cargo claims for the same vessel — metallurgical coal (0.91) citing berth assignment +
draught-change observations + terminal supply edges, versus thermal coal (0.34) citing
a contrary shipping-schedule observation. One objects to the other's claim; salience
and consensus resolve; the winning inference becomes `CORROBORATED`, the losing one
`CONTESTED`; both retain full evidence and contrary-evidence arrays. Runs offline
against `reference-server/` in `node --test`.

## 6. Registration (deferred)

Registering `au.tailor.intel.claim` in `spec/vX.Y/resource-types.yaml` follows the
registry's own process — a dedicated PR against that file, linking the reference
implementation, reviewed by the maintainer. **This workspace's PR deliberately does not
touch the registry** (or any file under `spec/`): per AGENTS.md, agents do not freehand
spec-adjacent surfaces; the registration PR is a follow-up under maintainer sign-off.
Proposed entry (informative draft):

```yaml
- type: au.tailor.intel.claim
status: registered
field_schema: "claim:{id} — evidence-carrying knowledge-claim identifier"
proposal_payload: "inference envelope per intelligence/schemas/inference.schema.json"
apply_semantics: Claim verified into the knowledge graph with evidence and lineage retained.
terminal_states: [Verified, Retracted]
content_format: application/json
maintainer: TailorAU
```

## 7. Relationship to issue #34

Issue #34 (extraction of the production KG product into this repo) is owner-only and
security-sensitive. This workspace is **not** that extraction: it is a new extension
implementation under GOVERNANCE.md D2=B, with its own data model and no dependency on
`tailor-app` internals. The only overlap is conceptual — both verify claims into a
knowledge graph — and this RFC is written so the production product could *adopt* the
schema'd payload later if the owner chooses.

## 8. Compatibility

- Servers that know only the built-in `fact` type receive the degraded
`{ claim, evidence, tier, sources }` mapping (§3) — no breakage.
- The payload adds no PACT operations, no endpoints, no `_meta` extensions; it rides
the existing propose/object/consensus machinery unchanged.
- Nothing in this RFC is normative for PACT. It becomes normative for the registry
only if/when the §6 registration PR is accepted.
94 changes: 94 additions & 0 deletions intelligence/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# PACT.TAILOR Intelligence Network

A real-time model of the physical economy — energy, industry, and trade — built as a
**Tailor extension workspace** on top of the PACT protocol. The machine interface is a
REST + SSE API over a bitemporal knowledge graph; the human interface is a GPU-rendered
globe; the coordination, evidence, and attestation layer is PACT.

This directory is an *extension implementation* in the sense of `GOVERNANCE.md`
(decision D2=B: domain-specific Tailor needs land as extensions, not PACT core). Nothing
under `spec/` is modified by this workspace. The design record lives at
`docs/v2-prep/rfc-intelligence-fact-evidence-extension.md`.

## The one rule

**Intelligence means live observable state — not AI prose.** Every piece of information
in this system belongs to exactly one of three classes, and the class is structurally
unforgeable (each class has its own envelope schema):

| Class | Envelope | Meaning |
|---|---|---|
| A. Structural | `schemas/entity.schema.json`, `relationship.schema.json` | The world model: what exists and how it connects |
| B. Live telemetry | `schemas/observation.schema.json` | What is being observed right now |
| C. Derived intelligence | `schemas/inference.schema.json` | Interpretations — **must** reference their evidence |

Derived intelligence can never be written into the observation store, and inferred
cargo/state/utilisation is never presented as reported fact. Where live observability
does not exist, the system records an explicit **observability gap**
(`schemas/observability-gap.schema.json`) — a first-class product output identifying
where Tailor/Sovrgn could deploy owned instrumentation.

## Honest data status

This workspace was developed in an environment whose network egress policy blocks all
energy-data endpoints. Consequently:

- Every connector is built against the *documented* wire format of its source and ships
with fixtures marked `fixture_provenance: synthetic-from-spec`.
- Every source registry record carries `verification_status: documented_only` until the
live-verification harness (`npm run verify:live`) has actually reached the endpoint.
- No claim of live verification is made anywhere in this tree. The replay path used in
tests is the *same code path* as live ingestion (`HttpGate` swaps `ReplayGate` for
`LiveGate`), so passing fixtures is meaningful but is not live proof.

## Layout

```
schemas/ Canonical JSON Schemas (2020-12) for all envelopes
data/ YAML registries: grids, sources, observability gaps + NEM structural seed
packages/
ontology TS types, ajv validators, ID grammar
registry YAML loaders, cross-reference validation, entity-resolution candidates
store Bronze/silver/gold medallion storage (embedded, zero external services)
graph In-process bitemporal property graph + traversal
state Real-time state engine: observations → current-state projections
connectors Connector SDK + AEMO / AIS / BOM / ENTSO-E / EIA connectors + fixtures
intel Rule-based derived intelligence (utilisation, cargo inference, anomalies)
api Zero-dep REST + SSE server + SDUI composer
pact-bridge Inference → PACT `fact` proposals; competing-hypotheses demo
globe deck.gl GPU globe prototype (Vite)
docs/ Architecture, data model, identifiers, storage, SDUI, globe, licensing
scripts/ End-to-end smoke test
var/ Runtime output (gitignored; bronze/silver/gold live here)
```

## Quick start

```bash
cd intelligence
npm ci
npm run build # builds all packages in dependency order
npm test # per-package node --test suites
npm run validate:data # ajv-validates every schema + every YAML registry record
npm run smoke # fixtures → ingest → state → graph → API → SSE, end to end
```

Run the API against replayed fixtures:

```bash
node packages/connectors/dist/cli.js ingest --replay
node packages/api/dist/cli.js --port 4200
# then: curl localhost:4200/api/intel/grids/grid:au-nem/summary
```

Run the globe dev server (expects the API on :4200):

```bash
npm run dev -w @pact-tailor/globe
```

## Status

All packages are `private: true` and are **never published to npm** (see repo issue #5).
First depth target: Australia's NEM, with the Gladstone industrial cluster as the worked
exemplar. Global breadth lives in the registries.
Loading
Loading