Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions packages/core/skills/shared/drift-detection.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Don't raise a `DriftIssue` for:

- Style differences (camelCase vs snake_case, English vs Chinese phrasing). Names mean the same thing.
- High-level intent vs low-level implementation detail (intent says "compute total price", code has 12 lines of arithmetic; that's expected, not drift).
- Code-only or intent-only existence at the *whole-node* level: that's already covered by `metadata.intentMissing` / `metadata.implementationMissing` flags on the node. Use `DriftIssue` for field-level drift on a shared concept.
- A role missing at the *whole-node* level (only some roles have evidence so far): that's already covered by `metadata.missingRoles` on the node. Use `DriftIssue` for field-level drift on a shared concept.
- Vague suspicions ("I think these might differ but couldn't verify"). Either confirm with a specific cite or skip. Drift is structured evidence, not impressions.

If the disagreement makes you unsure which concept these even *are* (two different `cancelOrder` candidates? same? distinct?), you don't have field-level drift; you have an identity question. Emit a `ClarifyTicket`, not a `DriftIssue`.
Expand Down Expand Up @@ -139,5 +139,5 @@ Before attaching a `DriftIssue`:
- [ ] Two specific source citations (file + line / anchor)
- [ ] Description names both sides and the impact in one sentence
- [ ] Severity matches the contradiction-vs-gap distinction above
- [ ] Not duplicating an `intentMissing` / `implementationMissing` flag at the whole-node level
- [ ] Not duplicating a `missingRoles` entry at the whole-node level
- [ ] If multiple dimensions disagree, one `DriftIssue` per dimension
5 changes: 2 additions & 3 deletions packages/core/skills/shared/proposal-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,14 +43,13 @@ Each entry in `operations[]` is one of:
"sourceReferences": [
{ "sourceId": "src-intent", "location": { "uri": "...", "anchor": "..." } }
],
"intentMissing": false, // optional; true means "code exists, no spec yet"
"implementationMissing": false, // optional; true means "spec exists, no code yet"
"missingRoles": [], // optional list of roles not yet evidenced
"driftIssues": [ /* DriftIssue[]; see drift-detection.md */ ]
}
}
```

`EvidenceValidator` (in `validators.md`) requires *some* evidence: at least one `sourceReferences` entry, or `intentMissing: true`, or `implementationMissing: true`.
`EvidenceValidator` (in `validators.md`) requires *some* evidence: at least one `sourceReferences` entry, or a non-empty `missingRoles`.

### Picking sourceReferences

Expand Down
7 changes: 3 additions & 4 deletions packages/core/skills/shared/validators.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,7 @@ Reads cardinality "one-to-many" left-to-right: `1:N` means each target accepts a
**What it checks.** Every node carries an evidence trail:

- At least one `metadata.sourceReferences[]` entry, **OR**
- `metadata.implementationMissing: true` (intent-only, code not built yet), **OR**
- `metadata.intentMissing: true` (code-only, intent not written yet).
- a non-empty `metadata.missingRoles[]`, the declared source roles whose evidence is missing.

Also: a node with `status: 'completed'` must have at least one `sourceReferences` entry (completion is a claim of fact and requires a citation).

Expand All @@ -56,11 +55,11 @@ And: every `DriftIssue` attached to a node's `metadata.driftIssues[]` is surface

| Code | When |
|---|---|
| `evidence.no-source-or-flag` | Node has no sources AND no `intentMissing`/`implementationMissing` flag |
| `evidence.no-source-or-missing-roles` | Node has no sources AND no `missingRoles` |
| `evidence.completed-no-source` | Node is `status: completed` but `sourceReferences[]` is empty |
| `evidence.drift` | A DriftIssue on the node's metadata (severity passed through; `error` blocks apply) |

**How to self-check.** Treat `metadata` as required on every node you emit. If you have nothing, decide which flag applies (extract from code only means `intentMissing`; extract from PRD only means `implementationMissing`). Never emit empty `metadata`.
**How to self-check.** Treat `metadata` as required on every node you emit. If a node has no source yet, list the declared roles whose evidence is missing in `metadata.missingRoles`. Never emit empty `metadata`.

## 4. OrphanEdgeValidator

Expand Down
17 changes: 15 additions & 2 deletions packages/core/src/domain/validation/OntologyTypeValidator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ import type {
import type { OntologyPlugin, OntologyValidator } from '../plugin/OntologyPlugin.js'

/**
* Generic engine. Reads `nodeTypes` / `edgeTypes` from an `OntologyPlugin`,
* and rejects nodes or edges whose `type` field isn't in the allow-list.
* Generic engine reading an `OntologyPlugin`'s declared vocabulary.
* Rejects any graph reference outside it, a node or edge `type`,
* or a node's `metadata.missingRoles` entry.
*
* Not a plugin: callers construct an instance bound to their ontology,
* typically `defineOntologyPlugin()` in the SDK,
Expand All @@ -18,10 +19,12 @@ import type { OntologyPlugin, OntologyValidator } from '../plugin/OntologyPlugin
export class OntologyTypeValidator implements OntologyValidator {
private readonly knownNodeTypes: ReadonlySet<string>
private readonly knownEdgeTypes: ReadonlySet<string>
private readonly knownSourceRoles: ReadonlySet<string>

constructor(private readonly ontology: OntologyPlugin) {
this.knownNodeTypes = new Set(ontology.nodeTypes.map(nodeType => nodeType.id))
this.knownEdgeTypes = new Set(ontology.edgeTypes.map(edgeType => edgeType.id))
this.knownSourceRoles = new Set(ontology.sourceRoles.map(role => role.id))
}

async validate(snapshot: ModelSnapshot): Promise<readonly ValidationIssue[]> {
Expand All @@ -35,6 +38,16 @@ export class OntologyTypeValidator implements OntologyValidator {
nodeId: node.id,
})
}
for (const role of node.metadata.missingRoles ?? []) {
if (!this.knownSourceRoles.has(role)) {
issues.push({
code: 'ontology.unknown-source-role' as ValidationCode,
severity: 'error',
message: `Node "${node.name}" declares a missing role "${role}" which is not a source role in the ${this.ontology.ontologyId} ontology. Valid roles: ${[...this.knownSourceRoles].join(', ')}.`,
nodeId: node.id,
})
}
}
}
for (const edge of snapshot.edges) {
if (!this.knownEdgeTypes.has(edge.type)) {
Expand Down
16 changes: 8 additions & 8 deletions packages/core/src/domain/validation/validateEvidence.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,12 @@ import type {
* Framework invariant: every node must declare some evidence trail.
* One of these must hold:
* - at least one `metadata.sourceReferences[]` entry,
* - `metadata.implementationMissing: true`, intent-only, code not built yet,
* - `metadata.intentMissing: true`, code-only, intent not written yet.
* - a non-empty `metadata.missingRoles[]`, declared roles whose evidence is missing.
*
* The rule is role-agnostic. It never names a role,
* it only asks for a source or an explicit list of roles not yet evidenced.
* Without this, the graph silently accepts wishful thinking,
* intent claimed as done with no code and no explicit "not yet" flag.
* a node claimed with no source and no explicit missing-roles list.
*
* Also catches the contradiction of `status: 'completed'` with zero references,
* completion is a claim of fact, and needs at least one source citation.
Expand All @@ -33,14 +34,13 @@ function checkNode(node: GraphNode): ValidationIssue[] {
const issues: ValidationIssue[] = []
const refs = node.metadata.sourceReferences
const hasSources = refs.length > 0
const intentMissing = node.metadata.intentMissing === true
const implementationMissing = node.metadata.implementationMissing === true
const hasMissingRoles = (node.metadata.missingRoles?.length ?? 0) > 0

if (!hasSources && !intentMissing && !implementationMissing) {
if (!hasSources && !hasMissingRoles) {
issues.push({
code: 'evidence.no-source-or-flag' as ValidationCode,
code: 'evidence.no-source-or-missing-roles' as ValidationCode,
severity: 'error',
message: `Node "${node.name}" has no sourceReferences and neither intentMissing nor implementationMissing is set. Every node must declare evidence (a source) or an explicit "not yet" flag.`,
message: `Node "${node.name}" has no sourceReferences and no missingRoles. Every node must declare evidence (a source) or the roles whose evidence is missing.`,
nodeId: node.id,
})
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import type {
NodeTypeId,
ProposalId,
SkillId,
SourceRole,
UserId,
WorkspaceId,
} from '@braidhq/schema'
Expand Down Expand Up @@ -112,7 +113,7 @@ describe('HITLService — workspace history hooks', () => {
name: 'voidTask',
id: mintTestId('n') as NodeId,
status: 'draft' as NodeStatus,
metadata: { sourceReferences: [], implementationMissing: true },
metadata: { sourceReferences: [], missingRoles: ['alpha' as SourceRole] },
},
}],
generatedBy: 'extract' as SkillId,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { EdgeId, EdgeTypeId, ModelSnapshot, NodeId, NodeStatus, NodeTypeId } from '@braidhq/schema'
import type { EdgeId, EdgeTypeId, ModelSnapshot, NodeId, NodeStatus, NodeTypeId, SourceRole } from '@braidhq/schema'
import { makeOntology } from '@braidhq/test-utils'
import { describe, expect, it } from 'vitest'
import { OntologyTypeValidator } from '../../../src/domain/validation/OntologyTypeValidator.js'
Expand All @@ -23,6 +23,7 @@ const tinyOntology = makeOntology({
edgeTypes: [
{ id: 'mounts' as EdgeTypeId, fromTypes: ['page'] as NodeTypeId[], toTypes: ['widget'] as NodeTypeId[] },
],
sourceRoles: [{ id: 'spec', label: 'Spec' }],
})

describe('OntologyTypeValidator', () => {
Expand All @@ -41,6 +42,25 @@ describe('OntologyTypeValidator', () => {
expect(issues).toEqual([])
})

it('accepts a node whose missingRoles are declared source roles', async () => {
const issues = await validator.validate(snapshot([
{ id: 'n1' as NodeId, type: 'page' as NodeTypeId, name: 'home', status: draft, metadata: { sourceReferences: [], missingRoles: ['spec' as SourceRole] } },
]))
expect(issues).toEqual([])
})

it('rejects a missingRoles entry not declared by the ontology', async () => {
const issues = await validator.validate(snapshot([
{ id: 'n1' as NodeId, type: 'page' as NodeTypeId, name: 'home', status: draft, metadata: { sourceReferences: [], missingRoles: ['bogus' as SourceRole] } },
]))
expect(issues).toHaveLength(1)
expect(issues[0]).toMatchObject({
code: 'ontology.unknown-source-role',
severity: 'error',
nodeId: 'n1',
})
})

it('rejects unknown node types with namespaced code', async () => {
const issues = await validator.validate(snapshot([
{ id: 'n1' as NodeId, type: 'unknown-type' as NodeTypeId, name: 'x', status: draft, metadata: { sourceReferences: [] } },
Expand Down
18 changes: 9 additions & 9 deletions packages/core/test/domain/validation/validateEvidence.test.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import type { DriftIssueId, ModelSnapshot, NodeId, NodeStatus, NodeTypeId, SourceId } from '@braidhq/schema'
import type { DriftIssueId, ModelSnapshot, NodeId, NodeStatus, NodeTypeId, SourceId, SourceRole } from '@braidhq/schema'
import { describe, expect, it } from 'vitest'
import { validateEvidence } from '../../../src/domain/validation/validateEvidence.js'

Expand All @@ -15,7 +15,7 @@ function snapshot(nodes: ModelSnapshot['nodes'], edges: ModelSnapshot['edges'] =
}

describe('validateEvidence', () => {
it('emits error when node has no sources and no missing-evidence flag', () => {
it('emits error when node has no sources and no missingRoles', () => {
const issues = validateEvidence(snapshot([
{
id: 'n1' as NodeId,
Expand All @@ -27,26 +27,26 @@ describe('validateEvidence', () => {
]))
expect(issues).toHaveLength(1)
expect(issues[0]).toMatchObject({
code: 'evidence.no-source-or-flag',
code: 'evidence.no-source-or-missing-roles',
severity: 'error',
nodeId: 'n1',
})
})

it('accepts node with implementationMissing flag', () => {
it('accepts sourceless node that declares a missing role', () => {
const issues = validateEvidence(snapshot([
{
id: 'n1' as NodeId,
type: aggregate,
name: 'Cart',
status: draft,
metadata: { sourceReferences: [], implementationMissing: true },
metadata: { sourceReferences: [], missingRoles: ['alpha' as SourceRole] },
},
]))
expect(issues).toEqual([])
})

it('accepts node with intentMissing flag', () => {
it('accepts a sourced node that also declares a missing role', () => {
const issues = validateEvidence(snapshot([
{
id: 'n1' as NodeId,
Expand All @@ -58,7 +58,7 @@ describe('validateEvidence', () => {
sourceId: 'code-a' as SourceId,
location: { uri: 'apps/api/cart.ts' },
}],
intentMissing: true,
missingRoles: ['alpha' as SourceRole],
},
},
]))
Expand All @@ -72,10 +72,10 @@ describe('validateEvidence', () => {
type: aggregate,
name: 'Cart',
status: completed,
metadata: { sourceReferences: [], implementationMissing: true },
metadata: { sourceReferences: [], missingRoles: ['alpha' as SourceRole] },
},
]))
// implementationMissing clears the no-source rule,
// missingRoles clears the no-source rule,
// but completed-no-source still fires.
expect(issues.map(i => i.code)).toEqual(['evidence.completed-no-source'])
})
Expand Down
8 changes: 4 additions & 4 deletions packages/ontology-ddd/skills/extract/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,12 +75,12 @@ For each candidate node compared to the current graph:

For each candidate node, set `metadata` according to where the evidence lives:

- Intent source only (no code yet, e.g. a fresh PRD): set `metadata.sourceReferences = [intent ref]` plus `metadata.implementationMissing = true`. Status stays `draft`.
- Code source only (running code with no spec): set `metadata.sourceReferences = [code ref]` plus `metadata.intentMissing = true`. Status `draft`.
- Intent source only (no code yet, e.g. a fresh PRD): set `metadata.sourceReferences = [intent ref]` plus `metadata.missingRoles = ['code']`. Status stays `draft`.
- Code source only (running code with no spec): set `metadata.sourceReferences = [code ref]` plus `metadata.missingRoles = ['intent']`. Status `draft`.
- Both sources agree: set `metadata.sourceReferences = [intent ref, code ref]`. Status `draft` (only the human can promote to `completed` on apply).
- Both sources disagree: distinguish identity-level disagreement from field-level drift (see below).

Every node you emit MUST have `metadata` set. A node with `metadata.sourceReferences: []` AND no `implementationMissing` AND no `intentMissing` will be rejected by the server validator.
Every node you emit MUST have `metadata` set. A node with `metadata.sourceReferences: []` AND an empty `metadata.missingRoles` will be rejected by the server validator.

When a node has multiple sources to cite (intent plus one or more code files, or several layers of code), order them by representativeness: see `proposal-format.md` § Picking sourceReferences.

Expand Down Expand Up @@ -125,7 +125,7 @@ Produced N proposals + M clarify tickets:

- [ ] Ontology fetched from `braid-core` before any operation was drafted; every `node.type` / `edge.type` matches an id in the response.
- [ ] Wiring rules in `ontology-ddd/concept.md` followed (parent edges, no Context Mapping auto-emit, policy has both edges, `dependsOn` is aggregate-to-aggregate).
- [ ] Every node has `metadata.sourceReferences` and / or an `implementationMissing` / `intentMissing` flag.
- [ ] Every node has `metadata.sourceReferences` and / or a non-empty `metadata.missingRoles`.
- [ ] Field-level disagreement between sources surfaces as a `DriftIssue` on the node (see `drift-detection.md`), not a ClarifyTicket.
- [ ] Each proposal was submitted via `braid-core` proposal-create and the final response was 201 (not 4xx).
- [ ] No `removeNode` of a node still referenced elsewhere; deprecate instead.
Expand Down
8 changes: 5 additions & 3 deletions packages/schema/src/model.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { z } from 'zod'
import { DriftIssueId, EdgeId, ExternalReference, NodeId, SkillId, SourceReference, Timestamp } from './common.js'
import { EdgeTypeId, NodeStatus, NodeTypeId } from './ontology.js'
import { SourceRole } from './source.js'

export const DriftSeverity = z.enum(['error', 'warning', 'info'])
export type DriftSeverity = z.infer<typeof DriftSeverity>
Expand Down Expand Up @@ -31,9 +32,10 @@ export const Embedding = z.object({
export type Embedding = z.infer<typeof Embedding>

export const GraphNodeMetadata = z.object({
intentMissing: z.boolean().optional(),
intentConflict: z.boolean().optional(),
implementationMissing: z.boolean().optional(),
// Declared source roles whose evidence is missing on this node.
// Role-agnostic, an ontology names its own roles, absent means none missing.
// A node with no sources stays valid while a role is still missing.
missingRoles: z.array(SourceRole).optional(),
sourceReferences: z.array(SourceReference).default([]),
lastTouchedBy: SkillId.optional(),
externalReferences: z.array(ExternalReference).optional(),
Expand Down
14 changes: 11 additions & 3 deletions packages/schema/src/proposal-preview.ts
Original file line number Diff line number Diff line change
Expand Up @@ -190,9 +190,7 @@ function nodesEqual(a: GraphNode, b: GraphNode): boolean {
&& a.name === b.name
&& a.description === b.description
&& a.status === b.status
&& a.metadata.intentMissing === b.metadata.intentMissing
&& a.metadata.intentConflict === b.metadata.intentConflict
&& a.metadata.implementationMissing === b.metadata.implementationMissing
&& stringArraysEqual(a.metadata.missingRoles, b.metadata.missingRoles)
&& a.metadata.lastTouchedBy === b.metadata.lastTouchedBy
&& sourceRefsEqual(a.metadata.sourceReferences, b.metadata.sourceReferences)
&& externalRefsEqual(a.metadata.externalReferences, b.metadata.externalReferences)
Expand All @@ -209,6 +207,16 @@ function edgesEqual(a: GraphEdge, b: GraphEdge): boolean {

// Structural comparison, not JSON.stringify. Stable across key order, no stringify cost. Array order matters,
// since skills typically append.
function stringArraysEqual(a: readonly string[] | undefined, b: readonly string[] | undefined): boolean {
if (a === undefined && b === undefined)
return true
if (a === undefined || b === undefined)
return false
if (a.length !== b.length)
return false
return a.every((value, index) => value === b[index])
}

function sourceRefsEqual(a: readonly SourceReference[], b: readonly SourceReference[]): boolean {
if (a.length !== b.length)
return false
Expand Down
6 changes: 3 additions & 3 deletions packages/schema/test/ontology.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,11 +75,11 @@ describe('OntologyResponse', () => {

it('carries the ontology-declared source roles and their capabilities', () => {
const res = OntologyResponse.parse({
ontologyId: 'ddd',
ontologyId: 'test',
nodeTypes: [],
edgeTypes: [],
sourceRoles: [{ id: 'intent', label: 'Intent', required: true, unitBearing: true, pathSegment: 'intents' }],
sourceRoles: [{ id: 'alpha', label: 'Alpha', required: true, unitBearing: true, pathSegment: 'alphas' }],
})
expect(res.sourceRoles).toEqual([{ id: 'intent', label: 'Intent', required: true, unitBearing: true, pathSegment: 'intents' }])
expect(res.sourceRoles).toEqual([{ id: 'alpha', label: 'Alpha', required: true, unitBearing: true, pathSegment: 'alphas' }])
})
})
Loading
Loading