feat: add pilot evidence and operator surfaces - #14
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 522a6ba35e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const rebuilt = buildEvidenceReceipt(found, { generatedAt: receipt.generatedAt }); | ||
| if (JSON.stringify(rebuilt.integrity) !== JSON.stringify(receipt.integrity)) { | ||
| issue(issues, "receipt-integrity-inventory-mismatch", "Receipt integrity inventory does not match current canonical evidence."); |
There was a problem hiding this comment.
If any receipt field outside integrity is edited—for example, the completeness claim, task results, or reviewer decision—run verify still passes because the rebuilt receipt is compared only through its integrity member, and the receipt itself is excluded from that inventory. Compare the complete rebuilt receipt (using the persisted generatedAt) so verification cannot accept altered evidence claims.
AGENTS.md reference: AGENTS.md:L5-L5
Useful? React with 👍 / 👎.
| const qualifyingEvidence = definition.source === "participants" | ||
| ? [...externalParticipantIds].sort() | ||
| : [...new Set(evidenceByType.get(definition.observationType) || [])].sort(); | ||
| const observed = qualifyingEvidence.length; |
There was a problem hiding this comment.
When one independent participant records three repeat-user observations with different observation IDs, this generic evidence count reaches the threshold of three even though there is only one repeat user. Derive this gate from distinct record.participant.id values that have qualifying repeat-use evidence rather than counting observations.
AGENTS.md reference: AGENTS.md:L5-L5
Useful? React with 👍 / 👎.
| const qualifyingEvidence = definition.source === "participants" | ||
| ? [...externalParticipantIds].sort() | ||
| : [...new Set(evidenceByType.get(definition.observationType) || [])].sort(); | ||
| const observed = qualifyingEvidence.length; |
There was a problem hiding this comment.
Five onboarding-remediation observations can all use rank: 1 with distinct codes and this count marks top-onboarding-failures-remediated as met. Since the validator only bounds each rank to 1–5, status must deduplicate by rank and require the complete rank set instead of treating any five observations as the top five failures.
AGENTS.md reference: AGENTS.md:L5-L5
Useful? React with 👍 / 👎.
| const recordPath = path.join(repoRoot, ".cewp", "pilots", pilotId, "record.json"); | ||
| if (fs.existsSync(recordPath)) { |
There was a problem hiding this comment.
If .cewp/pilots/<pilot-id> is pre-created as a symlink to a directory outside the repository, this path construction and the later atomic write follow it, allowing pilot create to create record.json outside the repository; pilot record can similarly rewrite a valid external record. Apply the same component-by-component symlink and realpath containment checks used for export before reading or writing canonical pilot records.
Useful? React with 👍 / 👎.
| const record = JSON.parse(fs.readFileSync(filePath, "utf8")); | ||
| if (record.schemaVersion !== PILOT_RECORD_SCHEMA_VERSION) { | ||
| throw new Error(`Unsupported pilot record schema for ${entry.name}: ${record.schemaVersion || "missing"}.`); |
There was a problem hiding this comment.
When any record.json is truncated, invalid JSON, or uses an unsupported schema, this unguarded parse/throw aborts the entire pilot status command, hiding every valid record and gate result. Load records independently and report failures through the existing records.invalid, exclusions, and warnings fields so one damaged pilot cannot make the authoritative status unavailable.
AGENTS.md reference: AGENTS.md:L5-L5
Useful? React with 👍 / 👎.
Summary
Validation
This is part 4 of the ordered Phase 11-14 publication series.