feat: promotion candidate detector and pk candidates list|accept|reject

openspec/changes/promotion-candidates/design.md

@@ -0,0 +1,41 @@
+## Where the detector runs
+
+Inside `run-once`, after jobs drain and before memory reconciliation, while the queue lock is held. The worker is the only writer of `lessons.jsonl` and `pending/`, so the lock serializes it. A detector error is logged and recorded in `status.json`'s existing `lastError`. It never fails the run, because memory delivery matters more than a proposal.
+
+## What a "lesson" is
+
+The worker's job carries one final assistant message, not structured lessons. The detector takes lines prefixed `LESSON:`, `GOTCHA:`, `PATTERN:` or `LEARNED:` (case-insensitive, list bullets allowed). When there are none, it takes the whole message (first 2,000 characters). It also takes typed `Lesson`/`Gotcha` entries from the project KB, because frontmatter tags are the only place a `global` tag is observable. Inline `#tag` words count as tags for session lessons.
+
+**The uncomfortable case:** a whole-message lesson is noisy. Two long, mostly different messages rarely reach Jaccard 0.6, which keeps false recurrence low. But a message that names a CLI in a code span and cites no repo path becomes a `portable-tooling` candidate on its own. The human gate absorbs that noise. If it proves too much, the fix is to tighten `names_dependency_or_cli`, not to remove the gate.
+
+## Fingerprints and ids
+
+- **Tokens:** lowercase alphanumeric runs. **Shingles:** word trigrams, hashed with the first 8 bytes of SHA-256. A text under three tokens is one shingle. SHA-256, not `DefaultHasher`, because the index outlives the binary and `DefaultHasher` output is not stable across Rust releases.
+- **Lesson id:** SHA-256 over `projectId \0 source \0 normalized tokens`, so re-indexing the same job or KB entry is a no-op.
+- **Clustering:** greedy, single pass, ordered by `(recordedAt, lessonId)`. Each record joins the first cluster whose anchor (first member) it matches at ≥ 0.6. New records always sort after existing ones, so an anchor never changes.
+- **Candidate id:** `promo-` plus 16 hex digits of SHA-256 over the anchor's lesson id. It stays stable as evidence grows.
+
+## Idempotency
+
+A candidate whose id exists in `accepted/` or `rejected/` is skipped. A pending candidate is rewritten, atomically and keeping `createdAt`, only when its evidence set changed. Otherwise the file is untouched.
+
+**Known race:** `pk candidates accept` does not take the worker's queue lock, because pk-cli has no `fs2` dependency. If a worker run rewrites a pending candidate at the moment `accept` renames it, the pending file can reappear. A second accept is then idempotent: the KB upsert is skipped when the content is unchanged, and the memory operation is skipped when its id exists in any outbox state. The window is the time between one `exists` check and one `rename`.
+
+## Accept semantics
+
+The steps run in a crash-safe order:
+
+1. Upsert the KB entry. It has a fixed id and is skipped when unchanged.
+2. Commit the shared snapshot.
+3. Queue the memory operation. Its id is deterministic, `sha256("promotion-candidate:<id>")`, and it is skipped when present in any `memory/*` state.
+4. Rewrite the candidate with `state: accepted`, then durably rename it to `accepted/`.
+
+A crash at any step leaves a pending candidate, and re-running `accept` completes it without duplicates.
+
+The outbox record matches the worker's own `enqueue_memory`: schemaVersion 2, `method: add_memory`, and `payloadHash` = SHA-256 of `serde_json::to_vec(arguments)`. The worker's `normalize_operation` re-derives that hash, and it passes `add_memory` arguments through unchanged.
+
+**User scope** is `@user:<hash>` with `agent_id: @user` (design table §2). The hash uses the same resolver as the skill system's `project_id.py`: `PROMETHEUS_USER_ID`, else the first 16 hex digits of SHA-256 over the lowercased global git email.
+
+## Timestamps in pk-cli
+
+pk-cli has no chrono dependency, and adding one would change `Cargo.lock`, which the `--locked` rule forbids without approval. `rfc3339_now()` formats `SystemTime` with the civil-from-days algorithm. The worker parses `queuedAt` as RFC 3339, so the format must stay `YYYY-MM-DDTHH:MM:SSZ`.

openspec/changes/promotion-candidates/proposal.md

@@ -0,0 +1,44 @@
+## Why
+
+The skill system's team-aware learning design (§3, "Visibility and promotion") makes promotion beyond a project auto-proposed and human-confirmed. Nothing proposed or applied promotion candidates:
+
+- **No detector.** A lesson learned in two projects stayed two private project lessons. Nothing noticed recurrence, a `global` tag, or a lesson about a tool rather than a repository.
+- **No confirmation step.** No command took a proposed lesson into the shared KB and the `@global`/`@user` memory scopes, so the only way to promote was a hand-run `pk ingest --scope shared`, which skips surreal-memory entirely.
+
+Adding a `pk` subcommand changes a frozen surface (the `pk` CLI). This proposal names the consumers.
+
+## What Changes
+
+- **Worker promotion detector (new, additive).** After draining jobs in `run-once`, `prometheus-learning-worker`:
+  - extracts lessons from each processed job (lines prefixed `LESSON:`, `GOTCHA:`, `PATTERN:` or `LEARNED:`, else the whole final assistant message) and from typed `Lesson`/`Gotcha` entries in the project KB;
+  - fingerprints them as word-trigram shingle sets into `~/.prometheus/learning-index/lessons.jsonl`, with project id, project root and source evidence. Append-only and idempotent on lesson id;
+  - proposes a candidate when a lesson recurs with Jaccard ≥ 0.6 across ≥ 2 distinct projects, is tagged global without a `[GLOBAL]`/`[USER]` marker, or names a dependency or CLI and contains no repo-relative path;
+  - writes candidates atomically to `~/.prometheus/promotion-candidates/pending/<id>.json` with an evidence list. Re-runs do not duplicate a candidate, and a candidate already accepted or rejected is never re-proposed.
+- **`pk candidates list|accept|reject --kind promotion|skill` (new subcommand, additive).**
+  - `list [--state pending|accepted|rejected] [--json]`.
+  - `accept <id>` (promotion): upserts the lesson into `~/.prometheus/knowledge/shared` as `promoted-<id>`, commits the shared prompt snapshot, queues an `add_memory` operation for `@global` (or `@user:<hash>` / `@user` for a user-scoped candidate) in the learning-queue memory outbox, and moves the file to `accepted/`.
+  - `reject <id> [--reason]`: moves the file to `rejected/`.
+  - `--kind skill` reads `~/.prometheus/skill-candidates/`. `list` and `reject` work. `accept` exits non-zero with "not yet supported" until skill promotion lands.
+
+## Consumers
+
+| Consumer | Affected surface | Effect |
+|---|---|---|
+| prometheus-skill-system (`kbd-open.sh`, reflector role, later skill-candidate work) | `pk candidates`, `~/.prometheus/promotion-candidates/` | Intended consumer. It surfaces pending candidates and asks a human to run `pk candidates accept|reject`. Needs a pin bump to this release; handoff note below. |
+| prometheus-skills-mini | `pk ingest` | Unchanged. |
+| forge-rs | `pk ingest` (shells out) | Unchanged. |
+| prometheus-cli | pk-core / pk-store / pk-librarian crates | No public API change. |
+| prometheus doctor | `status.json` | No field added. `lastError` may now carry a `promotion detector: …` message. |
+| surreal-memory-server | `/api/v2/operations` via the worker's outbox | New operations use the existing `add_memory` method and the existing schemaVersion 2 outbox format, keyed `@global`/`@global` or `@user:<hash>`/`@user`. No ledger contract change. |
+
+## Capabilities
+
+### New Capabilities
+- `promotion-candidates`: lesson fingerprinting, candidate proposal, and human-confirmed promotion to shared scopes.
+
+## Impact
+
+- **Code:** `pk-learning-worker/src/promotion.rs` (new), `pk-learning-worker/src/main.rs`, `pk-cli/src/candidates.rs` (new), `pk-cli/src/main.rs`.
+- **Tests:** `pk-learning-worker/tests/promotion.rs`, `pk-cli/tests/candidates.rs`.
+- **Dependencies:** none added. `Cargo.lock` is unchanged.
+- **Downstream handoff (prometheus-skill-system):** bump the pk pin to the release containing this change. Document `pk candidates` in `docs/guide/13-tools-reference.md`, and point `kbd-open.sh` and the reflector role at `~/.prometheus/promotion-candidates/pending`.

openspec/changes/promotion-candidates/specs/promotion-candidates/spec.md

@@ -0,0 +1,43 @@
+## ADDED Requirements
+
+### Requirement: The worker fingerprints lessons
+After processing jobs, `prometheus-learning-worker run-once` SHALL append a record for each newly seen lesson to `~/.prometheus/learning-index/lessons.jsonl`, carrying its project id, project root, source evidence and word-trigram fingerprint. Re-processing the same lesson SHALL NOT append a second record.
+
+#### Scenario: Re-running indexes nothing new
+- **WHEN** the worker runs again with no new jobs
+- **THEN** `lessons.jsonl` is byte-identical
+
+### Requirement: Recurring or portable lessons become promotion candidates
+The worker SHALL write a candidate to `~/.prometheus/promotion-candidates/pending/<id>.json`, atomically, with an evidence list, when a lesson recurs with Jaccard ≥ 0.6 across at least two distinct projects, is tagged global without a `[GLOBAL]`/`[USER]` marker, or names a dependency or CLI and contains no repo-relative path. A lesson carrying an immediate-promotion marker SHALL NOT be proposed.
+
+#### Scenario: The same lesson in two projects
+- **WHEN** two jobs from two different projects carry the same lesson
+- **THEN** exactly one pending candidate exists, with reason `recurrence` and two evidence entries citing two distinct project ids
+
+### Requirement: Candidate proposal is idempotent
+Re-running the worker SHALL NOT create a second candidate for the same cluster, SHALL NOT rewrite a pending candidate whose evidence is unchanged, and SHALL NOT re-propose a candidate that was accepted or rejected.
+
+#### Scenario: Re-run adds none
+- **WHEN** the worker runs a second time with no new jobs
+- **THEN** the pending directory holds the same single file, byte-identical
+
+### Requirement: Accepting a promotion candidate promotes it
+`pk candidates accept <id>` SHALL upsert the lesson into `~/.prometheus/knowledge/shared`, commit the shared prompt snapshot, queue one schemaVersion 2 `add_memory` operation whose `payloadHash` is the SHA-256 of its serialized arguments, keyed `@global`/`@global` (or `@user:<hash>`/`@user` for a user-scoped candidate), and move the candidate file to `accepted/`. Accepting an already-accepted candidate SHALL fail without queuing a second operation.
+
+#### Scenario: Accept a global candidate
+- **WHEN** `pk candidates accept <id>` runs against a pending global candidate
+- **THEN** the shared prompt snapshot contains `promoted-<id>`, the learning-queue outbox holds one `@global` `add_memory` operation, and the candidate is in `accepted/`
+
+### Requirement: Rejecting a candidate retires it
+`pk candidates reject <id>` SHALL move the candidate to `rejected/`, recording an optional reason.
+
+#### Scenario: Reject
+- **WHEN** `pk candidates reject <id> --reason noise` runs
+- **THEN** the file is in `rejected/` and no longer in `pending/`
+
+### Requirement: Skill candidates are listed but not yet accepted
+`pk candidates --kind skill` SHALL list and reject entries under `~/.prometheus/skill-candidates/`, and `accept` SHALL exit non-zero with a "not yet supported" message, leaving the file in place.
+
+#### Scenario: Skill accept
+- **WHEN** `pk candidates accept <id> --kind skill` runs
+- **THEN** it exits non-zero, stderr says "not yet supported", and the candidate is still pending

openspec/changes/promotion-candidates/tasks.md

@@ -0,0 +1,12 @@
+## 1. Promotion detector (learning-worker-engineer)
+- [x] 1.1 `pk-learning-worker/src/promotion.rs`: lesson extraction, trigram fingerprints, `lessons.jsonl` index, greedy clustering, candidate proposal with evidence.
+- [x] 1.2 Return observations from `process_job` and run the detector after the job loop in `run_once` (`pk-learning-worker/src/main.rs`).
+- [x] 1.3 `pk-learning-worker/tests/promotion.rs`: two projects with one lesson yield one candidate with two evidence entries, and a re-run changes nothing.
+
+## 2. `pk candidates` (surfaces-engineer)
+- [x] 2.1 `pk-cli/src/candidates.rs` + the `Candidates` subcommand in `pk-cli/src/main.rs`: list, accept (promotion), reject, `--kind skill` list/reject with accept refused.
+- [x] 2.2 `pk-cli/tests/candidates.rs`: accept writes the shared snapshot, queues one `@global` op with a valid payloadHash, and moves the file; reject moves the file; skill accept exits non-zero.
+
+## 3. Release and handoff (knowledge-lead)
+- [ ] 3.1 Verifier gate: `cargo fmt --all -- --check`, `cargo clippy --workspace --all-targets --locked -- -D warnings`, `cargo test --workspace --locked`.
+- [ ] 3.2 Downstream handoff note for prometheus-skill-system (pin, tools reference, `kbd-open.sh`/reflector surfacing).