Skip to content

feat(intelligence): first-class customer contacts → relationship churn signals (champion-at-risk, single-threaded) - #37

Closed
zaridan wants to merge 2 commits into
mainfrom
feat/rforge-contact-relationship-modeling
Closed

feat(intelligence): first-class customer contacts → relationship churn signals (champion-at-risk, single-threaded)#37
zaridan wants to merge 2 commits into
mainfrom
feat/rforge-contact-relationship-modeling

Conversation

@zaridan

@zaridan zaridan commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Why

We're building an accuracy benchmark that pits the Resonant IQ engine's evidence-fusion against a naive single-prompt LLM, using ResonantForge's planted corpus as ground truth. Two of the engine's 11 churn detectors — relationship_champion_at_risk (Rule 4) and relationship_single_threaded (Rule 5) — track specific customer contacts across many conversations, and were untestable because the corpus modeled accounts, agents, and conversations but not distinct customer contacts. These two are likely the engine's strongest differentiator vs a plain LLM. This PR models customer contacts as first-class entities and plants both signals with decoys and ground-truth labels.

Process note: the Atlassian (Jira) MCP requires interactive OAuth that isn't available in a non-interactive session, so no RFORGE ticket could be created from here. Landed on the descriptive branch feat/rforge-contact-relationship-modeling; please attach the ticket # on merge.

Engine semantics were verified against resonantiq/src/lib/intelligence/churn-detectors.ts (Rules 4 & 5) and cross-stream-helpers.ts before implementing — including confirming that the engine's distinctContactsPrior180d field is the 60–240d window (matches the spec).

What's in it

SchemaContact (+ContactRole), ContactRollup, RelationshipLabel (+RelationshipDetector); DaySnapshot gains pre-rolled contact aggregates (distinct_active_contacts_60d, distinct_active_contacts_prior_180d, total_contacts_all_time, champion_rollups) mirroring how health_score is pre-rolled; conversation→contact attribution on payload + ConversationRecord; manifest counts/hashes + relationship_signal_distribution.

Simulationlayer1/contact_planner.py: a deterministic post-simulation pass (seed-derived RNG that never perturbs the event stream) that knows each account's realized lifespan, so planted conditions land relative to the account's actual "now". It generates contacts, attributes every conversation, enriches snapshots, and emits labels. Wired into StateMachine.simulate().

  • Positives: champion silence ≥30d, champion frequency <1/mo (both Rule-4 OR branches), and narrowing from ≥3 prior contacts to a single active thread (Rule 5).
  • Decoys (near-misses, must NOT fire): active champion; account narrowed to two threads; effectively-single-threaded but only 2 contacts all-time (guard) — mirrors the RFORGE-73 paraphrase-trap discipline so FP rate is measurable.
  • Ground truth by construction: labels are computed by pure Python mirrors of the engine's Rule 4/5 logic against the actual rollup, so expected_fire is exactly what a correct detector must output.

Prose — conversations attributed to the specific contact; name + role threaded into the prose prompt and the ConversationRecord.

Artifacts + CLI — new contacts.jsonl and relationship_labels.jsonl (hashed in the manifest); validate / inspect / stats extended.

Validatorvalidators/relationship.py: an account-level feature-extractor + rule-engine that independently recomputes windowed features from raw attribution and agrees with the planted ground truth (extends disagreement-ledger coverage).

Determinism

rforge generate --dry-run --smoke is now bit-identical across independent processes (verified end-to-end). This surfaced a pre-existing determinism bug — _pick_off_brand_variant_id used the PYTHONHASHSEED-salted builtin hash(), making planted_quality.jsonl differ across runs — fixed in a separate commit (36a4af2) with a subprocess regression test. The determinism property test now also covers contacts_hash and relationship_labels_hash.

Audited distribution (60-account dry-run, seed 42)

See docs/relationship-signal-distribution-audit.md. Per detector: champion_at_risk 13 positive / 47 negative (7 decoy); single_threaded 10 positive / 50 negative (10 decoy). Positives are the minority (~17–22%), the correct base rate for a churn signal, with measurable near-miss decoys for FP rate.

Testing

  • 17 new tests (tests/test_contact_modeling.py, tests/test_injector_offbrand_determinism.py); determinism test extended.
  • Full suite: 501 passed, 3 skipped.
  • Spot-checked accounts end-to-end: contacts → attributed conversations → snapshot rollup → planted label all consistent.

🤖 Generated with Claude Code

zaridan and others added 2 commits July 14, 2026 14:54
`_pick_off_brand_variant_id` indexed the variant list with the builtin
`hash(conv_id)`, which Python salts per process via PYTHONHASHSEED. Two
otherwise-identical `rforge generate --dry-run --smoke` runs therefore produced
different `off_brand_variant_id`s (and a different `planted_quality.jsonl` /
`planted_quality_hash`), violating the documented bit-identical determinism
guarantee. The function's own docstring already promised deterministic selection.

Switch to a stable blake2b digest. Adds a subprocess regression test asserting
the pick is identical across PYTHONHASHSEED values.

Pre-existing bug, surfaced while adding contact modeling (whose corpus must be
seed-reproducible).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…relationship churn signals)

Adds first-class customer-contact modeling so the Resonant IQ engine's two
relationship churn detectors — `relationship_champion_at_risk` (Rule 4) and
`relationship_single_threaded` (Rule 5) — can be scored against planted ground
truth. Both require tracking specific customer contacts across many
conversations, which the corpus previously could not express.

Schema (schemas.py)
- Contact (account_id, contact_id, name, role, is_champion, engagement span)
  + ContactRole enum.
- ContactRollup + DaySnapshot rollup fields (distinct_active_contacts_60d,
  distinct_active_contacts_prior_180d, total_contacts_all_time,
  champion_rollups) — trailing-window aggregates pre-rolled like health_score.
- ConversationStartedPayload / ConversationRecord contact attribution.
- RelationshipLabel + RelationshipDetector ground-truth schema.
- Manifest: contact_count, relationship_label_count, hashes, and
  relationship_signal_distribution telemetry.

Simulation (layer1/contact_planner.py, state_machine.py, sim_events.py)
- ContactPlanner: a deterministic post-simulation pass (seed-derived RNG that
  never perturbs the event stream). Knows each account's realized lifespan, so
  planted conditions land relative to the account's actual "now". Generates
  contacts, attributes every conversation to a contact, enriches snapshots, and
  emits labels. Wired into StateMachine.simulate().
- Plants both positives (champion silence ≥30d and champion frequency <1/mo;
  narrowing to a single active thread from ≥3 prior) and near-miss decoys
  (active champion; narrowed-to-two; small-account guard) mirroring the
  RFORGE-73 distractor discipline, so false-positive rate is measurable.
- Labels are computed by pure Python mirrors of the engine's Rule 4/5 logic
  (champion_at_risk_fires / single_threaded_fires) against the actual rollup —
  ground truth by construction.

Prose (pipeline.py)
- Conversations attributed to the specific customer contact; contact name/role
  threaded into the prose prompt and ConversationRecord.

Artifacts + CLI
- New contacts.jsonl and relationship_labels.jsonl (hashed in the manifest).
- validate / inspect / stats extended to cover both.

Validator (validators/relationship.py)
- Account-level feature-extractor + rule-engine that independently recomputes
  windowed contact features from raw attribution and agrees with the planted
  ground truth — extends disagreement-ledger coverage to relationship signals.

Determinism / audit
- `rforge generate --dry-run --smoke` is bit-identical across independent
  processes (verified end-to-end; determinism property test extended to cover
  contacts + relationship labels).
- docs/relationship-signal-distribution-audit.md audits the planted
  positive/negative/decoy distribution for both detectors (60-account dry-run).

17 new contact-modeling tests; full suite green (501 passed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@zaridan

zaridan commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by the correctly-named stack-base branch feat/rforge-93-contact-relationship-modeling (this work is RFORGE-93, the base of the benchmark-hardening stack). Re-opening as a draft there.

@zaridan zaridan closed this Jul 14, 2026
@zaridan
zaridan deleted the feat/rforge-contact-relationship-modeling branch July 14, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant