feat(intelligence): first-class customer contacts → relationship churn signals (champion-at-risk, single-threaded) - #37
Closed
zaridan wants to merge 2 commits into
Closed
Conversation
`_pick_off_brand_variant_id` indexed the variant list with the builtin `hash(conv_id)`, which Python salts per process via PYTHONHASHSEED. Two otherwise-identical `rforge generate --dry-run --smoke` runs therefore produced different `off_brand_variant_id`s (and a different `planted_quality.jsonl` / `planted_quality_hash`), violating the documented bit-identical determinism guarantee. The function's own docstring already promised deterministic selection. Switch to a stable blake2b digest. Adds a subprocess regression test asserting the pick is identical across PYTHONHASHSEED values. Pre-existing bug, surfaced while adding contact modeling (whose corpus must be seed-reproducible). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…relationship churn signals) Adds first-class customer-contact modeling so the Resonant IQ engine's two relationship churn detectors — `relationship_champion_at_risk` (Rule 4) and `relationship_single_threaded` (Rule 5) — can be scored against planted ground truth. Both require tracking specific customer contacts across many conversations, which the corpus previously could not express. Schema (schemas.py) - Contact (account_id, contact_id, name, role, is_champion, engagement span) + ContactRole enum. - ContactRollup + DaySnapshot rollup fields (distinct_active_contacts_60d, distinct_active_contacts_prior_180d, total_contacts_all_time, champion_rollups) — trailing-window aggregates pre-rolled like health_score. - ConversationStartedPayload / ConversationRecord contact attribution. - RelationshipLabel + RelationshipDetector ground-truth schema. - Manifest: contact_count, relationship_label_count, hashes, and relationship_signal_distribution telemetry. Simulation (layer1/contact_planner.py, state_machine.py, sim_events.py) - ContactPlanner: a deterministic post-simulation pass (seed-derived RNG that never perturbs the event stream). Knows each account's realized lifespan, so planted conditions land relative to the account's actual "now". Generates contacts, attributes every conversation to a contact, enriches snapshots, and emits labels. Wired into StateMachine.simulate(). - Plants both positives (champion silence ≥30d and champion frequency <1/mo; narrowing to a single active thread from ≥3 prior) and near-miss decoys (active champion; narrowed-to-two; small-account guard) mirroring the RFORGE-73 distractor discipline, so false-positive rate is measurable. - Labels are computed by pure Python mirrors of the engine's Rule 4/5 logic (champion_at_risk_fires / single_threaded_fires) against the actual rollup — ground truth by construction. Prose (pipeline.py) - Conversations attributed to the specific customer contact; contact name/role threaded into the prose prompt and ConversationRecord. Artifacts + CLI - New contacts.jsonl and relationship_labels.jsonl (hashed in the manifest). - validate / inspect / stats extended to cover both. Validator (validators/relationship.py) - Account-level feature-extractor + rule-engine that independently recomputes windowed contact features from raw attribution and agrees with the planted ground truth — extends disagreement-ledger coverage to relationship signals. Determinism / audit - `rforge generate --dry-run --smoke` is bit-identical across independent processes (verified end-to-end; determinism property test extended to cover contacts + relationship labels). - docs/relationship-signal-distribution-audit.md audits the planted positive/negative/decoy distribution for both detectors (60-account dry-run). 17 new contact-modeling tests; full suite green (501 passed). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
|
Superseded by the correctly-named stack-base branch feat/rforge-93-contact-relationship-modeling (this work is RFORGE-93, the base of the benchmark-hardening stack). Re-opening as a draft there. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
We're building an accuracy benchmark that pits the Resonant IQ engine's evidence-fusion against a naive single-prompt LLM, using ResonantForge's planted corpus as ground truth. Two of the engine's 11 churn detectors —
relationship_champion_at_risk(Rule 4) andrelationship_single_threaded(Rule 5) — track specific customer contacts across many conversations, and were untestable because the corpus modeled accounts, agents, and conversations but not distinct customer contacts. These two are likely the engine's strongest differentiator vs a plain LLM. This PR models customer contacts as first-class entities and plants both signals with decoys and ground-truth labels.Engine semantics were verified against
resonantiq/src/lib/intelligence/churn-detectors.ts(Rules 4 & 5) andcross-stream-helpers.tsbefore implementing — including confirming that the engine'sdistinctContactsPrior180dfield is the 60–240d window (matches the spec).What's in it
Schema —
Contact(+ContactRole),ContactRollup,RelationshipLabel(+RelationshipDetector);DaySnapshotgains pre-rolled contact aggregates (distinct_active_contacts_60d,distinct_active_contacts_prior_180d,total_contacts_all_time,champion_rollups) mirroring howhealth_scoreis pre-rolled; conversation→contact attribution on payload +ConversationRecord; manifest counts/hashes +relationship_signal_distribution.Simulation —
layer1/contact_planner.py: a deterministic post-simulation pass (seed-derived RNG that never perturbs the event stream) that knows each account's realized lifespan, so planted conditions land relative to the account's actual "now". It generates contacts, attributes every conversation, enriches snapshots, and emits labels. Wired intoStateMachine.simulate().expected_fireis exactly what a correct detector must output.Prose — conversations attributed to the specific contact; name + role threaded into the prose prompt and the
ConversationRecord.Artifacts + CLI — new
contacts.jsonlandrelationship_labels.jsonl(hashed in the manifest);validate/inspect/statsextended.Validator —
validators/relationship.py: an account-level feature-extractor + rule-engine that independently recomputes windowed features from raw attribution and agrees with the planted ground truth (extends disagreement-ledger coverage).Determinism
rforge generate --dry-run --smokeis now bit-identical across independent processes (verified end-to-end). This surfaced a pre-existing determinism bug —_pick_off_brand_variant_idused the PYTHONHASHSEED-salted builtinhash(), makingplanted_quality.jsonldiffer across runs — fixed in a separate commit (36a4af2) with a subprocess regression test. The determinism property test now also coverscontacts_hashandrelationship_labels_hash.Audited distribution (60-account dry-run, seed 42)
See
docs/relationship-signal-distribution-audit.md. Per detector:champion_at_risk13 positive / 47 negative (7 decoy);single_threaded10 positive / 50 negative (10 decoy). Positives are the minority (~17–22%), the correct base rate for a churn signal, with measurable near-miss decoys for FP rate.Testing
tests/test_contact_modeling.py,tests/test_injector_offbrand_determinism.py); determinism test extended.🤖 Generated with Claude Code