51 lines (40 loc) · 2.07 KB

Event Contract — `llm.signals.v1`

AdaptLoop's spine is the signal event contract. Every producer and consumer speaks this schema. Topic name and version are frozen for v1.

Topic

Property	Value
Topic	`llm.signals.v1`
Key	`tenant_id` (UTF-8)
Value	JSON (`LLMSignalEvent`)
Idempotency	`event_id` (consumer dedup)

Schema

Field	Required	Notes
`event_id`	yes	UUID or deterministic id; unique per logical signal
`tenant_id`	yes	Multi-tenant partition key
`trace_id`	no	Gateway / OTel correlation
`signal_type`	yes	`trace` \| `feedback` \| `eval`
`model`	yes	Model id as seen by the gateway or demo
`prompt_hash`	yes	SHA-256 of normalized prompt, or `gw:<digest>` for metadata-only gateway events
`prompt_text`	no	Gated. Synthetic producer only in v1. Never invent prompts from gateway metadata.
`response_text`	no	Gated with `prompt_text`
`label`	no	`0.0`/`1.0` or continuous score
`latency_ms`	no	End-to-end latency
`cost_usd`	no	Request cost
`domain`	yes	Fixed `epc_qa` in v1
`source`	yes	`gateway` \| `synthetic`
`needs_review`	no	Low-confidence / poisoning flag (default `false`)
`occurred_at`	yes	UTC timestamp

PII policy (v1)

Gateway bridge emits metadata only (prompt_text / response_text always null).
Synthetic EPC producer may include demo Q&A text (non-PII by construction).
Production prompt capture requires an explicit later change (LLMOps side-channel).

Signal sources

Gateway bridge — polls LLMOps requests (+ token_usage) → source=gateway, signal_type=trace.
Synthetic EPC producer — seed corpus of EPC terminology Q&A → source=synthetic, may include text + labels.

Consumer expectations

Validate with Pydantic; quarantine/drop invalid payloads (do not crash the dataflow).
Dedup on event_id (Redis SET NX).
Append to training_examples idempotently on event_id.
Update Redis rollups keyed by tenant_id.