Skip to content

roadmap: deterministic incident evidence → runtime adapters #1

Description

@duhenri9

Canonical roadmap

This issue replaces the temporary Debug Evidence checkpoint that lived in wm3-request-protocol before this standalone repository existed.

Product thesis

Diagnose before you mutate: collect runtime evidence, preserve contradictory signals and produce a reproducible debug bundle before proposing code changes.

V0 — deterministic incident bundle

Build one offline incident fixture with one real root cause and plausible decoys.

Required mechanisms:

  • structured ingestion for logs, stack trace, environment metadata and recent-change evidence;
  • secret/environment redaction before persistence;
  • bounded hypothesis model;
  • evidence links classified as support, contradiction or unresolved;
  • allowlisted safe probes only;
  • SUPPORTED | CONTRADICTED | INDETERMINATE semantics;
  • deterministic input/bundle digests;
  • no source mutation in V0.

Required controls:

  1. misleading-log control — loudest error is not root cause;
  2. missing-evidence control — return INDETERMINATE rather than certainty;
  3. unsafe-probe control — policy blocks a proposed destructive/unapproved probe.

V0.2 — real local runtime inputs

  • Python and Node log/trace parsers;
  • local Git diff/recent-commit correlation;
  • runtime/environment fingerprint with allowlisted values only;
  • safe filesystem/process metadata adapters;
  • reproducible incident archive.

V0.3 — observability adapters

Only after the local evidence contract is stable:

  • OpenTelemetry traces;
  • structured JSON logs;
  • Sentry-compatible import where practical;
  • container/Kubernetes read-only diagnostic adapters.

V0.4 — optional model reasoning

  • provider-neutral hypothesis adapter;
  • redaction before model calls;
  • model suggestions remain untrusted hypotheses;
  • deterministic evidence engine owns result semantics;
  • prompt/model/version identity recorded.

V0.5 — remediation proposals

Proposal only first. Mutation requires a separately designed authority contract and must not be smuggled into the diagnosis layer.

Public-quality gate

  • one-command offline demo;
  • positive + all three negative controls;
  • tests/static checks;
  • pinned/minimal-permission CI;
  • evidence artifact uploaded by CI;
  • privacy/redaction threat model;
  • architecture/ADR docs;
  • no paid model key required;
  • benchmark bundle-generation latency/resource use before performance claims.

Non-goals

  • autonomous code repair in V0;
  • arbitrary shell agent;
  • replacing an observability platform;
  • presenting LLM explanation as evidence;
  • ingesting private/commercial project data as public fixtures;
  • production incident auto-remediation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions