Skip to content

Consolidate Entry Machine retry policy into catalog classes (needs review) #919

Closed as not planned

Description

@nick-potts

Independent of #917 and the fault-script experiment (#918). Needs its own review before any work starts; not ready-for-agent.

What

The CLI reaches an Entry Machine through five client primitives (call, call_repeatable, call_unretried, read, invoke), a wall-clock setup-retry loop, and per-call timeout helpers. Whether an RPC may be re-sent after a lost reply is decided at each call site; nine ops are called through more than one primitive. Resume guidance for an interrupted founder enrol is duplicated five times, one per step.

Proposed: one retry policy in one place; each catalog row declares its class (OneShot — never re-sent after a lost reply — or Repeatable) and, where needed, a per-attempt budget; a failed attempt is a Fault (Unreached / Lost / Refused) classified by which phase of the attempt failed; a stopped command returns Stopped { done, at, fault } and the handler renders guidance once. setup_retry and the five primitives are deleted.

Why this is not a bug fix

Reviewed: the inconsistency is not demonstrably incorrect today. ployz dns reserve retries ReserveDomain while the founder tail does not, but the daemon guards a second reservation with AlreadyReserved, so a retry cannot double-reserve — the worst case is ployz dns reserve printing an error after a reservation that succeeded. Start/Stop/Remove are idempotent daemon-side, so both call styles are safe.

The motivation is consistency and deletion (~600 lines, estimate), plus the one UX case above. Review should decide whether that is worth a production change now.

Proposed classes (for review, not decided)

OneShot: Initialize, Join, Reset, CreateContainer, CreateVolume, PrepareVolumes, ReserveDomain. Everything else Repeatable, including Register (daemon recognises replay by public key) and the container lifecycle rows.

Budget overrides: StopContainer = base + grace; GetContainerObservations = requested wait + base; InspectStorage = 3s. Default: none locally, 10s when targeted.

Open questions for the reviewer

  1. Is the dns reserve UX case enough to justify the change now, or does this wait for a second concrete case?
  2. PrepareVolumes: OneShot provisionally; the volume plugin documents create as idempotent. Settle by a daemon-side check, not by argument.
  3. Should Fault classification by attempt phase (connect vs send) be verified against tonic's actual behaviour with a round-trip test before anything depends on it?

Glossary

Fault is already defined in CONTEXT.md (uncommitted): "Why one attempt at a Machine produced no result: never sent, sent with no reply, or refused by the target." The "sent with no reply" case splits into effect-applied and effect-not-applied; the client cannot tell them apart, which is why OneShot rows are driven to completion by observe-then-act, never by a second send.

Activity

  1. nick-potts commented on Sep 29, 2026

    @nick-potts
    ContributorAuthor

    Not planned: its main user-facing case (dns reserve) was deleted in #1061. Reopen if a per-call timeout bug comes back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions