Skip to content

cloud: align provider read-retry policy with Midge (backoff, attempt budget, status set, mid-body disconnects) #341

Description

@smiggleworth

Gap cloud-27 · behavior-difference · confidence: verified (the no-retry-on-truncated-range part and the backoff); suspected (raw HttpIOException escaping) · tracked in #242
Baseline: Midge 7d39f86 (0.1.1) vs Pants 2507c66. Midge paths are relative to the midge repo, Pants paths to this repo. Midge behavior is used as the proxy for the not-yet-extracted lsm-spec behavioral contract.

Prior: none. #62 (related: transport indeterminacy) and #127 (related: flush backoff) are both closed.

Midge: src/storage/cloud/executor.rs:12-13, 441-521. There are 3 retries (4 attempts) with exponential backoff of
50ms × 2^n capped at 2^4. Transient statuses are exactly 408/425/429/500/502/503/504. Transport, body, and decode errors on reads count as
transient and are retried, including a partial body disconnect (should_retry_read_given_partial_response_body_disconnect, :1151).
Each attempt's timeout is clamped to the remaining deadline (:488-501).

Pants: S3ObjectStore.cs:325-369, AzureBlobObjectStore.cs:~340-396, GcsObjectStore.cs:~230-284. There are 3 attempts
total (MaximumAttempts = 3) with no backoff: only await Task.Yield() between attempts (S3:367, Azure:394, GCS:282).
IsRetryable treats 408, 429, and any status >= 500 as retryable (S3:617, Azure:398, GCS:858), so 425 is not retried while 501 and 505 are.
Only HttpRequestException is retried. For ranged GETs the body is read in CloudHttpResponseReader.BufferRangeAsync
(Protocol/CloudHttpResponseReader.cs:94-110), which converts only EndOfStreamException to PantsIOException. That exception
is not caught by the retry loop, so a truncated or disconnected range body fails the read on the first attempt
(CloudProviderRangeContractTests.ShouldRejectTruncatedRangeBodiesAndDisposeThem asserts failure with no retry). A mid-body
socket reset throws HttpIOException (an IOException, not EndOfStream), which likely escapes unwrapped and unretried (suspected).

Why it matters: cold SST point reads and scans issue many small ranged GETs (disk-resident reads). A single transient body reset
fails the user read, and with 429 throttling the three immediate retries are spent in microseconds, which makes the throttling worse.

Acceptance criteria:

  • Reads retry transient statuses (408/425/429/500/502/503/504) with bounded exponential backoff inside the operation deadline. Each backoff sleep is also bounded by that deadline.
  • Decide whether other 5xx statuses (501/505/…) are permanent and document the decision.
  • A read whose headers are valid but whose body is truncated or reset (full or ranged) is retried, then succeeds on a clean second response. The test asserts two requests were made and the correct bytes returned.
  • An HttpIOException from body reads never escapes unwrapped. It surfaces as PantsIOException or PantsTimeoutException.
  • Mutations remain non-retried (the existing ShouldNotRetryConditional* tests stay green).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:cloudReal cloud provider backendsarea:correctnessTransaction/CF/config behavioral hardeningparity-gapFeature not yet at parity with Midgepriority:P2Normal: bounded hardening, coverage, maintainability, or non-critical behavior gap

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions