Skip to content

cloud: emit per-HTTP-attempt provider cost observations (method, range, bytes, status, transport error, cancelled, elapsed) #345

Description

@smiggleworth

Gap cloud-31 · missing-capability · confidence: verified · tracked in #242
Baseline: Midge 7d39f86 (0.1.1) vs Pants 2507c66. Midge paths are relative to the midge repo, Pants paths to this repo. Midge behavior is used as the proxy for the not-yet-extracted lsm-spec behavioral contract.

Prior: none (the brief's telemetry domain may also claim this)

Midge: src/storage/cloud/executor/observation.rs emits one midge::cloud_io event per attempt with method, whether it was ranged,
request/response body bytes, status, transport_error, cancelled, and elapsed_ns. It never includes URLs, keys, or credentials. Added in 4fe106c
(#286) with executor/tests/observation.rs. The same commit reduced recovery HEADs by reusing one budgeted SST reader (streaming_recovery/coverage.rs).

Pants: the provider stores emit no metrics, logs, or activities (grep for ActivitySource, Meter, ILogger, or EventSource under
src/Pants.Core/Cloud finds nothing). Runtime metrics cover WAL uploads only (Observability/Internal/RuntimeMetricsSnapshotFactory.cs:147-155).
ProviderCloudSstSourceFactory.cs:15 issues a HEAD for every SST open, and Pants has no way to measure HEAD versus ranged-GET recovery cost.

Why it matters: without per-attempt cost data, Pants cannot qualify recovery request counts, cold-read cost, or retry amplification the way Midge's evidence documents do.

Acceptance criteria:

  • Each provider HTTP attempt, including retries and cancelled attempts, records method, ranged flag, request/response bytes, status, transport failure, cancellation, and elapsed time, with no URL, key, or secret.
  • A test counts the HEAD and ranged-GET attempts made during a cache-loss recovery and asserts an upper bound.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:cloudReal cloud provider backendsarea:telemetryMetrics and telemetry pipelineparity-gapFeature not yet at parity with Midgepriority:P3Deferred: blocked or intentionally postponed until an explicit prerequisite is met

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions