Skip to content

cloud: harden and prove SST range identity pinning and extend provider qualification coverage #346

Description

@smiggleworth

Gap cloud-32 · adaptation-decision (pinning) + missing-coverage (qualification) · confidence: verified · tracked in #242
Baseline: Midge 7d39f86 (0.1.1) vs Pants 2507c66. Midge paths are relative to the midge repo, Pants paths to this repo. Midge behavior is used as the proxy for the not-yet-extracted lsm-spec behavioral contract.

Prior: #222 (closed-but-incomplete for coverage: an unsafe OCI endpoint override is never tested through open), #7 (related)

Midge: conditional SST range reads send If-Match or x-goog-if-generation-match. The response must be 206, the Content-Range must match exactly
bytes s-e/<expected total size>, and a version mismatch is PreconditionFailed (storage/cloud/range.rs:6-48; s3.rs:1178-1240;
test should_reject_wrong_identity_or_range_when_body_has_valid_length). OCI endpoint validation at the backend boundary is
covered through OpenOptions::build (398add2; tests/cloud_provider_validation.rs should_reject_unsafe_oci_endpoint_given_open_options_when_endpoint_is_overridden). Opt-in real-S3 qualification uses
MIDGE_REAL_S3_* (providers/qualification.rs:20-25, :56; tests/cloud_provider_engine_qualification.rs:496). A partitioned-compaction
round-trip and a partial-upload rollback run over Sqrzl (cloud_provider_engine_qualification.rs:146, :165).

Pants: range reads send no precondition. Identity is checked afterwards by comparing the returned ETag/generation
(ProviderCloudSstSourceFactory.cs:94-98), and a mismatch raises PantsCorruptionException, not a precondition/conflict error. The
Content-Range check covers start and end but not the total size (CloudHttpResponseReader.cs:70-80). OCI endpoint validation runs at open
(PantsOpenOptions.cs:368-373BuiltInCloudProviderValidator.cs:193, 392-414), but
PantsCloudValidationTests.ShouldValidateFirstClassOciConfigurationAndRedactCredentials tests only a null endpoint. Qualification
(SqrzlCloudProviderQualificationTests, CloudProviderEngineQualificationTests) has no real-provider opt-in and no partitioned-compaction round-trip.

Why it matters: detection is equivalent today, but the error class differs (corruption vs precondition), and requests that are not pinned
spend bandwidth on replaced objects. The coverage items guard the fix from Midge 398add2 and the real-provider behavior.

Acceptance criteria:

  • Decide whether to send If-Match or generation-match on SST range reads and classify version changes as precondition failures. Record the decision in lsm-spec behavior/.
  • Reject a Content-Range whose total size differs from the manifest or HEAD size.
  • Test that PantsOpenOptions rejects OCI overrides with the ftp scheme, userinfo, a query, or a fragment, and that the error message contains no secret.
  • Add an opt-in real S3 qualification (environment-gated, skipped by default) and a partitioned-compaction round-trip over a Sqrzl provider protocol.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:cloudReal cloud provider backendsarea:read-pathBlock cache and bloom filter performanceparity-gapFeature not yet at parity with Midgepriority:P3Deferred: blocked or intentionally postponed until an explicit prerequisite is metstatus:needs-decisionDifference may be a legitimate adaptation; needs an explicit maintainer/spec decision

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions