Gap cloud-30 · missing-capability · confidence: suspected (may overlap with the admission/storage domain) · tracked in #242
Baseline: Midge 7d39f86 (0.1.1) vs Pants 2507c66. Midge paths are relative to the midge repo, Pants paths to this repo. Midge behavior is used as the proxy for the not-yet-extracted lsm-spec behavioral contract.
Prior: none
Midge: upload bodies carry a ResourceReservation that stays charged until the transport releases its body slices, and one admitted buffer is
shared across retries without being copied (executor.rs:55-70; tests should_keep_upload_memory_charged_until_transport_body_slices_are_released
:673, should_share_admitted_payload_across_retries_without_copying_or_releasing_it :694;
cloud/retained_tests.rs:36 should_keep_upload_charged_when_callback_adapter_times_out_before_backend_completion).
Range reads also take reservations (admitted.rs read_range_admitted) and a response limit equal to the range length (s3.rs:1178-1240).
Pants: S3ObjectStore.cs:382, GcsObjectStore.cs:304, and AzureBlobObjectStore.cs:252 copy the payload per request
(new ByteArrayContent(data.ToArray())). SST/WAL publication reads whole files (ProviderCloudPersistence.cs:625, 652;
ProviderCloudCompactionPublisher.cs:47, 88) and then fetches the full object back with GetAsync for verification (:118, :680). None of these
buffers is charged to Storage/Internal/ResourceBudget.cs. The payload stays referenced by the in-flight request after a deadline
cancellation, with no accounting.
Why it matters: one large compaction output can briefly hold about 3× its size unaccounted (the file, the copy, and the readback). This is
an unbounded-resource risk under memory budgets that Midge enforces.
Acceptance criteria:
- Upload payloads are reserved against the shared budget before submission and released only after the HTTP content is disposed, including after a deadline or cancellation.
- No per-attempt copy of the payload is made (use
ReadOnlyMemoryContent or a shared content).
- Readback verification is either bounded (ranged or streamed hash) or charged.
- A test shows the budget stays charged while the request is in flight after the caller's deadline fires, and is released after the transport completes.
Prior: none
Midge: upload bodies carry a
ResourceReservationthat stays charged until the transport releases its body slices, and one admitted buffer isshared across retries without being copied (
executor.rs:55-70; testsshould_keep_upload_memory_charged_until_transport_body_slices_are_released:673,
should_share_admitted_payload_across_retries_without_copying_or_releasing_it:694;cloud/retained_tests.rs:36 should_keep_upload_charged_when_callback_adapter_times_out_before_backend_completion).Range reads also take reservations (
admitted.rs read_range_admitted) and a response limit equal to the range length (s3.rs:1178-1240).Pants:
S3ObjectStore.cs:382,GcsObjectStore.cs:304, andAzureBlobObjectStore.cs:252copy the payload per request(
new ByteArrayContent(data.ToArray())). SST/WAL publication reads whole files (ProviderCloudPersistence.cs:625, 652;ProviderCloudCompactionPublisher.cs:47, 88) and then fetches the full object back withGetAsyncfor verification (:118, :680). None of thesebuffers is charged to
Storage/Internal/ResourceBudget.cs. The payload stays referenced by the in-flight request after a deadlinecancellation, with no accounting.
Why it matters: one large compaction output can briefly hold about 3× its size unaccounted (the file, the copy, and the readback). This is
an unbounded-resource risk under memory budgets that Midge enforces.
Acceptance criteria:
ReadOnlyMemoryContentor a shared content).