v1.x — deferred collections interfaces (uploads, versions, tags, bucket config, annotations, fsspec) #12

Deferred from v1 (ADR-0009). Each of these is a genuinely natural collections.abc fit; they wait because v1 scope is core + large-object I/O, and because over-scoping is the top risk (zarr's v3 rewrite replaced a MutableMapping store interface with a 15-method async ABC and killed five of its own backends in the process).

Roughly in the order they should arrive:

1. In-flight multipart uploads as a Mapping. ListMultipartUploads + AbortMultipartUpload is literally a mapping whose __delitem__ aborts: for k in uploads: del uploads[k] replaces the loop AWS's own docs tell you to write. Orphaned uploads are the classic silent S3 bill. Highest portability of this set, and client.exceptions.NoSuchUpload actually exists (one of only 13 modelled S3 error shapes), so it maps cleanly to KeyError. Key: (key, upload_id).
2. Object versions (+ a delete-marker view). Key: (key, version_id) or version_id.
3. Object tags (MutableMapping[str, str], ≤10 entries) and user metadata (x-amz-meta-). Both natural, both portability 2–4/5, so each needs capability detection. Note user-metadata writes are a CopyObject — surprising enough to document loudly.
4. Bucket configuration as one Mapping: cfg['lifecycle'] = {...} over the Get/Put/Delete triples (lifecycle, cors, policy, encryption, tagging, versioning, …). One spec-driven store rather than 20 methods.
5. Object annotations. Natively a per-object MutableMapping[str, bytes] — the first S3 feature that is a nested store rather than a flat blob store, which for a dol-shaped library is a gift: store.annotations['some/key']['embedding'] = .... Deferred on two hard facts, not design: no S3-compatible provider implements it (portability 0/5), and it needs a botocore floor too fresh for a storage library. Feature-detect with hasattr(client, 'put_object_annotation').
6. fsspec adapter — to_fsspec() / from_fsspec(). One adapter buys pandas, dask, pyarrow, and zarr-v3-via-FsspecStore. The mandatory surface is small: _strip_protocol + ls(detail=True) + _open/_fetch_range.
7. s3dol[fast] — obstore for the object level only (it has no bucket operations), if an independent re-measurement justifies it.
8. s3dol.aio — async mirroring the sync surface.

Explicitly never

• GetObjectTorrent — dead.
• SelectObjectContent — AWS closed it to new customers in July 2024 and points at Athena / Object Lambda. A Mapping over a service new users cannot enable is negative value.
• Native async Store ABC / zarr-v3 store. An s3dol store is already a working zarr v2 store for free. zarr v3 left MutableMapping behind deliberately, for async and byte-range coalescing; chasing it means becoming a different package. Adapt via fsspec instead.

Not ours at all → botodol

The line is mechanical — the endpoint test: can it be reached through the same boto3.client('s3', endpoint_url=...) a MinIO or R2 user already holds? If not, it isn't s3dol's. s3control is a separate service model with a required AccountId that no S3-compatible provider implements; same for s3tables, s3vectors, s3outposts, glacier. That test puts access points, batch jobs, Storage Lens, Access Grants and MRAP in botodol, and it never needs arguing.

Two cases the test resolves correctly rather than by exception: access points are consumed as bucket names (so s3dol accepts an access-point ARN wherever it accepts a bucket name — one line in the normalizer; administering them is botodol), and S3 Express / directory buckets are one row in the preset registry, not a new abstraction.

One obligation this creates: the spec-driven sub-resource store (item 4) and the paginated-list store are boto-generic, not S3-specific — the same code serves DynamoDB tables, SQS queues, Secrets Manager secrets. When built, they go in a documented public module and botodol depends on s3dol for them. Don't build a shared base package speculatively; don't let botodol reimplement them.