Skip to content

feat(providers): add managed S3 backend protocol + minIO support#144

Open
bherila wants to merge 4 commits into
gotempsh:mainfrom
bherila:feature/managed-s3-upstream-pr
Open

feat(providers): add managed S3 backend protocol + minIO support#144
bherila wants to merge 4 commits into
gotempsh:mainfrom
bherila:feature/managed-s3-upstream-pr

Conversation

@bherila

@bherila bherila commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Motivation

  • Add a reviewable managed S3 backend protocol without mixing it into the MariaDB service PR.
  • Keep rustfs as the default managed object-storage backend while allowing S3-compatible backend selectors for garage and minio.
  • Avoid compiling AGPL storage-engine code into the Temps binary while still giving Temps a stable lifecycle contract for provisioning, credentials, health, backup/restore, and migration hooks.

Description

  • Adds crates/temps-providers/src/externalsvc/managed_s3.rs with ManagedS3BackendKind, ManagedS3BackendSelection, request/response DTOs, and the ManagedS3Backend trait.
  • Exports the managed-S3 protocol through temps-providers.
  • Adds a read-only backend selector to managed s3 parameters with out-of-box choices rustfs (default), garage, and minio.
  • Provisions s3 services with backend=minio through the existing MinIO-compatible service implementation while keeping the stored service type as s3.
  • Rejects malformed backend values and rejects garage at service creation until out-of-process provider dispatch is wired, avoiding silently provisioning RustFS under a Garage label.
  • Keeps absent/null backend values defaulting to rustfs, while rejecting present-but-non-string backend values instead of silently treating malformed input as the default.
  • Updates managed-service docs to describe object storage as S3-compatible rather than MinIO-specific.

Stack hygiene

  • This branch is based directly on main and is independent of the existing MariaDB PR (Add opt-in MariaDB-compatible services #138).
  • A separate integration branch in the fork combines MariaDB, flat public hostnames, managed S3, and other downstream release features only for Amazon Linux release builds.

Testing

  • cargo fmt --check
  • cargo test --lib -p temps-providers managed_s3
  • cargo check --lib -p temps-providers

Validated on tempsdev1 (Amazon Linux 2023, cargo 1.95.0).

bherila and others added 2 commits June 19, 2026 13:23
from_parameters used as_str() which silently fell back to the rustfs
default when "backend" was present but not a string (e.g. a number or
object). Reject present-but-non-string values explicitly while still
treating absent/null as the default, so malformed backend selections
surface an error instead of being reinterpreted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 966899b)
@bherila bherila force-pushed the feature/managed-s3-upstream-pr branch from 29aa2d6 to 65e5bac Compare June 19, 2026 20:23
… exists

Mirror of the fork-side fix: an S3/Blob service created with backend: garage
(plus a provider_socket) passed validation but was still provisioned as RustFS,
since create_service_instance maps both S3 and Blob to RustfsService and nothing
dispatches to the out-of-process provider. Reject Garage at creation time until
that path is wired up rather than silently misrepresenting what was provisioned.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bherila

bherila commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up fix — reject managed Garage backend until provider dispatch exists

dba375a0 mirrors a fix applied to the same code on the downstream fork: an S3/Blob service created with backend: "garage" (plus a provider_socket) passed validate_for_service_create but was still provisioned as RustFS, since create_service_instance maps both S3 and Blob to RustfsService and nothing dispatches to the out-of-process provider. That silently misrepresents what was provisioned. Garage is now rejected at creation time until the dispatch path is wired up; the provider_socket field stays as part of the contract.

Test results

Ran on a self-hosted runner (cargo 1.95.0, Amazon Linux 2023) against dba375a0:

$ cargo test --lib -p temps-providers managed_s3

running 6 tests
test externalsvc::managed_s3::tests::garage_without_socket_is_also_rejected ... ok
test externalsvc::managed_s3::tests::default_backend_is_rustfs ... ok
test externalsvc::managed_s3::tests::non_string_backend_is_rejected ... ok
test externalsvc::managed_s3::tests::garage_is_rejected_until_provider_dispatch_exists ... ok
test externalsvc::managed_s3::tests::null_backend_falls_back_to_default ... ok
test externalsvc::managed_s3::tests::parses_garage_backend_and_buckets ... ok

test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 234 filtered out; finished in 0.00s

@dviejokfs

Copy link
Copy Markdown
Contributor

Thanks for the PR! MinIO also needs to be supported even though is deprecated, could you add it?

@bherila

bherila commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

Yeah I can add MinIO. Separate PR for that or want it in here?

@bherila bherila changed the title feat(providers): add managed S3 backend protocol feat(providers): add managed S3 backend protocol + minIO support Jun 20, 2026
@bherila

bherila commented Jun 20, 2026

Copy link
Copy Markdown
Contributor Author

MinIO added here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants