Skip to content

feat(remote-storage): add more follower target connectors through connector adapters #543

Description

@AptS-1547

Context

#461 migrates follower-side remote storage targets from the fixed Local/S3 contract to a connector-owned registry, versioned config envelopes, separately encrypted credentials, generic capability negotiation, and descriptor-driven frontend behavior.

The initial migration intentionally registers only:

  • asterdrive.remote-target.local
  • asterdrive.remote-target.s3

Ordinary storage policies already provide additional connectors such as SFTP, Azure Blob, Tencent COS, Alibaba OSS, Qiniu Kodo, and OneDrive. Those implementations are useful sources of reusable schemas, validation, driver construction, and request behavior, but they are not directly interchangeable with follower targets because remote targets have separate persistence, credential, reconciliation, capability, and runtime-context lifecycles.

Goal

Add more follower remote storage target connectors through narrow adapters that reuse existing storage connector and driver mechanisms where the ownership contracts actually match.

A provider should become available to the remote-target UI by registering one remote-target connector. The frontend must remain generic and descriptor-driven.

Direction

  • define the reusable boundary between an ordinary storage connector and a remote-target connector adapter
  • keep remote-target applicability, config conversion, credential conversion, persisted validation, and runtime construction connector-owned
  • reuse existing typed provider config, validation helpers, SDK driver construction, error mapping, and real request tests where their contracts match
  • keep target persistence, encrypted credential rows, revision/reconciliation, default-target rules, follower runtime context, and capability negotiation in the remote-target subsystem
  • do not carry ordinary policy behavior, OAuth actions, saved-policy actions, upload-strategy controls, multipart policy settings, or storage-native processing fields into remote targets unless a target-specific product requirement proves they apply
  • preserve unknown or temporarily unavailable connector payloads without constructing them at runtime
  • keep direct, reverse-tunnel, and auto transport behavior connector-neutral
  • do not add provider switches or capability matrices to the frontend

Candidate order

  1. SFTP
  2. Tencent COS, Alibaba OSS, Qiniu Kodo, and Azure Blob
  3. OneDrive, after separately defining delegated OAuth credential ownership and follower-side token refresh

The exact order may change based on provider applicability and test-environment readiness.

Acceptance criteria

  • adding a supported provider requires a remote-target connector registration, not a core enum or frontend branch
  • descriptor fields, defaults, required rules, help text, and credential scopes round-trip through create and update
  • saved secrets are retained on no-op edits and replaced or removed explicitly on connector changes
  • persisted config and credential connector/schema mismatches fail deterministically
  • target reconciliation builds the expected provider driver and preserves revision semantics
  • direct, reverse-tunnel, and auto paths negotiate the connector ID through remote_storage_target.connector_ids
  • unavailable connector data is retained and presented predictably
  • ordinary requests and presigned behavior are tested separately where the provider supports presigning
  • provider-specific real request tests or credential-gated smoke tests cover the reused driver contract
  • frontend tests prove descriptors appear without provider-specific conditionals

Non-goals

  • folding follower targets into ordinary storage policies
  • sharing ordinary policy behavior or action lifecycles by default
  • restoring Local/S3 enums, flattened target fields, or V5 protocol compatibility
  • implementing every ordinary storage provider in one change

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    EnhancementNew feature or requestPriority: MediumMedium priority issueRustPull requests that update Rust codeScope: Remote NodesPrimary/follower nodes, remote storage targets, and internal routingScope: StorageStorage policies, connectors, drivers, provider capabilities, and storage backendsStatus: DeferredTriaged outside the active release train; revisit before moving into a release milestoneTypeScriptPull requests that update JavaScript code

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions