Skip to content

Implement StorageLockClient for ADLS Gen2 for StorageBasedLockProvider #17708

@chrevanthreddy

Description

@chrevanthreddy

Feature Description

Implement native distributed locking mechanism for hudi in Azure ADLS using conditional writes.
Storage based locking is available in hudi for S3, GCS without using external systems like zookeeper. Implement similar functionality for Azure ADLS

User Experience

How users will use this feature:

  • Configuration changes needed
    yes. Additional configs for adls
  • API changes
    None.
  • Usage examples

Hudi RFC Requirements

(https://github.com/apache/hudi/blob/master/rfc/rfc-91/rfc-91.md)

  • Does this change public interfaces/APIs?
  • No. Leverages existing APIs and extends it for azure
  • Does this change storage format?
  • No
  • Justification:
    RFC already exists and implemented for S3/GCS.

Metadata

Metadata

Labels

type:featureNew features and enhancements

Type

No type
No fields configured for issues without a type.

Projects

Status

Open

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions