Feature Description
Implement native distributed locking mechanism for hudi in Azure ADLS using conditional writes.
Storage based locking is available in hudi for S3, GCS without using external systems like zookeeper. Implement similar functionality for Azure ADLS
User Experience
How users will use this feature:
- Configuration changes needed
yes. Additional configs for adls
- API changes
None.
- Usage examples
Hudi RFC Requirements
(https://github.com/apache/hudi/blob/master/rfc/rfc-91/rfc-91.md)
- Does this change public interfaces/APIs?
- No. Leverages existing APIs and extends it for azure
- Does this change storage format?
- No
- Justification:
RFC already exists and implemented for S3/GCS.
Feature Description
Implement native distributed locking mechanism for hudi in Azure ADLS using conditional writes.
Storage based locking is available in hudi for S3, GCS without using external systems like zookeeper. Implement similar functionality for Azure ADLS
User Experience
How users will use this feature:
yes. Additional configs for adls
None.
Hudi RFC Requirements
(https://github.com/apache/hudi/blob/master/rfc/rfc-91/rfc-91.md)
RFC already exists and implemented for S3/GCS.