[refactor] Register storage backend for greater scalability#103
Conversation
Signed-off-by: fy2462 <fy2462@gmail.com>
CLA Signature Guide@fy2462 , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
|
/check-cla |
CLA Signature Passfy2462, thanks for your pull request. All authors of the commits have signed the CLA. 👍 |
There was a problem hiding this comment.
Pull request overview
Refactors TransferQueue’s storage initialization to a registry-based backend mechanism, moving backend-specific initialization into dedicated modules under transfer_queue/storage/backends to improve scalability and extensibility.
Changes:
- Introduces
StorageBackendFactoryand registers backends via decorators. - Moves backend initialization code for SimpleStorage / MooncakeStore / Yuanrong into
transfer_queue/storage/backends/*. - Updates imports in interface/tests/scripts to point to the new backend module locations.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| transfer_queue/utils/yuanrong_utils.py | Removes Yuanrong backend init code from utils and keeps shared cleanup helpers. |
| transfer_queue/storage/backends/yuanrong_storage.py | Adds Yuanrong backend init implementation and registers it in the factory. |
| transfer_queue/storage/backends/simple_storage.py | Adds SimpleStorage backend init implementation and registers it in the factory. |
| transfer_queue/storage/backends/mooncake_storage.py | Adds MooncakeStore backend init implementation and registers it in the factory. |
| transfer_queue/storage/backends/base.py | Adds StorageBackendFactory registry/decorator. |
| transfer_queue/storage/backends/init.py | Imports backend modules for registration side effects and re-exports symbols. |
| transfer_queue/storage/init.py | Switches exports to source SimpleStorageUnit/StorageUnitData from storage.backends. |
| transfer_queue/interface.py | Switches storage creation logic to use the backend registry. |
| tests/test_simple_storage_unit.py | Updates imports to new SimpleStorage module path. |
| tests/test_metadata.py | Updates imports to new StorageUnitData module path. |
| scripts/put_benchmark.py | Updates import to new SimpleStorage module path. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: fy2462 <fy2462@gmail.com>
CLA Signature Passfy2462, thanks for your pull request. All authors of the commits have signed the CLA. 👍 |
Signed-off-by: fy2462 <fy2462@gmail.com>
CLA Signature Passfy2462, thanks for your pull request. All authors of the commits have signed the CLA. 👍 |
Signed-off-by: fy2462 <fy2462@gmail.com>
CLA Signature Passfy2462, thanks for your pull request. All authors of the commits have signed the CLA. 👍 |
Signed-off-by: fy2462 <fy2462@gmail.com>
CLA Signature Passfy2462, thanks for your pull request. All authors of the commits have signed the CLA. 👍 |
Description
As the issue(#102) I created, fix it with this PR.
Changes
boostrapsubfolder in storage and move the initialization code for different storage into the relevant files within this folder.StorageBootstrapProviderclass in boostrap/provider.py for bootstrap code registration.yuanrong_utils.pyand move it to theboostrapfolder.