Skip to content

[refactor] Register storage backend for greater scalability#103

Merged
0oshowero0 merged 5 commits into
Ascend:mainfrom
fy2462:main
May 18, 2026
Merged

[refactor] Register storage backend for greater scalability#103
0oshowero0 merged 5 commits into
Ascend:mainfrom
fy2462:main

Conversation

@fy2462
Copy link
Copy Markdown
Contributor

@fy2462 fy2462 commented May 17, 2026

Description

As the issue(#102) I created, fix it with this PR.

Changes

  1. Create a boostrap subfolder in storage and move the initialization code for different storage into the relevant files within this folder.
  2. Add the StorageBootstrapProvider class in boostrap/provider.py for bootstrap code registration.
  3. Split the storeage instantiation code in yuanrong_utils.py and move it to the boostrap folder.

Signed-off-by: fy2462 <fy2462@gmail.com>
@ascend-robot
Copy link
Copy Markdown

CLA Signature Guide

@fy2462 , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit Reason
[7892ac0 [refactor] Register storage bac...](7892ac0) the email used in the commit is not linked to a signed CLA!
please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

@fy2462
Copy link
Copy Markdown
Contributor Author

fy2462 commented May 17, 2026

/check-cla

@ascend-robot
Copy link
Copy Markdown

CLA Signature Pass

fy2462, thanks for your pull request. All authors of the commits have signed the CLA. 👍

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors TransferQueue’s storage initialization to a registry-based backend mechanism, moving backend-specific initialization into dedicated modules under transfer_queue/storage/backends to improve scalability and extensibility.

Changes:

  • Introduces StorageBackendFactory and registers backends via decorators.
  • Moves backend initialization code for SimpleStorage / MooncakeStore / Yuanrong into transfer_queue/storage/backends/*.
  • Updates imports in interface/tests/scripts to point to the new backend module locations.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
transfer_queue/utils/yuanrong_utils.py Removes Yuanrong backend init code from utils and keeps shared cleanup helpers.
transfer_queue/storage/backends/yuanrong_storage.py Adds Yuanrong backend init implementation and registers it in the factory.
transfer_queue/storage/backends/simple_storage.py Adds SimpleStorage backend init implementation and registers it in the factory.
transfer_queue/storage/backends/mooncake_storage.py Adds MooncakeStore backend init implementation and registers it in the factory.
transfer_queue/storage/backends/base.py Adds StorageBackendFactory registry/decorator.
transfer_queue/storage/backends/init.py Imports backend modules for registration side effects and re-exports symbols.
transfer_queue/storage/init.py Switches exports to source SimpleStorageUnit/StorageUnitData from storage.backends.
transfer_queue/interface.py Switches storage creation logic to use the backend registry.
tests/test_simple_storage_unit.py Updates imports to new SimpleStorage module path.
tests/test_metadata.py Updates imports to new StorageUnitData module path.
scripts/put_benchmark.py Updates import to new SimpleStorage module path.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread transfer_queue/interface.py Outdated
Comment thread transfer_queue/interface.py Outdated
Comment thread transfer_queue/storage/backends/mooncake_storage.py Outdated
Comment thread transfer_queue/storage/bootstrap/mooncake_bootstrap.py
Comment thread transfer_queue/storage/__init__.py Outdated
Comment thread scripts/put_benchmark.py Outdated
Comment thread transfer_queue/storage/backends/base.py Outdated
Signed-off-by: fy2462 <fy2462@gmail.com>
@ascend-robot
Copy link
Copy Markdown

CLA Signature Pass

fy2462, thanks for your pull request. All authors of the commits have signed the CLA. 👍

Comment thread transfer_queue/storage/bootstrap/simple_storage_bootstrap.py
Signed-off-by: fy2462 <fy2462@gmail.com>
@ascend-robot
Copy link
Copy Markdown

CLA Signature Pass

fy2462, thanks for your pull request. All authors of the commits have signed the CLA. 👍

Signed-off-by: fy2462 <fy2462@gmail.com>
@ascend-robot
Copy link
Copy Markdown

CLA Signature Pass

fy2462, thanks for your pull request. All authors of the commits have signed the CLA. 👍

Signed-off-by: fy2462 <fy2462@gmail.com>
@ascend-robot
Copy link
Copy Markdown

CLA Signature Pass

fy2462, thanks for your pull request. All authors of the commits have signed the CLA. 👍

Copy link
Copy Markdown
Collaborator

@0oshowero0 0oshowero0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!

@0oshowero0 0oshowero0 merged commit 94baa19 into Ascend:main May 18, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants