Skip to content

perf: improve share creation by using SQL aggregates in server limit checks - #349

Open
chuan137 wants to merge 1 commit into
stable/2025.1-m3from
share-server-lock-perf
Open

chuan137 wants to merge 1 commit into
stable/2025.1-m3from
share-server-lock-perf

Conversation

@chuan137

@chuan137 chuan137 commented Aug 11, 2026

Copy link
Copy Markdown
Member

When creating a share, the scheduler evaluates all available share
servers to find one within configured per-server limits. Previously this
fetched full ORM objects for every share instance and snapshot on each
server, loading large result sets into Python just to compute a count
and a size sum. On busy backends with many shares per server this adds
significant latency to every share creation request.

Replace those fetches with two SQL aggregate queries (COUNT + SUM) per
share server, returning only the scalar values needed.

  • share_instance_count_and_size_sum_by_server: returns (count, size_sum)
    in a single query joining ShareInstanceShare
  • share_snapshot_size_sum_by_server: returns total snapshot size using
    COALESCE(instance_size, snapshot.size), mirroring the ORM size
    property; share-group snapshot members (snapshot_id=NULL) are excluded
    as their size cannot be resolved
  • Missing ShareInstance.deleted filter added to the snapshot query to
    exclude snapshots belonging to soft-deleted instances

Change-Id: Idcd378c654c2fb7fbf867e9eb62991cd2cbcb017

@chuan137
chuan137 force-pushed the share-server-lock-perf branch from 78ec6e7 to 9753d82 Compare August 11, 2026 13:24
@chuan137 chuan137 changed the title wip: sql aggregates for _check_share_server_backend_limits perf: use SQL aggregates in _check_share_server_backend_limits Aug 11, 2026
@chuan137
chuan137 force-pushed the share-server-lock-perf branch 2 times, most recently from 45b0262 to f4f881c Compare August 11, 2026 13:47
@chuan137

Copy link
Copy Markdown
Member Author

@chuan137 chuan137 changed the title perf: use SQL aggregates in _check_share_server_backend_limits perf: improve share creation by using SQL aggregates in server limit checks Aug 11, 2026
…checks

When creating a share, the scheduler evaluates all available share
servers to find one within configured per-server limits. Previously this
fetched full ORM objects for every share instance and snapshot on each
server, loading large result sets into Python just to compute a count
and a size sum. On busy backends with many shares per server this adds
significant latency to every share creation request.

Replace those fetches with two SQL aggregate queries (COUNT + SUM) per
share server, returning only the scalar values needed.

- share_instance_count_and_size_sum_by_server: returns (count, size_sum)
  in a single query joining ShareInstance -> Share
- share_snapshot_size_sum_by_server: returns total snapshot size using
  COALESCE(instance_size, snapshot.size), mirroring the ORM size
  property; share-group snapshot members (snapshot_id=NULL) are excluded
  as their size cannot be resolved
- Missing ShareInstance.deleted filter added to the snapshot query to
  exclude snapshots belonging to soft-deleted instances

Change-Id: Idcd378c654c2fb7fbf867e9eb62991cd2cbcb017
@chuan137
chuan137 force-pushed the share-server-lock-perf branch from f4f881c to c3bd319 Compare August 11, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant