Skip to content

Show real shard count for custom sharding in collections list - #440

Merged
KShivendu merged 7 commits into
masterfrom
fix/custom-sharding-shard-count
Sep 2, 2026
Merged

Show real shard count for custom sharding in collections list#440
KShivendu merged 7 commits into
masterfrom
fix/custom-sharding-shard-count

Conversation

@KShivendu

@KShivendu KShivendu commented Sep 2, 2026

Copy link
Copy Markdown
Member

Why

The Shards column showed config.params.shard_number, which for sharding_method: "custom" is not the total shard count — it is only the default number of shards given to each new shard key. So every custom-sharded collection reported the default (usually 1) no matter how many shards it really had.

Demo collections on a distributed v1.16.3 cluster: b_tenants_default has 3 shard keys × 1 shard, c_tenants_uneven has 3 shard keys created with shards_number 2 / 3 / 1.

Before — 3 shard keys read as 1, and 2 / 3 / 1 shards read as 2:

After — the real totals, with the shard key count as context:

Tooltip on a custom-sharded row:

Thanks to @nleroy917 for discovering this

Key decisions

  • Total comes from GET /collections/{name}/clustershard_count, never computed as shard_keys × shard_number. PUT /collections/{name}/shards takes a per-key shards_number, so keys of 2 / 3 / 1 shards total 6 while a 2 / 2 / 1 split totals 5 — both with shard_number: 2 and 3 keys.
  • Cluster info is fetched only for sharding_method: "custom". Auto-sharded collections keep the existing single request per row and render unchanged.
  • Number of shard keys is shown as a caption, since "6 shards" alone is hard to interpret for a custom-sharded collection.

Worth knowing

  • Latency: one extra request per custom-sharded row (~196 bytes, same server cost as getCollection). It has to follow getCollection because sharding_method is only known from its response, so such a row costs 2 sequential round trips instead of 1; rows are still fetched in parallel, so a page adds ~1 round trip, not one per row. Auto-sharded collections are untouched.
  • Skipped when the client is isRestricted, and any failure falls back to shard_number rendered as N / key, so a per-key default is never mistaken for a total.
  • A custom-sharded collection with no shard keys yet now correctly shows 0 (it really has no shards) instead of 1.
  • The search path is not paginated, so a query matching many custom-sharded collections issues one extra request per match — it already did one getCollection per match.

🤖 Generated with Claude Code

KShivendu and others added 2 commits September 2, 2026 16:22
The Shards column rendered `config.params.shard_number`, which for
`sharding_method: custom` is only the *default* number of shards per
shard key, not the total: each shard key can be created with its own
`shards_number`, and a collection with no shard keys yet has no shards
at all. A custom-sharded collection therefore showed "1" regardless of
how many shards it actually had.

Read the total from the collection cluster info (`shard_count`) for
custom-sharded collections instead, and show the number of shard keys
alongside it. Cluster info is only requested for custom sharding, and
falls back to the `shard_number` default (marked as such) when it is
unavailable, e.g. for restricted API keys.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@KShivendu
KShivendu force-pushed the fix/custom-sharding-shard-count branch from 573ea6d to 74f77a0 Compare September 2, 2026 10:56
@KShivendu
KShivendu requested review from generall and trean September 2, 2026 10:58
Replace the nested ternaries with early returns per case, drop the
redundant "default" caption (the `N / key` value and the tooltip
already say it), and only attach `shard_count` when it parses as a
finite number, so the cell needs a single null check.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@KShivendu

Copy link
Copy Markdown
Member Author

CI is failing due to unrelated stuff (dependency audit)

@qdrant-cloud-bot

Copy link
Copy Markdown
Contributor

CI failure here is unrelated to the shard-count changes — the ESLint workflow's Audit step fails on a newly disclosed high-severity browserslist advisory (<=4.28.6).

Fix PR: #441 (browserslist 4.28.2 → 4.28.8 in package-lock.json). After that merges, rebasing this branch onto master should turn CI green.

@trean trean left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@KShivendu
KShivendu merged commit b11d754 into master Sep 2, 2026
1 check passed
@KShivendu
KShivendu deleted the fix/custom-sharding-shard-count branch September 2, 2026 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants