Grouped diagnostics and reusable value clustering

[devnull03]

Purpose

Group repeated diagnostics by stable producer keys while preserving every exact location. Move value-variant matching from spellcheck into a reusable clustering crate.

Scope

• Add explicit diagnostic group metadata and named validator findings.
• Use the existing dataset, column, row, and cell location scopes.
• Show repeated findings as one expandable Problems row with an occurrence count.
• Keep navigation and one-cell fixes on expanded occurrences.
• Group repeated spelling, capitalization, and value-variant findings.
• Extract value normalization, blocking, and similarity logic into crates/clustering.
• Credit OpenRefine as a workflow and research influence.

Safety

Grouping changes presentation only. Atomic diagnostics remain the source of truth, and similarity never changes data automatically.

Follow-up

The full cluster-review workspace is tracked in #125.

Plan

See PLAN.md for contracts, crate boundaries, implementation order, tests, and the OpenRefine license boundary.

Status

Planning complete. Implementation has not started for the grouped diagnostic UI.

Notion: The source task still needs to be created and cross-linked.

[Copilot]

🟡 Changes recommended

There are compile-blocking uses of str::starts_with(char::is_uppercase) in crates/spellcheck/src/lib.rs that must be fixed before this can be safely merged.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR implements grouped diagnostics in the Problems panel using explicit producer-provided grouping metadata (while retaining atomic diagnostics as the source of truth), and extracts value-variant matching into a new reusable clustering crate, wired into per-column settings and fix providers.

Changes:

• Add diagnostic grouping metadata (DiagnosticGroup) and richer column-validator output (ColumnFinding + ColumnValues) to support stable grouping and subdelimited-cell iteration without per-cell allocations.
• Implement grouped rendering + expansion in ProblemsPanel (including multi-select source filtering) while preserving navigation and one-cell fixes on expanded occurrences.
• Introduce crates/clustering (value-variant pairing + fixes) and add a per-column variant_review setting with UI/config/docs updates.

File summaries

File	Description
sample/column_config.csv	Adds Variant Review column to sample column config for enabling value-variant review per column.
PLAN.md	Adds a detailed implementation plan/spec for grouped diagnostics and clustering extraction (now marked as complete).
docs/diagnostics.md	Documents spelling/capitalization/value-variant checks, grouping behavior, scopes, counts, and OpenRefine credit.
docs/columns.md	Documents per-column spellcheck behavior and new value-variant review option.
crates/table/src/panel.rs	Adds a GPUI test validating variant-fix behavior (revalidate + undo semantics) and updates diagnostics literals for new group field.
crates/table/src/note.rs	Adds column context-menu toggle for “Review value variants” and updates diagnostics literals for new group field.
crates/table/src/file_links.rs	Adapts file-link findings pipeline to new ColumnFinding addressing.
crates/table/src/agent.rs	Ensures staged agent findings set group: None (bridge findings remain atomic/ungrouped).
crates/table/Cargo.toml	Adds clustering as a dev-dependency for table tests.
crates/spellcheck/src/lib.rs	Refactors spellcheck to load all installed dictionaries, adds capitalization validator + fixes, emits per-token findings with grouping metadata, and supports typographic apostrophes.
crates/spellcheck/src/catalogue.rs	Updates dictionary round-trip test to account for multi-dictionary storage.
crates/settings/src/columns.rs	Adds persisted per-column variant_review setting with defaults/tests.
crates/project-wizard/src/steps/review.rs	Applies variant_review from column config during project creation.
crates/project-wizard/src/data.rs	Extends column config parsing to accept Variant Review column.
crates/plugin-host/src/plugin.rs	Adapts plugin validators to ColumnValues input and ColumnFinding output.
crates/plugin-host/src/lib.rs	Passes column subdelimiter into ColumnValues for plugin validation runs.
crates/diagnostics/src/validator.rs	Introduces ColumnValues, CellValue, ColumnFinding, and begin_run; addresses findings with correct scope + group metadata and passes subdelimiter into snapshots.
crates/diagnostics/src/panel.rs	Implements grouped Problems panel projection + expansion state and replaces single-source dropdown with multi-select source filter.
crates/diagnostics/src/lib.rs	Adds Location::scope + constructors, DiagnosticGroup, Diagnostic.group, and friendly source labels.
crates/diagnostics/src/fixes.rs	Updates tests for new Diagnostic.group field.
crates/clustering/src/lib.rs	Adds ValueVariants column validator and fix provider gated by per-column variant_review, emitting grouped findings and stale-safe fixes.
crates/clustering/src/core.rs	Implements normalized tokenization, candidate blocking, pair evidence, and stable pair keys without transitive clustering.
crates/clustering/Cargo.toml	Defines new clustering crate dependencies (diagnostics/settings/gpui + caseless/unicode-normalization/strsim).
crates/checks/src/dates.rs	Updates date validator to validate per logical value via ColumnValues and return ColumnFinding.
crates/checks/src/authority/mod.rs	Adapts authority publish path to feed ColumnFinding into diagnostics::address.
crates/app/src/main.rs	Registers capitalization validator + fixes and registers the value-variant checker (validator + fixes) at app startup.
crates/app/src/app_settings/mod.rs	Updates spelling language setting text and adds Settings UI for selecting variant-reviewed columns.
crates/app/src/app_settings/config.rs	Adds Variant Review to the column config export schema and tests.
crates/app/Cargo.toml	Adds clustering dependency for app integration.
CLAUDE.md	Updates crate map description to include clustering and refreshes checks description.
Cargo.toml	Adds crates/clustering to the workspace members.
Cargo.lock	Locks new dependencies (clustering, caseless, strsim, unicode-normalization).
AGENTS.md	Clarifies that bridge findings remain ungrouped atomic diagnostics regardless of built-in grouping UI.

Review details

Suppressed comments (2)

crates/spellcheck/src/lib.rs:470

• str::starts_with does not accept char::is_uppercase; this will not compile. Use a first-character uppercase check instead.

    !ignore_capitalized || !token.starts_with(char::is_uppercase)

crates/spellcheck/src/lib.rs:504

• str::starts_with does not accept a predicate; word.starts_with(char::is_uppercase) will not compile. Use chars().next() to detect initial uppercase for the proper-noun rule.

        None if ignore_capitalized && word.starts_with(char::is_uppercase) => {
            WordOutcome::ProperNoun
        }

• Files reviewed: 32/33 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.