Skip to content

Fix: Unique key error#123

Open
sophiascaglioni-whoop wants to merge 10 commits intomainfrom
fix-unique-key-error
Open

Fix: Unique key error#123
sophiascaglioni-whoop wants to merge 10 commits intomainfrom
fix-unique-key-error

Conversation

@sophiascaglioni-whoop
Copy link
Copy Markdown
Contributor

Description

Aligns SST relationship validation with Snowflake semantic view relationship rules: referenced (right) join columns must match the model’s primary key or declared meta.sst.unique_keys, with clearer multiline errors and direction hints. Fixes the case where catalog-declared unique keys were ignored and valid joins were rejected.

Related Issue

Closes #119

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Performance improvement
  • Code refactoring
  • Test improvements
  • Other (please describe):

Changes Made

  • validator.py: Surface unique_keys from dbt catalog (sm_tables) so relationship validation can treat them like Snowflake’s unique-key option for the referenced side.
  • references.py:
    • Right-side join columns must match the full PK set or the full unique_keys set (set equality, order-independent).
    • Reject self-references (same logical left and right table).
    • Reject circular relationship graphs (directed edges left → right), including cycles longer than two hops.
    • Richer error text (newlines + relationship direction reminder: RIGHT = referenced, LEFT = referencing).
  • Tests (test_relationship_validation.py): unique_keys pass-through, self-ref, 2-node cycle, 3-node cycle, acyclic fan-out, composite PK column order, composite unique_keys when PK differs, wrong unique subset still errors; existing assertions updated for new messaging where needed.

Testing

  • Unit tests pass (pytest tests/unit/)
  • All existing tests pass
  • New tests added for new functionality
  • Tested locally with Python 3.9-3.11
  • Manual testing completed (if applicable)
  • Test coverage maintained or improved (target: >90%)

Test Results

Unit tests (repo)

pytest tests/unit/ -v 
=========== 1197 passed in 2.11s ===========

pytest tests/unit/core/validation/ -q
=========== 254 passed in 0.83s ===========

pytest tests/unit/core/validation/test_relationship_validation.py -v
=========== 28 passed in 0.50s ===========

Manual (within an external dbt project)

Scenario A — will PASS after this PR (join on unique_keys, not PK, on the referenced/right table)
Catalog / model table1 (right / referenced):

config:
  meta:
    sst:
      primary_key: [response_id]
      unique_keys: [user_id, response_date]

Model table2 (referencing) with composite PK, relationship right_table = table1, left_table = table2, conditions equating user_id / calendar_dateuser_id / response_date (so the join is on the unique rather than the primary key in the right table)

Before fix (representative sst validate output — message implied UNIQUE could be valid, but validation still failed; unique_keys in catalog was ignored):

12:04:01  Validation completed in 0.6s [ERROR]

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
VALIDATION SUMMARY
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Status: FAILED - Fix errors before deployment
Errors: 1
Warnings: 0

1 Error(s) - Must be fixed:

  📁 snowflake_semantic_models/relationships/memberships.yml
     1. [ERROR] Relationship 'table1_to_table2' references column(s) [user_id, response_date] in right table 'table1', but the primary key is 'response_id'. Snowflake documentation states relationships MUST reference PRIMARY KEY or UNIQUE columns. If 'user_id' has a UNIQUE constraint, this is valid. Otherwise, consider reversing the relationship direction.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Next steps:
  1. Fix errors listed above
  2. Run 'sst validate' again
  3. When clean, run 'sst extract' to deploy

Error: Validation failed with errors

After fix (same project / same YAML — join columns match meta.sst.unique_keys on the right-hand model):

12:04:02  Validation completed in 0.6s [OK]

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
VALIDATION SUMMARY
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Status: PASSED - No issues found
Errors: 0
Warnings: 0
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Ready for deployment:
  sst extract                    # Uses database/schema from dbt profile
  sst extract --target prod      # Use a specific profile target

Scenario B — should still FAIL, with a clearer error (right table PK not used; no unique_keys covering the join columns)

Right table PK e.g. cycle_id; relationship joins on columns like user_id and cycle_start_date_app_view without a matching unique_keys declaration.

Example dbt models (generic table1 = left, table2 = right for this failure case)

table1 — referencing / LEFT (composite PK matches the join grain you use on this side):

version: 2
models:
  - name: table1
    description: Referencing side (e.g. daily status grain).
    config:
      meta:
        sst:
          primary_key: [user_id, calendar_date]

table2 — referenced / RIGHT: PK is not the join columns, and there is no meta.sst.unique_keys that matches the join:

version: 2
models:
  - name: table2
    description: Referenced side — surrogate PK only; join targets non-key columns.
    config:
      meta:
        sst:
          primary_key: cycle_id

Relationship (invalid): left_table = table1, right_table = table2, conditions equating table1.user_id / table1.calendar_date to table2.user_id / table2.cycle_start_date_app_view. Validation should still fail with a clear error (sample output below).

Representative output (table2 on the right, PK cycle_id, join on user_id + cycle_start_date_app_view; SST shows logical table names in uppercase):

12:04:05  Validation completed in 0.5s [ERROR]

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
VALIDATION SUMMARY
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Status: FAILED - Fix errors before deployment
Errors: 1
Warnings: 0

1 Error(s) - Must be fixed:

  📁 snowflake_semantic_models/relationships/memberships.yml
     1. [ERROR] Relationship 'table1_to_table2' references column(s) [user_id, cycle_start_date_app_view] in right table 'TABLE2', but the primary key is 'cycle_id'.

Snowflake requires the referenced column(s) in the RIGHT table to be the primary or unique key in 'TABLE2'.

To fix:
  - Swap left/right so the keyed table is on the right, or
  - Add meta.sst.unique_keys if that column or column set is unique on 'TABLE2'.

Relationship direction reminder:
  - RIGHT = referenced table: join column(s) must match that model's meta.sst.primary_key or meta.sst.unique_keys (remember: these keys must be unique in THIS table).
  - LEFT = referencing table: many left rows can match one right row.

  If the uniquely keyed table is on the left, swap left_table and right_table.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Next steps:
  1. Fix errors listed above
  2. Run 'sst validate' again
  3. When clean, run 'sst extract' to deploy

Error: Validation failed with errors

Checklist

Code Quality

  • Code follows the project's style guidelines (Black, line length 120)
  • Imports sorted with isort (black profile)
  • Type hints added for new code (mypy snowflake_semantic_tools/ passes)
  • Docstrings added for public functions/classes
  • No linting errors
  • Pre-commit hooks pass (if using pre-commit)

Documentation & Compatibility

  • Documentation updated (if needed)
  • Backward compatibility maintained (if applicable)
  • Breaking changes discussed with maintainer first (see CONTRIBUTING.md)

Performance

  • Performance impact considered
  • No significant performance regressions

Screenshots / Examples

See Test Results — unit commands plus manual sst validate scenarios and expected exit codes / messages.

Additional Notes

Not at this time.

@greptile-apps
Copy link
Copy Markdown

greptile-apps bot commented Mar 20, 2026

Last reviewed commit: "black / isort / type..."

Comment thread snowflake_semantic_tools/core/validation/rules/references.py Outdated
Comment thread snowflake_semantic_tools/core/validation/rules/references.py Outdated
sophiascaglioni-whoop and others added 3 commits March 20, 2026 13:39
…ule as single-column PK (key column ∈ right join columns), make sure nested circular relationships are all surfaced at once. new tests added "

"
…rcuit, remove redundant import

- Allow unique_keys to validate joins even when primary_key is not declared
  (previously fell through to "no primary key metadata" error unconditionally)
- Add `continue` after self-reference error to avoid redundant downstream errors
- Remove redundant `import re` inside has_sql_transformation (already at module level)

.... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)

Co-Authored-By: Cortex Code <noreply@snowflake.com>
…kflows

Adds project-level CoCo configuration:
- .cortex/skills/sst-validate: SST validation workflow
- .cortex/skills/sst-test: Unit test workflow
- .cortex/skills/sst-pr-review: PR review checklist
- .cortex/agents/sst-validator.md: E2E validation agent
- AGENTS.md: Project context for AI assistants

.... Generated with [Cortex Code](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)

Co-Authored-By: Cortex Code <noreply@snowflake.com>
@sophiascaglioni-whoop
Copy link
Copy Markdown
Contributor Author

I'm on mobile so I can't confirm but wanted to flag in case you plan to merge -- I think the CoCo skills you added in the last commit (great idea) won't work on other ppl’s machines since they’re hard coded file paths to your cd

@mluizzi-whoop

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Unique Key Error Msg Inconsistency

2 participants