Skip to content

Example PR: GitHub Action Behaviour - validate semguard PR comment and annotations#2

Open
yeaight7 wants to merge 5 commits into
mainfrom
test/semguard-pr-comment
Open

Example PR: GitHub Action Behaviour - validate semguard PR comment and annotations#2
yeaight7 wants to merge 5 commits into
mainfrom
test/semguard-pr-comment

Conversation

@yeaight7
Copy link
Copy Markdown
Owner

@yeaight7 yeaight7 commented Apr 26, 2026

Purpose

This is a deliberately artificial PR created to demonstrate how dbt-semguard behaves when a pull request introduces semantic changes in a dbt Semantic Layer definition.

The goal is not to merge this PR. The goal is to leave it open as a public example showing:

  • GitHub Action execution
  • Generated dbt-semguard report
  • Structured action outputs
  • When source diagnostics are available, the inline check-run annotations

This PR can be used by potential users to understand what the tool looks like in practice before installing it in their own dbt projects.


What this PR changes

This PR modifies the example semantic model fixture at:

tests/fixtures/projects/base/models/orders.yml

The changes intentionally alter the semantic contract exposed by the fixture.


Breaking changes introduced

1. gross_revenue aggregation changed

- agg: sum
+ agg: avg

This changes the meaning of gross_revenue from total revenue to average order revenue. Breaking change.


2. country dimension removed

- name: country
- dimension:
-   type: categorical

This removes a dimension that downstream BI users could previously use to slice or group the metric. Breaking change.


3. Ratio metric denominator changed

- denominator: order_count
+ denominator: gross_revenue

This changes the definition of the aov ratio metric.

Previously, aov meant:

gross_revenue / order_count

After this PR, it means:

gross_revenue / gross_revenue

That completely changes the business meaning of the metric and should be blocked by default.


4. ordered_at time granularity changed

- granularity: day
+ granularity: week

This changes the time grain of the ordered_at dimension from daily to weekly. This causes loss of information. Breaking change.
Changing from week to day would be risky instead of breaking (no loss of information).


Risky changes introduced (click to expand subsections)

5. New `sales_rep` entity added

- name: sales_rep_id
  entity:
    name: sales_rep
    type: foreign

This adds a new foreign entity to the semantic model.

Adding an entity is not necessarily breaking, but it changes the available semantic join surface and should be visible during review.


6. New `sales_channel` dimension added

- name: sales_channel
  dimension:
    type: categorical

This adds a new categorical dimension to the semantic model.

This is useful as an example of a non-blocking semantic expansion: it changes the available analytical surface, but does not remove or redefine an existing field.


7. `gross_revenue` filter changed

- filter: order_status = 'completed'
+ filter: order_status in ('completed', 'refunded')

This changes which rows are included in the gross_revenue metric.

The metric now includes refunded orders, which may be intentional, but it changes the population behind the metric and should be reviewed explicitly.


8. New `refunded_orders` metric added

- name: refunded_orders
  type: simple
  agg: count
  expr: 1
  filter: order_status = 'refunded'

This adds a new metric to the semantic layer.

Adding a metric is generally not breaking, but it is still a semantic contract change and should appear in the review report.


Expected dbt-semguard behavior

The dbt-semguard GitHub Action should detect both breaking and risky semantic changes.

Expected result:

highest-severity = breaking
blocking = true
breaking-count > 0
risky-count > 0

Because the workflow is configured with:

fail-on: breaking

the action itself is expected to return a failing status for the semantic guard step.

For demo purposes, the workflow may use continue-on-error: true so that a later assertion step can print and verify the action outputs clearly.


Why this matters

Normal dbt CI can tell you whether a project parses or builds.

It does not necessarily tell you whether a PR changed the business meaning of a metric.

This PR demonstrates the class of issue dbt-semguard is designed to catch:

dbt still accepts the project, but the semantic contract exposed to downstream users has changed.

Examples of downstream impact include:

  • dashboards silently showing different KPI values,
  • notebooks returning unexpected metric definitions,
  • reverse ETL jobs syncing changed business logic,
  • API consumers depending on removed dimensions,
  • analysts grouping metrics by a changed time grain,
  • and reviewers missing a semantic change because the YAML diff looks small.

What to look at in this PR

Reviewers should inspect:

  1. The changed orders.yml file.
  2. The dbt-semguard GitHub Action run.
  3. The generated PR comment.
  4. The action outputs:
    • highest-severity
    • blocking
    • breaking-count
    • risky-count
    • safe-count
  5. Any inline annotations created through the GitHub Check Runs API.

The important part is not the fixture itself. The important part is the review experience around the semantic diff.


Merge status

This PR is intentionally not meant to be merged.

It should remain open as a live demonstration of how dbt-semguard behaves on a realistic semantic change review.

If this PR is eventually closed, it should be closed without merging.

@yeaight7 yeaight7 self-assigned this Apr 26, 2026
@github-actions
Copy link
Copy Markdown

dbt-semguard report

Breaking changes

  • Ratio metric aov changed denominator from order_count to gross_revenue. (models/orders.yml:45)
  • Simple metric gross_revenue changed aggregation from sum to avg. (models/orders.yml:28)
  • Dimension country in semantic model orders was removed. (models/orders.yml:21)
  • Dimension ordered_at in semantic model orders changed granularity from day to week. (models/orders.yml:22)

Risky changes

  • Metric gross_revenue changed filter from order_status='completed' to order_status in ('completed', 'refunded'). (models/orders.yml:28)
  • Metric refunded_orders was added. (models/orders.yml:38)
  • Dimension sales_channel in semantic model orders was added. (models/orders.yml:25)
  • Entity sales_rep in semantic model orders was added. (models/orders.yml:17)

Status: blocking

Repository owner locked and limited conversation to collaborators Apr 26, 2026
@yeaight7 yeaight7 changed the title Example PR: test - validate semguard PR comment and annotations Example PR: GitHub Action Behaviour - validate semguard PR comment and annotations Apr 27, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant