Sigma GitOps

GitOps for Sigma Computing. Manage data models and workbooks as code, with version control, code review, drift checks, and automated deployment.

Overview

┌─────────────────┐     Pull      ┌─────────────────┐
│     Sigma       │ ────────────► │     GitHub      │
│   Computing     │               │   Repository    │
│                 │ ◄──────────── │                 │
└─────────────────┘  Push/Merge   └─────────────────┘
                          │
                          │ PR + Code Review
                          ▼
                    ┌───────────┐
                    │  CI/CD    │
                    │ Automated │
                    │   Sync    │
                    └───────────┘

Quick Start

1. Clone and Setup

git clone https://github.com/YOUR_ORG/sigma-gitops.git
cd sigma-gitops
pip install requests pyyaml

2. Configure Environment

export SIGMA_CLIENT_ID="your-client-id"
export SIGMA_SECRET="your-client-secret"
export SIGMA_CLOUD="aws"  # or azure, gcp

3. Pull Existing Data Models

python scripts/pull_from_sigma.py

4. Push Changes to Sigma

python scripts/sync_to_sigma.py data-models/my-model.json

Repository Structure

├── .github/
│   └── workflows/
│       ├── sync-to-sigma.yml    # Push data models + workbooks to Sigma
│       └── pull-from-sigma.yml  # Pull data models + workbooks from Sigma
├── data-models/
│   ├── sales-model.json         # Data model JSON specs
│   ├── inventory-model.json
│   └── ...
├── workbooks/                    # Workbooks as code (optional, opt-in)
│   ├── _template.json           # Starter workbook spec
│   └── ...
├── scripts/
│   ├── sync_to_sigma.py             # Push data models to Sigma
│   ├── pull_from_sigma.py           # Pull data models from Sigma
│   ├── sync_workbooks_to_sigma.py   # Push spec-created workbooks to Sigma
│   ├── pull_workbooks_from_sigma.py # Pull spec-created workbooks from Sigma
│   └── generate_diff_report.py      # PR diff reports
├── config.yml                    # ID mappings & settings
└── README.md

GitHub Actions Setup

Required Secrets

Add these in your repository settings (Settings → Secrets and variables → Actions):

Secret	Description
SIGMA_CLIENT_ID	Your Sigma API client ID
SIGMA_SECRET	Your Sigma API client secret

Optional Variables

Variable	Description	Default
SIGMA_CLOUD	Cloud provider (aws, azure, gcp)	aws

Workflow Triggers

The workflow runs automatically when:

• Push to main: Syncs changed data models to Sigma
• Pull Request: Validates JSON and posts a diff report comment
• Manual trigger: Use "Run workflow" in Actions tab

Workflow

Making Changes

1. Create a branch

   git checkout -b feature/add-revenue-metrics

2. Edit data model JSON

   # Edit existing model
   vim data-models/sales-model.json
   
   # Or create new model
   cp data-models/_template.json data-models/new-model.json

3. Commit and push

   git add data-models/
   git commit -m "Add revenue metrics to sales model"
   git push origin feature/add-revenue-metrics

4. Open PR for review

   • The workflow validates JSON syntax
   • A diff report is posted as a PR comment
   • Team can review changes before merge

5. Merge to deploy

   • On merge to main, changes are automatically synced to Sigma

Pulling Updates from Sigma UI

If changes are made directly in the Sigma UI:

# Pull all data models
python scripts/pull_from_sigma.py

# Pull specific model
python scripts/pull_from_sigma.py --name "Sales Model"

# Commit the updates
git add data-models/ config.yml
git commit -m "Sync changes from Sigma UI"
git push

Workbooks as code (limited private beta)

🧪 Workbooks as code is in a limited private beta. The workbook spec API doesn't cover everything yet. To get added to the access list, reach out to your CSM with your use case. Data models as code is generally available.

The same pull/sync mechanics work for workbooks. Sigma exposes a workbook spec via GET /v2/workbooks/{id}/spec and accepts one via POST /v2/workbooks/spec (create) and PUT /v2/workbooks/{id}/spec (update). It mirrors data models.

This is off by default and scoped deliberately:

• Folder selection is the trust boundary. Sigma exposes no "created via API vs UI" flag, so discovery is folder-scoped. pull_workbooks_from_sigma.py pulls every workbook whose folder path is within one of your workbook_folders. Point it only at folders you know are built through the spec. UI-only features (trellis, tooltip, and so on) don't round-trip through the spec API.
• A round-trip guard protects post-back. On update, sync_workbooks_to_sigma.py fetches the live spec and blocks the push if it would remove pages, elements, or top-level fields present in the live workbook (the signature of a spec-coverage gap or a stale local copy). Pure value edits pass. Override intentionally with ALLOW_WORKBOOK_REMOVALS=true.
• Managed workbooks are tracked in config.yml under workbooks:, mirroring data_models:. Duplicate workbook names are disambiguated with a short id suffix so no file silently overwrites another.

Enable it

# config.yml
manage_workbooks: true
workbook_folders:
  - Inventory Workbooks       # pull every workbook under these folder paths
  - My Documents/Test         # (prefix match, includes subfolders)

The flow

# 1. Pull every workbook spec in the configured folders into workbooks/.
python scripts/pull_workbooks_from_sigma.py
python scripts/pull_workbooks_from_sigma.py --folder "My Documents/Test"  # one-off folder
python scripts/pull_workbooks_from_sigma.py --id <workbookId>             # one workbook

# 2. Review/edit a workbook spec in git and push → sync-to-sigma PUTs the
#    update, after the round-trip guard confirms it removes nothing live.

# 3. The daily pull-from-sigma re-pulls the configured folders (+ refreshes
#    anything already tracked), so Sigma-side changes flow back to git.

# Push a single file locally:
python scripts/sync_workbooks_to_sigma.py workbooks/my-workbook.json

The pull-from-sigma workflow can also be dispatched manually with include_workbooks: true to force a one-off folder pull without flipping manage_workbooks in config.

Data Model JSON Structure

{
  "dataModelId": "uuid-assigned-by-sigma",
  "name": "Sales Model",
  "schemaVersion": 1,
  "pages": [
    {
      "id": "page-uuid",
      "name": "Main",
      "elements": [
        {
          "id": "element-uuid",
          "name": "Sales Table",
          "kind": "table",
          "source": {
            "connectionId": "connection-uuid",
            "kind": "warehouse-table",
            "path": ["DATABASE", "SCHEMA", "TABLE"]
          },
          "columns": [
            {
              "id": "col-uuid",
              "name": "Revenue",
              "formula": "[Sales/Amount]"
            }
          ]
        }
      ]
    }
  ]
}

Supported Elements

• Tables
• Calculated columns
• Metrics
• Relationships
• Custom SQL elements
• Controls (List, Checkbox, Date, Number, Text, Range)
• Filters (List, Top N, Number range, Date range, Text)

Local Development

Validate JSON

# Validate all files
for f in data-models/*.json; do python3 -m json.tool "$f" > /dev/null && echo "✓ $f"; done

Test Sync (Dry Run)

# See what would be synced without making changes
python scripts/sync_to_sigma.py --dry-run data-models/*.json

Force Re-sync

# Re-sync all data models (even if unchanged)
python scripts/sync_to_sigma.py --all

Troubleshooting

Authentication Errors

Authentication failed: unauthorized

• Verify SIGMA_CLIENT_ID and SIGMA_SECRET are correct
• Ensure the API credentials haven't expired
• Check that your account has "Create, edit, and publish datasets" permission

Data Model Not Found

Failed to update data model: not found

• The dataModelId in the JSON may be stale
• Remove the dataModelId field to create a new model
• Or update config.yml with the correct mapping

Schema Version Mismatch

Failed to update: schema version mismatch

• Pull the latest from Sigma: python scripts/pull_from_sigma.py --id <uuid>
• Merge your changes with the updated spec
• Push again

API Reference

This repo uses these Sigma API endpoints:

Endpoint	Method	Description
/v2/auth/token	POST	Get access token
/v2/datamodels	GET	List data models
/v3alpha/datamodels/{id}/spec	GET	Get JSON representation
/v3alpha/datamodels/{id}/spec	PUT	Update data model
/v3alpha/datamodels/spec	POST	Create data model

See Sigma API Docs for details.

License

MIT