Skip to content

ci: update SDK compliance harness - #121

Merged
marandaneto merged 2 commits into
mainfrom
ci/update-sdk-compliance-harness
Jul 29, 2026
Merged

ci: update SDK compliance harness#121
marandaneto merged 2 commits into
mainfrom
ci/update-sdk-compliance-harness

Conversation

@marandaneto

@marandaneto marandaneto commented Jul 29, 2026

Copy link
Copy Markdown
Member

💡 Motivation and Context

The SDK compliance workflow is pinned to an older reusable workflow whose actions target Node.js 20, producing deprecation warnings on GitHub Actions. The CI and local Docker Compose compliance harness images are also one release behind and do not include the latest /flags retry contract coverage.

This updates the reusable workflow and aligns both CI and local compliance harness pins while keeping the compliance setup otherwise unchanged.

💚 How did you test it?

  • Parsed .github/workflows/sdk-compliance.yml successfully as YAML.
  • Ran docker compose -f sdk_compliance_adapter/docker-compose.yml config --quiet.
  • Ran git diff --check.
  • Verified the pinned reusable workflow uses the Node.js 24-compatible action versions.
  • Verified the ghcr.io/posthog/sdk-test-harness:0.10.0 image manifest exists.
  • CI SDK compliance tests passed 46/46 on the initial commit.

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran pnpm changeset to generate a changeset file

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

A maintainer directed Pi coding agent (GPT-5.6) to audit PR #120's CI logs and update the stale SDK compliance pins. The agent used shell, Git, GitHub CLI, and Docker tooling; after review feedback, the local Docker Compose harness pin was aligned with CI.

@marandaneto marandaneto self-assigned this Jul 29, 2026
@marandaneto
marandaneto marked this pull request as ready for review July 29, 2026 07:26
@marandaneto
marandaneto requested a review from a team as a code owner July 29, 2026 07:26
@marandaneto
marandaneto enabled auto-merge (squash) July 29, 2026 07:26
Comment thread .github/workflows/sdk-compliance.yml
@greptile-apps

greptile-apps Bot commented Jul 29, 2026

Copy link
Copy Markdown
Prompt To Fix All With AI
### Issue 1
.github/workflows/sdk-compliance.yml:21
**Local harness pin remains stale**

CI now runs compliance harness 0.10.0, while `sdk_compliance_adapter/docker-compose.yml` still runs 0.9.0 locally. Local compliance runs therefore skip the new contract coverage and can pass despite a different CI result.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "ci: update SDK compliance harness" | Re-trigger Greptile

@github-actions

github-actions Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

posthog-unity Compliance Report

Date: 2026-07-29 07:33:32 UTC
Duration: 105889ms

✅ All Tests Passed!

46/46 tests passed


Capture Tests

29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 84ms
Format Validation.Event Has Uuid 11ms
Format Validation.Event Has Lib Properties 8ms
Format Validation.Distinct Id Is String 7ms
Format Validation.Token Is Present 8ms
Format Validation.Custom Properties Preserved 13ms
Format Validation.Event Has Timestamp 8ms
Retry Behavior.Retries On 503 6520ms
Retry Behavior.Does Not Retry On 400 2015ms
Retry Behavior.Does Not Retry On 401 2013ms
Retry Behavior.Respects Retry After Header 8017ms
Retry Behavior.Implements Backoff 18525ms
Retry Behavior.Retries On 500 5515ms
Retry Behavior.Retries On 502 5518ms
Retry Behavior.Retries On 504 5516ms
Retry Behavior.Max Retries Respected 18525ms
Deduplication.Generates Unique Uuids 14ms
Deduplication.Preserves Uuid On Retry 5515ms
Deduplication.Preserves Uuid And Timestamp On Retry 11519ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 5520ms
Deduplication.No Duplicate Events In Batch 14ms
Deduplication.Different Events Have Different Uuids 8ms
Compression.Sends Gzip When Enabled 9ms
Batch Format.Uses Proper Batch Structure 7ms
Batch Format.Flush With No Events Sends Nothing 4ms
Batch Format.Multiple Events Batched Together 12ms
Error Handling.Does Not Retry On 403 2008ms
Error Handling.Does Not Retry On 413 2011ms
Error Handling.Retries On 408 5516ms

Feature_Flags Tests

17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 21ms
Request Payload.Flags Request Uses V2 Query Param 10ms
Request Payload.Flags Request Hits Flags Path Not Decide 6ms
Request Payload.Flags Request Omits Authorization Header 5ms
Request Payload.Token In Flags Body Matches Init 6ms
Request Payload.Groups Round Trip 5ms
Request Payload.Groups Default To Empty Object 6ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 5ms
Request Payload.Disable Geoip Omitted Defaults To False 5ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 5ms
Request Lifecycle.No Flags Request On Init Alone 3ms
Request Lifecycle.No Flags Request On Normal Capture 7ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 8ms
Request Lifecycle.Mock Response Value Is Returned To Caller 7ms
Retry Behavior.Retries Flags On 502 508ms
Retry Behavior.Retries Flags On 504 509ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 9ms

@marandaneto
marandaneto disabled auto-merge July 29, 2026 07:29
@marandaneto
marandaneto enabled auto-merge (squash) July 29, 2026 07:33
@marandaneto
marandaneto merged commit e3c1ebc into main Jul 29, 2026
16 checks passed
@marandaneto
marandaneto deleted the ci/update-sdk-compliance-harness branch July 29, 2026 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants