Skip to content

fix(mi355): harden exact-SHA evidence worker - #12

Merged
jhinpan merged 2 commits into
mainfrom
fix/mi355-worker-hardening
Jul 23, 2026
Merged

fix(mi355): harden exact-SHA evidence worker#12
jhinpan merged 2 commits into
mainfrom
fix/mi355-worker-hardening

Conversation

@jhinpan

@jhinpan jhinpan commented Jul 23, 2026

Copy link
Copy Markdown
Owner

Summary

  • fail closed when the fetched or current PR head differs from the maintainer-approved SHA
  • publish content-addressed evidence without overwriting an existing bundle
  • run validation as the service identity with only GPU device groups, disable runtime pulls, and retain the executable tmpfs required for probes

Test plan

  • python3 tests/test_evolution.py
  • repository validation, retrieval/answer evaluations, provenance, corpus, indices, and shell syntax checks
  • digest-pinned MI355X sandbox harness: 26 pass, 0 fail, 1 source-recorded
  • live container inspection: no network, credentials, Docker socket, capabilities, or writable controller/candidate mounts

Made with Cursor

Summary by Sourcery

Harden the MI355 evidence worker by enforcing exact SHA approvals, making evidence bundles content-addressed and immutable, and tightening sandbox isolation and identity usage.

New Features:

  • Enforce exact SHA matching between the approved PR head and both the current and fetched commit before running or publishing validation.
  • Publish MI355 evidence bundles to content-addressed, SHA-identified directories and record corresponding artifact URIs.

Enhancements:

  • Prevent overwriting existing MI355 evidence bundles by treating them as immutable once created.
  • Run MI355 validation sandboxes as the calling service user with only required GPU device groups, disabled runtime pulls, and an executable tmpfs for probes.
  • Clarify MI355 README to document the stricter SHA checks, sandbox isolation model, content-addressed evidence bundles, and required GitHub App permissions.

Tests:

  • Add tests to verify SHA mismatch rejection for fetched PR heads, immutability and content addressing of evidence bundles, and sandbox configuration for isolation and caller identity.

Fail closed on fetch races and missing pinned images while preserving immutable, content-addressed evidence.

Co-authored-by: Cursor <cursoragent@cursor.com>
@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@sourcery-ai

sourcery-ai Bot commented Jul 23, 2026

Copy link
Copy Markdown

Reviewer's Guide

Hardens the MI355 evidence worker by enforcing exact approved SHAs throughout the workflow, making the compact evidence bundle content‑addressed and immutable, tightening sandbox execution to use the caller identity with least-privilege GPU access and no runtime pulls, and extending tests/docs to cover the new guarantees.

Sequence diagram for MI355 PR processing with exact SHA enforcement

sequenceDiagram
    participant Worker as mi355_worker
    participant GitHub as GitHubAPI
    participant Repo as GitRepo

    Worker->>GitHub: _authorize_pull(repo, pr_number, required_label)
    GitHub-->>Worker: pull, head_sha, approval

    Worker->>Repo: _run git fetch origin +pull/pr_number/head:fetch_ref
    Worker->>Repo: _run git rev-parse fetch_ref^{commit}
    Repo-->>Worker: fetched_sha
    Worker->>Worker: require_exact_sha(fetched_sha, head_sha, source=current PR head)

    Worker->>GitHub: _authorize_pull(repo, pr_number, required_label, expected_head_sha=head_sha)
    GitHub-->>Worker: pull, head_sha (unchanged), approval

    Worker->>Repo: _run git worktree add --detach candidate head_sha
    Worker->>Worker: _run_validation(candidate, gpu, run_output)

    Worker->>GitHub: _authorize_pull(repo, pr_number, required_label, expected_head_sha=head_sha)
    GitHub-->>Worker: approval

    Worker->>Worker: _copy_compact_bundle(run_output, evidence_root, pr_number, head_sha, controller_sha, approval, artifact_uri)
    Worker-->>GitHub: _publish_check(repo, pr_number, head_sha, evidence)
Loading

Flow diagram for content-addressed immutable evidence bundle creation

flowchart TD
    A[collect provenance: pr, head_sha, controller_sha, approval] --> B[create staging directory in evidence_root]
    B --> C[copy manifest.json, verdicts.json, summary.txt to staging]
    C --> D[copy health-before.txt, health-after.txt if present]
    D --> E[_bundle_digest staging and provenance]
    E --> F[build destination path: pr-head_sha_sha256-digest]
    F --> G{destination exists?}
    G -->|yes| H[raise ValueError immutable evidence bundle already exists]
    G -->|no| I[resolve artifact_uri to include destination.name]
    I --> J[write EVIDENCE.yaml in staging]
    J --> K[rename staging to destination]
    K --> L[return destination, evidence]
Loading

File-Level Changes

Change Details Files
Enforce exact maintainer-approved SHA for PR processing and refetches to fail closed on races or mismatches.
  • Introduce a reusable require_exact_sha helper that raises if an observed SHA differs from the approved SHA.
  • Add an _authorize_pull helper that validates the required label, optionally re-validates the current head SHA, and locates a valid approval comment.
  • Refactor process_pr to use _authorize_pull initially, after the git fetch, after creating the candidate worktree, and after validation to ensure the head SHA and approval are still valid.
  • Force-update the local fetch ref for the PR head and verify the fetched commit SHA exactly matches the previously observed head SHA before creating the worktree.
scripts/evolve/mi355_worker.py
ops/mi355/README.md
Make MI355 compact evidence bundles content-addressed, immutable, and safer to produce.
  • Extend _bundle_digest to incorporate a JSON-serialized provenance object into the aggregate SHA, binding content to provenance.
  • Change _copy_compact_bundle to stage files in a temporary directory, compute the content/provenance digest, and name the final bundle directory with the PR/head prefix plus the SHA-256 digest.
  • Prevent overwriting an existing evidence bundle directory by raising if the computed destination already exists, and only then atomically rename the staged directory into place.
  • Move artifact URI resolution into _copy_compact_bundle so the stored artifact_uri embeds the content-addressed bundle directory name.
scripts/evolve/mi355_worker.py
ops/mi355/README.md
tests/test_evolution.py
Tighten MI355 sandbox security and isolation while running containers as the calling service identity with GPU-specific groups only.
  • Update run-sandbox.sh and run-command-sandbox.sh to compute caller UID/GID and run containers as that user instead of the runtime default.
  • Derive group-add arguments dynamically from the GIDs of /dev/kfd and the selected DRI device, deduplicating groups while keeping explicit device grants.
  • Disallow runtime image pulls by adding --pull=never and strengthen tmpfs by allowing exec on /tmp while keeping nosuid/nodev and a bounded size.
  • Set HOME to /tmp in the container, preserve existing GPU visibility environment variables, and ensure network-none, read-only root, dropped capabilities, and no-new-privileges remain enforced.
ops/mi355/run-sandbox.sh
ops/mi355/run-command-sandbox.sh
tests/test_evolution.py
ops/mi355/README.md
Extend tests and documentation to cover SHA race handling, content-addressed evidence, sandbox isolation, and GitHub App permissions.
  • Add tests to ensure require_exact_sha rejects mismatched SHAs and that _copy_compact_bundle produces content-addressed, immutable evidence bundles with stable metadata.
  • Add tests asserting the sandbox scripts include required hardening flags, run as the caller identity with GPU device groups, and avoid exposing sensitive environment variables or docker.sock.
  • Update README to describe the strengthened SHA verification (including fetched commits and post-validation recheck), content-addressed evidence bundles, and strict sandbox isolation properties.
  • Document the minimal GitHub App permissions required and note the need to rotate short-lived installation tokens safely.
tests/test_evolution.py
ops/mi355/README.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="tests/test_evolution.py" line_range="761-770" />
<code_context>
             raise AssertionError(f"MI355 control-plane change accepted: {path}")


+def test_mi355_worker_rejects_fetched_sha_race():
+    from evolve.mi355_worker import require_exact_sha
+
+    approved = "a" * 40
+    require_exact_sha(approved, approved, source="test")
+    try:
+        require_exact_sha("b" * 40, approved, source="fetched PR head")
+    except ValueError as error:
+        assert "does not match approved SHA" in str(error)
+    else:
+        raise AssertionError("worker accepted a fetched SHA that was not approved")
+
+
</code_context>
<issue_to_address>
**suggestion (testing):** Strengthen `require_exact_sha` test by asserting the full error message and including the source label

This test only checks for the generic substring "does not match approved SHA". Since `require_exact_sha` includes the `source` in its error message, please also assert that the message contains the specific source (e.g. "fetched PR head"), and ideally the mismatching SHA values. This will better lock in the error contract and help detect regressions in the message structure or content.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread tests/test_evolution.py
Co-authored-by: Cursor <cursoragent@cursor.com>
@jhinpan
jhinpan merged commit 10abc27 into main Jul 23, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant