Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions .github/workflows/copilot-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
name: copilot tests

on:
pull_request:
paths:
- "copilot/**"
- ".github/workflows/copilot-tests.yml"
push:
branches: [main]
paths:
- "copilot/**"
- ".github/workflows/copilot-tests.yml"

permissions:
contents: read

jobs:
# Standard library only, so the composite action needs no install step. 3.9 is
# the floor the shared core supports.
stdlib:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9", "3.11", "3.12", "3.13"]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}
- name: Run tests with no dependencies installed
working-directory: copilot
run: |
pip install pytest
python -m pytest tests -q

# The check runs against this repository, which carries the surfaces it looks
# for: AGENTS.md files, .github/instructions, and skill directories. So it is
# both a smoke test and a dogfood.
self-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.12"
- name: Snapshot this repository's Copilot composition
run: python copilot/engine/capture.py snapshot
- name: Verify against the baseline, reporting without failing
# No baseline is committed for this repo yet, so verify exits 0 and says
# so. Once one is committed this becomes a real gate on ourselves.
run: python copilot/engine/capture.py verify
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,17 @@ TRACE only works as a standard if it is genuinely neutral. Integrations are list
| [claude-code](claude-code/) | agentrust-io | agent-manifest, trace | community |
| [agentrust-codex](plugins/agentrust-codex/) | agentrust-io | agent-manifest, trace | community |
| [scheduled-agents](scheduled-agents/) | agentrust-io | trace | community |
| [copilot](copilot/) | agentrust-io | (drift check only, see note) | community |

All four engines share [`agentrust-capture-core`](packages/agentrust-capture-core),
which owns fingerprinting, comparison, baseline sealing and the report honesty rules.

**Note on the Copilot entry.** It is a pull-request status check rather than a
session hook, because Copilot's composition lives in the repository. It emits no
TRACE record and no Agent Manifest yet, so it claims neither: `integrates_with` in
the manifest schema offers only `cmcp`, `trace` and `agent-manifest`, and asserting
one of those today would be an unverifiable claim. Emitting a TRACE record per
checked pull request is the intended next step and is what would make one true.

## Community

Expand Down
122 changes: 122 additions & 0 deletions copilot/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# AgenTrust for GitHub Copilot

**Review changes to your coding agent the way you review changes to your code.**

Copilot is not just a model. In this repository it is a model plus the instructions
you wrote it, the skills you gave it, and the MCP servers you connected. Those files
decide what the agent will do to your codebase, and every one of them arrives by
pull request.

So this integration is not a local warning. It is a status check:

> **Does this pull request change what Copilot reads, without saying so?**

## Why this differs from the other integrations here

The Claude Code and Codex integrations watch a developer's machine and warn at
session start, after the fact, one developer at a time. They have to, because that
composition lives in a home directory.

Copilot's composition lives in the repository. That is a better place to defend:

- **One baseline, shared.** Committed at `.agentrust/copilot-baseline.json`, not one
per laptop.
- **Reviewed like code.** A change to the agent's instructions shows up in a diff
with an author, and can require a reviewer.
- **Enforceable.** As a required status check, a pull request that changes the
agent's behaviour without updating the baseline does not merge.
- **Caught on entry.** At the moment it enters the codebase, rather than on some
developer's next session.

It also means **this integration does not seal its baseline**, unlike the others.
They do, because a local baseline can be rewritten with nothing to show for it. A
committed baseline gets provenance from git. Adding a digest on top would be
ceremony.

## Quickstart

```yaml
# .github/workflows/copilot-integrity.yml
name: Copilot integrity
on: pull_request

permissions:
contents: read
pull-requests: write # only needed for the comment

jobs:
integrity:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: agentrust-io/integrations/copilot@main
```

Then create the baseline and commit it:

```bash
python copilot/engine/capture.py approve
git add .agentrust/copilot-baseline.json
```

Adopting this on a busy repository? Start with `fail-on-drift: false`. You get the
comment and the summary without blocking anyone, and you can flip it on once the
baseline is settled.

## What it measures

Verified against GitHub's documentation for what Copilot actually reads.

| Category | Paths |
|---|---|
| Instructions | `.github/copilot-instructions.md`, `.github/instructions/**/*.instructions.md`, **`AGENTS.md` anywhere in the tree**, root `CLAUDE.md` and `GEMINI.md` |
| Skills | `.github/skills/<name>/`, `.claude/skills/<name>/`, `.agents/skills/<name>/` |
| MCP | `copilot/mcp-config.json`, `.vscode/mcp.json` |

Two of those deserve a note.

**`AGENTS.md` is matched anywhere**, because Copilot resolves the nearest one. A
file added three directories down changes how the agent behaves in that subtree
without touching anything at the root, and that is exactly the change worth
catching. Vendored directories (`node_modules`, `vendor`, `.venv` and friends) are
skipped, so a dependency shipping its own `AGENTS.md` is not counted as yours.

**Skills are digested across the whole directory**, not just `SKILL.md`. A skill's
`scripts/` decide what it does. Digesting the manifest alone was a live bypass in
two other engines in this repo, so the shared core covers the tree.

## What it does not do

- **It does not read your model or your tool roster.** Those are session facts, not
repository files. This integration measures what the repository gives Copilot.
- **It does not evaluate whether an instruction is good.** It tells you one changed
and who changed it. Judgement is the reviewer's.
- **It does not cover organisation-level or personal instructions.** Those are set
outside the repository and are invisible to a check that runs inside it. If your
organisation sets Copilot instructions centrally, this check does not see them.
- **It is not a sandbox.** It reports composition, it does not constrain execution.

## Inputs

| Input | Default | Notes |
|---|---|---|
| `root` | `.` | Repository root to inspect |
| `comment` | `true` | One comment per pull request, edited in place rather than appended per push |
| `fail-on-drift` | `true` | Set `false` to report without blocking |
| `github-token` | `${{ github.token }}` | Only used to post the comment |

## Commands

```bash
python copilot/engine/capture.py snapshot # print the composition as JSON
python copilot/engine/capture.py verify # diff against the baseline, exit 1 on drift
python copilot/engine/capture.py approve # write the baseline
```

No install step. The engine and its vendored copy of
[`agentrust-capture-core`](../packages/agentrust-capture-core) are standard library
only.

## License

Apache-2.0.
86 changes: 86 additions & 0 deletions copilot/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
name: AgenTrust Copilot integrity check
description: >-
Fail a pull request that changes what GitHub Copilot reads in this repository
(instructions, skills, MCP configuration) without updating the approved baseline
in the same change.
author: AgenTrust Contributors
branding:
icon: shield
color: purple

inputs:
root:
description: Repository root to inspect.
required: false
default: "."
comment:
description: >-
Post the result as a pull-request comment, updating the same comment on each
run rather than adding one per push. Needs pull-requests: write.
required: false
default: "true"
fail-on-drift:
description: >-
Fail the check when the composition changed. Set false to report without
blocking, which is the sensible first step when adopting this on a busy repo.
required: false
default: "true"
github-token:
description: Token used to post the comment.
required: false
default: ${{ github.token }}

outputs:
changed:
description: "true when the composition drifted from the baseline"
value: ${{ steps.check.outputs.changed }}

runs:
using: composite
steps:
# No install step. The engine and its vendored core are standard library
# only, which is the whole reason the core carries no dependencies.
- id: check
shell: bash
run: |
set -o pipefail
comment_file="${RUNNER_TEMP}/agentrust-copilot-comment.md"
if python "${{ github.action_path }}/engine/capture.py" verify \
--root "${{ inputs.root }}" --comment-file "$comment_file"; then
echo "changed=false" >> "$GITHUB_OUTPUT"
else
echo "changed=true" >> "$GITHUB_OUTPUT"
fi
echo "comment-file=$comment_file" >> "$GITHUB_OUTPUT"
{
echo "## AgenTrust Copilot integrity check"
echo
cat "$comment_file"
} >> "$GITHUB_STEP_SUMMARY"

- if: ${{ inputs.comment == 'true' && github.event_name == 'pull_request' }}
shell: bash
env:
GH_TOKEN: ${{ inputs.github-token }}
BODY_FILE: ${{ steps.check.outputs.comment-file }}
PR: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: |
# One comment per pull request, edited in place. A comment per push turns
# a useful signal into noise people mute.
marker="<!-- agentrust-copilot-integrity -->"
body="$(printf '%s\n\n' "$marker"; cat "$BODY_FILE")"
existing="$(gh api "repos/$REPO/issues/$PR/comments" --paginate \
--jq "[.[] | select(.body | contains(\"$marker\")) | .id] | first // empty")"
if [ -n "$existing" ]; then
gh api --method PATCH "repos/$REPO/issues/comments/$existing" -f body="$body" >/dev/null
else
gh api --method POST "repos/$REPO/issues/$PR/comments" -f body="$body" >/dev/null
fi

- if: ${{ inputs.fail-on-drift == 'true' && steps.check.outputs.changed == 'true' }}
shell: bash
run: |
echo "::error::This pull request changes what Copilot reads without updating"\
"the approved baseline. See the comment on this pull request."
exit 1
7 changes: 7 additions & 0 deletions copilot/engine/_vendor/agentrust_capture_core/VENDORED.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Generated by scripts/sync_vendored_core.py. Do not edit.
#
# Pinned copy of agentrust-capture-core, used when the package is not installed.
# The engines run from shell hooks before anything is installed, so this fallback
# is what makes drift detection work on a bare plugin install. Edit
# packages/agentrust-capture-core and re-run the sync script; CI fails if this
# copy and the package disagree.
97 changes: 97 additions & 0 deletions copilot/engine/_vendor/agentrust_capture_core/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
"""Shared core for AgenTrust agent-integrity capture engines.

Each engine answers one question about a different coding agent: is this the
composition I approved, with nothing added and nothing subtracted? What differs
between agents is where to look and what to call things. What must not differ is
how content is fingerprinted, how snapshots are compared, how a baseline is sealed,
and the rules that keep a report honest.

Those lived in three copies before this package existed, and the cost was not
theoretical: the same skill-fingerprinting bypass had to be found and fixed twice,
independently, and a reporting defect once. This package is the single source of
truth for the parts that are genuinely identical.

Standard library only, because the engines run from shell hooks at session start
and must work before anything is installed.
"""

from __future__ import annotations

from .compare import (
Change,
diff_hash,
diff_maps,
diff_scalar,
diff_sets,
observed_categories,
scope_change,
)
from .hashing import (
EXCLUDE_DIRS,
EXCLUDE_SUFFIXES,
now_iso,
safe_sha_file,
sha_bytes,
sha_file,
sha_mapping,
tree_digest,
uuid7,
)
from .report import (
UNMEASURED,
change_lines,
clean_verdict,
measured_or,
seal_section,
unmeasured_footnote,
)
from .seal import (
INTEGRITY_BROKEN,
INTEGRITY_OK,
INTEGRITY_UNSEALED,
SEAL_FIELD,
attach_seal,
check_seal,
state_digest,
)
from .state import StatePaths, atomic_write, load_state, save_baseline, save_state

__version__ = "0.1.0"

__all__ = [
"Change",
"EXCLUDE_DIRS",
"EXCLUDE_SUFFIXES",
"INTEGRITY_BROKEN",
"INTEGRITY_OK",
"INTEGRITY_UNSEALED",
"SEAL_FIELD",
"StatePaths",
"UNMEASURED",
"__version__",
"atomic_write",
"attach_seal",
"change_lines",
"check_seal",
"clean_verdict",
"diff_hash",
"diff_maps",
"diff_scalar",
"diff_sets",
"load_state",
"measured_or",
"now_iso",
"observed_categories",
"safe_sha_file",
"save_baseline",
"save_state",
"scope_change",
"seal_section",
"sha_bytes",
"sha_file",
"sha_mapping",
"state_digest",
"tree_digest",
"unmeasured_footnote",
"uuid7",
]
Loading