From 1d666f35abe9534682dda9059a550fbe3b7efc81 Mon Sep 17 00:00:00 2001 From: Qwynn Marcelle Date: Sun, 26 Jul 2026 00:45:04 -0400 Subject: [PATCH] fix(migration): make the parity harnesses portable and self-contained MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The harnesses carried absolute paths into a session-scoped scratchpad: SCRATCH="/private/tmp/claude-502/.../scratchpad" They ran correctly where they were written, but they could not run from a checkout, and the directory they pointed at is temporary. They also assumed someone had already cloned the frozen source, built both sides, and packed candidates by hand — none of which was written down. As the compatibility gate for every downstream producer change, that made them effectively unrunnable. Every path is now derived: - new side = this repository, resolved from the script's own location - old side = a clone of the frozen pre-migration source, pinned to the SHA recorded in migration/PROVENANCE.md - output = .parity-cache/ (gitignored) New migration/parity-lib.sh resolves, clones, pins, builds and packs both sides. First run clones and builds the frozen source; the clone is cached, so later runs take ~30s. It refuses to proceed if the old checkout is at any commit other than the frozen SHA — comparing against a different commit measures drift, not migration fidelity. Overridable: WORKSPACEJSON_OLD_CHECKOUT, WORKSPACEJSON_PARITY_OUT, WORKSPACEJSON_PARITY_CACHE, WORKSPACEJSON_SKIP_BUILD. The Node harness reuses the same cache convention and, when the old side is missing or unbuilt, prints the exact command to fix it instead of throwing a path error. If packages/datahub-adapter is gone — which META-248 will do — it says so and points at workspacejson/datahub-agent rather than failing obscurely. Portability fixes found while testing against the system shell: - `declare -A` needs bash 4+; macOS ships 3.2. Replaced with a plain branch. - empty-array expansion under `set -u` errors on bash 3.2. Guarded with ${arr[@]+"${arr[@]}"}. Verified from a clean checkout with no cache present: runtime 27/29 (the two recorded META-236 vendor-notice differences), 33s shim 35/35 pack runs; identity fields 11/11 identical. Its remaining differences are all expected META-247 consequences: the content-hashed tsup chunk name, the description, repository/bugs URLs, the added @workspacejson/cli dependency and @types/node. CONTRIBUTING now documents how to run them and what results to expect, so a third difference reads as a regression rather than as noise. --- .gitignore | 3 + CONTRIBUTING.md | 32 +++++- migration/parity-agents-audit-pack.sh | 21 ++-- migration/parity-agents-audit-runtime.sh | 17 ++- migration/parity-datahub-shim.mjs | 35 ++++++- migration/parity-lib.sh | 128 +++++++++++++++++++++++ 6 files changed, 216 insertions(+), 20 deletions(-) create mode 100644 migration/parity-lib.sh diff --git a/.gitignore b/.gitignore index 3738e52..f64bb52 100644 --- a/.gitignore +++ b/.gitignore @@ -22,3 +22,6 @@ audit-output/ agents-workspace-cannon-repo/ audit-findings/ shared-baseline-audit-2026-05-10.md + +# parity harness cache (cloned frozen source + packed candidates) +.parity-cache/ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e40ab38..5b4fc78 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -27,12 +27,40 @@ node packages/agents-audit-compat/dist/cli.js scan . node packages/cli/dist/cli.js generate --check ``` +## Parity harnesses — the compatibility gate + +`agents-audit` is a frozen compatibility bridge. Anything touching its command +surface, exit codes, output or exports must be measured against the frozen +pre-migration source, not just against the current tests. + +```bash +migration/parity-agents-audit-runtime.sh # command + perturbed behavior parity +migration/parity-agents-audit-pack.sh # packed-artifact and manifest parity +node migration/parity-datahub-shim.mjs # DataHub adapter parity +``` + +They are self-contained. On first run the bash harnesses clone the frozen source +at its recorded SHA, build it, build this repository, and pack both sides; the +clone is cached under `.parity-cache/` (gitignored), so later runs take seconds. +The Node harness reuses that same cache and tells you what to run if it is +missing. + +Overridable via environment: `WORKSPACEJSON_OLD_CHECKOUT` to point at an +existing clone, `WORKSPACEJSON_PARITY_OUT` for the working directory, +`WORKSPACEJSON_SKIP_BUILD=1` to reuse existing builds. + +**Expected results today:** runtime `27/29` with two recorded intentional +differences from META-236's vendor-notice ruling, and the DataHub adapter +`35/35`. A third difference means you changed something you should not have — +or you owe it an explicit intentional-difference record. + ## Change Expectations - Update package READMEs when public APIs change - Update `CHANGELOG.md` for repository-level changes and - `packages/agents-audit/CHANGELOG.md` for package release notes -- Keep the CLI contract documented in `packages/agents-audit/README.md` + `packages/agents-audit-compat/CHANGELOG.md` for package release notes +- Keep the CLI contract documented in `packages/cli/README.md` +- Re-run the parity harnesses before changing anything `agents-audit` exposes - Add a changeset for anything user-facing in `agents-audit` ## Boundaries enforced in CI diff --git a/migration/parity-agents-audit-pack.sh b/migration/parity-agents-audit-pack.sh index d9aaad1..77878ab 100644 --- a/migration/parity-agents-audit-pack.sh +++ b/migration/parity-agents-audit-pack.sh @@ -1,20 +1,21 @@ #!/usr/bin/env bash # agents-audit packed-artifact parity: OLD (frozen source) vs NEW (workspacejson/cli) +# +# Usage: migration/parity-agents-audit-pack.sh +# +# Self-contained: see migration/parity-lib.sh for the overridable paths. set -uo pipefail -SCRATCH="/private/tmp/claude-502/-Users-user1-dev-cli/ed967700-e9b4-4202-b983-6faf9cee9f6d/scratchpad" -OLD="$SCRATCH/source-agents-audit/packages/agents-audit" -NEW="$SCRATCH/cli-extract/packages/agents-audit-compat" -OUT="$SCRATCH/parity" -mkdir -p "$OUT/old" "$OUT/new" +# shellcheck source=./parity-lib.sh +source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/parity-lib.sh" -echo "### Packing OLD candidate (frozen source e47eb1b) ###" -(cd "$OLD" && npm pack --ignore-scripts --pack-destination "$OUT/old" --json > "$OUT/old/pack.json" 2>"$OUT/old/pack.err") -echo "### Packing NEW candidate (workspacejson/cli) ###" -(cd "$NEW" && npm pack --ignore-scripts --pack-destination "$OUT/new" --json > "$OUT/new/pack.json" 2>"$OUT/new/pack.err") +# Packs both sides (and resolves/builds the frozen source on first run). +parity_prepare_agents_audit +# Associative arrays need bash 4+; macOS still ships 3.2, so keep this portable. for side in old new; do - tgz="$OUT/$side/agents-audit-0.4.4.tgz" + if [ "$side" = "old" ]; then tgz="$OLD_AGENTS_AUDIT_TGZ"; else tgz="$NEW_AGENTS_AUDIT_TGZ"; fi + mkdir -p "$OUT/$side" tar -tzf "$tgz" | sed 's|^\./||' | sort > "$OUT/$side/files.txt" tar -xOzf "$tgz" package/package.json > "$OUT/$side/manifest.json" shasum -a 256 "$tgz" | cut -d' ' -f1 > "$OUT/$side/tarball.sha256" diff --git a/migration/parity-agents-audit-runtime.sh b/migration/parity-agents-audit-runtime.sh index ee49c4d..fc187bc 100644 --- a/migration/parity-agents-audit-runtime.sh +++ b/migration/parity-agents-audit-runtime.sh @@ -4,10 +4,19 @@ # # Every load-bearing behavior is PERTURBED: we do not just assert exit 0, we # break the precondition and assert the expected failure/movement, on both sides. +# +# Usage: migration/parity-agents-audit-runtime.sh +# +# Self-contained: resolves and builds the frozen pre-migration source, builds +# this repository, packs both, then compares. See migration/parity-lib.sh for +# the overridable paths. set -uo pipefail -SCRATCH="/private/tmp/claude-502/-Users-user1-dev-cli/ed967700-e9b4-4202-b983-6faf9cee9f6d/scratchpad" -OUT="$SCRATCH/parity" +# shellcheck source=./parity-lib.sh +source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/parity-lib.sh" + +parity_prepare_agents_audit + RUN="$OUT/runtime"; rm -rf "$RUN"; mkdir -p "$RUN" PASS=0; FAIL=0 @@ -27,8 +36,8 @@ install_side () { # $1=side $2=tarball $3...=extra sibling tarballs echo "$dir" } -OLD_DIR=$(install_side old "$OUT/oldpnpm/agents-audit-0.4.4.tgz") -NEW_DIR=$(install_side new "$OUT/newpnpm/agents-audit-0.4.4.tgz" "$OUT/newpnpm/workspacejson-cli-0.1.0.tgz") +OLD_DIR=$(install_side old "$OLD_AGENTS_AUDIT_TGZ") +NEW_DIR=$(install_side new "$NEW_AGENTS_AUDIT_TGZ" ${NEW_SIBLING_TGZS[@]+"${NEW_SIBLING_TGZS[@]}"}) # Build one canonical fixture repo, then clone it per invocation so old and new # always see byte-identical input. diff --git a/migration/parity-datahub-shim.mjs b/migration/parity-datahub-shim.mjs index 7b0c065..38f8d0c 100644 --- a/migration/parity-datahub-shim.mjs +++ b/migration/parity-datahub-shim.mjs @@ -7,15 +7,42 @@ import { mkdtempSync, mkdirSync, writeFileSync, rmSync, readFileSync, existsSync } from "node:fs"; import { tmpdir } from "node:os"; -import { join } from "node:path"; +import { dirname, join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; import { spawnSync } from "node:child_process"; -const SCRATCH = "/private/tmp/claude-502/-Users-user1-dev-cli/ed967700-e9b4-4202-b983-6faf9cee9f6d/scratchpad"; +// Paths are derived, never hardcoded. The old side is a clone of the frozen +// pre-migration source; migration/parity-lib.sh resolves, pins and builds it, +// and caches it under .parity-cache/ so repeat runs are cheap. +const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), ".."); +const PARITY_CACHE = process.env.WORKSPACEJSON_PARITY_CACHE ?? join(REPO_ROOT, ".parity-cache"); +const OLD_CHECKOUT = process.env.WORKSPACEJSON_OLD_CHECKOUT ?? join(PARITY_CACHE, "source-agents-audit"); + const SIDES = { - old: join(SCRATCH, "source-agents-audit/packages/cli"), - new: join(SCRATCH, "cli-extract/packages/datahub-adapter"), + old: join(OLD_CHECKOUT, "packages/cli"), + new: join(REPO_ROOT, "packages/datahub-adapter"), }; +for (const [side, dir] of Object.entries(SIDES)) { + if (!existsSync(join(dir, "package.json"))) { + console.error(`\nERROR: missing the ${side} side at ${dir}\n`); + console.error(side === "old" + ? "Run migration/parity-lib.sh's bootstrap first — the simplest way is:\n" + + " bash -c 'source migration/parity-lib.sh && parity_resolve_old_checkout && parity_build_old'\n" + + "or point WORKSPACEJSON_OLD_CHECKOUT at an existing clone of the frozen source.\n" + : "The DataHub adapter has been extracted from this repository (META-248).\n" + + "Re-run this harness from workspacejson/datahub-agent against its candidate.\n"); + process.exit(1); + } + if (!existsSync(join(dir, "dist/index.js"))) { + console.error(`\nERROR: ${side} side is not built (${join(dir, "dist/index.js")} missing).`); + console.error(side === "old" + ? "Build the frozen source: (cd " + OLD_CHECKOUT + " && pnpm install --no-frozen-lockfile && pnpm -r build)\n" + : "Build this repository: pnpm install && pnpm -r build\n"); + process.exit(1); + } +} + let pass = 0, fail = 0; const failures = []; diff --git a/migration/parity-lib.sh b/migration/parity-lib.sh new file mode 100644 index 0000000..7928b98 --- /dev/null +++ b/migration/parity-lib.sh @@ -0,0 +1,128 @@ +#!/usr/bin/env bash +# Shared bootstrap for the workspace.json CLI parity harnesses. +# +# The harnesses compare this repository ("new") against the frozen pre-migration +# source ("old"). Both sides have to be built and packed before any comparison +# is meaningful, and the old side is a different repository entirely — so this +# script resolves, builds and packs both rather than assuming someone did it by +# hand. Every path is derived; nothing is hardcoded to a machine. +# +# Overridable: +# WORKSPACEJSON_OLD_CHECKOUT path to an existing clone of the frozen source +# (default: cached clone under .parity-cache/) +# WORKSPACEJSON_PARITY_OUT working directory for packed candidates +# (default: .parity-cache/out) +# WORKSPACEJSON_SKIP_BUILD set to 1 to reuse existing dist/ and tarballs + +set -uo pipefail + +PARITY_LIB_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(cd "$PARITY_LIB_DIR/.." && pwd)" + +# Recorded in migration/PROVENANCE.md. The old side must be this exact commit — +# comparing against anything else measures drift, not migration fidelity. +FROZEN_SOURCE_REPO="https://github.com/workspace-json/agents-audit.git" +FROZEN_SOURCE_SHA="e47eb1b8556c4f361db9a78190a2f36b400756e8" + +PARITY_CACHE="${WORKSPACEJSON_PARITY_CACHE:-$REPO_ROOT/.parity-cache}" +OLD_CHECKOUT="${WORKSPACEJSON_OLD_CHECKOUT:-$PARITY_CACHE/source-agents-audit}" +OUT="${WORKSPACEJSON_PARITY_OUT:-$PARITY_CACHE/out}" + +parity_log () { printf ' [parity] %s\n' "$*" >&2; } + +parity_die () { printf '\nERROR: %s\n\n' "$*" >&2; exit 1; } + +# --- old side --------------------------------------------------------------- + +parity_resolve_old_checkout () { + if [ -d "$OLD_CHECKOUT/.git" ]; then + local actual + actual="$(git -C "$OLD_CHECKOUT" rev-parse HEAD 2>/dev/null || echo unknown)" + if [ "$actual" = "$FROZEN_SOURCE_SHA" ]; then + parity_log "old checkout ready at $OLD_CHECKOUT" + return 0 + fi + parity_die "$OLD_CHECKOUT is at $actual, not the frozen SHA $FROZEN_SOURCE_SHA. +Point WORKSPACEJSON_OLD_CHECKOUT at the frozen source, or remove that directory +so it can be re-cloned. Comparing against any other commit measures drift, not +migration fidelity." + fi + + if [ -e "$OLD_CHECKOUT" ]; then + parity_die "$OLD_CHECKOUT exists but is not a git checkout. Remove it or set WORKSPACEJSON_OLD_CHECKOUT." + fi + + parity_log "cloning frozen source $FROZEN_SOURCE_SHA -> $OLD_CHECKOUT" + mkdir -p "$(dirname "$OLD_CHECKOUT")" + git init -q "$OLD_CHECKOUT" || parity_die "git init failed" + git -C "$OLD_CHECKOUT" remote add origin "$FROZEN_SOURCE_REPO" + # Fetch just the frozen commit when the server allows it; fall back to a full + # clone. Either way the checkout ends up pinned to the same SHA. + if git -C "$OLD_CHECKOUT" fetch -q --depth 1 origin "$FROZEN_SOURCE_SHA" 2>/dev/null; then + git -C "$OLD_CHECKOUT" checkout -q FETCH_HEAD + else + parity_log "shallow fetch by SHA unavailable; falling back to a full fetch" + git -C "$OLD_CHECKOUT" fetch -q origin || parity_die "could not fetch $FROZEN_SOURCE_REPO" + git -C "$OLD_CHECKOUT" checkout -q "$FROZEN_SOURCE_SHA" \ + || parity_die "frozen SHA $FROZEN_SOURCE_SHA not found in $FROZEN_SOURCE_REPO" + fi + parity_log "old checkout pinned to $(git -C "$OLD_CHECKOUT" rev-parse HEAD)" +} + +parity_build_old () { + [ "${WORKSPACEJSON_SKIP_BUILD:-0}" = "1" ] && { parity_log "skipping old build"; return 0; } + parity_log "installing + building the frozen source (first run is slow, then cached)" + (cd "$OLD_CHECKOUT" && pnpm install --no-frozen-lockfile >/dev/null 2>&1 && pnpm -r build >/dev/null 2>&1) \ + || parity_die "could not build the frozen source at $OLD_CHECKOUT" +} + +# --- new side --------------------------------------------------------------- + +parity_build_new () { + [ "${WORKSPACEJSON_SKIP_BUILD:-0}" = "1" ] && { parity_log "skipping new build"; return 0; } + parity_log "installing + building this repository" + (cd "$REPO_ROOT" && pnpm install --no-frozen-lockfile >/dev/null 2>&1 && pnpm -r build >/dev/null 2>&1) \ + || parity_die "could not build this repository at $REPO_ROOT" +} + +# --- packing ---------------------------------------------------------------- + +# Packs a package directory into a destination, echoing the tarball path. +parity_pack () { # $1=package dir $2=destination dir + local package_dir="$1" destination="$2" name version tarball + [ -f "$package_dir/package.json" ] || parity_die "no package.json in $package_dir" + name="$(node -p "require('$package_dir/package.json').name")" + version="$(node -p "require('$package_dir/package.json').version")" + tarball="$destination/$(printf '%s' "$name" | sed 's|^@||; s|/|-|g')-$version.tgz" + mkdir -p "$destination" + rm -f "$tarball" + (cd "$package_dir" && pnpm pack --pack-destination "$destination" >/dev/null 2>&1) \ + || parity_die "pnpm pack failed for $package_dir" + [ -f "$tarball" ] || parity_die "expected $tarball after packing $name" + echo "$tarball" +} + +# Resolves the old side, builds both, and packs the candidates the runtime +# harness installs. Sets: OLD_AGENTS_AUDIT_TGZ, NEW_AGENTS_AUDIT_TGZ, +# NEW_SIBLING_TGZS (array). +parity_prepare_agents_audit () { + parity_resolve_old_checkout + parity_build_old + parity_build_new + + rm -rf "$OUT/old" "$OUT/new" + OLD_AGENTS_AUDIT_TGZ="$(parity_pack "$OLD_CHECKOUT/packages/agents-audit" "$OUT/old")" + NEW_AGENTS_AUDIT_TGZ="$(parity_pack "$REPO_ROOT/packages/agents-audit-compat" "$OUT/new")" + + # agents-audit depends on @workspacejson/cli, which is deliberately + # unpublished until the authority cutover (META-243), so the new side has to + # install it from a packed sibling or the install fails on a package that + # simply is not released yet. + NEW_SIBLING_TGZS=() + if [ -d "$REPO_ROOT/packages/cli" ]; then + NEW_SIBLING_TGZS+=("$(parity_pack "$REPO_ROOT/packages/cli" "$OUT/new")") + fi + + parity_log "old candidate: $OLD_AGENTS_AUDIT_TGZ" + parity_log "new candidate: $NEW_AGENTS_AUDIT_TGZ" +}