Skip to content

[CLI] META-247 — land the ratified neutral package architecture - #3

Merged
qmarcelle merged 3 commits into
mainfrom
feat/meta-247-neutral-package-architecture
Jul 26, 2026
Merged

qmarcelle merged 3 commits into
mainfrom
feat/meta-247-neutral-package-architecture

Conversation

@qmarcelle

Copy link
Copy Markdown
Contributor

Lands the neutral package architecture ratified in META-236, preserving the historical agents-audit surface.

Linear: META-247. Gates META-195, META-235 and META-245 — those issues reference directories that did not exist until this merges.

What landed

packages/cli/                  @workspacejson/cli, bin `workspacejson`   PUBLIC (not published)
  src/producer/                generation, scanning, reconciliation, drift
  src/commands/                command routing and UX
packages/agents-audit-compat/  agents-audit, bin `agents-audit`          PUBLIC, frozen bridge
packages/datahub-adapter/      @workspacejson/datahub-adapter            PRIVATE, staging only

Every module move is a git rename, so history follows the code.

From To
agents-audit/src/generate.ts cli/src/producer/generate.ts
agents-audit/src/internal/fs.ts cli/src/producer/fs.ts
agents-audit/src/internal/config.ts split — detectCiProvider → producer, DEFAULT_AUDIT_CONFIG → compat
agents-audit/src/cli.ts generate route cli/src/commands/generate.ts
packages/cli/** (old shim) packages/datahub-adapter/**

One implementation, two binaries

agents-audit generate delegates to the same runGenerate the neutral CLI uses. The caller supplies only its own producer identity and commandName, which is what keeps agents-audit output byte-identical while workspacejson correctly names itself. The two commands cannot drift during the compatibility window.

packages/datahub-adapter/ is staging, not architecture

Per the mid-flight course correction: this is a DataHub consumer adapter and belongs to workspacejson/datahub-agent. It is parked here only because META-240 had to preserve it somewhere.

It is renamed for accuracy, kept private: true, excluded from the durable package map in the README, and the neutral CLI does not depend on it. A new neutral-producer-purity guard prevents DataHub/dbt/vendor logic from leaking into packages/cli on its way out. Its 35/35 harness travels with it. Extraction is tracked separately.

Three defects the move exposed

Each fixed at the root rather than worked around:

  1. generated.by hardcoded 'agents-audit' inside what is now the neutral producer. by is provenance — it must record which producer actually ran — so it became a caller-supplied identity defaulting to this package. Compat passes its historical value, so its artifacts are unchanged.
  2. Refusal messages hardcoded agents-audit generate …, which would have told a workspacejson user to run a different tool. Command name is now parameterized; compat's messages stay byte-identical.
  3. generate.ts resolved its own package.json by fixed relative path. Source depth (src/producer/) no longer matches bundled depth (dist/), so the path was right for this package's own tests and wrong for every consumer importing dist — a failure invisible to the package that owns the file. It now walks up to find its manifest.

Plus a latent defect in the pack verifier: it left the packed manifest as the raw string tar returns, so every assertion indexing into it — bin targets, pinned dependencies — read undefined and passed vacuously. Parsing it makes those checks real for the first time. They pass.

Guards

  • local-dependency refined. It rejected workspace: in any published package — a rule written for cross-repository links. Intra-repository links are normal and pnpm rewrites them at pack time. It now distinguishes the two by checking whether this repository defines the package.
  • neutral-producer-purity added. Rejects DataHub, dbt or vendor content in packages/cli, by filename and content. Its first red test failed — \bdatahub\b does not match joinDataHubUrn, which is exactly how vendor logic arrives — so the rule now matches substrings and checks filenames too.

Red tests 15 → 19, all passing, including the baseline that proves an unmodified repository is still accepted.

Parity — the gate

Producer: 27/29, with 2 deliberate differences. Structural move alone was 29/29; the two differences come from the separately committed Vreko change.

Intentional difference Evidence
scan . human output: vendor upsell → neutral producer notice exit code unchanged; scan --json byte-identical, so machine-readable output is unaffected
exports gained renderMissingArtifactNotice purely additive — all nine historical exports remain, including renderVrekoUpsell

Everything else green: generate, --dry-run, --check, --force, invalid-artifact refusal, --force move-aside, verbatim manual-evidence preservation, atomic writes, generated-artifact equivalence, producer stamp {agents-audit, 0.4.4}.

DataHub adapter: 35/35, with the package rename and bin surrender documented in the harness itself.

The Vreko ruling

META-236 ratified that vendor promotion never enters the neutral package and is removed or opt-in in the compatibility package.

Deleting it outright would have discarded a useful signal, so the signal and validation errors are kept and only the promotion changes:

before:  Vreko generates it automatically from real codebase structure
         and activity, unlocking richer audit findings.  vreko.dev
after:   Generate it with: agents-audit generate
         A committed artifact unlocks richer audit findings.

This also serves META-182, which is about the producer never being named. renderVrekoUpsell remains exported and unchanged — the "opt-in" the ruling allows — and is simply no longer called by the CLI.

Committed separately from the structural move, so the behavior-preserving change is verifiable at 29/29 on its own.

Verification

typecheck                    3/3 packages
build                        3/3 packages
tests                        80  (32 neutral CLI + 42 compat + 6 adapter); was 74
architecture guard           pass — 72 files, 4 manifests
architecture red tests       19/19
pack verification            pass for both public packages, incl. a real
                             generate smoke test resolving the unpublished
                             sibling from disk
producer parity              27/29 + 2 documented intentional differences
DataHub adapter parity       35/35

Not in scope

  • No producer enrichment (META-195) and no dual emission (META-235).
  • Nothing published. @workspacejson/cli has public metadata but has never been on npm; agents-audit@0.4.4 on the registry is untouched; no publish authority moved; the repository still holds no secrets.
  • No README narrative, quickstart or install commands finalized — that is META-245, which should now block on this issue rather than on META-236 alone. The README was updated only to stop describing a two-package structure that no longer exists.

Implements Architecture B, ratified in META-236. Behavior-preserving: the
public surface of `agents-audit` is unchanged and both parity harnesses are
green.

  packages/cli/                 @workspacejson/cli, bin `workspacejson`
    src/producer/               generation, scanning, reconciliation
    src/commands/               command routing and UX
  packages/agents-audit-compat/ agents-audit, bin `agents-audit` (frozen bridge)
  packages/datahub-adapter/     private, MIGRATION STAGING ONLY

Module moves are git renames, so history follows the code:
  agents-audit/src/generate.ts        -> cli/src/producer/generate.ts
  agents-audit/src/internal/fs.ts     -> cli/src/producer/fs.ts
  agents-audit/src/internal/config.ts -> split: detectCiProvider to the producer,
                                        DEFAULT_AUDIT_CONFIG stays in compat
  packages/cli/**                     -> packages/datahub-adapter/**

One implementation, two binaries
`agents-audit generate` delegates to the same `runGenerate` the neutral CLI
uses, so the two commands cannot drift during the compatibility window. The
caller supplies its own `producer` identity and `commandName`, which is what
keeps `agents-audit` output byte-identical.

Three defects the move exposed, each fixed rather than worked around:

1. `generated.by` hardcoded `'agents-audit'` inside what is now the neutral
   producer. `by` is provenance — it must record which producer actually ran —
   so it became a caller-supplied identity defaulting to this package. compat
   passes its historical value, so its artifacts are unchanged.
2. Refusal messages hardcoded `agents-audit generate ...`, which would have told
   a `workspacejson` user to run a different tool. Command name is now
   parameterized; compat's messages are byte-identical.
3. `generate.ts` resolved its own package.json by fixed relative path. Source
   depth (`src/producer/`) no longer matches bundled depth (`dist/`), so the
   path was correct for tests and wrong for consumers. It now walks up to find
   the manifest, which is depth-independent.

Also fixed: scripts/verify-package-tarball.mjs left the packed manifest as the
raw string `tar` returns, so every assertion indexing into it — bin targets,
pinned dependencies — read `undefined` and passed vacuously. Parsing it makes
those checks real for the first time.

Guards
- `local-dependency` now distinguishes intra-repository workspace links (legal,
  rewritten by pnpm at pack time) from cross-repository ones (never resolvable).
  The old rule would have rejected compat depending on the neutral package.
- New `neutral-producer-purity` rejects DataHub, dbt or vendor content in
  packages/cli by filename or content. Its first red test failed — `\bdatahub\b`
  does not match `joinDataHubUrn` — so the rule now matches substrings and
  checks filenames.
- Red tests 15 -> 19, all passing including baseline.

packages/datahub-adapter is staging, not architecture
It is a DataHub *consumer* adapter and belongs to workspacejson/datahub-agent.
It is renamed for accuracy, kept private, excluded from the durable package map,
and the neutral CLI does not depend on it. Its 35/35 harness travels with it.

Verification
- typecheck 3/3, build 3/3
- tests 80 (32 neutral CLI + 42 compat + 6 adapter); was 74
- architecture guard pass; red tests 19/19
- pack verification pass for both public packages, incl. a real generate smoke
  test resolving the unpublished sibling from disk
- META-240 producer parity 29/29 — all 9 exports identical, producer stamp
  preserved, generated artifacts identical
- META-240 DataHub parity 35/35, with two documented intentional differences:
  the package rename and surrendering the `workspacejson` bin

Nothing published. @workspacejson/cli has public metadata but is not on npm;
agents-audit@0.4.4 on the registry is untouched; no publish authority moved.
…e (META-236)

META-236 ratified that vendor promotion never enters the neutral package and is
removed or made opt-in in the compatibility package.

Deleting the notice outright would have thrown away a genuinely useful signal —
"workspace.json is missing or stale" — so the signal and the validation-error
display are kept and only the promotion changes. The remediation now names the
command this tool actually provides:

  before:  Vreko generates it automatically from real codebase structure
           and activity, unlocking richer audit findings.  vreko.dev
  after:   Generate it with: agents-audit generate
           A committed artifact unlocks richer audit findings.

That also serves META-182, which is about the producer never being named.

`renderVrekoUpsell` remains exported and unchanged — it is part of the nine-export
compatibility surface — and is simply no longer called by the CLI. Callers who
want it can still invoke it, which is the "opt-in" the ruling allows.

INTENTIONAL PARITY DIFFERENCES — 27/29, both deliberate and evidenced:

1. `agents-audit scan .` human-readable output differs by exactly this box.
   Exit code unchanged. `scan . --json` is byte-identical, so nothing consuming
   machine-readable output is affected.
2. Public exports gained `renderMissingArtifactNotice`. Purely additive — all
   nine historical exports remain, including `renderVrekoUpsell`.

Everything else in the harness stays green: generate/--dry-run/--check/--force,
invalid-artifact refusal, --force move-aside, manual-evidence preservation,
atomic writes, generated-artifact equivalence and the producer stamp.

This is a separate commit from the structural move deliberately: the
restructure is behavior-preserving at 29/29, and this is the one place the
ratified decision changes what a user sees.
Copilot AI review requested due to automatic review settings July 26, 2026 04:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

agents-audit consumes @workspacejson/cli's emitted declarations, so on a clean
checkout tsc cannot resolve them until the neutral package has been built. This
passed locally only because dist/ was already present. pnpm builds in
topological order, so reordering is sufficient.

Documented in CONTRIBUTING so a fresh clone does not hit the same wall.
@qmarcelle
qmarcelle merged commit c60447f into main Jul 26, 2026
2 checks passed
@qmarcelle
qmarcelle deleted the feat/meta-247-neutral-package-architecture branch July 27, 2026 03:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants