You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Follow-up to #160 and #162 after the production path was proven successfully with v1.2.379.
The core release architecture is now considered correct and operationally proven:
build once
identify once
test exact release
promote exact release
rollback exact previous release
This issue does not reopen #162 and does not introduce another round of generic release hardening.
The remaining goal is operational simplification:
make it harder for an operator to start an invalid promotion
make staging exercise the same deploy executor as production
make promotion creation reproducible
make historical deployment identity trivial to recover
The work is intentionally bounded to four improvements:
pre-tag production-readiness preflight;
one canonical promotion entrypoint;
staging/production convergence on the same hermetic executor;
durable bounded deployment ledger.
No blue/green, Kubernetes, GitOps migration, new state machine or automatic rollback selection is required.
Context
#160 established immutable Release Bundle v2 and exact release identity.
The successful production promotion of v1.2.379 proved the normal production path end-to-end.
Do not weaken or redesign those invariants.
Operational friction still exists:
operator creates production tag
↓
workflow discovers missing/invalid promotion prerequisites
↓
production remains safe
BUT
immutable tag/version has already been consumed
A concrete example was a promotion that failed pre-boundary because PRODUCTION_RECOVERY_SHA was not configured. Another pre-boundary failure exposed a workflow-only recovery bundle assertion defect.
Both failed safely, but both should be discoverable before creating the production tag.
The current workflow also intentionally has an asymmetry:
staging -> local npm run deploy:staging path
production -> tag-only GitHub Actions + hermetic production executor
Now that the production executor is proven, staging should converge on that same executor semantics so normal release traffic continuously exercises the production path.
Core invariants
Invariant 1 — no tag before production readiness
A production tag must not be created until a non-mutating readiness check proves all prerequisites required to start the production promotion.
Conceptually:
RC selected
-> exact release identity resolved
-> production readiness preflight
-> operator sees exact C/R/release identity
-> only then production tag may be created
A failed readiness preflight must leave:
no production mutation
no production tag
no new release candidate
Invariant 2 — promotion remains exact and immutable
The preflight/promotion helper must not select latest, rebuild artifacts, recompute OpenPath pins independently or choose an alternative release.
It must consume one exact RC and resolve exactly:
C
RC_RUN_ID
RELEASE_ID
OpenPath SHA
OpenPath contract SHA256
OCI digests
Windows template tuple where present
The annotated tag identity must match those exact values.
Invariant 3 — production remains tag-only
Do not change production deployment into a branch push, manual mutable checkout or direct SSH command.
The canonical promotion entrypoint may create the annotated tag, but the production deploy itself must remain triggered by the immutable v* tag and use the existing #160/#162 semantics.
Invariant 4 — R remains explicit authority
Do not infer recovery authority from:
previous tag
previous main
latest known-good
candidate parent
PRODUCTION_RECOVERY_SHA remains an explicit independently validated authority.
The readiness preflight must prove it before tag creation.
Invariant 5 — staging and production share executor semantics
Staging and production should consume the same canonical release payload and the same hermetic deployment executor/state semantics.
They must not maintain separate implementations of release projection, switch, verification or transaction semantics.
Invariant 6 — deployment evidence is durable but bounded
Persist a small safe deployment ledger sufficient to answer:
What exact release was deployed?
From which RC/tag?
What was previous?
Which recovery authority was prepared?
Did the transaction COMMIT or ROLLBACK?
What immutable images/contract were live?
Which workflow run proves it?
current production tag creation/operator procedure;
scripts/deploy-production-remote.sh;
scripts/lib/deploy-production-runtime.sh;
production recovery authority/bundle scripts;
deployment-state / transaction helpers;
staging deployment scripts and helpers;
release evidence/promotion evidence tooling;
relevant deployment/release automation tests.
Do not modify OpenPath from this issue.
Do not deploy staging/production, create tags or release anything during implementation unless separately authorized.
Phase 1 — pre-tag production readiness
Create one canonical non-mutating production-readiness command/workflow callable before a production tag exists.
Repository conventions may determine whether this is primarily a script, reusable workflow, or both, but there must be one authoritative implementation.
Suggested conceptual interface:
production-readiness --rc-run-id <RC>
or equivalent.
It must resolve the exact RC and fail closed if any required prerequisite is unavailable.
The selected RC must be successful and promotion-eligible according to the current repository contract.
Do not select another RC automatically.
Staging eligibility
Require the exact release identity to have the staging evidence currently required by production promotion.
The preflight must prove whatever the current production workflow would later require from verify-staging-release-state.
Do not duplicate staging release-selection logic; reuse the existing canonical evidence parser/contract.
Production recovery authority
Before tag creation prove:
PRODUCTION_RECOVERY_SHA present
full lowercase SHA
R != C
R checkout/source resolvable
contract/source versions supported
exact recovery bundle packageable
recovery executor present
artifact/executor hashes valid
recovery authority preflight passes
Use the same recovery packaging/preflight code that production uses.
Do not create a second approximation of recovery readiness.
The preflight must catch a missing/invalid PRODUCTION_RECOVERY_SHA before tag creation.
Production configuration contract
Validate all configuration that can be safely checked without mutating production, including at minimum the mandatory variables/secrets/identities required by the deploy job.
Do not print secret values.
For secret-backed requirements, only report presence/absence and safe metadata.
Host/readiness contract
Where practical and non-mutating, run the same production host preflight/read-only connectivity checks used by the deploy path:
Do not emit generic exit 1 when a safe diagnostic can be provided.
Phase 2 — one canonical promotion entrypoint
Add one operator-facing promotion command/workflow that consumes an explicit RC and is the normal way to create a production tag.
Conceptually:
promote-production <RC_RUN_ID>
Required sequence:
resolve exact RC
↓
run Phase 1 production readiness
↓
render exact proposed identity
↓
require explicit operator approval where repository policy requires it
↓
create exactly one annotated v* tag
↓
push tag
↓
existing tag-only production workflow executes
Tag identity
Reuse the current canonical annotated-tag schema.
The entrypoint must not hand-build a divergent schema.
Before pushing, verify locally/in-run that the tag identity contains and matches the exact resolved values required by the current deploy workflow, including at least:
CLASSROOMPATH_SHA
RC_RUN_ID
RELEASE_ID
If additional canonical identity fields already exist, preserve them.
Version allocation
Use the repository's existing release version policy.
Do not silently retag an existing version.
Do not move existing tags.
If version allocation remains operator-supplied, validate uniqueness before creation.
If automated, make the allocation deterministic and collision-safe.
Failure semantics
Anything failing before the tag push must result in no tag.
Once the tag is pushed, the existing immutable production workflow remains authoritative.
Do not add logic that deletes a pushed production tag after a failed deploy.
Phase 3 — converge staging on the hermetic production executor
Refactor staging so it uses the same canonical deployment runtime/executor semantics proven by #162.
This does not mean production and staging share mutable state or credentials.
A post-boundary staging failure should produce the same truthful failure/recovery semantics appropriate to staging.
No duplicated runtime projection
There must be one canonical implementation/list for release runtime projection.
Do not keep separate production and staging copies that can drift.
Migration path
Remove or reduce the legacy npm run deploy:staging implementation only after the shared executor path is covered and proven.
A thin wrapper command may remain for operator UX, but it should call the canonical executor rather than implement separate deploy semantics.
Phase 4 — durable deployment ledger
Add a bounded safe deployment ledger independent of short-lived GitHub artifact retention.
Prefer the smallest repository-consistent storage mechanism.
Do not introduce a new external database solely for this issue unless already available infrastructure makes that clearly simpler.
One record per completed/failed production promotion is sufficient.
Minimum fields
Persist safe values such as:
timestamp
environment
tag
CANDIDATE_SHA
RC_RUN_ID
RELEASE_ID
OPENPATH_SHA
OPENPATH_CONTRACT_SHA256
PRODUCTION_RECOVERY_SHA
transactionId
phase/result
previous releaseId/SHA where canonical
current releaseId/SHA where canonical
OCI image digests
health result
ready result
rollback attempted
rollback result
GitHub workflow run ID/URL
Use current canonical names where they differ.
Security
Never persist:
secrets
tokens
Authorization/Cookie values
private keys
download refs
personalized installer payloads
full environment dumps
raw config containing credentials
Integrity
Ledger entries should be immutable/append-only in normal operation or otherwise tamper-evident according to existing repository conventions.
Do not allow a failed later job to rewrite an earlier successful deployment as a different identity.
Retention
Keep existing bounded debug artifacts as appropriate.
The ledger is the long-lived compact identity/audit summary; it is not a replacement for all debug artifacts.
Phase 5 — tests and regression coverage
Write failing tests before or alongside implementation for each new invariant.
Pre-tag readiness tests
Required cases:
exact valid RC passes;
missing RC fails;
wrong releaseId/C mismatch fails;
staging evidence for a different release fails;
missing PRODUCTION_RECOVERY_SHA fails before tag creation;
ROLLED_BACK promotion records rollback truthfully;
pre-boundary failure records no false production mutation;
no secrets included;
records cannot silently overwrite an unrelated deployment identity;
workflow run locator preserved.
Existing suites
Run at minimum the repository-current equivalents of:
npm run test:deployment
npm run test:release-automation
npm run test:ci-regression
npm test
verify:static / relevant static gates
shell syntax/shellcheck for modified shell
Follow AGENTS.md for exact required commands.
Operational proof
Implementation completion and operational adoption are separate.
Before claiming implemented
All local/CI tests above must pass and main must contain the implementation.
Staging proof
Only with explicit authorization, run one normal staging deployment through the new shared executor.
Prove:
exact selected release
COMMITTED
runtime/checkout/images exact
health 200
ready 200 / true
live identity exact
ledger entry exact
A deliberate staging fault is only required if the implementation changes recovery/failure semantics materially; do not repeat #162 K automatically for UI/refactoring-only changes.
Production proof
Do not deploy production as part of implementation without separate explicit authorization.
The first authorized production promotion using the new entrypoint should demonstrate:
pre-tag readiness PASSED
tag created only after readiness
normal tag-only deploy
COMMITTED
exact live identity
health/ready/smoke green
ledger record written
Do not deliberately inject production faults.
Acceptance criteria
One canonical non-mutating production-readiness preflight exists.
Missing/invalid recovery authority is detected before production tag creation.
Recovery bundle/package/preflight is exercised before production tag creation.
Exact RC/C/releaseId/OpenPath/contract/image identity is resolved before tagging.
Current staging eligibility is checked before tagging without duplicating release-selection logic.
Production config/host prerequisites that can be checked safely are verified before tagging.
Preflight failures produce clear blocker classification and no tag.
One canonical operator promotion entrypoint exists.
Promotion entrypoint runs readiness first and creates exactly one immutable annotated tag.
Production remains tag-only.
No latest, rebuild or alternative RC selection is introduced.
R remains explicit; it is never inferred automatically from previous/current releases.
Staging uses the same canonical hermetic deploy/runtime semantics as production.
Staging and production retain strict environment/authorization separation.
Runtime projection has one canonical implementation/list.
Staging state progression matches the proven transactional semantics where applicable.
Prefer deletion/convergence over adding another abstraction layer.
Examples:
GOOD:
existing release resolver + existing recovery authority + one readiness orchestrator
BAD:
new parallel release resolver
new recovery model
new deployment state machine
GOOD:
npm run deploy:staging -> thin wrapper -> canonical hermetic executor
BAD:
production executor + staging executor that merely happen to look similar
The goal after this issue is that a routine promotion is boring:
select RC
-> readiness green
-> create tag
-> deploy
-> COMMITTED
-> smoke
-> ledger
and an invalid promotion stops before an immutable production version is consumed.
Completion report required
The implementing agent must report separately:
PRE-TAG READINESS
command/workflow;
exact checks;
failure classifications;
proof that failure creates no tag.
PROMOTION ENTRYPOINT
exact invocation;
tag schema reuse;
version collision behavior;
no rebuild/latest behavior.
SHARED EXECUTOR
code paths removed/reused;
staging vs production environment adapters;
runtime projection ownership;
state transition evidence.
DEPLOYMENT LEDGER
storage location/format;
fields;
retention/integrity behavior;
secret exclusion tests.
TESTS
focused commands/results;
complete deployment/release suite results.
OPERATIONAL PROOF
staging only if separately authorized/executed;
production only if separately authorized/executed.
Do not collapse implementation completion into production proof.
Summary
Follow-up to #160 and #162 after the production path was proven successfully with
v1.2.379.The core release architecture is now considered correct and operationally proven:
This issue does not reopen #162 and does not introduce another round of generic release hardening.
The remaining goal is operational simplification:
The work is intentionally bounded to four improvements:
No blue/green, Kubernetes, GitOps migration, new state machine or automatic rollback selection is required.
Context
#160 established immutable Release Bundle v2 and exact release identity.
#162 established and operationally proved:
The successful production promotion of
v1.2.379proved the normal production path end-to-end.Do not weaken or redesign those invariants.
Operational friction still exists:
A concrete example was a promotion that failed pre-boundary because
PRODUCTION_RECOVERY_SHAwas not configured. Another pre-boundary failure exposed a workflow-only recovery bundle assertion defect.Both failed safely, but both should be discoverable before creating the production tag.
The current workflow also intentionally has an asymmetry:
Now that the production executor is proven, staging should converge on that same executor semantics so normal release traffic continuously exercises the production path.
Core invariants
Invariant 1 — no tag before production readiness
A production tag must not be created until a non-mutating readiness check proves all prerequisites required to start the production promotion.
Conceptually:
A failed readiness preflight must leave:
Invariant 2 — promotion remains exact and immutable
The preflight/promotion helper must not select
latest, rebuild artifacts, recompute OpenPath pins independently or choose an alternative release.It must consume one exact RC and resolve exactly:
The annotated tag identity must match those exact values.
Invariant 3 — production remains tag-only
Do not change production deployment into a branch push, manual mutable checkout or direct SSH command.
The canonical promotion entrypoint may create the annotated tag, but the production deploy itself must remain triggered by the immutable
v*tag and use the existing #160/#162 semantics.Invariant 4 — R remains explicit authority
Do not infer recovery authority from:
PRODUCTION_RECOVERY_SHAremains an explicit independently validated authority.The readiness preflight must prove it before tag creation.
Invariant 5 — staging and production share executor semantics
Staging and production should consume the same canonical release payload and the same hermetic deployment executor/state semantics.
Environment-specific differences may include:
They must not maintain separate implementations of release projection, switch, verification or transaction semantics.
Invariant 6 — deployment evidence is durable but bounded
Persist a small safe deployment ledger sufficient to answer:
Do not retain secrets or full runtime envs.
Phase 0 — read before editing
Read completely:
AGENTS.md;.github/workflows/deploy.yml;scripts/deploy-production-remote.sh;scripts/lib/deploy-production-runtime.sh;Do not modify OpenPath from this issue.
Do not deploy staging/production, create tags or release anything during implementation unless separately authorized.
Phase 1 — pre-tag production readiness
Create one canonical non-mutating production-readiness command/workflow callable before a production tag exists.
Repository conventions may determine whether this is primarily a script, reusable workflow, or both, but there must be one authoritative implementation.
Suggested conceptual interface:
or equivalent.
It must resolve the exact RC and fail closed if any required prerequisite is unavailable.
Required checks
Exact candidate identity
Resolve and verify:
The selected RC must be successful and promotion-eligible according to the current repository contract.
Do not select another RC automatically.
Staging eligibility
Require the exact release identity to have the staging evidence currently required by production promotion.
The preflight must prove whatever the current production workflow would later require from
verify-staging-release-state.Do not duplicate staging release-selection logic; reuse the existing canonical evidence parser/contract.
Production recovery authority
Before tag creation prove:
Use the same recovery packaging/preflight code that production uses.
Do not create a second approximation of recovery readiness.
The preflight must catch a missing/invalid
PRODUCTION_RECOVERY_SHAbefore tag creation.Production configuration contract
Validate all configuration that can be safely checked without mutating production, including at minimum the mandatory variables/secrets/identities required by the deploy job.
Do not print secret values.
For secret-backed requirements, only report presence/absence and safe metadata.
Host/readiness contract
Where practical and non-mutating, run the same production host preflight/read-only connectivity checks used by the deploy path:
Do not cross the production mutation boundary.
Do not stop/recreate containers.
Output
On success emit one bounded human-readable and machine-readable summary, conceptually:
On failure identify the concrete blocker class:
Do not emit generic
exit 1when a safe diagnostic can be provided.Phase 2 — one canonical promotion entrypoint
Add one operator-facing promotion command/workflow that consumes an explicit RC and is the normal way to create a production tag.
Conceptually:
Required sequence:
Tag identity
Reuse the current canonical annotated-tag schema.
The entrypoint must not hand-build a divergent schema.
Before pushing, verify locally/in-run that the tag identity contains and matches the exact resolved values required by the current deploy workflow, including at least:
If additional canonical identity fields already exist, preserve them.
Version allocation
Use the repository's existing release version policy.
Do not silently retag an existing version.
Do not move existing tags.
If version allocation remains operator-supplied, validate uniqueness before creation.
If automated, make the allocation deterministic and collision-safe.
Failure semantics
Anything failing before the tag push must result in no tag.
Once the tag is pushed, the existing immutable production workflow remains authoritative.
Do not add logic that deletes a pushed production tag after a failed deploy.
Phase 3 — converge staging on the hermetic production executor
Refactor staging so it uses the same canonical deployment runtime/executor semantics proven by #162.
This does not mean production and staging share mutable state or credentials.
They must remain isolated environments.
Desired model
The shared path should own:
Environment adapters/config should own:
Preserve production fences
Production-only protections must remain explicit and fail closed.
Do not make a staging environment capable of satisfying production identity accidentally.
The existing K environment fencing concepts should guide the separation, but do not introduce K-only coupling into normal staging.
Staging transaction model
Normal staging success should follow the same relevant state progression as production:
A post-boundary staging failure should produce the same truthful failure/recovery semantics appropriate to staging.
No duplicated runtime projection
There must be one canonical implementation/list for release runtime projection.
Do not keep separate production and staging copies that can drift.
Migration path
Remove or reduce the legacy
npm run deploy:stagingimplementation only after the shared executor path is covered and proven.A thin wrapper command may remain for operator UX, but it should call the canonical executor rather than implement separate deploy semantics.
Phase 4 — durable deployment ledger
Add a bounded safe deployment ledger independent of short-lived GitHub artifact retention.
Prefer the smallest repository-consistent storage mechanism.
Do not introduce a new external database solely for this issue unless already available infrastructure makes that clearly simpler.
One record per completed/failed production promotion is sufficient.
Minimum fields
Persist safe values such as:
Use current canonical names where they differ.
Security
Never persist:
Integrity
Ledger entries should be immutable/append-only in normal operation or otherwise tamper-evident according to existing repository conventions.
Do not allow a failed later job to rewrite an earlier successful deployment as a different identity.
Retention
Keep existing bounded debug artifacts as appropriate.
The ledger is the long-lived compact identity/audit summary; it is not a replacement for all debug artifacts.
Phase 5 — tests and regression coverage
Write failing tests before or alongside implementation for each new invariant.
Pre-tag readiness tests
Required cases:
PRODUCTION_RECOVERY_SHAfails before tag creation;R == Cfails;Promotion entrypoint tests
latestfallback.Use fixtures or an isolated test repository where appropriate; tests must not create real production tags.
Shared executor tests
Prove staging and production use the same canonical runtime/deploy helper for:
Add a regression that fails if staging reintroduces a divergent projection/switch implementation.
Test environment fences so staging cannot identify as production.
Ledger tests
Existing suites
Run at minimum the repository-current equivalents of:
Follow
AGENTS.mdfor exact required commands.Operational proof
Implementation completion and operational adoption are separate.
Before claiming implemented
All local/CI tests above must pass and
mainmust contain the implementation.Staging proof
Only with explicit authorization, run one normal staging deployment through the new shared executor.
Prove:
A deliberate staging fault is only required if the implementation changes recovery/failure semantics materially; do not repeat #162 K automatically for UI/refactoring-only changes.
Production proof
Do not deploy production as part of implementation without separate explicit authorization.
The first authorized production promotion using the new entrypoint should demonstrate:
Do not deliberately inject production faults.
Acceptance criteria
latest, rebuild or alternative RC selection is introduced.Non-goals
Do not in this issue:
main/branch pushes;Implementation guidance
Prefer deletion/convergence over adding another abstraction layer.
Examples:
The goal after this issue is that a routine promotion is boring:
and an invalid promotion stops before an immutable production version is consumed.
Completion report required
The implementing agent must report separately:
PRE-TAG READINESS
PROMOTION ENTRYPOINT
SHARED EXECUTOR
DEPLOYMENT LEDGER
TESTS
OPERATIONAL PROOF
Do not collapse implementation completion into production proof.