Persist lifecycle JSON atomically

[Anionix]

Fixes #475
Fixes #478
Fixes #479
Fixes #481
Fixes #482
Fixes #483
Fixes #484
Fixes #485
Fixes #487
Fixes #489
Fixes #490
Fixes #493
Fixes #494
Fixes #496

Contract

• Serialize the complete strict JSON document before creating or changing a destination.
• Write through a regular same-directory temporary file, handle short writes, flush, fsync, and call os.replace only after those steps succeed.
• Preserve the previous destination byte-for-byte on serialization, write, flush, and replacement failures.
• Reject destination symlinks and cross-directory temporary replacement.
• Preserve existing evidence permissions, honor the process umask for first
  writes, and anchor replacement to one retained directory descriptor.
• Keep the temporary file at 0600 through payload write and atomic
  publication, restore the intended final mode through the retained descriptor
  only after replacement, and retain failed temps rather than performing a
  racy name-based unlink.
• Publish only inside a directory owned by the effective user and not writable
  by group/other principals, and reject macOS extended ACLs; same-principal
  processes form the trust boundary.
• Create fresh run, input, and shard-output publication directories at 0700
  so collaborative umasks cannot violate that boundary.

Scope

• Add one typed atomic_write_json persistence primitive.
• Route workflow, fair, equivalence, profile, shard, and report lifecycle JSON through it.
• Preserve the existing sorted, indented UTF-8 JSON bytes and trailing newline.
• Keep stream output and write-once staging artifacts out of this change.

Verification

• TDD red: the atomic persistence tests failed to import before the primitive existed.
• Affected suites: test_json_contract.py, test_workflow.py, test_fair_run.py, test_equivalence_run.py, test_shards.py, and test_report.py all pass in one focused run.
• Ruff lint and git diff --check: PASS.
• CI found a same-directory path spelling with ..; the regression now
  canonicalizes and retains the destination directory identity, and the exact
  failing native-suite test passes.
• Review found mode drift and a parent-retargeting TOCTOU; deterministic
  regressions now cover both.
• A follow-up review found a first-write umask bypass; new temporary files now
  use descriptor-relative O_CREAT|O_EXCL|O_NOFOLLOW with requested mode
  0666, leaving the effective mode to the process umask.
• Exact-SHA review then found unsafe path fallback and stale post-publication
  ownership; absolute-path cleanup was removed and descriptor closure plus
  same-name replacement behavior gained deterministic regressions.
• GitHub review found a pre-write permission window; replacement temps are now
  created at the existing destination mode (still masked by umask), then
  restored to the exact mode before any payload write.
• Final adversarial review proved that stat followed by unlink cannot provide
  an atomic inode-bound cleanup guarantee. Name-based failure cleanup is now
  absent: failed temps remain 0600 while writing, completed replacement temps
  use the intended final mode, and unrelated same-name entries are never a
  cleanup target.
• GitHub review then found the analogous publish-by-name swap. Because portable
  Python/POSIX also lacks compare-and-rename, shared-writable destination
  directories are rejected before any temporary or destination mutation. A
  follow-up exact-SHA review found the macOS ACL bypass; Darwin now queries the
  extended ACL from the retained directory fd and conservatively rejects it.
• Two final review findings are covered by Create lifecycle directories with non-shared modes #489 and Keep completed atomic JSON temps private until publication #490: lifecycle directories
  request 0700 under umask 002/000, and complete temps remain 0600
  through replace. Run/input/artifact/observation and recursive shard output
  directories all request 0700. A mode-restoration failure leaves the new
  document at 0600; a later fsync failure leaves it at the intended mode.
• Create missing default run ancestors privately #493 covers the missing-parent case: the default root/runs ancestor is
  created explicitly at 0700 before the private run leaf.
• Create missing shard-output ancestors privately #494 applies the same invariant to every missing shard-output ancestor,
  creating each level explicitly from the nearest existing parent.
• Reject race-created shard output destination #496 restores the final output leaf's exclusive-create boundary after the
  recursive ancestor change, rejecting a destination created between initial
  validation and publication.
• Focused final gate: 16 shard tests, Ruff, and git diff --check: PASS.
• Exact final SHA: 210f0e65df0b015c7400a818190627fce1a5603e.
• Targeted exact-SHA correctness/scope review: in progress; formal role remains
  NO_LEAN with native executable linkage.

Formal boundary

This is an executable filesystem contract, not a Lean theorem. The tests connect
the lifecycle invariant to Python and OS behavior. The code claims an atomic
rename boundary after successful os.replace; it does not claim directory
metadata or power-loss durability.

Primary sources

• Python os.replace: https://docs.python.org/3/library/os.html#os.replace
• Python os.open: https://docs.python.org/3/library/os.html#os.open
• Python os.fchmod: https://docs.python.org/3/library/os.html#os.fchmod
• Python os.fsync: https://docs.python.org/3/library/os.html#os.fsync
• Apple acl_get_fd: https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/acl_get_fd.3.html

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a8b2a09c7b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f3f4815e85

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c6c7bfd0b9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cccf8babc2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e848e725d6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5ef7c1ffe5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7a1be7cad7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 05e72a7a68

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4f1c84efdb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".