restore and verify: a revision recomputed from itself, or not claimed (0.8.11)

[sotashimozono]

This is the Archeion half of making a revision rebuildable from what it holds. The DataVault half is QAtlasHub/DataVault.jl#58. The design and the measurements behind it are in Vault/infra/REGISTRY-REVIEW-ROUND3.md §8.

What it adds

• Archeion.restore(rev, dest) lays out the study, developed packages, depot packages, artifacts and environment from repro/ alone. Nothing is resolved or downloaded.

  • Each depot package and artifact is checked against the tree its Manifest pins, using git_tree_hash. That function is stdlib only, and a test checks it against git write-tree.
  • Points computed from more than one source state are refused rather than restored as one.

• Archeion.verify(rev; dest) runs the restored study:

  • with the restored depot only, an empty HOME and Pkg offline;
  • with the network namespace unshared (unshare -rn) where the host allows it;
  • with the recorded Julia binary, checked by digest, and the recorded BLAS thread count.

  It compares each point's result file with result_sha256. Only when every point matches does it write a capability.verified event, which records depot, network, host, julia_version, executable_sha256, threads and blas_threads.

• deposit accepts a study outside git. In that case it omits source (§5.1), and refuses when no provenance would hold the code. It also warns when the source is dirty.

• CLI: julia -m Archeion restore|verify.

• SPEC: §5.5 and §6 now list the optional observation and event fields. These are additions allowed in registry/2.

Measured

On panza, with an empty depot, an empty HOME and an unshared network, a real 5-point WilsonNRG study was verified against its own revision:

• all 5 result files matched byte for byte;
• 21 depot packages and 1 artifact matched their pins.

Eight of those packages matched only after removing *.jl.<pid>.cov files, which coverage runs had written into the local depot after install. restore removes them only when a tree would otherwise fail its pin, and says so. It does not strip them up front, because DataVault's own main tree has such files committed.

Local check

Only test/test_restore.jl (4 testsets) and test/test_provenance.jl (12 testsets, including the new "study outside git") were run; all pass. The full suite is left to CI.

After review (ed7ade1)

• Untrusted revisions: restore now refuses a revision whose token, snapshot id, root name (package name, uuid, tree) or file path would leave the restore, meaning an absolute path or a .. segment. validate does not read a snapshot's rows, so a registry it accepts could carry such a revision. A symlink whose target leaves its root is not made, and is listed in missing.
• Distributed runs: verify compares every point across all observations of the restored source. A sweep run across processes has one observation per process, and before this fix only one process's points were compared. A point that cannot be compared is listed in excluded and withholds the event. The event now records observations, not_held and julia_as_recorded.
• Non-git studies: deposit decides whether the code is recorded from what repro/ actually holds, not from source_contents alone. If an observation is missing from the store, the deposit is refused instead of writing a README that says the code is held.
• Tests: each of the above has a hostile or failing fixture. Also tested: a depot package at its slug, a depot package against a wrong pin, and a Julia binary that is not the recorded one.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 92.68293% with 18 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/restore.jl	91.08%	18 Missing ⚠️

📢 Thoughts on this report? Let us know!