fix(store): a not-determined marker no longer overwrites a determination

[sotashimozono]

Refs #583. The issue is corrected in a comment there — its central claim was wrong, and this fixes what is actually broken.

The permission was conditional, and the condition stopped holding

docs/STORE.md §6 lets doiget rewrite the [doiget] table on a re-fetch:

A doiget re-fetch of an entry that previously had a PDF but is now metadata-only ... rewrites the [doiget] table (source, size_bytes, …) in place. This is intentional, not silent: as of issue #118 the blocked-PDF reason is surfaced to the caller ... so the operator always learns the entry was downgraded and why.

Since #539, metadata_only only runs the OA lookup when include_oa_location is set. Omit it — the ordinary call shape — and the record carries oa_status: None and license: "unknown". Those won, and the caller got no note: line, no pdf.status, no log row. The clause the permission rests on was not met.

What the fix rests on

Both values are markers, and the schema already says so: oa_status is "omitted when not determined (#281)", license is "an OA license string, or the literal unknown".

• a paper that genuinely stops being open access reports oa_status: Some("closed")
• a license that genuinely changes reports the new string

Neither reports the marker. So a call carrying None / "unknown" did not look, and preferring the stored value is not a guess about which is newer — only one of the two is a value.

merge_metadata's [doiget] arm keeps the stored value in exactly that case. Everything else in the table still follows §6. "unknown" gained a name (LICENSE_UNDETERMINED) so the check does not hang off a bare literal matching the others scattered through the orchestrator.

I got the issue wrong, and measured instead of re-reading

I filed #583 from build_metadata_only_metadata and stopped before FsStore::write, which reads the existing file and merges. Probed it:

field	existing	after a default re-write
url (where oa_url lands)	Some("https://example.org/paper.pdf")	Some(…) — preserved
[doiget].oa_status	Some("gold")	None
[doiget].license	"CC-BY-4.0"	"unknown"

merge_opt!(url) already protected oa_url. So "the store file can be overwritten with nulls" was not true of the field the title named, and the store-partition / schema-change options the issue proposed were sized against a problem that is not there. Only the two [doiget] fields were affected; the fix is a merge rule.

Tests

• a_default_rewrite_does_not_downgrade_a_known_oa_status_or_license — the preserve, and it pins url too so the two halves of A default metadata-only re-write silently downgrades [doiget].oa_status and .license to their not-determined markers #583 cannot drift apart
• a_rewrite_that_determined_a_new_oa_status_or_license_still_wins — the other half; a guard that suppressed gold → closed would be worse than the bug

Mutation-checked: with the [doiget] arm reverted, the first fails on left: None, right: Some("gold").

Also restored a #[test] attribute I had knocked off merge_metadata_preserves_existing_arxiv_categories while working here — clippy's dead-code lint caught it, but a test that silently stops running is the same class of defect #462 was about.

Docs

ADR-0056, and docs/STORE.md §6's note is amended to scope its claim to determinations rather than markers — a NORMATIVE change, hence the ADR (ADR-0014).

Nothing BiblioFetch.jl reads changes shape: no field added, removed or renamed, schema_version unmoved, reserved top-level fields keep the merge_opt! behaviour they had.

Verified locally

clippy -D warnings clean at oa-only,citation; doiget-core suite green (482 tests).

[codecov]

Codecov Report

❌ Patch coverage is 97.56098% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
crates/doiget-core/src/store/fs_store.rs	97.56%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!