feat(record): QuantityRecord, the inert interchange form (#82)

devguide/pending_proposals/README.md

@@ -5,6 +5,10 @@ Edit report front matter and regenerate this index; do not edit the generated bl
 
 <!-- generated: devguide_index -->
 
+### Active (1)
+
+- [`quantity_record.md`](quantity_record.md) — [#82](https://github.com/uibcdf/pyunitwizard/issues/82) — QuantityRecord — an inert interchange form that never loses or misreads a unit (provisional MVP implemented). *(measured)*
+
 ### Open (2)
 
 - [`forallpeople_integration.md`](forallpeople_integration.md) — [#44](https://github.com/uibcdf/pyunitwizard/issues/44) — Decide whether forallpeople should become a supported quantity form. *(inspected)*

devguide/pending_proposals/quantity_record.md

@@ -0,0 +1,136 @@
+---
+summary: QuantityRecord — an inert interchange form that never loses or misreads a unit (provisional MVP implemented).
+issue: uibcdf/pyunitwizard#82
+status: active
+opened: 2026-09-24
+closed:
+verification: measured
+area: [serialization, interoperability, forms]
+guard:
+normative:
+blocked_by: []
+supersedes: []
+---
+
+# QuantityRecord: PyUnitWizard's inert interchange form
+
+## What
+
+A form, `"record"`, whose objects (`QuantityRecord`) carry values together with a verified
+description of their unit, so that a number is never read in a unit it was not written in.
+It is inert like `"string"`: it converts to and from every form and does not compute.
+`QuantityRecordBundle` seals many small quantities of one document (a card) together.
+
+The design, its motivation, the alternatives evaluated (per-value `{value, unit}`, unit
+strings, pint tuples, unenforced container manifests, ASDF, CF/UDUNITS, OpenFF, HDF5 and
+Parquet checksums, UCUM, QUDT, UO, D-SI) and the prior art (ArgDigest's removed passport
+and declined value certification) are recorded in uibcdf/pyunitwizard#83. It supersedes
+`devguide/serialization_contract_draft.md`, whose safety rules it meets and whose
+promotion gate (two real integration cases) is met by uibcdf/sabueso#32 and
+uibcdf/molsysmt#240.
+
+## How
+
+Implemented in this MVP (`pyunitwizard/record.py`, `forms/api_record.py`):
+
+- **Manifest**: `format` (`qrec/0.3`), `field` (or null), `unit` (canonical name: pint's
+  default long form, whatever display format a registry is configured with), `si`
+  (`factor`, `offset`, `exponents` over `m, kg, s, A, K, mol, cd`, with QUDT's semantics),
+  optional `kind` (quantity-kind IRI), `dtype`, `shape` and `blocks`.
+- **Seal**: blake2b-128 per block, and a record digest over the canonical manifest and the
+  block digests. The manifest is encoded with a tagged little-endian binary encoding,
+  because JSON text is not canonical across languages; the values are C-order
+  little-endian bytes with NaN canonicalized.
+- **Reading**: `from_dict` refuses a missing manifest (there is no default unit), an
+  unknown format, a malformed dtype or shape, a broken seal, or a unit whose canonical name
+  disagrees with its SI description beyond a relative 1e-6. `to_quantity(field=, unit=,
+  dimensionality=, kind=, form=)` is the handshake.
+- **Encodings**: strict JSON, which refuses NaN and infinity because other languages
+  cannot read them, and base64.
+- **Form integration**: `get_form` detects records. `convert` delegates whenever the
+  source or the target is `"record"`. The form's dispatch entries make `get_value`,
+  `get_unit`, `get_dimensionality`, `change_value` and `quantity(..., form="record")` work.
+  Records never import a backend at load time.
+- **Diagnostics**: `RecordError` (`PUW-ERR-REC-001`) in the SMonitor catalog.
+- **Third parties**: `_private/record_reference_reader.py` reads and verifies records with
+  the Python standard library only. `tests/quantity_record_vectors/` holds frozen vectors,
+  with SI values written by hand, and `devtools/generate_quantity_record_vectors.py`
+  regenerates them.
+
+Deferred, each to be justified by a consumer case:
+
+- the HDF5 binding with a CF `units` attribute (needed by uibcdf/molsysmt#240);
+- the tagged layout for mixed units (TaggedQuantities; #83);
+- UCUM and CF spellings derived with real parsers (#85);
+- Arrow and Zarr bindings;
+- growing records by appended blocks;
+- OpenFF (#86);
+- foreign pint registries (#84);
+- a live, computing form (#87).
+
+## Why
+
+A unit lost at a boundary produces a wrong number with no error: the Mars Climate Orbiter
+case, uibcdf/molsysviewer#96 (10× under a user Å policy), and the latent H5MSM mechanisms of
+uibcdf/molsysmt#240. PyUnitWizard owns units across MOLI and MolSysSuite
+(uibcdf/moli#13, uibcdf/molsyssuite#46), so one implementation here prevents every
+consumer from inventing a format.
+
+## What is measured and what is assumed
+
+Measured with the prototype (#82, `qrec/0.2`; Python 3.13, pint 0.25.3):
+
+- JSON base64: 1.33× raw size; 100k values written in 5 ms and read in 4 ms.
+- A bundle holds 50 scalars at 166 B each, against 347 B for separate records.
+- The digest runs at about 0.3–0.6 GB/s.
+- 20 deliberate slips were refused across records, HDF5 and bundles.
+- CF strings were validated with cf-units 3.3.1.
+
+Measured with this implementation:
+
+- 58 tests: exact round trips (NaN, −0.0, float32, int32, n-D, compound and affine
+  units); 11 slips refused; a resealed lie caught by the SI redundancy; the CODATA
+  tolerance; records independent of the session policy; canonical spelling independent of
+  pint's display format; the form's integration; bundles; published vectors reproduced
+  byte for byte; the reference reader agreeing.
+- Full suite: 626 passed.
+
+Assumed: the 1e-6 tolerance separates definition drift (≤ ~7e-7 observed between UDUNITS
+and pint) from different units. It must be re-measured if a consumer meets a larger
+legitimate difference.
+
+## What was refuted
+
+- **Per-value `{value, unit}` for collections**: 4.3× the size, 5× the load time.
+- **A container manifest without a seal**: this is the H5MSM failure mode.
+- **Container-native checksums** (Fletcher32, Parquet CRC, ASDF MD5): any writer
+  recomputes them.
+- **A global unit-code catalogue**: codes that change meaning between versions misread old
+  files.
+- **A separate TaggedQuantities class**: two classes for one concept diverge, as the
+  passport showed.
+
+## Scope and exclusions
+
+- Protection against deliberate falsification needs signatures and is out of scope.
+- A writer that is consistently wrong (meant nM, wrote pM) cannot be detected by any
+  format. That is covered by API contracts (ArgDigest), cross-tool canaries and
+  conformance tests under a non-default session policy.
+
+## Acceptance criteria
+
+- MVP: `record` form, bundles, strict JSON and base64 encodings, seal, handshake, no
+  defaults, `RecordError`, reference reader, vectors and user documentation. Done in this
+  change.
+- Closure of #82: the deferred items above are either implemented or split into their own
+  issues; Sabueso (#32) and one MolSysSuite member consume the form; the 1.0 checklist
+  decides between promotion and continued provisional status. The guard at closure will be
+  `tests/test_quantity_record.py::test_every_change_outside_the_codec_is_refused`.
+
+## Dependencies and risks
+
+- **The format may change before promotion.** Consumers pin the `qrec/<version>` they
+  read, and the vectors make any change visible.
+- **pint is required to build or read records into quantities.** Verification alone needs
+  only hashlib (see the reference reader).
+- **The first record in a process builds pint's registry** (0.3–0.5 s).

devguide/release_1.0.0_checklist.md

@@ -40,6 +40,7 @@ Use these documents as the primary evidence sources while closing each section:
 - [ ] Transparent frontend mode contract is frozen and test-backed (`devguide/frontend_transparent_mode_contract.md`).
 - [ ] Legacy `pyunitwizard.main` deprecation contract is tested and documented.
 - [ ] No open blockers in conversion, standardization, parsing, or compatibility workflows.
+- [ ] Provisional `QuantityRecord` / `QuantityRecordBundle` API and `qrec` format (#82): either promoted (frozen, contract-tested, format version fixed) or explicitly kept provisional in the release notes. Admitted as a provisional surface on 2026-09-24 under the change-control policy of `api_freeze_pre_1.0_decision.md`.
 
 ## 3. Diagnostics and observability
 
@@ -61,7 +62,7 @@ Use these documents as the primary evidence sources while closing each section:
 - [ ] Linux and macOS build and installation workflows are green (`sdist`, wheel, conda).
 - [ ] Release artifacts do not claim or publish Windows support.
 - [ ] README and devguide documents are synchronized with shipped behavior.
-- [ ] Serialization draft status is explicit in release notes (`devguide/serialization_contract_draft.md`: draft/non-stable unless promoted).
+- [ ] Serialization status is explicit in release notes: `devguide/serialization_contract_draft.md` is superseded by `QuantityRecord` (#82), whose API and `qrec` format are provisional unless promoted.
 - [ ] Tag/release notes prepared with migration notes (if any).
 
 ## 6. Final go/no-go

devguide/serialization_contract_draft.md

@@ -1,6 +1,13 @@
 # Serialization Contract Draft (Post-1.0 Candidate)
 
-Status: draft for future promotion. Not a committed public API yet.
+Status: **superseded** (2026-09-24) by `QuantityRecord`, the inert interchange form
+(`pyunitwizard/record.py`; issue uibcdf/pyunitwizard#82; design record #83; report
+`pending_proposals/quantity_record.md`). Kept as history. Its safety rules are all met
+by the new design: versioned format (`qrec/0.3`), dimensional validation on read, explicit
+errors for missing or malformed fields, and no implicit backend import. Its promotion
+gate is met by two real integration cases: uibcdf/sabueso#32 and uibcdf/molsysmt#240. The
+`form` field of the payload below was dropped: a record is independent of the backend it
+came from.
 
 ## Goal
 

devtools/generate_quantity_record_vectors.py

@@ -0,0 +1,63 @@
+"""Regenerate the published QuantityRecord test vectors (tests/quantity_record_vectors/).
+
+The vectors freeze the format: tests assert that the library reproduces them byte for
+byte, and third parties use them to check their own readers. The expected SI values are
+written by hand here, independently of the implementation. Regenerate only with a format
+version change, and say so in the changelog.
+"""
+
+import json
+from pathlib import Path
+
+import numpy as np
+
+import pyunitwizard as puw
+from pyunitwizard.record import QuantityRecord
+
+FOLDER = Path(__file__).resolve().parents[1] / "tests" / "quantity_record_vectors"
+
+CASES = {
+    "scalar_concentration_with_kind": (
+        puw.quantity(3.0, "nM", form="pint"),
+        "bioactivity.ic50",
+        "http://qudt.org/vocab/quantitykind/AmountOfSubstanceConcentration",
+        [3.0e-6],  # 3 nmol/L = 3e-9 mol / 1e-3 m^3
+    ),
+    "coordinates_float32_2d": (
+        puw.quantity(np.array([[0.5, 1.0, 1.5], [2.0, 2.5, 3.0]], dtype=np.float32), "nm", form="pint"),
+        "structures.coordinates",
+        None,
+        [0.5e-9, 1.0e-9, 1.5e-9, 2.0e-9, 2.5e-9, 3.0e-9],
+    ),
+    "counts_int32_dalton": (
+        puw.quantity(np.array([180, 18], dtype=np.int32), "dalton", form="pint"),
+        "properties.molecular_weight",
+        None,
+        [180 * 1.66053906892e-27, 18 * 1.66053906892e-27],
+    ),
+    "affine_celsius": (
+        puw.quantity(np.array([20.0, 37.0]), "degC", form="pint"),
+        "assay.temperature",
+        None,
+        [293.15, 310.15],
+    ),
+    "compound_unit": (
+        puw.quantity(np.array([1.0e-3]), "N*s/m**2", form="pint"),
+        "fluid.viscosity",
+        None,
+        [1.0e-3],
+    ),
+}
+
+
+def main() -> None:
+    FOLDER.mkdir(parents=True, exist_ok=True)
+    for name, (quantity, field, kind, si_values) in CASES.items():
+        record = QuantityRecord.from_quantity(quantity, field=field, kind=kind)
+        case = {"record": record.to_dict(), "si_values": si_values}
+        (FOLDER / f"{name}.json").write_text(json.dumps(case, indent=2, sort_keys=True) + "\n", encoding="utf-8")
+        print("wrote", name)
+
+
+if __name__ == "__main__":
+    main()

docs/api/users/api_user.rst

@@ -53,6 +53,19 @@ Core Introspection and Extraction
    pyunitwizard.get_value_and_unit
    pyunitwizard.change_value
 
+Interchange Form (provisional)
+------------------------------
+
+Quantity records carry values with a verified description of their unit, for storage
+and exchange between tools. See :doc:`../../content/user/quantity-records`. The API is
+provisional until the 1.0 checklist promotes it.
+
+.. autosummary::
+   :toctree: autosummary
+
+   pyunitwizard.QuantityRecord
+   pyunitwizard.QuantityRecordBundle
+
 Standardization and Context
 ---------------------------
 

docs/content/user/index.md

@@ -29,6 +29,11 @@ In this route you will learn how to:
 - diagnose common failures quickly,
 - communicate reproducible unit issues to library maintainers.
 
+## Storing and Exchanging Quantities
+
+To write quantities to files, databases or messages without ever losing or misreading
+their unit, read [Quantity Records](quantity-records.md).
+
 Both routes converge on the same objective: unit-safe workflows across
 scientific Python libraries without hidden assumptions.
 
@@ -68,6 +73,7 @@ numpy-pandas-matplotlib.md
 quickstart.md
 mini-library-walkthrough.md
 configuration.md
+quantity-records.md
 integrating-your-library.md
 troubleshooting.md
 production-checklist.md

docs/content/user/quantity-records.md

@@ -0,0 +1,103 @@
+# Quantity Records
+
+A number without its unit is the most dangerous value in a scientific pipeline. One tool
+stores *3 nanomolar*; the next reads *3 picomolar*; nothing fails and the result is simply
+wrong. A dimension check cannot catch it, because both are concentrations. Only a unit
+that never leaves its values, and a reader that never assumes one, prevent it.
+
+`QuantityRecord` is PyUnitWizard's form for storing and exchanging quantities. It is
+**inert**, like the `"string"` form: it converts to and from every other form, but it does
+not compute. To compute, convert it to pint, openmm.unit, astropy.units or unyt.
+
+```{note}
+Provisional API. `QuantityRecord` and the `qrec/0.3` format may still change before the
+1.0 checklist promotes them. The design, the alternatives evaluated and the measurements
+are recorded in [uibcdf/pyunitwizard#83](https://github.com/uibcdf/pyunitwizard/issues/83).
+```
+
+## Writing and reading
+
+```python
+import json
+import pyunitwizard as puw
+from pyunitwizard import QuantityRecord
+
+ic50 = puw.quantity([3.0, 12.5, 0.4], "nM")
+
+record = QuantityRecord.from_quantity(ic50, field="bioactivity.ic50")
+text = json.dumps(record.to_dict())          # store or send it
+
+record = QuantityRecord.from_dict(json.loads(text))   # verified, or refused
+q = record.to_quantity(field="bioactivity.ic50", unit="uM")
+```
+
+`record` is also a form, so the generic API works on it:
+
+```python
+record = puw.convert(ic50, to_form="record")
+puw.get_form(record)                       # 'record'
+puw.get_unit(record)                       # 'nanomolar'
+puw.convert(record, to_form="openmm.unit", to_unit="uM")
+```
+
+## What a record guarantees
+
+| Guarantee | How |
+|---|---|
+| The unit travels with the values | Every record holds its unit's canonical name *and* its description in SI base units (factor, offset, exponents). Readers check that both agree. |
+| No change goes unnoticed | A blake2b digest covers the unit description and the values' bytes. Only `QuantityRecord` produces it, so a hand edit, a raw append, reordered or truncated values, or a renamed unit all fail on read. |
+| The reader says what it expects | `to_quantity(field=..., unit=..., dimensionality=..., kind=...)` refuses a record bound to another field, of another dimension, or of another kind. |
+| No defaults | A record without its manifest is refused. There is no fallback unit, ever. |
+| Independent of the session | A record states its own unit; the session's standard units never change what it means. |
+| Readable without PyUnitWizard | A reader needs a hash function, JSON and IEEE-754 floats. `pyunitwizard/_private/record_reference_reader.py` does it with the Python standard library alone, and `tests/quantity_record_vectors/` holds test vectors for readers in other languages. |
+
+What no format can detect is a writer that is wrong *and* consistent: it meant nM, built pM,
+and wrote pM. Contracts at your API boundaries and tests that compare values across tools
+cover that case.
+
+## Kinds
+
+Some different quantities share SI dimensions: hertz and becquerel, joule and newton-metre.
+Record the kind when it matters, and ask for it when reading:
+
+```python
+rate = QuantityRecord.from_quantity(puw.quantity(1.0, "1/s"), field="rate",
+                                    kind="http://qudt.org/vocab/quantitykind/Frequency")
+rate.to_quantity(kind="http://qudt.org/vocab/quantitykind/Frequency")
+```
+
+## Many small values: bundles
+
+A document such as a data card holds many scalars. A `QuantityRecordBundle` seals them
+together, at about half the size of one record each:
+
+```python
+from pyunitwizard import QuantityRecordBundle
+
+card = QuantityRecordBundle.from_quantities({
+    "molecular_weight": puw.quantity(180.16, "dalton"),
+    "tpsa": puw.quantity(63.6, "angstrom**2"),
+})
+data = card.to_dict()
+values = QuantityRecordBundle.from_dict(data).to_quantities(
+    expected={"molecular_weight": {"[M]": 1}, "tpsa": {"[L]": 2}}
+)
+```
+
+## Encodings
+
+- `to_dict(encoding="json")` writes readable lists. It refuses NaN and infinity, which JSON
+  cannot hold, so that every JSON reader, not only Python's, can load the file.
+- `to_dict(encoding="base64")` writes the values' little-endian bytes (about 1.33 times
+  their raw size), holds any value exactly, and is faster for large arrays.
+
+Bindings for HDF5 (with a CF-compatible `units` attribute), Arrow and Zarr, and a layout
+for collections that mix units, are planned; see
+[uibcdf/pyunitwizard#82](https://github.com/uibcdf/pyunitwizard/issues/82).
+
+## Not a passport
+
+PyUnitWizard's integration guide lists "passports" as an anti-pattern: registries or caches
+that remember a quantity was once canonical, so that a check can be skipped. A quantity
+record is the opposite. It remembers nothing about live objects, and it is verified, never
+trusted, every time it is read.

pyunitwizard/__init__.py

@@ -69,6 +69,9 @@ def __print_version__() -> None:
     "to_string": (".api", "to_string"),
     "unit": (".api", "unit"),
     "context": (".api", "context"),
+    # Inert interchange form (provisional until the 1.0 checklist promotes it; #82)
+    "QuantityRecord": (".record", "QuantityRecord"),
+    "QuantityRecordBundle": (".record", "QuantityRecordBundle"),
 }
 
 __all__ = sorted(list(_LAZY_ATTRIBUTES.keys()))