From df7a0163ce4f217533830eb98a7e3d21883f2aca Mon Sep 17 00:00:00 2001 From: OpenCode Date: Fri, 10 Jul 2026 11:54:44 +0700 Subject: [PATCH] docs: Phase J J2 schema version migration guide -- MAJOR/MINOR/PATCH conventions, field add/remove/rename procedures, agent rules, migration notes table (#913) --- docs/SCHEMA_MIGRATION_GUIDE.md | 154 +++++++++++++++++++++++++++++++++ 1 file changed, 154 insertions(+) create mode 100644 docs/SCHEMA_MIGRATION_GUIDE.md diff --git a/docs/SCHEMA_MIGRATION_GUIDE.md b/docs/SCHEMA_MIGRATION_GUIDE.md new file mode 100644 index 00000000..6bb5f7de --- /dev/null +++ b/docs/SCHEMA_MIGRATION_GUIDE.md @@ -0,0 +1,154 @@ +# Schema Version Migration Guide + +This document explains how to update OpenAMP evidence schemas without breaking existing artifacts. + +An artifact is any JSON file or Python dict that was serialized under a specific schema version. Once an artifact exists — in a results directory, a partner review packet, or an external database — its field names and validation rules are frozen. The schema may evolve; the artifact cannot. + +## Versioning conventions + +All OpenAMP schemas follow this numbering: + +``` +vMAJOR.MINOR.PATCH +``` + +| Increment | Meaning | Artifact impact | +|-----------|---------|----------------| +| MAJOR | Breaking change — field removed, type changed, validation rule tightened | Existing artifacts fail new validation | +| MINOR | Additive change — new optional field, new controlled vocab term | Existing artifacts remain valid | +| PATCH | Non-structural change — docstring, comment, formatting | No impact | + +A schema's version is stored in its `$schema_version` field (for Python modules) or `$schema` / `"version"` field (for JSON schemas). + +## Adding a field (MINOR) + +1. Add the field with a default value or mark it optional. +2. Bump the MINOR version in the schema file. +3. Do not add the field to `REQUIRED_FIELDS` until the next MAJOR release. +4. Update the `validate_*()` function to accept artifacts that omit the field. +5. Add tests for both the old artifact shape (without the field) and the new shape (with the field). + +Example: adding `synthesis_priority` to a batch outcome record: + +```python +# Before (v1.0.0) +@dataclass +class BatchOutcomeSummary: + batch_id: str + total_candidates: int + n_hits: int + +# After (v1.1.0) — MINOR bump, field is optional +@dataclass +class BatchOutcomeSummary: + batch_id: str + total_candidates: int + n_hits: int + synthesis_priority: str = "" # new optional field +``` + +## Removing a field (MAJOR) + +Field removal is a breaking change. Follow this two-step process: + +**Step 1 — deprecate (current release):** + +```python +@dataclass +class OldSchema: + legacy_field: str = "" # DEPRECATED since v2.0. Use new_field instead. + new_field: str = "" +``` + +Validation must continue to accept `legacy_field` without error. + +**Step 2 — remove (next release):** + +- Delete the field. +- Bump MAJOR version. +- Add a migration note to this document (see "Migration notes" below). +- Update `validate_*()` to reject or ignore the removed field. + +## Renaming a field (MAJOR) + +Renaming looks like removal + addition. Follow the same two-step deprecation process: + +1. Add the new name alongside the old name. +2. In `validate_*()`, accept either name but warn when the old name is present. +3. In the next MAJOR release, remove the old name. + +Do not rename a field in a PATCH or MINOR release. + +## Tightening a validation rule (MAJOR) + +Examples of tightening: +- Reducing the allowed vocabulary (`VALID_OUTCOMES` loses a term) +- Changing a field from optional to required +- Reducing a minimum length or count + +Tightening is always a MAJOR bump because existing artifacts may fail the new rule. + +Before tightening, audit all artifacts in `outputs/` and `tests/` for the old value. If any existing artifact uses the soon-to-be-invalid value: +1. Migrate it. +2. Open a PR for the migration. +3. Merge the migration PR before the schema PR. + +## Loosening a validation rule (MINOR) + +Examples of loosening: +- Adding a term to a controlled vocabulary +- Making a required field optional +- Increasing an allowed maximum + +Loosening is safe; existing artifacts remain valid. + +## Schema ID stability + +Every schema has a stable `$id` or identifier prefix (e.g., `NRR-`, `BMC-`, `FAE-`). These IDs appear in external artifacts and cannot be changed without breaking cross-references. + +Rules: +- Never change a schema `$id`. +- Never change a schema prefix (e.g., `FAE-` → `FASTA-`). +- Never rename a module at import path level without leaving a deprecated shim. + +If a schema is so broken that its `$id` must change, create a new schema alongside the old one and deprecate the old one following [DEPRECATION_POLICY.md](DEPRECATION_POLICY.md). + +## Migration notes + +Append a row to this table whenever a MAJOR version bump removes or renames a field. + +| Schema | Old version | New version | Removed/renamed field | Migration action | +|--------|-------------|-------------|----------------------|-----------------| +| (none yet) | — | — | — | — | + +When filling in a migration row: +- **Schema**: the module name or schema file, e.g., `negative_result_record.py` +- **Old version**: the version before the breaking change +- **New version**: the version after +- **Removed/renamed field**: the old field name +- **Migration action**: what callers must do, e.g., "rename `legacy_id` to `record_id` in all NRR- artifacts" + +## Testing migrations + +Every MAJOR schema change must include: + +1. A test that validates an artifact in the *old* format and confirms it now fails validation (the change is real). +2. A test that validates the migrated artifact and confirms it now passes. +3. An updated `test_test_count_regression.py` BASELINE. + +Do not merge a MAJOR schema change without both tests. + +## Agent rules + +An agent MUST NOT: +- Bump the MAJOR version without a corresponding migration note in this document. +- Change a schema `$id` or controlled-vocabulary term without human review. +- Remove a field from a schema without first deprecating it in a prior PR. + +An agent MAY: +- Add optional fields (MINOR bump). +- Add terms to controlled vocabularies (MINOR bump). +- Write migration tests. +- Append a row to the migration notes table when removing a deprecated field. + +When in doubt about whether a change is MAJOR or MINOR, treat it as MAJOR and ask for human review.