From 2018785a4fdd6f57ff9d9d39335bcb88eaf259c9 Mon Sep 17 00:00:00 2001 From: DX23876 <176692557+DX23876@users.noreply.github.com> Date: Thu, 17 Sep 2026 12:18:23 +0200 Subject: [PATCH] chore(tools): retire the Swift/Kotlin parity-governance stack, drop the Android steps-locale test, fix a stale brand-phrase assertion MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The fork retired the cross-platform parity contract when `android/` was removed on 2026-08-14 (docs/FORK_GUIDE.md: "It does not bind here"), but the tooling that enforces exactly that contract was never retired alongside it. It kept absorbing upstream's daily re-derivation churn on every sync while silently failing here, because its own `KOTLIN_GLOBS` point at `android/app/src/main/java/com/noop/...` — a directory this fork doesn't have. **Removed wholesale**, confirmed to have no callers or references outside itself (`Tools/parity_ledger.py`, `Tools/parity_ratchet.py`, `Tools/issue_ref.py`, `Tools/parity_dispositions.json`, `Tools/parity_ledger_baseline.json`, `Tools/parity_twin_map.json`, their six test files under `Tools/tests/`, and `.github/workflows/parity-governance.yml`, which is not even in this fork's own documented CI table). Every commit touching these files was upstream's own (`Fanboynz`/`Bernd Helm`, re-deriving the baseline after their Swift/Kotlin changes) — nothing here was ever fork-owned, and none of it can pass without the tree it audits. 97 of the 103 removed tests were pure unit tests of the scanner's own lexical logic against synthetic fixtures (no repo dependency, all green) — removed anyway because a scanner for a contract this fork doesn't enforce has no job left to unit-test. The other 6 were the repository-level acceptance tests that actually broke: comparing the checked-in baseline/authority against current sources (`test_parity_governance_acceptance.py`), and asserting specific `android/` files exist (`test_rr_legacy_preservation_contract.py`). Both already failed on `main` before today, independent of the current upstream-sync PR — confirmed by running them against `f0d530d51`. **`Tools/test_steps_i18n.py`**: same shape, smaller — `test_android_all_shipped_locales` read `android/app/src/main/res/values/steps_view.xml`, which doesn't exist. Removed just that method (and the now-unused `RES` constant / `ElementTree` import); `test_ios_all_shipped_locales` is untouched and still the real coverage for this fork's one shipped platform. **`Tools/test_i18n_audit.py::test_two_word_brand_is_flagged`**: found while re-running the full suite green, unrelated to the above and NOT a removal — a real 3-day-old regression. `0c18441e4` ("make training load evidence-aware") added "Apple Health" to `BRAND_PHRASES` so a brand-only string is no longer flagged as a translation gap, but the one test that named this exact string kept asserting the PRE-change behavior and was never updated. Renamed to `test_two_word_brand_phrase_is_not` with the assertion flipped to match the now-current, deliberate behavior, plus a new sibling (`test_brand_phrase_with_a_real_word_is_still_flagged`) pinning the one case the module's own comment promises but nothing tested: "Apple Health sync" still has "sync" to translate, so it must stay flagged. Not a locale artifact — verified as a genuine code/test mismatch by reading the exempting logic and its git blame, not assumed. Verified: `python3 -m unittest discover -p "test_*.py"` in Tools/ — 124 tests (was 278), 0 failures, 16 skipped (unchanged skip set). `doc_comment_lint.py` and `i18n_audit.py --ci origin/main` both exit 0. No Swift/Kotlin/product source touched; this is Tools/ and CI config only. --- .github/workflows/parity-governance.yml | 84 - Tools/issue_ref.py | 55 - Tools/parity_dispositions.json | 4 - Tools/parity_ledger.py | 2851 ----------------- Tools/parity_ledger_baseline.json | 29 - Tools/parity_ratchet.py | 711 ---- Tools/parity_twin_map.json | 34 - Tools/test_i18n_audit.py | 13 +- Tools/test_steps_i18n.py | 32 +- Tools/tests/test_parity_arity.py | 137 - Tools/tests/test_parity_artefact_filter.py | 101 - Tools/tests/test_parity_disposition_kinds.py | 64 - .../test_parity_governance_acceptance.py | 1134 ------- Tools/tests/test_parity_ledger.py | 1279 -------- .../test_rr_legacy_preservation_contract.py | 58 - 15 files changed, 11 insertions(+), 6575 deletions(-) delete mode 100644 .github/workflows/parity-governance.yml delete mode 100644 Tools/issue_ref.py delete mode 100644 Tools/parity_dispositions.json delete mode 100644 Tools/parity_ledger.py delete mode 100644 Tools/parity_ledger_baseline.json delete mode 100644 Tools/parity_ratchet.py delete mode 100644 Tools/parity_twin_map.json delete mode 100644 Tools/tests/test_parity_arity.py delete mode 100644 Tools/tests/test_parity_artefact_filter.py delete mode 100644 Tools/tests/test_parity_disposition_kinds.py delete mode 100644 Tools/tests/test_parity_governance_acceptance.py delete mode 100644 Tools/tests/test_parity_ledger.py delete mode 100644 Tools/tests/test_rr_legacy_preservation_contract.py diff --git a/.github/workflows/parity-governance.yml b/.github/workflows/parity-governance.yml deleted file mode 100644 index d68dc01cab..0000000000 --- a/.github/workflows/parity-governance.yml +++ /dev/null @@ -1,84 +0,0 @@ -# These are the scanner's own unit and repository-acceptance tests, plus the -# explicitly named legacy R-R preservation contract. Keep that companion test -# exact rather than widening back to every Tools/tests file. -# -# On a PULL REQUEST, run the multi-minute suite only when its implementation, -# authority, named protection tests or workflow change; ordinary product-source -# changes belong to the later ledger/ratchet gate, not to repeated self-testing -# of the scanner. -# -# It also runs DAILY on main. One of these tests is not about the scanner at -# all: the repository acceptance asserts that the checked-in authority still -# reproduces from the CURRENT product source. Product changes are what -# invalidates it, and product changes are exactly what both filters exclude, so -# main could carry stale authority indefinitely with nothing red until some -# later PR touched Tools/ and inherited the whole accumulated drift. -# -# That happened four times, and three landed on an outside contributor's PR that -# had caused none of it. The schedule finds it within a day, on main, where -# re-deriving is a two-minute maintainer action. -# -# Deliberately a schedule rather than an unfiltered push: this suite is pinned -# NOT to run on product source (see test_core_tools_filter_covers_every_ -# governance_tool_path, which asserts both filters and forbids product globs), -# and a periodic check keeps that true. See the history in #1534. - -name: Parity Governance CI - -on: - pull_request: - branches: [main] - paths: - - 'Tools/issue_ref.py' - - 'Tools/parity_*.py' - - 'Tools/parity_*.json' - - 'Tools/tests/test_parity_*.py' - - 'Tools/tests/test_rr_legacy_preservation_contract.py' - - '.github/workflows/parity-governance.yml' - push: - branches: [main] - paths: - - 'Tools/issue_ref.py' - - 'Tools/parity_*.py' - - 'Tools/parity_*.json' - - 'Tools/tests/test_parity_*.py' - - 'Tools/tests/test_rr_legacy_preservation_contract.py' - - '.github/workflows/parity-governance.yml' - # Daily, so authority staleness is found on a schedule instead of by whoever - # next edits Tools/. See the header note. - schedule: - - cron: '17 4 * * *' - workflow_dispatch: - -concurrency: - group: parity-governance-${{ github.ref }} - cancel-in-progress: true - -jobs: - parity-governance: - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v5 - - - name: Set up Python - uses: actions/setup-python@v5 - with: - python-version: '3.12' - - - name: Run parity-governance tests - run: | - set -o pipefail - python3 -m unittest -v \ - tests.test_parity_ledger \ - tests.test_parity_governance_acceptance \ - tests.test_rr_legacy_preservation_contract \ - tests.test_parity_disposition_kinds \ - 2>&1 | tee "$RUNNER_TEMP/out.txt" - ran=$(grep -oE '^Ran [0-9]+ test' "$RUNNER_TEMP/out.txt" | grep -oE '[0-9]+') - echo "collected ${ran:-0} tests" - if [ "${ran:-0}" -lt 122 ]; then - echo "::error::expected at least 122 parity-governance tests, collected ${ran:-0} — discovery is broken, not the suite" - exit 1 - fi - working-directory: Tools diff --git a/Tools/issue_ref.py b/Tools/issue_ref.py deleted file mode 100644 index 9af8b717a6..0000000000 --- a/Tools/issue_ref.py +++ /dev/null @@ -1,55 +0,0 @@ -"""Strict, repository-qualified references for governed GitHub issues.""" - -from __future__ import annotations - -import re -from dataclasses import dataclass - - -_CURRENT = re.compile( - r"(?P[A-Za-z0-9](?:[A-Za-z0-9-]{0,37}[A-Za-z0-9])?)/" - r"(?P[A-Za-z0-9][A-Za-z0-9_.-]*)#(?P[1-9][0-9]*)" -) -class IssueRefError(ValueError): - """Raised when governed issue metadata is not canonical.""" - - -@dataclass(frozen=True, order=True) -class IssueRef: - repo: str - number: int - - def __str__(self) -> str: - return f"{self.repo}#{self.number}" - - -def parse_current(value: object) -> IssueRef: - """Parse only the canonical current ``owner/repo#N`` representation.""" - if not isinstance(value, str): - raise IssueRefError("issue must be a canonical owner/repo#N string") - match = _CURRENT.fullmatch(value) - if match is None: - raise IssueRefError(f"invalid issue reference {value!r}; expected owner/repo#N") - return IssueRef( - f"{match.group('owner')}/{match.group('name')}", - int(match.group("number")), - ) - - -def validate_current_issue_fields(value: object, location: str = "JSON") -> None: - """Validate every governed field literally named ``issue`` in current JSON.""" - def walk(node: object, pointer: str) -> None: - if isinstance(node, dict): - for key, child in node.items(): - child_pointer = f"{pointer}/{key}" - if key == "issue": - try: - parse_current(child) - except IssueRefError as exc: - raise IssueRefError(f"{location}{child_pointer}: {exc}") from exc - walk(child, child_pointer) - elif isinstance(node, list): - for index, child in enumerate(node): - walk(child, f"{pointer}/{index}") - - walk(value, "") diff --git a/Tools/parity_dispositions.json b/Tools/parity_dispositions.json deleted file mode 100644 index 036071b6f1..0000000000 --- a/Tools/parity_dispositions.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "schema_version": 1, - "dispositions": [] -} diff --git a/Tools/parity_ledger.py b/Tools/parity_ledger.py deleted file mode 100644 index e297408693..0000000000 --- a/Tools/parity_ledger.py +++ /dev/null @@ -1,2851 +0,0 @@ -#!/usr/bin/env python3 -"""Inventory and gate declared Swift/Kotlin parity contracts. - -The ledger is deliberately lexical. It does not compile either language and uses only -the Python standard library. Compact checked metadata hashes exact semantic sets and -accepted finding identities; normal runs independently rederive both from source. - -Usage: - python3 Tools/parity_ledger.py - python3 Tools/parity_ledger.py --no-baseline - python3 Tools/parity_ledger.py --bootstrap-map --write-baseline - -New semantic debt is governed by exact, manually reviewed typed dispositions. -""" - -from __future__ import annotations - -import argparse -import hashlib -import io -import json -import os -import re -import subprocess -import sys -import tarfile -import tempfile -from collections import Counter, defaultdict -from dataclasses import dataclass -from decimal import Decimal, InvalidOperation -from pathlib import Path -from typing import Iterable - -import issue_ref - - -ROOT = Path(__file__).resolve().parent.parent -DEFAULT_MAP = ROOT / "Tools/parity_twin_map.json" -DEFAULT_BASELINE = ROOT / "Tools/parity_ledger_baseline.json" - -SWIFT_GLOBS = ( - "Packages/StrandAnalytics/Sources/**/*.swift", - "Packages/StrandImport/Sources/**/*.swift", - "Packages/WhoopStore/Sources/**/*.swift", - "Packages/WhoopProtocol/Sources/**/*.swift", - "Packages/OuraProtocol/Sources/**/*.swift", -) -KOTLIN_GLOBS = ( - "android/app/src/main/java/com/noop/analytics/**/*.kt", - "android/app/src/main/java/com/noop/ingest/**/*.kt", - "android/app/src/main/java/com/noop/data/**/*.kt", - "android/app/src/main/java/com/noop/protocol/**/*.kt", - "android/app/src/main/java/com/noop/oura/**/*.kt", -) -SWIFT_EXCLUDED_GLOBS = ( - "Packages/NoopLocalAccess/Sources/**/*.swift", - "Packages/PolarProtocol/Sources/**/*.swift", - "Packages/StrandDesign/Sources/**/*.swift", - "Strand/**/*.swift", - "StrandiOS*/**/*.swift", - "NOOPWatch*/**/*.swift", -) -KOTLIN_EXCLUDED_GLOBS = ( - "android/app/src/main/java/com/noop/*.kt", - "android/app/src/main/java/com/noop/ai/**/*.kt", - "android/app/src/main/java/com/noop/alarm/**/*.kt", - "android/app/src/main/java/com/noop/ble/**/*.kt", - "android/app/src/main/java/com/noop/location/**/*.kt", - "android/app/src/main/java/com/noop/notif/**/*.kt", - "android/app/src/main/java/com/noop/polar/**/*.kt", - "android/app/src/main/java/com/noop/testcentre/**/*.kt", - "android/app/src/main/java/com/noop/ui/**/*.kt", - "android/app/src/main/java/com/noop/update/**/*.kt", - "android/app/src/main/java/com/noop/widget/**/*.kt", -) -PRODUCTION_GLOBS = ( - "Packages/**/Sources/**/*.swift", - "Strand/**/*.swift", - "StrandiOS*/**/*.swift", - "NOOPWatch*/**/*.swift", - "android/app/src/main/java/**/*.kt", -) -TEST_GLOBS = ( - "Packages/**/Tests/**/*.swift", - "StrandTests/**/*.swift", - "android/app/src/test/**/*.kt", - "android/app/src/androidTest/**/*.kt", -) -REFERENCE_GLOBS = ( - "Packages/**/*.swift", - "Strand/**/*.swift", - "StrandTests/**/*.swift", - "StrandiOS*/**/*.swift", - "NOOPWatch*/**/*.swift", - "android/**/*.kt", -) - -# These constants intentionally describe different platform-local persistence schemas. Their normalized -# names happen to match, but their migration generations do not and must not be compared as parity twins. -# Keep the exclusion exact so unrelated schema constants still go through the normal pairing audit. -CONSTANT_NON_TWIN_PAIRS = frozenset({ - ( - "Packages/WhoopStore/Sources/WhoopStore/WhoopStore.swift::schemaVersion", - "android/app/src/main/java/com/noop/data/WhoopDatabase.kt::SCHEMA_VERSION", - ), -}) - -HARD_FINDING_RULES = frozenset({ - "malformed-twin-map", "duplicate-twin-target", "twin-map-overlap", - "stale-twin-file", "stale-twin-function", "stale-twin-property", - "unmapped-declared-function-pair", "stale-declared-function-pair", - "unmapped-declared-file-pair", "stale-declared-file-pair", - "unmapped-constant-pair", "stale-constant-pair", "dead-twin-reference", - "unresolved-attached-function-claim", "ambiguous-attached-function-claim", - "stale-bootstrap-exemption", -}) - - -@dataclass(frozen=True) -class Declaration: - language: str - path: str - name: str - arity: int - line: int - ordinal: int = 1 - kind: str = "function" - owner_name: str | None = None - offset: int = -1 - opening: int = -1 - parameter_labels: tuple[str, ...] = () - required_arity: int = 0 - - @property - def key(self) -> str: - if self.kind == "property": - return f"{self.path}::{self.name}@property#{self.ordinal}" - return f"{self.path}::{self.name}/{self.arity}#{self.ordinal}" - - @property - def owner(self) -> str: - if self.owner_name: - return self.owner_name - stem = Path(self.path).stem - return stem.replace("+Trace", "Trace") - - -@dataclass(frozen=True) -class Constant: - language: str - path: str - name: str - value: str | None - display_value: str - line: int - owner_name: str | None = None - - @property - def key(self) -> str: - return f"{self.path}::{self.name}" - - @property - def owner(self) -> str: - return self.owner_name or Path(self.path).stem - - -@dataclass(frozen=True) -class Finding: - rule: str - path: str - line: int - text: str - identity: str - - def output(self) -> str: - return f"{self.path}:{self.line}: {self.rule}: {self.text}" - - -@dataclass(frozen=True) -class ScanError: - rule: str - path: str - line: int - text: str - - def output(self) -> str: - return f"{self.path}:{self.line}: {self.rule}: {self.text}" - - -@dataclass -class ScanResult: - findings: list[Finding] - counters: dict[str, int] - stats: dict[str, int] - errors: list[ScanError] - missing_attached_claimants: set[str] - bootstrap_unpaired_debts: set[str] - - -class _InvalidSourceEncoding(ValueError): - def __init__(self, path: Path, detail: str): - super().__init__(detail) - self.path = path - - -@dataclass(frozen=True) -class TwinReference: - language: str - path: str - line: int - raw_target: str - target_name: str - target_owner: str | None - attached_function: str | None - claim_ordinal: int - - -@dataclass(frozen=True) -class CallSite: - name: str - arity: int - owner: str | None - path: str - lexical_owner: str | None = None - - -# Directories that hold BUILD OUTPUT rather than source. Every glob here ends in `**`, and `**` -# descends into these exactly as happily as into a source tree: a dependency checked out under -# `Packages/StrandAnalytics/.build/checkouts/...` matches `Packages/**/Sources/**/*.swift` whenever the -# dependency happens to lay itself out with a `Sources` directory, which SwiftPM packages do by -# convention. -# -# Left unfiltered this is worse than noise, because the pollution is SUBTRACTIVE. It does not add -# findings a reader would question; it REMOVES them, by handing a declaration a callsite that only -# exists in a vendored copy of somebody else's library. That is how a scan on a working tree came back -# one `test-only-callsite` short for Packages/StrandAnalytics, against a baseline derived on a clean -# checkout, and the local acceptance test passed anyway because both sides of its comparison came from -# the same polluted tree. CI checks out clean, so it never saw any of it and could not warn. -# -# Filtering here rather than in each glob because `_paths` is the ONLY place this module globs: the -# declaration scan, the reference scan and both callsite corpora all come through it. -# -# The invariant that makes this safe is checked by a test rather than asserted here: everything this -# drops is untracked by git, so no file the repository actually contains can be hidden by it. -_ARTEFACT_DIRS = frozenset({ - ".build", # SwiftPM: checkouts, index-build, the lot - ".swiftpm", - "DerivedData", # Xcode - "build", # Gradle output, incl. anything KSP or the AGP generates - ".gradle", - "node_modules", - "Pods", # CocoaPods, if it ever appears -}) - - -def _is_build_artefact(root: Path, path: Path) -> bool: - """True when `path` sits inside a build-output directory rather than the source tree.""" - try: - parts = path.relative_to(root).parts - except ValueError: - parts = path.parts - return any(part in _ARTEFACT_DIRS for part in parts) - - -def _paths(root: Path, globs: Iterable[str]) -> list[Path]: - found: set[Path] = set() - for pattern in globs: - found.update(path for path in root.glob(pattern) - if path.is_file() and not _is_build_artefact(root, path)) - return sorted(found) - - -def _relative(root: Path, path: Path) -> str: - return path.relative_to(root).as_posix() - - -def _read(path: Path) -> str: - try: - return path.read_text(encoding="utf-8", errors="strict") - except UnicodeDecodeError as exc: - raise _InvalidSourceEncoding(path, str(exc)) from exc - - -class _SourceSnapshot: - """Operation-local immutable source view; discarded after one top-level scan.""" - - def __init__(self) -> None: - self._text: dict[Path, str] = {} - self._masked: dict[tuple[Path, bool], str] = {} - self._functions: dict[tuple[Path, str], tuple[Declaration, ...]] = {} - self._properties: dict[tuple[Path, str], tuple[Declaration, ...]] = {} - self._constants: dict[tuple[Path, str], tuple[Constant, ...]] = {} - - def text(self, path: Path) -> str: - resolved = path.resolve() - if resolved not in self._text: - self._text[resolved] = _read(resolved) - return self._text[resolved] - - def masked(self, path: Path, *, kotlin_templates: bool = False) -> str: - resolved = path.resolve() - key = (resolved, kotlin_templates) - if key not in self._masked: - self._masked[key] = _mask_non_code( - self.text(resolved), kotlin_templates=kotlin_templates - ) - return self._masked[key] - - def functions(self, root: Path, path: Path, language: str) -> tuple[Declaration, ...]: - key = (path.resolve(), language) - if key not in self._functions: - self._functions[key] = _parse_functions_content( - str(root.resolve()), str(key[0]), language, self.text(key[0]), self.masked(key[0]) - ) - return self._functions[key] - - def properties(self, root: Path, path: Path, language: str) -> tuple[Declaration, ...]: - key = (path.resolve(), language) - if key not in self._properties: - self._properties[key] = _parse_properties_content( - str(root.resolve()), str(key[0]), language, self.text(key[0]), self.masked(key[0]) - ) - return self._properties[key] - - def constants(self, root: Path, path: Path, language: str) -> tuple[Constant, ...]: - key = (path.resolve(), language) - if key not in self._constants: - self._constants[key] = _parse_constants_content( - str(root.resolve()), str(key[0]), language, self.text(key[0]), self.masked(key[0]) - ) - return self._constants[key] - - -def _mask_non_code(text: str, kotlin_templates: bool = False) -> str: - """Replace comments and string contents with spaces, preserving newlines.""" - if kotlin_templates: - return _mask_kotlin_template_code(text) - out = list(text) - i = 0 - state = "code" - block_depth = 0 - quote = "" - while i < len(text): - if state == "code": - if text.startswith("//", i): - out[i] = out[i + 1] = " " - i += 2 - state = "line" - elif text.startswith("/*", i): - out[i] = out[i + 1] = " " - i += 2 - block_depth = 1 - state = "block" - elif text.startswith('"""', i): - out[i : i + 3] = " " - i += 3 - quote = '"""' - state = "string" - elif text[i] in "\"'": - quote = text[i] - out[i] = " " - i += 1 - state = "string" - else: - i += 1 - elif state == "line": - if text[i] == "\n": - state = "code" - else: - out[i] = " " - i += 1 - elif state == "block": - if text.startswith("/*", i): - out[i] = out[i + 1] = " " - block_depth += 1 - i += 2 - elif text.startswith("*/", i): - out[i] = out[i + 1] = " " - block_depth -= 1 - i += 2 - if block_depth == 0: - state = "code" - else: - if text[i] != "\n": - out[i] = " " - i += 1 - else: - if quote == '"""' and text.startswith(quote, i): - out[i : i + 3] = " " - i += 3 - state = "code" - elif quote != '"""' and text[i] == "\\" and i + 1 < len(text): - if text[i] != "\n": - out[i] = " " - if text[i + 1] != "\n": - out[i + 1] = " " - i += 2 - elif quote != '"""' and text[i] == quote: - out[i] = " " - i += 1 - state = "code" - else: - if text[i] != "\n": - out[i] = " " - i += 1 - return "".join(out) - - -class _MalformedKotlinTemplate(ValueError): - def __init__(self, offset: int): - super().__init__("unterminated Kotlin string template") - self.offset = offset - - -def _mask_kotlin_template_code(text: str) -> str: - """Mask Kotlin non-code while retaining balanced ``${...}`` expressions. - - Kotlin string templates contain executable callsites, but their surrounding text—and strings or - comments nested inside an expression—must remain invisible to the lexical call scanner. The - context stack keeps braces balanced and supports nested strings/templates. A plain unterminated - literal is masked through EOF; an unterminated template raises an explicit scan error because - silently discarding its executable expression could hide a production callsite. - """ - - out = list(text) - stack: list[dict[str, object]] = [{"kind": "code"}] - i = 0 - - def blank(start: int, end: int) -> None: - for index in range(start, min(end, len(out))): - if text[index] != "\n": - out[index] = " " - - def fail_closed() -> str: - string_starts = [ - int(context["start"]) - for context in stack - if context["kind"] == "string" - ] - if string_starts: - blank(min(string_starts), len(text)) - template_starts = [ - int(context["template_start"]) - for context in stack - if context["kind"] == "string" and "template_start" in context - ] + [ - int(context["start"]) - for context in stack - if context["kind"] == "template" - ] - if template_starts: - raise _MalformedKotlinTemplate(min(template_starts)) - return "".join(out) - - while i < len(text): - context = stack[-1] - kind = context["kind"] - if kind in {"code", "template"}: - if text.startswith("//", i): - blank(i, i + 2) - stack.append({"kind": "line"}) - i += 2 - elif text.startswith("/*", i): - blank(i, i + 2) - stack.append({"kind": "block", "depth": 1}) - i += 2 - elif text.startswith('"""', i): - blank(i, i + 3) - out[i] = "0" # Occupy a containing call argument without exposing literal text. - stack.append({"kind": "string", "quote": '"""', "start": i}) - i += 3 - elif text[i] in "\"'": - blank(i, i + 1) - out[i] = "0" # String/character literals are one lexical argument token. - stack.append({"kind": "string", "quote": text[i], "start": i}) - i += 1 - elif kind == "template" and text[i] == "{": - context["depth"] = int(context["depth"]) + 1 - i += 1 - elif kind == "template" and text[i] == "}": - context["depth"] = int(context["depth"]) - 1 - i += 1 - if context["depth"] == 0: - stack.pop() - else: - i += 1 - elif kind == "line": - if text[i] == "\n": - stack.pop() - else: - blank(i, i + 1) - i += 1 - elif kind == "block": - if text.startswith("/*", i): - blank(i, i + 2) - context["depth"] = int(context["depth"]) + 1 - i += 2 - elif text.startswith("*/", i): - blank(i, i + 2) - context["depth"] = int(context["depth"]) - 1 - i += 2 - if context["depth"] == 0: - stack.pop() - else: - blank(i, i + 1) - i += 1 - else: - quote = str(context["quote"]) - if quote == '"""' and text.startswith(quote, i): - blank(i, i + 3) - stack.pop() - i += 3 - elif quote != '"""' and text[i] == "\n": - return fail_closed() - elif quote != '"""' and text[i] == "\\" and i + 1 < len(text): - blank(i, i + 2) - i += 2 - elif quote != '"""' and text[i] == quote: - blank(i, i + 1) - stack.pop() - i += 1 - elif quote != "'" and text.startswith("${", i): - blank(i, i + 1) - context.setdefault("template_start", i) - stack.append({"kind": "template", "depth": 1, "start": i}) - i += 2 - else: - blank(i, i + 1) - i += 1 - - if any(context["kind"] in {"string", "template"} for context in stack): - return fail_closed() - return "".join(out) - - -def _arity(masked: str, opening: int, *, angles_are_brackets: bool = True) -> int | None: - stack: list[str] = [] - pairs = {")": "(", "]": "[", "}": "{", ">": "<"} - segments = 0 - segment_has_token = False - i = opening + 1 - while i < len(masked): - char = masked[i] - if char == "(" or char == "[" or char == "{": - stack.append(char) - elif char == "<": - # Parameter lists use angle brackets for types. Do not treat Kotlin/Swift arrows as generics. - # - # Nor Swift's half-open range operator. `a[x..= len(masked) or masked[i + 1] not in "= "): - stack.append(char) - elif char in pairs: - if char == ")" and not stack: - return segments + (1 if segment_has_token else 0) - if stack and stack[-1] == pairs[char]: - stack.pop() - elif char == "," and not stack: - if segment_has_token: - segments += 1 - segment_has_token = False - elif not char.isspace() and not stack: - segment_has_token = True - i += 1 - - # The walk never balanced, which means some `<` was pushed that nothing closed. Falling out of - # here returns None, and every caller answers None with `continue` -- so an argument this walk - # cannot parse does not merely lose its arity, it ERASES THE ENTIRE CALLSITE and the scan reports - # a declaration nobody calls. - # - # `<` is the only genuinely ambiguous character: a generic argument list needs it treated as a - # bracket, while `a << b`, `a tuple[str, ...]: - labels: list[str] = [] - stack: list[str] = [] - start = opening + 1 - pairs = {")": "(", "]": "[", "}": "{"} - i = start - while i < len(masked): - char = masked[i] - if char in "([{": - stack.append(char) - elif char == ")" and not stack: - segment = masked[start:i].strip() - if segment: - match = re.match(r"(_|[A-Za-z_][A-Za-z0-9_]*)\b", segment) - labels.append(match.group(1) if match else "") - return tuple(labels) - elif char in pairs and stack and stack[-1] == pairs[char]: - stack.pop() - elif char == "," and not stack: - segment = masked[start:i].strip() - match = re.match(r"(_|[A-Za-z_][A-Za-z0-9_]*)\b", segment) - labels.append(match.group(1) if match else "") - start = i + 1 - i += 1 - return () - - -def _parameter_segments(masked: str, opening: int) -> tuple[str, ...]: - segments: list[str] = [] - stack: list[str] = [] - start = opening + 1 - pairs = {")": "(", "]": "[", "}": "{"} - i = start - while i < len(masked): - char = masked[i] - if char in "([{": - stack.append(char) - elif char == ")" and not stack: - segment = masked[start:i].strip() - if segment: - segments.append(segment) - return tuple(segments) - elif char in pairs and stack and stack[-1] == pairs[char]: - stack.pop() - elif char == "," and not stack: - segment = masked[start:i].strip() - if segment: - segments.append(segment) - start = i + 1 - i += 1 - return () - - -def _required_arity(masked: str, opening: int) -> int: - required = 0 - for segment in _parameter_segments(masked, opening): - stack: list[str] = [] - has_default = False - pairs = {")": "(", "]": "[", "}": "{"} - for char in segment: - if char in "([{": - stack.append(char) - elif char in pairs and stack and stack[-1] == pairs[char]: - stack.pop() - elif char == "=" and not stack: - has_default = True - break - if not has_default: - required += 1 - return required - - -SWIFT_FUNC = re.compile( - r"\bfunc\s+(`?[A-Za-z_][A-Za-z0-9_]*`?|[=!<>+\-*/%&|^~?.]+)\s*(?:<[^\n{}()]*>\s*)?\(" -) -KOTLIN_FUNC = re.compile( - r"\bfun\s+(?:<[^\n{}()]*>\s*)?([^\n{}()=]+?)\s*\(" -) - -TYPE_DECLARATION = { - "swift": re.compile(r"\b(?:struct|class|enum|actor|protocol|extension)\s+([A-Za-z_][A-Za-z0-9_]*)"), - "kotlin": re.compile( - r"\b(?:(?:data|sealed|enum|annotation|value)\s+)?(?:class|object|interface)\s+([A-Za-z_][A-Za-z0-9_]*)" - ), -} - - -def _matching_brace(masked: str, opening: int) -> int: - depth = 0 - for index in range(opening, len(masked)): - if masked[index] == "{": - depth += 1 - elif masked[index] == "}": - depth -= 1 - if depth == 0: - return index - return len(masked) - - -def _type_spans(masked: str, language: str) -> list[tuple[int, int, str]]: - spans: list[tuple[int, int, str]] = [] - for match in TYPE_DECLARATION[language].finditer(masked): - opening = masked.find("{", match.end()) - if opening < 0: - continue - # Do not attach a type to a later, unrelated declaration when its body is absent. - next_decl = TYPE_DECLARATION[language].search(masked, match.end()) - if next_decl and next_decl.start() < opening: - continue - spans.append((opening, _matching_brace(masked, opening), match.group(1))) - return spans - - -def _owner_at(spans: list[tuple[int, int, str]], offset: int, fallback: str) -> str: - containing = [item for item in spans if item[0] < offset < item[1]] - return max(containing, key=lambda item: item[0])[2] if containing else fallback - - -def _swift_module_owner(path: str) -> str | None: - parts = Path(path).parts - if len(parts) >= 4 and parts[0] == "Packages" and parts[2] == "Sources": - return parts[1] - return None - - -def _receiver_owner(header: str) -> str | None: - name_match = re.search(r"(`?[A-Za-z_][A-Za-z0-9_]*`?)\s*$", header) - if not name_match: - return None - prefix = header[: name_match.start()].rstrip() - if not prefix.endswith("."): - return None - receiver = prefix[:-1].strip().rstrip("?") - # The receiver can contain nested generics; the leading nominal type is the useful owner. - names = re.findall(r"[A-Za-z_][A-Za-z0-9_]*", receiver) - return names[0] if names else None - - -def _parse_functions_content( - root_string: str, path_string: str, language: str, text: str, masked: str | None = None -) -> tuple[Declaration, ...]: - root = Path(root_string) - path = Path(path_string) - masked = masked if masked is not None else _mask_non_code(text) - pattern = SWIFT_FUNC if language == "swift" else KOTLIN_FUNC - rel = _relative(root, path) - fallback_owner = path.stem.replace("+Trace", "Trace") - spans = _type_spans(masked, language) - out: list[Declaration] = [] - ordinals: Counter[tuple[str, int]] = Counter() - for match in pattern.finditer(masked): - arity = _arity(masked, match.end() - 1) - if arity is None: - continue - header = match.group(1) - if language == "kotlin": - name_match = re.search(r"(`?[A-Za-z_][A-Za-z0-9_]*`?)\s*$", header) - if not name_match: - continue - name = name_match.group(1).strip("`") - name_offset = match.start(1) + name_match.start(1) - receiver = _receiver_owner(header) - else: - name = header.strip("`") - name_offset = match.start(1) - receiver = None - ordinals[(name, arity)] += 1 - out.append( - Declaration( - language=language, - path=rel, - name=name, - arity=arity, - line=text.count("\n", 0, match.start()) + 1, - ordinal=ordinals[(name, arity)], - owner_name=receiver or _owner_at(spans, match.start(), fallback_owner), - offset=name_offset, - opening=match.end() - 1, - parameter_labels=( - _swift_parameter_labels(masked, match.end() - 1) - if language == "swift" else () - ), - required_arity=_required_arity(masked, match.end() - 1), - ) - ) - return tuple(out) - - -def parse_functions(root: Path, path: Path, language: str) -> list[Declaration]: - return list( - _parse_functions_content( - str(root.resolve()), str(path.resolve()), language, _read(path) - ) - ) - - -def _parse_properties_content( - root_string: str, path_string: str, language: str, text: str, masked: str | None = None -) -> tuple[Declaration, ...]: - """Inventory computed properties/getters, excluding stored fields.""" - root = Path(root_string) - path = Path(path_string) - masked = masked if masked is not None else _mask_non_code(text) - rel = _relative(root, path) - fallback_owner = path.stem.replace("+Trace", "Trace") - spans = _type_spans(masked, language) - if language == "swift": - pattern = re.compile( - r"\bvar\s+([A-Za-z_][A-Za-z0-9_]*)\s*:\s*[^=\n{]+\{" - ) - else: - pattern = re.compile( - r"\b(?:val|var)\s+([A-Za-z_][A-Za-z0-9_]*)\b" - r"(?:\s*:\s*[^=\n{]+)?\s*(?:\n[ \t]*)?get\s*\(\s*\)" - ) - out: list[Declaration] = [] - ordinals: Counter[str] = Counter() - for match in pattern.finditer(masked): - name = match.group(1) - ordinals[name] += 1 - out.append( - Declaration( - language=language, - path=rel, - name=name, - arity=0, - line=text.count("\n", 0, match.start()) + 1, - ordinal=ordinals[name], - kind="property", - owner_name=_owner_at(spans, match.start(), fallback_owner), - offset=match.start(1), - ) - ) - return tuple(out) - - -def parse_properties(root: Path, path: Path, language: str) -> list[Declaration]: - return list( - _parse_properties_content( - str(root.resolve()), str(path.resolve()), language, _read(path) - ) - ) - - -NUMBER_PATTERN = ( - r"(?:0[xX][0-9A-Fa-f_]+[lL]?|0[bB][01_]+[lL]?|0[oO][0-7_]+[lL]?|" - r"(?:\d[\d_]*(?:\.[\d_]*)?|\.[\d_]+)(?:[eE][-+]?\d[\d_]*)?[fFdDlL]?)" -) -NUMBER_TOKEN = re.compile(NUMBER_PATTERN) - - -class _NumberExpression: - def __init__(self, raw: str): - self.raw = raw - self.tokens = re.findall( - NUMBER_PATTERN + r"|[()+\-*/]", - raw, - ) - self.index = 0 - - def parse(self) -> Decimal: - compact = re.sub(r"\s+", "", self.raw) - if "".join(self.tokens) != compact or not self.tokens: - raise InvalidOperation - value = self._sum() - if self.index != len(self.tokens): - raise InvalidOperation - return value - - def _sum(self) -> Decimal: - value = self._product() - while self._peek() in {"+", "-"}: - operator = self._take() - right = self._product() - value = value + right if operator == "+" else value - right - return value - - def _product(self) -> Decimal: - value = self._unary() - while self._peek() in {"*", "/"}: - operator = self._take() - right = self._unary() - if operator == "*": - value *= right - else: - if right == 0: - raise InvalidOperation - value /= right - return value - - def _unary(self) -> Decimal: - if self._peek() in {"+", "-"}: - operator = self._take() - value = self._unary() - return value if operator == "+" else -value - if self._peek() == "(": - self._take() - value = self._sum() - if self._take() != ")": - raise InvalidOperation - return value - token = self._take() - if not NUMBER_TOKEN.fullmatch(token): - raise InvalidOperation - number = token.replace("_", "") - if number.lower().startswith(("0x", "0b", "0o")): - # f/F/d/D are hex digits here; radix literals only take the l/L suffix. - return Decimal(int(number.rstrip("lL"), 0)) - return Decimal(number.rstrip("fFdDlL")) - - def _peek(self) -> str | None: - return self.tokens[self.index] if self.index < len(self.tokens) else None - - def _take(self) -> str: - if self.index >= len(self.tokens): - raise InvalidOperation - token = self.tokens[self.index] - self.index += 1 - return token - - -def _initializer(text: str, start: int) -> str: - """Return exactly one single-line constant initializer, without its terminator/comment.""" - out: list[str] = [] - quote: str | None = None - escaped = False - depth = 0 - index = start - while index < len(text): - char = text[index] - if quote: - out.append(char) - if escaped: - escaped = False - elif char == "\\": - escaped = True - elif char == quote: - quote = None - index += 1 - continue - if text.startswith("//", index): - break - if char in {'"', "'"}: - quote = char - out.append(char) - elif char == "(": - depth += 1 - out.append(char) - elif char == ")": - depth = max(0, depth - 1) - out.append(char) - elif char in "\n;," and depth == 0: - break - elif char == "}" and depth == 0: - break - else: - out.append(char) - index += 1 - return "".join(out).strip().rstrip(",").strip() - - -def _literal(raw: str) -> tuple[str, str] | None: - value = raw.strip() - if re.fullmatch(r'"(?:\\.|[^"\\])*"', value): - token = value - token_for_json = re.sub( - r"\\u\{([0-9A-Fa-f]{1,8})\}", - lambda item: "\\u" + item.group(1).zfill(4), - token, - ) - try: - decoded = json.loads(token_for_json) - except json.JSONDecodeError: - decoded = token[1:-1] - return f"string:{decoded}", token - if value in {"true", "false"}: - return f"bool:{value}", value - if value in {"nil", "null"}: - return "null", value - try: - number = _NumberExpression(value).parse() - return f"number:{number.normalize()}", value - except (InvalidOperation, ZeroDivisionError): - return None - - -def _parse_constants_content( - root_string: str, path_string: str, language: str, text: str, masked: str | None = None -) -> tuple[Constant, ...]: - root = Path(root_string) - path = Path(path_string) - masked = masked if masked is not None else _mask_non_code(text) - if language == "swift": - pattern = re.compile(r"\blet\s+([A-Za-z_][A-Za-z0-9_]*)\s*(?::[^=\n]+)?=") - else: - pattern = re.compile(r"\bconst\s+val\s+([A-Za-z_][A-Za-z0-9_]*)\s*(?::[^=\n]+)?=") - rel = _relative(root, path) - fallback_owner = path.stem.replace("+Trace", "Trace") - spans = _type_spans(masked, language) - out: list[Constant] = [] - for match in pattern.finditer(masked): - if language == "swift": - line_start = text.rfind("\n", 0, match.start()) + 1 - prefix = text[line_start : match.start()] - # Kotlin's `const val` has static storage. Pair it only with a Swift `static - # let` or a file-scope declaration, never with a local/instance `let`. - if "static" not in prefix.split() and len(prefix) - len(prefix.lstrip()) > 4: - continue - raw = _initializer(text, match.end()) - parsed = _literal(raw) - canonical, display = parsed if parsed is not None else (None, raw or "") - out.append( - Constant( - language, - rel, - match.group(1), - canonical, - display, - text.count("\n", 0, match.start()) + 1, - _owner_at(spans, match.start(), fallback_owner), - ) - ) - return tuple(out) - - -def parse_constants(root: Path, path: Path, language: str) -> list[Constant]: - return list( - _parse_constants_content( - str(root.resolve()), str(path.resolve()), language, _read(path) - ) - ) - - -def _inventory( - root: Path, - snapshot: _SourceSnapshot | None = None, -) -> tuple[ - list[Path], - list[Path], - list[Declaration], - list[Declaration], - list[Declaration], - list[Declaration], - list[Constant], - list[Constant], -]: - snapshot = snapshot or _SourceSnapshot() - swift_files = _paths(root, SWIFT_GLOBS) - kotlin_files = _paths(root, KOTLIN_GLOBS) - swift_functions: list[Declaration] = [] - kotlin_functions: list[Declaration] = [] - swift_properties: list[Declaration] = [] - kotlin_properties: list[Declaration] = [] - swift_constants: list[Constant] = [] - kotlin_constants: list[Constant] = [] - for paths, language, functions, properties, constants in ( - (swift_files, "swift", swift_functions, swift_properties, swift_constants), - (kotlin_files, "kotlin", kotlin_functions, kotlin_properties, kotlin_constants), - ): - for path in paths: - functions.extend(snapshot.functions(root, path, language)) - properties.extend(snapshot.properties(root, path, language)) - constants.extend(snapshot.constants(root, path, language)) - return ( - swift_files, - kotlin_files, - swift_functions, - kotlin_functions, - swift_properties, - kotlin_properties, - swift_constants, - kotlin_constants, - ) - - -def _annotation_count(files: list[Path], snapshot: _SourceSnapshot | None = None) -> int: - snapshot = snapshot or _SourceSnapshot() - pattern = re.compile(r"\b(?:twin|parity)\b", re.I) - return sum(len(pattern.findall(snapshot.text(path))) for path in files) - - -def _normal_name(name: str) -> str: - return re.sub(r"[^a-z0-9]", "", name.lower()) - - -def _comment_blocks(text: str, language: str) -> list[tuple[int, int, str]]: - out: list[tuple[int, int, str]] = [] - for match in re.finditer(r"/\*[\s\S]*?\*/", text): - start = text.count("\n", 0, match.start()) + 1 - end = text.count("\n", 0, match.end()) + 1 - out.append((start, end, match.group(0))) - for match in re.finditer(r"(?m)(?:^[ \t]*//[^\n]*(?:\n|$))+", text): - start = text.count("\n", 0, match.start()) + 1 - end = text.count("\n", 0, match.end()) + 1 - out.append((start, end, match.group(0))) - return sorted(out) - - -REFERENCE_PATTERNS = ( - re.compile(r"\b(?:Kotlin|Swift)(?:'s)?\s+twin\s*(?:is\s*|of\s*|:\s*)?(?:the\s+)?`([^`]+)`", re.I), - re.compile(r"\btwin\s+of\s+(?:the\s+)?(?:Kotlin|Swift)(?:'s)?\s+`([^`]+)`", re.I), - re.compile(r"\bmirrors\s+(?:Kotlin|Swift)(?:'s)?\s+`([^`]+)`", re.I), - re.compile( - r"\b(?:Kotlin|Swift)(?:'s)?\s+twin\s*(?:is\s*|:\s*)?" - r"([A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*)+)", - re.I, - ), -) - - -def _target(raw: str) -> tuple[str, str | None] | None: - value = raw.strip() - if "/" in value and value.lower().endswith((".swift", ".kt")): - return Path(value).stem, None - if not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*(?:\([^)]*\))?)*", value): - return None - pieces = value.split(".") - if pieces[-1].lower() in {"swift", "kt"} and len(pieces) >= 2: - return pieces[-2], None - name = pieces[-1].split("(", 1)[0] - owner = pieces[-2] if len(pieces) > 1 else None - if len(pieces) > 2 and all(piece[:1].islower() for piece in pieces[:-1]): - owner = None # package-qualified type, e.g. com.noop.protocol.DeviceConfigWriteGate - elif name[:1].isupper(): - owner = None # module/type-qualified type, not an Owner.member reference - return name, owner - - -def parse_twin_references( - root: Path, - files: list[Path], - language: str, - functions: list[Declaration], - snapshot: _SourceSnapshot | None = None, -) -> list[TwinReference]: - snapshot = snapshot or _SourceSnapshot() - by_path: dict[str, list[Declaration]] = defaultdict(list) - for declaration in functions: - by_path[declaration.path].append(declaration) - out: list[TwinReference] = [] - expected = "kotlin" if language == "swift" else "swift" - for path in files: - rel = _relative(root, path) - text = snapshot.text(path) - claim_ordinals: Counter[tuple[str, str | None]] = Counter() - for start, end, comment in _comment_blocks(text, language): - if "twin" not in comment.lower() or expected not in comment.lower(): - continue - # Remove line-doc decoration without changing offsets, so wrapped unquoted - # references remain machine-readable and line reporting stays exact. - searchable = re.sub( - r"(?m)^(\s*)(?:///|//|/\*\*?|\*) ?", - lambda match: " " * len(match.group(0)), - comment, - ) - raw_targets: list[tuple[str, int]] = [] - for pattern in REFERENCE_PATTERNS: - raw_targets.extend((match.group(1), match.start(1)) for match in pattern.finditer(searchable)) - raw_targets.sort(key=lambda item: item[1]) - - parsed_targets: list[tuple[str, int, str, str | None]] = [] - seen: set[str] = set() - for raw, offset in raw_targets: - if raw in seen: - continue - seen.add(raw) - parsed = _target(raw) - if parsed is None: - continue - name, owner = parsed - parsed_targets.append((raw, offset, name, owner)) - - nearby_declaration = next( - (decl.key for decl in by_path[rel] if end <= decl.line <= end + 4), - None, - ) - # A prose block can mention older alternatives before stating the - # authoritative twin. Only its nearest function-shaped claim is - # attached to the following declaration. File/type references - # remain repository-wide references and never claim a function. - attachable = [ - item for item in parsed_targets - if item[3] is not None or item[2][:1].islower() - ] - attached_target = attachable[-1] if nearby_declaration and attachable else None - - for raw, offset, name, owner in parsed_targets: - attached = ( - nearby_declaration - if attached_target is not None and (raw, offset, name, owner) == attached_target - else None - ) - claim_key = (raw, attached) - claim_ordinals[claim_key] += 1 - line = start + searchable.count("\n", 0, offset) - out.append( - TwinReference( - language, - rel, - line, - raw, - name, - owner, - attached, - claim_ordinals[claim_key], - ) - ) - return out - - -def _resolve(reference: TwinReference, targets: list[Declaration]) -> list[Declaration]: - matches = [decl for decl in targets if _normal_name(decl.name) == _normal_name(reference.target_name)] - if reference.target_owner: - wanted = _normal_name(reference.target_owner) - matches = [ - decl for decl in matches - if wanted in { - _normal_name(decl.owner), - _normal_name(_swift_module_owner(decl.path) or ""), - } - ] - selector = re.search(r"\(([^)]*)\)\s*$", reference.raw_target) - if selector: - labels = tuple( - part.strip().rstrip(":") - for part in selector.group(1).split(":") - if part.strip() - ) - if labels: - labelled = [decl for decl in matches if decl.parameter_labels[: len(labels)] == labels] - matches = labelled - return matches - - -def attached_function_resolutions( - references: Iterable[TwinReference], - swift_functions: list[Declaration], - kotlin_functions: list[Declaration], -) -> dict[TwinReference, tuple[Declaration, ...]]: - """Resolve every attached claim, including missing and ambiguous claims.""" - result: dict[TwinReference, tuple[Declaration, ...]] = {} - for reference in references: - if reference.attached_function is None: - continue - targets = kotlin_functions if reference.language == "swift" else swift_functions - candidates = _resolve(reference, targets) - result[reference] = tuple(candidates) - return result - - -def resolved_attached_function_pairs( - references: Iterable[TwinReference], - swift_functions: list[Declaration], - kotlin_functions: list[Declaration], - *, - resolutions: dict[TwinReference, tuple[Declaration, ...]] | None = None, -) -> dict[tuple[str, str], list[TwinReference]]: - """Resolve attached source claims into the exact pairs they declare. - - Both map bootstrap and normal scans use this function so a source comment - cannot be retargeted, added, or removed independently of the checked map. - Ambiguous and unresolved claims are deliberately omitted here; the normal - dead-reference audit reports those claims separately. - """ - pairs: dict[tuple[str, str], list[TwinReference]] = defaultdict(list) - if resolutions is None: - resolutions = attached_function_resolutions( - references, swift_functions, kotlin_functions - ) - for reference, resolved in resolutions.items(): - if len(resolved) != 1: - continue - pair = ( - (reference.attached_function, resolved[0].key) - if reference.language == "swift" - else (resolved[0].key, reference.attached_function) - ) - pairs[pair].append(reference) - return pairs - - -def resolved_file_pairs( - function_pairs: Iterable[tuple[str, str]], - swift_functions: list[Declaration], - kotlin_functions: list[Declaration], -) -> set[tuple[str, str]]: - """Derive file authority from the same resolved function pairs everywhere.""" - swift_by_key = {item.key: item for item in swift_functions} - kotlin_by_key = {item.key: item for item in kotlin_functions} - return { - (swift_by_key[swift_key].path, kotlin_by_key[kotlin_key].path) - for swift_key, kotlin_key in function_pairs - if swift_key in swift_by_key and kotlin_key in kotlin_by_key - } - - -def _symbol_owners( - root: Path, - files: list[Path], - declarations: list[Declaration], - snapshot: _SourceSnapshot | None = None, -) -> dict[str, set[str]]: - snapshot = snapshot or _SourceSnapshot() - symbols: dict[str, set[str]] = defaultdict(set) - for item in declarations: - owners = {_normal_name(item.owner), _normal_name(Path(item.path).stem)} - module = _swift_module_owner(item.path) - if module: - owners.add(_normal_name(module)) - symbols[_normal_name(item.name)].update(owners) - for path in files: - file_owner = _normal_name(path.stem) - symbols[file_owner].add(file_owner) - language = "swift" if path.suffix == ".swift" else "kotlin" - text = snapshot.text(path) - masked = snapshot.masked(path) - spans = _type_spans(masked, language) - for _, _, name in spans: - symbols[_normal_name(name)].add(file_owner) - declaration = re.compile( - r"\b(?:typealias|let|var|val)\s+([A-Za-z_][A-Za-z0-9_]*)\b" - ) - for match in declaration.finditer(masked): - owner = _owner_at(spans, match.start(), path.stem) - symbols[_normal_name(match.group(1))].update((_normal_name(owner), file_owner)) - if language == "kotlin": - for type_match in TYPE_DECLARATION[language].finditer(masked): - opening = masked.find("{", type_match.end()) - header_end = opening if opening >= 0 else masked.find("\n", type_match.end()) - if header_end < 0: - header_end = len(masked) - header = masked[type_match.end():header_end] - for property_match in re.finditer(r"\b(?:val|var)\s+([A-Za-z_][A-Za-z0-9_]*)\b", header): - symbols[_normal_name(property_match.group(1))].update( - (_normal_name(type_match.group(1)), file_owner) - ) - return symbols - - -def _reference_resolves(reference: TwinReference, symbols: dict[str, set[str]]) -> bool: - owners = symbols.get(_normal_name(reference.target_name), set()) - if not owners: - return False - return reference.target_owner is None or _normal_name(reference.target_owner) in owners - - -def _symbol_names(files: list[Path], functions: list[Declaration]) -> set[str]: - """Compatibility helper retained for callers outside this module.""" - names = {_normal_name(item.name) for item in functions} - names.update(_normal_name(path.stem) for path in files) - declaration = re.compile( - r"\b(?:struct|class|enum|actor|protocol|object|interface|typealias|let|var|val)\s+([A-Za-z_][A-Za-z0-9_]*)\b" - ) - for path in files: - masked = _mask_non_code(_read(path)) - names.update(_normal_name(match.group(1)) for match in declaration.finditer(masked)) - return names - - -def _constant_pairing( - swift: list[Constant], kotlin: list[Constant], file_pairs: set[tuple[str, str]] | None = None -) -> tuple[list[tuple[Constant, Constant]], list[tuple[str, list[Constant], list[Constant]]]]: - """Pair normalized constant names, preferring type owner and then mapped/same-stem files.""" - file_pairs = file_pairs or set() - sw_by_name: dict[str, list[Constant]] = defaultdict(list) - kt_by_name: dict[str, list[Constant]] = defaultdict(list) - for item in swift: - sw_by_name[_normal_name(item.name)].append(item) - for item in kotlin: - kt_by_name[_normal_name(item.name)].append(item) - pairs: list[tuple[Constant, Constant]] = [] - ambiguous: list[tuple[str, list[Constant], list[Constant]]] = [] - for name in sorted(sw_by_name.keys() & kt_by_name.keys()): - left = list(sw_by_name[name]) - right = list(kt_by_name[name]) - - def consume(predicate) -> None: - nonlocal left, right - edges = [ - (sw, kt) - for sw in left - for kt in right - if predicate(sw, kt) and (sw.key, kt.key) not in CONSTANT_NON_TWIN_PAIRS - ] - left_degree = Counter(id(sw) for sw, _ in edges) - right_degree = Counter(id(kt) for _, kt in edges) - chosen = [(sw, kt) for sw, kt in edges if left_degree[id(sw)] == 1 and right_degree[id(kt)] == 1] - pairs.extend(chosen) - chosen_left = {id(sw) for sw, _ in chosen} - chosen_right = {id(kt) for _, kt in chosen} - left = [item for item in left if id(item) not in chosen_left] - right = [item for item in right if id(item) not in chosen_right] - - consume(lambda sw, kt: _normal_name(sw.owner) == _normal_name(kt.owner)) - consume(lambda sw, kt: (sw.path, kt.path) in file_pairs) - if (len(left) == 1 and len(right) == 1 - and (left[0].key, right[0].key) not in CONSTANT_NON_TWIN_PAIRS): - pairs.append((left.pop(), right.pop())) - allowed_edges = [ - (sw, kt) - for sw in left - for kt in right - if (sw.key, kt.key) not in CONSTANT_NON_TWIN_PAIRS - ] - if allowed_edges: - allowed_left = {id(sw) for sw, _ in allowed_edges} - allowed_right = {id(kt) for _, kt in allowed_edges} - ambiguous.append(( - name, - sorted((item for item in left if id(item) in allowed_left), key=lambda item: item.key), - sorted((item for item in right if id(item) in allowed_right), key=lambda item: item.key), - )) - return sorted(pairs, key=lambda pair: (pair[0].key, pair[1].key)), ambiguous - - -def _property_candidates( - swift: list[Declaration], kotlin: list[Declaration] -) -> list[tuple[Declaration, Declaration]]: - sw_by_identity: dict[tuple[str, str, str], list[Declaration]] = defaultdict(list) - kt_by_identity: dict[tuple[str, str, str], list[Declaration]] = defaultdict(list) - for item in swift: - sw_by_identity[(_normal_name(item.name), _normal_name(item.owner), _normal_name(Path(item.path).stem))].append(item) - for item in kotlin: - kt_by_identity[(_normal_name(item.name), _normal_name(item.owner), _normal_name(Path(item.path).stem))].append(item) - return [ - (sw_by_identity[key][0], kt_by_identity[key][0]) - for key in sorted(sw_by_identity.keys() & kt_by_identity.keys()) - if len(sw_by_identity[key]) == 1 and len(kt_by_identity[key]) == 1 - ] - - -def _reference_declarations( - root: Path, - inventory_declarations: tuple[ - list[Declaration], list[Declaration], list[Declaration], list[Declaration] - ] | None = None, - snapshot: _SourceSnapshot | None = None, -) -> tuple[list[Path], list[Declaration]]: - snapshot = snapshot or _SourceSnapshot() - if inventory_declarations is None: - inventory = _inventory(root, snapshot) - inventory_declarations = (inventory[2], inventory[3], inventory[4], inventory[5]) - files = _paths(root, REFERENCE_GLOBS) - governed: dict[str, list[Declaration]] = defaultdict(list) - for source_declarations in inventory_declarations: - for item in source_declarations: - governed[item.path].append(item) - declarations: list[Declaration] = [] - for path in files: - relative = _relative(root, path) - existing = governed.get(relative, []) - if existing: - declarations.extend(existing) - continue - language = "swift" if path.suffix == ".swift" else "kotlin" - declarations.extend(snapshot.functions(root, path, language)) - declarations.extend(snapshot.properties(root, path, language)) - return files, declarations - - -def build_twin_map( - root: Path, - inventory: tuple | None = None, - snapshot: _SourceSnapshot | None = None, -) -> dict: - """Derive the full internal semantic inventory from source.""" - root = root.resolve() - snapshot = snapshot or _SourceSnapshot() - inventory = inventory or _inventory(root, snapshot) - ( - sw_files, - kt_files, - sw_funcs, - kt_funcs, - sw_properties, - kt_properties, - sw_consts, - kt_consts, - ) = inventory - reference_files, reference_declarations = _reference_declarations( - root, (sw_funcs, kt_funcs, sw_properties, kt_properties), snapshot - ) - reference_functions = [item for item in reference_declarations if item.kind == "function"] - repo_sw_funcs = [item for item in reference_functions if item.language == "swift"] - repo_kt_funcs = [item for item in reference_functions if item.language == "kotlin"] - refs = parse_twin_references(root, sw_files, "swift", sw_funcs, snapshot) + parse_twin_references(root, kt_files, "kotlin", kt_funcs, snapshot) - pairs = set(resolved_attached_function_pairs(refs, repo_sw_funcs, repo_kt_funcs)) - - paired_sw = {left for left, _ in pairs} - paired_kt = {right for _, right in pairs} - file_pairs = sorted(resolved_file_pairs(pairs, repo_sw_funcs, repo_kt_funcs)) - property_pairs = _property_candidates(sw_properties, kt_properties) - constant_pairs, _ = _constant_pairing(sw_consts, kt_consts, set(file_pairs)) - paired_sw_files = {left for left, _ in file_pairs} - paired_kt_files = {right for _, right in file_pairs} - - sw_unpaired = [item for item in sw_funcs if item.key not in paired_sw] - kt_unpaired = [item for item in kt_funcs if item.key not in paired_kt] - return { - "file_pairs": [ - {"swift": left, "kotlin": right} - for left, right in file_pairs - ], - "function_pairs": [ - {"swift": left, "kotlin": right} - for left, right in sorted(pairs) - ], - "property_pairs": [ - {"swift": left.key, "kotlin": right.key} - for left, right in property_pairs - ], - "constant_pairs": [ - {"swift": left.key, "kotlin": right.key} - for left, right in constant_pairs - ], - "unpaired_files": { - "swift": [_relative(root, path) for path in sw_files if _relative(root, path) not in paired_sw_files], - "kotlin": [_relative(root, path) for path in kt_files if _relative(root, path) not in paired_kt_files], - }, - "unpaired_functions": { - "swift": [item.key for item in sw_unpaired], - "kotlin": [item.key for item in kt_unpaired], - }, - "unpaired_properties": { - "swift": [item.key for item in sw_properties if item.key not in {left.key for left, _ in property_pairs}], - "kotlin": [item.key for item in kt_properties if item.key not in {right.key for _, right in property_pairs}], - }, - } - - -SEMANTIC_AUTHORITY_SETS = ( - "files", - "functions", - "properties", - "constants", - "file_pairs", - "function_pairs", - "property_pairs", - "constant_pairs", - "unpaired_files", - "unpaired_functions", - "unpaired_properties", - "unpaired_constants", -) - - -def _canonical_sha256(value: object) -> str: - encoded = json.dumps( - value, ensure_ascii=False, sort_keys=True, separators=(",", ":") - ).encode("utf-8") - return hashlib.sha256(encoded).hexdigest() - - -def semantic_authority( - root: Path, - *, - expanded: dict | None = None, - inventory: tuple | None = None, - snapshot: _SourceSnapshot | None = None, -) -> dict[str, list[str]]: - """Return exact canonical semantic sets, excluding descriptions and suggestions.""" - root = root.resolve() - snapshot = snapshot or _SourceSnapshot() - inventory = inventory or _inventory(root, snapshot) - expanded = expanded or build_twin_map(root, inventory, snapshot) - ( - sw_files, - kt_files, - sw_funcs, - kt_funcs, - sw_properties, - kt_properties, - sw_constants, - kt_constants, - ) = inventory - - def pairs(name: str) -> list[str]: - return sorted( - f"{item['swift']}\u0000{item['kotlin']}" - for item in expanded[name] - ) - - def both(name: str) -> list[str]: - group = expanded[name] - return sorted([f"swift\u0000{item}" for item in group["swift"]] - + [f"kotlin\u0000{item}" for item in group["kotlin"]]) - - paired_constant_keys = { - item[side] - for item in expanded["constant_pairs"] - for side in ("swift", "kotlin") - } - - return { - "files": sorted( - [f"swift\u0000{_relative(root, path)}" for path in sw_files] - + [f"kotlin\u0000{_relative(root, path)}" for path in kt_files] - ), - "functions": sorted( - [f"swift\u0000{item.key}" for item in sw_funcs] - + [f"kotlin\u0000{item.key}" for item in kt_funcs] - ), - "properties": sorted( - [f"swift\u0000{item.key}" for item in sw_properties] - + [f"kotlin\u0000{item.key}" for item in kt_properties] - ), - "constants": sorted( - [f"swift\u0000{item.key}" for item in sw_constants] - + [f"kotlin\u0000{item.key}" for item in kt_constants] - ), - "file_pairs": pairs("file_pairs"), - "function_pairs": pairs("function_pairs"), - "property_pairs": pairs("property_pairs"), - "constant_pairs": pairs("constant_pairs"), - "unpaired_files": both("unpaired_files"), - "unpaired_functions": both("unpaired_functions"), - "unpaired_properties": both("unpaired_properties"), - "unpaired_constants": sorted( - [f"swift\u0000{item.key}" for item in sw_constants if item.key not in paired_constant_keys] - + [f"kotlin\u0000{item.key}" for item in kt_constants if item.key not in paired_constant_keys] - ), - } - - -def authority_manifest(sets: dict[str, list[str]]) -> dict[str, dict[str, object]]: - return { - name: {"count": len(sets[name]), "sha256": _canonical_sha256(sets[name])} - for name in SEMANTIC_AUTHORITY_SETS - } - - -def _base_semantic_state( - root: Path, -) -> tuple[dict[str, list[str]], set[tuple[str, str, str, int]]] | None: - """Return semantic sets and function identities from the comparison base.""" - try: - try: - base = subprocess.check_output( - ["git", "merge-base", "HEAD", "origin/main"], - cwd=root, - text=True, - stderr=subprocess.DEVNULL, - ).strip() - except subprocess.CalledProcessError: - base = "HEAD" - archive = subprocess.check_output( - ["git", "archive", "--format=tar", base], - cwd=root, - stderr=subprocess.DEVNULL, - ) - with tempfile.TemporaryDirectory() as directory: - # Resolve before anything is derived from it: build_twin_map resolves its root, and an - # inventory taken from the unresolved spelling then fails relative_to (#2143, macOS). - base_root = Path(directory).resolve() - with tarfile.open(fileobj=io.BytesIO(archive), mode="r:") as bundle: - bundle.extractall(base_root, filter="data") - inventory = _inventory(base_root) - expanded = build_twin_map(base_root, inventory) - semantic_sets = semantic_authority( - base_root, expanded=expanded, inventory=inventory - ) - except (OSError, subprocess.CalledProcessError, tarfile.TarError, TypeError, - _InvalidSourceEncoding): - return None - function_identities = { - (declaration.language, declaration.owner, declaration.name, declaration.arity) - for declaration in [*inventory[2], *inventory[3]] - } - return semantic_sets, function_identities - - -def _authority_change_requires_refresh( - section: str, - base_sets: dict[str, list[str]], - current_sets: dict[str, list[str]], -) -> bool: - before = set(base_sets[section]) - after = set(current_sets[section]) - if section.startswith("unpaired_"): - return not after.issubset(before) - return not before.issubset(after) - - -def _new_unpaired_diagnostics( - semantic_sets: dict[str, list[str]], - declarations: Iterable[Declaration], - base_functions: set[tuple[str, str, str, int]], -) -> list[Finding]: - """Name locally added one-sided declarations while compact authority is stale.""" - unpaired = set(semantic_sets["unpaired_functions"]) - return [ - _finding( - "add-unpaired-function", - declaration.path, - declaration.line, - f"new one-sided {declaration.language} function {declaration.key}", - f"add-unpaired-function|{declaration.language}|{declaration.key}", - ) - for declaration in declarations - if f"{declaration.language}\0{declaration.key}" in unpaired - and ( - declaration.language, - declaration.owner, - declaration.name, - declaration.arity, - ) not in base_functions - ] - - -def build_compact_twin_map(root: Path) -> dict: - """Freeze derived semantic sets without checking in repeated inventory rows.""" - manifest = authority_manifest(semantic_authority(root)) - return { - "schema_version": 3, - "derivation": "parity_ledger.build_twin_map/v3", - "scope": { - "swift_roots": [glob.split("/**", 1)[0] for glob in SWIFT_GLOBS], - "kotlin_roots": [glob.split("/**", 1)[0] for glob in KOTLIN_GLOBS], - }, - "authority": manifest, - } - - -def expand_twin_map(root: Path, twin_map: dict) -> tuple[dict, list[str]]: - """Expand v3 from source and report every frozen-authority mismatch.""" - # Resolve before taking the inventory, as build_twin_map and semantic_authority do: both - # receive this inventory and resolve their own root, so the spellings must already agree. - root = root.resolve() - if twin_map.get("schema_version") != 3: - return twin_map, [] - inventory = _inventory(root) - expanded = build_twin_map(root, inventory) - checked = twin_map.get("authority", {}) - current = authority_manifest( - semantic_authority(root, expanded=expanded, inventory=inventory) - ) - drift = [ - key for key in SEMANTIC_AUTHORITY_SETS - if not isinstance(checked, dict) or checked.get(key) != current[key] - ] - return expanded, drift - - -def _finding(rule: str, path: str, line: int, text: str, identity: str) -> Finding: - return Finding(rule, path, line, text, f"{rule}|{identity}") - - -def _twin_map_consistency_findings( - twin_map: dict, - *, - files: set[str], - functions: set[str], - properties: set[str], -) -> list[Finding]: - """Validate the curated map itself instead of trusting it as inventory truth. - - Map regeneration is intentionally not used here: hand-curated evidence and - cross-name pairs are authoritative, but every asserted or explicitly unpaired - key must still resolve exactly and occur in only one state. - """ - - findings: list[Finding] = [] - categories = ( - ("file_pairs", "unpaired_files", files, "file"), - ("function_pairs", "unpaired_functions", functions, "function"), - ("property_pairs", "unpaired_properties", properties, "property"), - ) - for pair_name, unpaired_name, inventory, kind in categories: - pairs = twin_map.get(pair_name, []) - unpaired = twin_map.get(unpaired_name, {}) - if not isinstance(pairs, list) or not isinstance(unpaired, dict): - findings.append( - _finding( - "malformed-twin-map", DEFAULT_MAP.relative_to(ROOT).as_posix(), 1, - f"{pair_name}/{unpaired_name} must be an array/object", - f"{pair_name}|{unpaired_name}", - ) - ) - continue - for side in ("swift", "kotlin"): - paired = [entry.get(side) for entry in pairs if isinstance(entry, dict)] - paired_keys = [key for key in paired if isinstance(key, str)] - unpaired_keys = unpaired.get(side, []) - if not isinstance(unpaired_keys, list): - findings.append( - _finding( - "malformed-twin-map", DEFAULT_MAP.relative_to(ROOT).as_posix(), 1, - f"{unpaired_name}.{side} must be an array", - f"{unpaired_name}|{side}", - ) - ) - continue - for key, count in sorted(Counter(paired_keys).items()): - # A source file may intentionally contain declarations paired - # with more than one counterpart file. Declaration targets, - # however, are one-to-one and may never be reused. - if kind != "file" and count > 1: - path = key.split("::", 1)[0] - findings.append( - _finding( - "duplicate-twin-target", path, 1, - f"{kind} target {key} appears in {count} {pair_name} entries", - f"{pair_name}|{side}|{key}", - ) - ) - for key in sorted(set(paired_keys) & set(unpaired_keys)): - path = key.split("::", 1)[0] - findings.append( - _finding( - "twin-map-overlap", path, 1, - f"{kind} {key} is both paired and explicitly unpaired", - f"{kind}|{side}|{key}", - ) - ) - for key in sorted(set(paired_keys) | set(unpaired_keys)): - if key not in inventory: - path = key.split("::", 1)[0] - findings.append( - _finding( - f"stale-twin-{kind}", path, 1, - f"{kind} key {key} does not resolve exactly in the current inventory", - f"{side}|{key}", - ) - ) - return findings - - -def _mapped_sets(twin_map: dict) -> tuple[set[str], set[str], set[str]]: - files: set[str] = set() - functions: set[str] = set() - properties: set[str] = set() - for entry in twin_map.get("file_pairs", []): - files.update((entry["swift"], entry["kotlin"])) - unpaired_files = twin_map.get("unpaired_files", {}) - if isinstance(unpaired_files, dict): - files.update(unpaired_files.get("swift", [])) - files.update(unpaired_files.get("kotlin", [])) - for entry in twin_map.get("function_pairs", []): - functions.update((entry["swift"], entry["kotlin"])) - unpaired_functions = twin_map.get("unpaired_functions", {}) - if isinstance(unpaired_functions, dict): - functions.update(unpaired_functions.get("swift", [])) - functions.update(unpaired_functions.get("kotlin", [])) - for entry in twin_map.get("property_pairs", []): - properties.update((entry["swift"], entry["kotlin"])) - unpaired_properties = twin_map.get("unpaired_properties", {}) - if isinstance(unpaired_properties, dict): - properties.update(unpaired_properties.get("swift", [])) - properties.update(unpaired_properties.get("kotlin", [])) - return files, functions, properties - - -def _bootstrap_unpaired_debts( - identities: set[str], - twin_map: dict, - swift_functions: list[Declaration], - kotlin_functions: list[Declaration], -) -> set[str]: - declarations = { - "swift": {item.key: item for item in swift_functions}, - "kotlin": {item.key: item for item in kotlin_functions}, - } - unpaired = { - language: set(twin_map.get("unpaired_functions", {}).get(language, [])) - for language in ("swift", "kotlin") - } - debts: set[str] = set() - for identity in identities: - if not isinstance(identity, str) or "\0" not in identity: - continue - language, key = identity.split("\0", 1) - opposite = "kotlin" if language == "swift" else "swift" - declaration = declarations.get(language, {}).get(key) - if declaration is None or key not in unpaired.get(language, set()): - continue - counterparts = [ - item for item in declarations[opposite].values() - if _normal_name(item.name) == _normal_name(declaration.name) - and _normal_name(item.owner) == _normal_name(declaration.owner) - ] - if not counterparts: - debts.add(identity) - return debts - - -def bootstrap_unpaired_debts(root: Path, identities: set[str]) -> set[str]: - """Independently derive reviewed existing declaration debt without comment reliance.""" - inventory = _inventory(root.resolve()) - return _bootstrap_unpaired_debts( - identities, - build_twin_map(root, inventory), - inventory[2], - inventory[3], - ) - - -def _call_sites( - root: Path, - globs: tuple[str, ...], - errors: list[ScanError], - snapshot: _SourceSnapshot | None = None, -) -> dict[str, list[CallSite]]: - snapshot = snapshot or _SourceSnapshot() - calls: dict[str, list[CallSite]] = {"swift": [], "kotlin": []} - for path in _paths(root, globs): - language = "swift" if path.suffix == ".swift" else "kotlin" - text = snapshot.text(path) - try: - masked = snapshot.masked(path, kotlin_templates=language == "kotlin") - except _MalformedKotlinTemplate as error: - errors.append( - ScanError( - "malformed-kotlin-template", - _relative(root, path), - text.count("\n", 0, error.offset) + 1, - str(error), - ) - ) - continue - declarations = snapshot.functions(root, path, language) - declaration_openings = {item.opening for item in declarations} - spans = _type_spans(masked, language) - fallback_owner = path.stem.replace("+Trace", "Trace") - for match in re.finditer(r"\b(`?[A-Za-z_][A-Za-z0-9_]*`?)\s*\(", masked): - opening = match.end() - 1 - if opening in declaration_openings: - continue - arity = _arity(masked, opening) - if arity is None: - continue - prefix = masked[max(0, match.start() - 100) : match.start()] - receiver_match = re.search(r"(`?[A-Za-z_][A-Za-z0-9_]*`?)\s*\.\s*$", prefix) - lexical_owner = _owner_at(spans, match.start(), fallback_owner) - owner = receiver_match.group(1).strip("`") if receiver_match else None - if owner in {"self", "this", "Self"}: - owner = lexical_owner - elif owner is not None and owner[:1].islower(): - # `burst.codesWithTimes(...)`: the receiver is an instance variable; its - # type is not recoverable lexically, so resolve like an unqualified call. - owner = None - calls[language].append( - CallSite(match.group(1).strip("`"), arity, owner, _relative(root, path), lexical_owner) - ) - return calls - - -def _declaration_call_counts( - declarations: list[Declaration], sites: dict[str, list[CallSite]] -) -> dict[str, int]: - by_name: dict[tuple[str, str], list[Declaration]] = defaultdict(list) - for declaration in declarations: - by_name[(declaration.language, _normal_name(declaration.name))].append(declaration) - counts: Counter[str] = Counter() - for language, language_sites in sites.items(): - for site in language_sites: - # A call may omit defaulted parameters, so besides exact-arity declarations it - # can target any same-name declaration with MORE parameters (false test-only - # finding for HrvAnalyzer.rollingRmssd/4: its only production call leaves - # minBeatsPerWindow defaulted). Exact-arity candidates win outright so that the - # relaxed pool cannot introduce owner ambiguity where none existed before. - named = by_name.get((language, _normal_name(site.name)), []) - exact = [item for item in named if item.arity == site.arity] - relaxed = [ - item for item in named - if item.required_arity <= site.arity < item.arity - ] - - def _owner_filtered(pool: list[Declaration]) -> list[Declaration]: - if site.owner: - owner = _normal_name(site.owner) - return [ - item - for item in pool - if owner in {_normal_name(item.owner), _normal_name(Path(item.path).stem)} - ] - # Unqualified call: same-file declarations first (self-scope calls) — - # within the file, prefer the type whose span the call sits in, so three - # same-named members of sibling types don't all take credit. Then the - # global pool, only when its owner is unambiguous. - local = [item for item in pool if item.path == site.path] - if local: - if site.lexical_owner: - lexical = _normal_name(site.lexical_owner) - scoped = [item for item in local if _normal_name(item.owner) == lexical] - if scoped: - return scoped - return local - owners = {_normal_name(item.owner) for item in pool} - return pool if len(owners) == 1 else [] - - candidates = _owner_filtered(exact) or _owner_filtered(relaxed) - for candidate in candidates: - counts[candidate.key] += 1 - return dict(counts) - - -def scan(root: Path, twin_map: dict) -> ScanResult: - root = root.resolve() - snapshot = _SourceSnapshot() - compact_exemptions = twin_map.get("exemptions", []) - accepted_missing_claimants = { - item.get("identity") - for item in compact_exemptions - if isinstance(item, dict) and item.get("kind") == "bootstrap-unpaired-function" - } - try: - inventory = _inventory(root, snapshot) - if twin_map.get("schema_version") == 3: - expanded = build_twin_map(root, inventory, snapshot) - semantic_sets = semantic_authority( - root, expanded=expanded, inventory=inventory, snapshot=snapshot - ) - current = authority_manifest(semantic_sets) - checked = twin_map.get("authority", {}) - authority_drift = [ - key for key in SEMANTIC_AUTHORITY_SETS - if not isinstance(checked, dict) or checked.get(key) != current[key] - ] - base_state = _base_semantic_state(root) if authority_drift else None - if base_state is not None: - base_sets, base_functions = base_state - authority_drift = [ - section for section in authority_drift - if _authority_change_requires_refresh( - section, base_sets, semantic_sets - ) - ] - twin_map = expanded - else: - authority_drift = [] - base_state = None - base_functions = set() - semantic_sets = {} - ( - sw_files, - kt_files, - sw_funcs, - kt_funcs, - sw_properties, - kt_properties, - sw_consts, - kt_consts, - ) = inventory - reference_files, all_reference_declarations = _reference_declarations( - root, (sw_funcs, kt_funcs, sw_properties, kt_properties), snapshot - ) - repo_sw_funcs = [item for item in all_reference_declarations if item.language == "swift" and item.kind == "function"] - repo_kt_funcs = [item for item in all_reference_declarations if item.language == "kotlin" and item.kind == "function"] - except _InvalidSourceEncoding as error: - return ScanResult( - [], {}, {}, - [ScanError("invalid-utf8", _relative(root, error.path), 1, str(error))], - set(), set(), - ) - findings: list[Finding] = [ - _finding( - "twin-map-authority-drift", - DEFAULT_MAP.relative_to(ROOT).as_posix(), - 1, - f"derived twin-map authority changed in {section}", - section, - ) - for section in authority_drift - ] - if "unpaired_functions" in authority_drift and base_state is not None: - findings.extend( - _new_unpaired_diagnostics( - semantic_sets, [*sw_funcs, *kt_funcs], base_functions - ) - ) - errors: list[ScanError] = [] - findings.extend( - _twin_map_consistency_findings( - twin_map, - files={_relative(root, path) for path in reference_files}, - functions={item.key for item in all_reference_declarations if item.kind == "function"}, - properties={item.key for item in all_reference_declarations if item.kind == "property"}, - ) - ) - mapped_files, mapped_functions, mapped_properties = _mapped_sets(twin_map) - bootstrap_unpaired_debts = _bootstrap_unpaired_debts( - accepted_missing_claimants, twin_map, sw_funcs, kt_funcs - ) - - source_refs = ( - parse_twin_references(root, sw_files, "swift", sw_funcs, snapshot) - + parse_twin_references(root, kt_files, "kotlin", kt_funcs, snapshot) - ) - source_resolutions = attached_function_resolutions( - source_refs, repo_sw_funcs, repo_kt_funcs - ) - missing_attached_claimants = { - f"{reference.language}\0{reference.attached_function}" - for reference, candidates in source_resolutions.items() - if not candidates and reference.attached_function is not None - } - declared_pair_claims = resolved_attached_function_pairs( - source_refs, - repo_sw_funcs, - repo_kt_funcs, - resolutions=source_resolutions, - ) - declared_pairs = set(declared_pair_claims) - mapped_function_pairs = { - (entry["swift"], entry["kotlin"]) - for entry in twin_map.get("function_pairs", []) - if isinstance(entry, dict) - and isinstance(entry.get("swift"), str) - and isinstance(entry.get("kotlin"), str) - } - for reference, candidates in source_resolutions.items(): - if len(candidates) == 1: - continue - claimant = f"{reference.language}\0{reference.attached_function}" - if not candidates and claimant in accepted_missing_claimants: - continue - candidate_keys = tuple(item.key for item in candidates) - targets = repo_kt_funcs if reference.language == "swift" else repo_sw_funcs - if not candidates: - rule = "unresolved-attached-function-claim" - detail = "does not resolve to an inventory function" - else: - rule = "ambiguous-attached-function-claim" - detail = f"resolves to {len(candidates)} inventory functions: {', '.join(candidate_keys)}" - findings.append( - _finding( - rule, - reference.path, - reference.line, - f"attached twin claim {reference.raw_target} {detail}", - ( - f"{reference.path}|{reference.raw_target}|" - f"{reference.attached_function}|{reference.claim_ordinal}|" - f"{'|'.join(candidate_keys)}" - ), - ) - ) - for identity in sorted(accepted_missing_claimants - bootstrap_unpaired_debts): - findings.append( - _finding( - "stale-bootstrap-exemption", - identity.split("\0", 1)[-1].split("::", 1)[0], - 1, - f"bootstrap unpaired-function exemption no longer matches a declaration without a counterpart: {identity}", - identity, - ) - ) - for swift_key, kotlin_key in sorted(declared_pairs - mapped_function_pairs): - claim = declared_pair_claims[(swift_key, kotlin_key)][0] - findings.append( - _finding( - "unmapped-declared-function-pair", - claim.path, - claim.line, - f"attached twin claim {swift_key} -> {kotlin_key} is absent from the twin map", - f"{swift_key}|{kotlin_key}", - ) - ) - for swift_key, kotlin_key in sorted(mapped_function_pairs - declared_pairs): - findings.append( - _finding( - "stale-declared-function-pair", - swift_key.split("::", 1)[0], - 1, - f"mapped function pair {swift_key} -> {kotlin_key} has no resolved attached source claim", - f"{swift_key}|{kotlin_key}", - ) - ) - - declared_file_pairs = resolved_file_pairs( - declared_pairs, repo_sw_funcs, repo_kt_funcs - ) - checked_file_pairs = { - (entry["swift"], entry["kotlin"]) - for entry in twin_map.get("file_pairs", []) - if isinstance(entry, dict) - and isinstance(entry.get("swift"), str) - and isinstance(entry.get("kotlin"), str) - } - for swift_path, kotlin_path in sorted(declared_file_pairs - checked_file_pairs): - findings.append( - _finding( - "unmapped-declared-file-pair", - swift_path, - 1, - f"source-declared file pair {swift_path} -> {kotlin_path} is absent from the twin map", - f"{swift_path}|{kotlin_path}", - ) - ) - for swift_path, kotlin_path in sorted(checked_file_pairs - declared_file_pairs): - findings.append( - _finding( - "stale-declared-file-pair", - swift_path, - 1, - f"mapped file pair {swift_path} -> {kotlin_path} has no resolved source function claim", - f"{swift_path}|{kotlin_path}", - ) - ) - - for language, files in (("Swift", sw_files), ("Kotlin", kt_files)): - for path in files: - rel = _relative(root, path) - if rel not in mapped_files: - findings.append(_finding("unmapped-file", rel, 1, f"new {language} file has no twin-map entry", rel)) - for declaration in sw_funcs + kt_funcs: - if declaration.key not in mapped_functions: - findings.append( - _finding( - "unmapped-function", - declaration.path, - declaration.line, - f"{declaration.name}/{declaration.arity} has no twin-map entry", - declaration.key, - ) - ) - for declaration in sw_properties + kt_properties: - if declaration.key not in mapped_properties: - findings.append( - _finding( - "unmapped-property", - declaration.path, - declaration.line, - f"{declaration.owner}.{declaration.name} has no twin-map entry", - declaration.key, - ) - ) - - all_production_files = _paths(root, PRODUCTION_GLOBS) - all_functions: list[Declaration] = [] - all_properties: list[Declaration] = [] - for path in all_production_files: - language = "swift" if path.suffix == ".swift" else "kotlin" - all_functions.extend(snapshot.functions(root, path, language)) - all_properties.extend(snapshot.properties(root, path, language)) - all_swift_files = [path for path in reference_files if path.suffix == ".swift"] - all_kotlin_files = [path for path in reference_files if path.suffix == ".kt"] - all_swift_functions = [item for item in all_functions if item.language == "swift"] - all_kotlin_functions = [item for item in all_functions if item.language == "kotlin"] - - reference_swift_declarations = [item for item in all_reference_declarations if item.language == "swift"] - reference_kotlin_declarations = [item for item in all_reference_declarations if item.language == "kotlin"] - refs = parse_twin_references(root, all_swift_files, "swift", reference_swift_declarations, snapshot) + parse_twin_references( - root, - all_kotlin_files, - "kotlin", - reference_kotlin_declarations, - snapshot, - ) - swift_symbols = _symbol_owners( - root, - all_swift_files, - reference_swift_declarations, - snapshot, - ) - kotlin_symbols = _symbol_owners( - root, - all_kotlin_files, - reference_kotlin_declarations, - snapshot, - ) - resolved_refs = 0 - for reference in refs: - target_symbols = kotlin_symbols if reference.language == "swift" else swift_symbols - if _reference_resolves(reference, target_symbols): - resolved_refs += 1 - continue - claimant = ( - f"{reference.language}\0{reference.attached_function}" - if reference.attached_function is not None else None - ) - if claimant in accepted_missing_claimants: - continue - target_language = "Kotlin" if reference.language == "swift" else "Swift" - findings.append( - _finding( - "dead-twin-reference", - reference.path, - reference.line, - f"{target_language} target {reference.raw_target} does not resolve", - ( - f"{reference.path}|{reference.raw_target}|" - f"{reference.attached_function or ''}|{reference.claim_ordinal}" - ), - ) - ) - - constants_by_key = {item.key: item for item in sw_consts + kt_consts} - paired_constants, ambiguous_constants = _constant_pairing( - sw_consts, kt_consts, declared_file_pairs - ) - dynamic_pairs = {(left.key, right.key) for left, right in paired_constants} - mapped_pairs = {(entry["swift"], entry["kotlin"]) for entry in twin_map.get("constant_pairs", [])} - for left_key, right_key in sorted(dynamic_pairs - mapped_pairs): - left = constants_by_key[left_key] - right = constants_by_key[right_key] - findings.append( - _finding( - "unmapped-constant-pair", - left.path, - left.line, - f"mirrored constant pair {left_key} -> {right_key} is absent from the twin map", - f"{left_key}|{right_key}", - ) - ) - for left_key, right_key in sorted(dynamic_pairs | mapped_pairs): - left = constants_by_key.get(left_key) - right = constants_by_key.get(right_key) - if left is None or right is None: - if (left_key, right_key) in mapped_pairs: - present = left or right - missing = left_key if left is None else right_key - findings.append( - _finding( - "stale-constant-pair", - present.path if present is not None else left_key.split("::", 1)[0], - present.line if present is not None else 1, - f"mapped constant {missing} does not resolve", - f"{left_key}|{right_key}", - ) - ) - continue - if left.value is None or right.value is None: - findings.append( - _finding( - "constant-unverifiable", - left.path, - left.line, - f"cannot fully evaluate {left.name}={left.display_value} against Kotlin {right.name}={right.display_value} ({right.path}:{right.line})", - f"{left_key}|{left.display_value}|{right_key}|{right.display_value}", - ) - ) - continue - if left.value == right.value: - continue - findings.append( - _finding( - "constant-value-mismatch", - left.path, - left.line, - f"{left.name}={left.display_value} differs from Kotlin {right.name}={right.display_value} ({right.path}:{right.line})", - f"{left_key}|{left.value}|{right_key}|{right.value}", - ) - ) - - for normal_name, left, right in ambiguous_constants: - left_keys = ", ".join(item.key for item in left) - right_keys = ", ".join(item.key for item in right) - first = left[0] - findings.append( - _finding( - "constant-ambiguous", - first.path, - first.line, - f"{normal_name} cannot be paired uniquely: Swift [{left_keys}] vs Kotlin [{right_keys}]", - f"{normal_name}|{'|'.join(item.key for item in left)}|{'|'.join(item.key for item in right)}", - ) - ) - - inventory_functions = sw_funcs + kt_funcs - prod_calls = _declaration_call_counts( - inventory_functions, _call_sites(root, PRODUCTION_GLOBS, errors, snapshot) - ) - test_calls = _declaration_call_counts( - inventory_functions, _call_sites(root, TEST_GLOBS, errors, snapshot) - ) - for declaration in sw_funcs + kt_funcs: - if test_calls.get(declaration.key, 0) > 0 and prod_calls.get(declaration.key, 0) == 0: - findings.append( - _finding( - "test-only-callsite", - declaration.path, - declaration.line, - f"{declaration.owner}.{declaration.name}/{declaration.arity} has {test_calls[declaration.key]} test callsite(s) and no production callsite", - declaration.key, - ) - ) - - day_string = [item for item in all_functions if item.name == "dayString"] - ui_pearson = [ - item for item in all_functions - if item.language == "kotlin" and item.name == "pearson" and item.path.startswith("android/app/src/main/java/com/noop/ui/") - ] - resting = [item for item in sw_funcs + kt_funcs if item.name in {"restingHR", "sessionRestingHR"}] - for declaration in day_string: - findings.append( - _finding( - "duplicate-implementation", - declaration.path, - declaration.line, - f"dayString implementation {declaration.owner}.{declaration.name}/{declaration.arity}", - f"dayString|{declaration.key}", - ) - ) - for declaration in ui_pearson: - findings.append( - _finding( - "duplicate-implementation", - declaration.path, - declaration.line, - f"independent Android UI Pearson implementation {declaration.owner}.pearson/{declaration.arity}", - f"android-ui-pearson|{declaration.key}", - ) - ) - for declaration in resting: - findings.append( - _finding( - "duplicate-implementation", - declaration.path, - declaration.line, - f"resting-HR path {declaration.owner}.{declaration.name}/{declaration.arity}", - f"resting-hr|{declaration.key}", - ) - ) - - findings.sort(key=lambda item: (item.path, item.line, item.rule, item.identity)) - counters = { - "day_string_implementations": len(day_string), - "resting_hr_paths": len({item.name for item in resting}), - "android_ui_pearson_implementations": len(ui_pearson), - } - stats = { - "swift_files": len(sw_files), - "kotlin_files": len(kt_files), - "swift_functions": len(sw_funcs), - "kotlin_functions": len(kt_funcs), - "swift_properties": len(sw_properties), - "kotlin_properties": len(kt_properties), - "swift_constants": len(sw_consts), - "kotlin_constants": len(kt_consts), - "swift_parity_annotations": _annotation_count(sw_files, snapshot), - "kotlin_parity_annotations": _annotation_count(kt_files, snapshot), - "declared_twin_references": len(refs), - "resolved_twin_references": resolved_refs, - "constant_pairs": len(dynamic_pairs | mapped_pairs), - } - errors.extend( - ScanError(item.rule, item.path, item.line, item.text) - for item in findings - if item.rule in HARD_FINDING_RULES - ) - errors = sorted( - set(errors), - key=lambda item: (item.path, item.line, item.rule, item.text), - ) - return ScanResult( - findings, counters, stats, errors, missing_attached_claimants, - bootstrap_unpaired_debts, - ) - - -def _load_json(path: Path, default: dict) -> dict: - if not path.exists(): - return default - value = json.loads(path.read_text()) - issue_ref.validate_current_issue_fields(value, str(path)) - return value - - -def _write_json(path: Path, value: dict) -> None: - path.parent.mkdir(parents=True, exist_ok=True) - if value.get("schema_version") == 3: - def render(node: object, level: int = 0) -> list[str]: - pad = " " * level - if isinstance(node, dict): - if node and all(not isinstance(child, (dict, list)) for child in node.values()): - return [pad + json.dumps(node, ensure_ascii=False, separators=(", ", ": "))] - lines = [pad + "{"] - items = list(node.items()) - for index, (key, child) in enumerate(items): - rendered = render(child, level + 1) - prefix = " " * (level + 1) + json.dumps(key) + ": " - rendered[0] = prefix + rendered[0].lstrip() - if index + 1 < len(items): - rendered[-1] += "," - lines.extend(rendered) - lines.append(pad + "}") - return lines - if isinstance(node, list): - if not node: - return [pad + "[]"] - lines = [pad + "["] - for index, child in enumerate(node): - rendered = render(child, level + 1) - if index + 1 < len(node): - rendered[-1] += "," - lines.extend(rendered) - lines.append(pad + "]") - return lines - return [pad + json.dumps(node, ensure_ascii=False)] - - path.write_text("\n".join(render(value)) + "\n", encoding="utf-8") - else: - path.write_text(json.dumps(value, indent=2, sort_keys=False) + "\n", encoding="utf-8") - - -def _publish_json_pair(first_path: Path, first: dict, second_path: Path, second: dict) -> None: - """Publish two JSON snapshots as one rollback-safe operation.""" - paths = (first_path, second_path) - values = (first, second) - old = [(path.exists(), path.read_bytes() if path.exists() else b"") for path in paths] - temporary: list[Path] = [] - try: - for path, value in zip(paths, values): - path.parent.mkdir(parents=True, exist_ok=True) - descriptor, name = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent) - os.close(descriptor) - candidate = Path(name) - temporary.append(candidate) - _write_json(candidate, value) - os.replace(temporary[0], first_path) - temporary.pop(0) - os.replace(temporary[0], second_path) - temporary.pop(0) - except BaseException: - for path, (existed, content) in zip(paths, old): - if existed: - descriptor, name = tempfile.mkstemp(prefix=f".{path.name}.rollback.", dir=path.parent) - try: - with os.fdopen(descriptor, "wb") as handle: - handle.write(content) - os.replace(name, path) - finally: - Path(name).unlink(missing_ok=True) - else: - path.unlink(missing_ok=True) - raise - finally: - for path in temporary: - path.unlink(missing_ok=True) - - -BASELINE_REASONS = { - "test-only-callsite": "Exact declaration is reached only from tests in the conservative lexical call graph; dynamic, callback and external entry points are intentionally not inferred.", - "duplicate-implementation": "Exact platform-local implementation is retained as visible parity debt pending a source-level consolidation decision.", - "constant-unverifiable": "Exact mirrored constant uses a lexical expression that the standard-library scanner cannot evaluate safely.", - "constant-value-mismatch": "Exact mirrored constant values differ on current upstream and remain visible as parity debt.", - "constant-ambiguous": "Exact normalized constant name has multiple viable cross-language matches and remains visible until explicitly disambiguated.", -} - - -def _finding_domain(path: str) -> str: - pieces = path.split("/") - if path.startswith("android/app/src/main/java/com/noop/") and len(pieces) > 7: - return f"android/{pieces[7]}" - if path.startswith("Packages/") and len(pieces) > 1: - return f"Packages/{pieces[1]}" - return pieces[0] - - -def baseline_group_provenance(rule: str, scope: str) -> str: - return ( - f"Exact identities emitted by parity_ledger rule {rule} " - f"for current upstream scope {scope}; no wildcard matching." - ) - - -def build_compact_baseline(result: ScanResult) -> dict: - """Group exact accepted identities while retaining narrow review provenance.""" - grouped: dict[tuple[str, str], list[str]] = defaultdict(list) - for finding in result.findings: - if finding.rule in HARD_FINDING_RULES: - continue - if finding.rule not in BASELINE_REASONS: - raise ValueError( - f"finding {finding.identity} has no reviewed compact-baseline disposition" - ) - grouped[(finding.rule, _finding_domain(finding.path))].append(finding.identity) - return { - "schema_version": 3, - "accepted_findings": [ - { - "rule": rule, - "scope": domain, - "reason": BASELINE_REASONS[rule], - "provenance": baseline_group_provenance(rule, domain), - "count": len(identities), - "identities_sha256": _canonical_sha256(sorted(identities)), - } - for (rule, domain), identities in sorted(grouped.items()) - ], - "counters": result.counters, - } - - -def compact_baseline_drift(result: ScanResult, baseline: dict) -> list[str]: - try: - current = build_compact_baseline(result) - except ValueError as exc: - return [str(exc)] - checked_groups = { - (item.get("rule"), item.get("scope")): item - for item in baseline.get("accepted_findings", []) - if isinstance(item, dict) - } - current_groups = { - (item["rule"], item["scope"]): item - for item in current["accepted_findings"] - } - return [ - f"{rule}|{scope}" - for rule, scope in sorted(set(checked_groups) | set(current_groups)) - if checked_groups.get((rule, scope), {}).get("count") - != current_groups.get((rule, scope), {}).get("count") - or checked_groups.get((rule, scope), {}).get("identities_sha256") - != current_groups.get((rule, scope), {}).get("identities_sha256") - ] - - -def compact_baseline_changes( - result: ScanResult, baseline: dict -) -> tuple[list[str], list[str]]: - """Return (regressions, improvements) for compact finding groups.""" - try: - current = build_compact_baseline(result) - except ValueError as exc: - return [str(exc)], [] - checked_groups = { - (item.get("rule"), item.get("scope")): item - for item in baseline.get("accepted_findings", []) - if isinstance(item, dict) - } - current_groups = { - (item["rule"], item["scope"]): item - for item in current["accepted_findings"] - } - regressions: list[str] = [] - improvements: list[str] = [] - for key in sorted(set(checked_groups) | set(current_groups)): - old = checked_groups.get(key) - new = current_groups.get(key) - label = f"{key[0]}|{key[1]}" - if old is None: - regressions.append(label) - elif new is None or new["count"] < old.get("count", 0): - improvements.append(label) - elif (new["count"] > old.get("count", 0) - or new["identities_sha256"] != old.get("identities_sha256")): - regressions.append(label) - return regressions, improvements - - -def finding_identities_at_git_ref(root: Path, ref: str) -> set[str]: - """Independently scan an exact git tree for monotonic baseline proof.""" - try: - resolved = subprocess.check_output( - ["git", "rev-parse", "--verify", f"{ref}^{{commit}}"], - cwd=root, - text=True, - stderr=subprocess.DEVNULL, - ).strip() - if re.fullmatch(r"[0-9a-f]{40}|[0-9a-f]{64}", resolved) is None: - raise ValueError(f"git returned an invalid commit for base {ref!r}") - archive = subprocess.check_output( - ["git", "archive", "--format=tar", resolved], cwd=root - ) - except (FileNotFoundError, subprocess.CalledProcessError) as exc: - raise ValueError(f"cannot scan exact base {ref!r}") from exc - with tempfile.TemporaryDirectory() as directory: - # Same resolution as _base_semantic_state, so both temp checkouts spell their root one way (#2143). - base_root = Path(directory).resolve() - try: - with tarfile.open(fileobj=io.BytesIO(archive), mode="r:") as bundle: - bundle.extractall(base_root, filter="data") - except (tarfile.TarError, TypeError) as exc: - raise ValueError(f"cannot materialize exact base {ref!r}") from exc - base_map = build_compact_twin_map(base_root) - base_result = scan(base_root, base_map) - if base_result.errors: - raise ValueError(f"cannot prove improvement against invalid base {ref!r}") - return {finding.identity for finding in base_result.findings} - - -def _summary(result: ScanResult) -> str: - stats = result.stats - return ( - f"{stats['swift_files']} Swift files, {stats['kotlin_files']} Kotlin files; " - f"{stats['swift_functions']} Swift functions, {stats['kotlin_functions']} Kotlin functions; " - f"{stats['swift_properties']} Swift properties, {stats['kotlin_properties']} Kotlin properties; " - f"{stats['swift_constants']} Swift constants, {stats['kotlin_constants']} Kotlin constants; " - f"annotations Swift={stats['swift_parity_annotations']}, Kotlin={stats['kotlin_parity_annotations']}; " - f"{stats['declared_twin_references']} declared twin references " - f"({stats['resolved_twin_references']} resolved); {stats['constant_pairs']} constant pairs; " - f"counters dayString={result.counters['day_string_implementations']}, " - f"resting-HR={result.counters['resting_hr_paths']}, " - f"Android-UI-Pearson={result.counters['android_ui_pearson_implementations']}" - ) - - -def main(argv: list[str] | None = None) -> int: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--root", type=Path, default=ROOT, help="repository root") - parser.add_argument("--map", dest="map_path", type=Path, help="twin-map JSON path") - parser.add_argument("--baseline", dest="baseline_path", type=Path, help="baseline JSON path") - parser.add_argument("--no-baseline", action="store_true", help="show every current finding") - parser.add_argument("--bootstrap-map", action="store_true", help="write a fresh inventory map before scanning") - parser.add_argument("--write-baseline", action="store_true", help="rewrite the baseline with current findings") - parser.add_argument("--refresh-derived", action="store_true", help="refresh existing derived snapshots only if the ratchet accepts the result") - parser.add_argument( - "--repair-stale-base", - action="store_true", - help="with --refresh-derived, repair metadata drift already present in the exact base", - ) - parser.add_argument("--base", default="origin/main", help="exact git ref used to prove debt reductions") - parser.add_argument( - "--migrate-authority", - action="store_true", - help="re-base onto a freshly derived base authority when the base's stored one cannot be " - "reproduced; new debt still requires issue-bound dispositions", - ) - args = parser.parse_args(argv) - - root = args.root.resolve() - map_path = args.map_path or root / "Tools/parity_twin_map.json" - baseline_path = args.baseline_path or root / "Tools/parity_ledger_baseline.json" - if args.bootstrap_map != args.write_baseline: - print("FAIL --bootstrap-map and --write-baseline must be used together") - return 2 - if args.repair_stale_base and not args.refresh_derived: - print("FAIL --repair-stale-base requires --refresh-derived") - return 2 - if args.migrate_authority and not args.refresh_derived: - print("FAIL --migrate-authority requires --refresh-derived") - return 2 - if args.migrate_authority and args.repair_stale_base: - print("FAIL --repair-stale-base and --migrate-authority are different remedies; use one") - return 2 - if args.refresh_derived: - if args.bootstrap_map or args.write_baseline or args.no_baseline: - print("FAIL --refresh-derived cannot be combined with bootstrap, baseline, or display modes") - return 2 - if not map_path.exists() or not baseline_path.exists(): - print("FAIL --refresh-derived requires existing authority; use --bootstrap-map --write-baseline once") - return 2 - if args.map_path is not None or args.baseline_path is not None: - print("FAIL --refresh-derived only supports the canonical checked-in snapshot paths") - return 2 - old_map = map_path.read_bytes() - old_baseline = baseline_path.read_bytes() - candidate_map = build_compact_twin_map(root) - candidate_result = scan(root, candidate_map) - if candidate_result.errors: - print(f"FAIL {len(candidate_result.errors)} parity ledger scan error(s); snapshots unchanged") - return 1 - accepted = False - try: - _write_json(map_path, candidate_map) - _write_json(baseline_path, build_compact_baseline(candidate_result)) - command = [ - sys.executable, str(Path(__file__).with_name("parity_ratchet.py")), - "--root", str(root), "--base", args.base, "--offline", - ] - if args.repair_stale_base: - command.append("--repair-stale-base") - if args.migrate_authority: - command.append("--migrate-authority") - completed = subprocess.run(command, cwd=root, text=True, capture_output=True) - if completed.returncode: - print("FAIL derived refresh rejected; snapshots restored") - print((completed.stderr or completed.stdout).strip()) - return 1 - accepted = True - finally: - if not accepted: - map_path.write_bytes(old_map) - baseline_path.write_bytes(old_baseline) - print(f"WROTE reviewed derived snapshots ({len(candidate_result.findings)} known findings)") - return 0 - if args.bootstrap_map: - if map_path.exists() or baseline_path.exists(): - print("FAIL --bootstrap-map is for initial authority creation only; use the reviewed refresh workflow for existing authority") - return 2 - twin_map = build_compact_twin_map(root) - result = scan(root, twin_map) - if result.errors: - print(f"FAIL {len(result.errors)} parity ledger scan error(s); snapshots unchanged") - for error in result.errors: - print(f" {error.output()}") - return 1 - baseline = build_compact_baseline(result) - _publish_json_pair(map_path, twin_map, baseline_path, baseline) - expanded, drift = expand_twin_map(root, twin_map) - assert not drift - print(f"WROTE {map_path} and {baseline_path} ({len(expanded['function_pairs'])} derived function pairs; {len(result.findings)} known findings)") - return 0 - else: - twin_map = _load_json(map_path, {}) - - result = scan(root, twin_map) - if result.errors: - print(f"FAIL {len(result.errors)} parity ledger scan error(s):\n") - for error in result.errors: - print(f" {error.output()}") - noun = "error" if len(result.errors) == 1 else "errors" - print(f"\nBaseline not evaluated: {len(result.errors)} scan {noun}.") - return 1 - if args.no_baseline: - if result.findings: - print(f"FAIL {len(result.findings)} current parity ledger finding(s):\n") - for item in result.findings: - print(f" {item.output()}") - print(f"\nScanned {_summary(result)}") - return 1 - print(f"OK no parity ledger findings ({_summary(result)})") - return 0 - - baseline = _load_json(baseline_path, {}) - compact_drift, improvements = compact_baseline_changes(result, baseline) - baseline_counters = baseline.get("counters", {}) - counter_regressions = [ - (name, baseline_counters[name], count) - for name, count in result.counters.items() - if name in baseline_counters and count > baseline_counters[name] - ] - if compact_drift: - print(f"FAIL compact baseline drift in {', '.join(compact_drift)}") - actionable = [item for item in result.findings if item.rule.startswith("add-unpaired-")] - if actionable: - print("\nActionable source drift:") - for item in actionable: - print(f" {item.output()}") - print(f"\nScanned {_summary(result)}") - return 1 - - if improvements: - try: - base_identities = finding_identities_at_git_ref(root, args.base) - except ValueError as exc: - print(f"FAIL {exc}; cannot prove that compact baseline drift is only a decrease") - return 1 - new_identities = { - item.identity for item in result.findings - } - base_identities - if new_identities: - print("FAIL debt count decreased but replacement findings are new against the exact base:") - for identity in sorted(new_identities): - print(f" {identity}") - return 1 - for improvement in improvements: - print(f"WARNING debt decreased in {improvement}; baseline cleanup is optional") - - if counter_regressions: - total = len(counter_regressions) - print(f"FAIL {total} parity ledger finding(s) beyond the baseline:\n") - for name, was, now in counter_regressions: - print(f" {baseline_path.relative_to(root)}:1: duplicate-counter: {name} increased from {was} to {now}") - print(f"\nScanned {_summary(result)}") - return 1 - - print(f"OK no NEW parity ledger findings ({len(result.findings)} baselined; {_summary(result)})") - return 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/Tools/parity_ledger_baseline.json b/Tools/parity_ledger_baseline.json deleted file mode 100644 index 5701d3dc31..0000000000 --- a/Tools/parity_ledger_baseline.json +++ /dev/null @@ -1,29 +0,0 @@ -{ - "schema_version": 3, - "accepted_findings": [ - {"rule": "constant-ambiguous", "scope": "Packages/StrandImport", "reason": "Exact normalized constant name has multiple viable cross-language matches and remains visible until explicitly disambiguated.", "provenance": "Exact identities emitted by parity_ledger rule constant-ambiguous for current upstream scope Packages/StrandImport; no wildcard matching.", "count": 2, "identities_sha256": "a09cc40da0a1374a5b83a5575bbf5c37e8a452da6b8a2230ec20a024e05620a2"}, - {"rule": "constant-unverifiable", "scope": "Packages/StrandAnalytics", "reason": "Exact mirrored constant uses a lexical expression that the standard-library scanner cannot evaluate safely.", "provenance": "Exact identities emitted by parity_ledger rule constant-unverifiable for current upstream scope Packages/StrandAnalytics; no wildcard matching.", "count": 9, "identities_sha256": "adb72d0c29519b98895644b5be8359eade019012349d43159e83490597db5080"}, - {"rule": "constant-unverifiable", "scope": "Packages/StrandImport", "reason": "Exact mirrored constant uses a lexical expression that the standard-library scanner cannot evaluate safely.", "provenance": "Exact identities emitted by parity_ledger rule constant-unverifiable for current upstream scope Packages/StrandImport; no wildcard matching.", "count": 3, "identities_sha256": "337509f798a8f5d4b5175ca24b3a6bcf8c905079fed63dd0f3f799f9b3af4c95"}, - {"rule": "constant-unverifiable", "scope": "Packages/WhoopProtocol", "reason": "Exact mirrored constant uses a lexical expression that the standard-library scanner cannot evaluate safely.", "provenance": "Exact identities emitted by parity_ledger rule constant-unverifiable for current upstream scope Packages/WhoopProtocol; no wildcard matching.", "count": 1, "identities_sha256": "ec2d804fac6265dd622a47119b60aad9125b8120acad624401852e05f922601a"}, - {"rule": "constant-value-mismatch", "scope": "Packages/StrandAnalytics", "reason": "Exact mirrored constant values differ on current upstream and remain visible as parity debt.", "provenance": "Exact identities emitted by parity_ledger rule constant-value-mismatch for current upstream scope Packages/StrandAnalytics; no wildcard matching.", "count": 2, "identities_sha256": "bd128022f805b1054a2d27ef711b56b766b0026b2e0a0996f59a24e7ba1051a3"}, - {"rule": "duplicate-implementation", "scope": "Packages/NoopLocalAccess", "reason": "Exact platform-local implementation is retained as visible parity debt pending a source-level consolidation decision.", "provenance": "Exact identities emitted by parity_ledger rule duplicate-implementation for current upstream scope Packages/NoopLocalAccess; no wildcard matching.", "count": 1, "identities_sha256": "93ac143d045e2adecdb4def57935476d9103e26336627f39c98bbf02f75f9154"}, - {"rule": "duplicate-implementation", "scope": "Packages/StrandAnalytics", "reason": "Exact platform-local implementation is retained as visible parity debt pending a source-level consolidation decision.", "provenance": "Exact identities emitted by parity_ledger rule duplicate-implementation for current upstream scope Packages/StrandAnalytics; no wildcard matching.", "count": 3, "identities_sha256": "bf4e358f9ff146278c782011d341a661bdbfd6ccd72784d4605d001382b5bbe1"}, - {"rule": "duplicate-implementation", "scope": "Packages/StrandImport", "reason": "Exact platform-local implementation is retained as visible parity debt pending a source-level consolidation decision.", "provenance": "Exact identities emitted by parity_ledger rule duplicate-implementation for current upstream scope Packages/StrandImport; no wildcard matching.", "count": 1, "identities_sha256": "f4174a7246cb82ac8a9e363fb0706d6c46cc118f64ae8842790d7fc2fb29a156"}, - {"rule": "duplicate-implementation", "scope": "Strand", "reason": "Exact platform-local implementation is retained as visible parity debt pending a source-level consolidation decision.", "provenance": "Exact identities emitted by parity_ledger rule duplicate-implementation for current upstream scope Strand; no wildcard matching.", "count": 3, "identities_sha256": "b4cdd17c98994f4fe2dda3a1a33e26df1f89981803fba600a42310c683a984b0"}, - {"rule": "duplicate-implementation", "scope": "StrandiOS", "reason": "Exact platform-local implementation is retained as visible parity debt pending a source-level consolidation decision.", "provenance": "Exact identities emitted by parity_ledger rule duplicate-implementation for current upstream scope StrandiOS; no wildcard matching.", "count": 1, "identities_sha256": "d92bf876aaac5bf751ce872c8aaabf4df2d4b6a2e1940f5e2e20804ad34869fa"}, - {"rule": "duplicate-implementation", "scope": "android/analytics", "reason": "Exact platform-local implementation is retained as visible parity debt pending a source-level consolidation decision.", "provenance": "Exact identities emitted by parity_ledger rule duplicate-implementation for current upstream scope android/analytics; no wildcard matching.", "count": 3, "identities_sha256": "524488fb20b5edfabb6a34acfb3b98844baf2b8bf158aac15cbc42436cd69be8"}, - {"rule": "duplicate-implementation", "scope": "android/ingest", "reason": "Exact platform-local implementation is retained as visible parity debt pending a source-level consolidation decision.", "provenance": "Exact identities emitted by parity_ledger rule duplicate-implementation for current upstream scope android/ingest; no wildcard matching.", "count": 2, "identities_sha256": "8b3126b470191bb155e6d7d3caf111423e5738edab66643558805020a0b1123d"}, - {"rule": "duplicate-implementation", "scope": "android/ui", "reason": "Exact platform-local implementation is retained as visible parity debt pending a source-level consolidation decision.", "provenance": "Exact identities emitted by parity_ledger rule duplicate-implementation for current upstream scope android/ui; no wildcard matching.", "count": 4, "identities_sha256": "b9f579c3d30221ff72e40282c29baab20d57d95acf044e0f71d2338e37c52b1b"}, - {"rule": "test-only-callsite", "scope": "Packages/OuraProtocol", "reason": "Exact declaration is reached only from tests in the conservative lexical call graph; dynamic, callback and external entry points are intentionally not inferred.", "provenance": "Exact identities emitted by parity_ledger rule test-only-callsite for current upstream scope Packages/OuraProtocol; no wildcard matching.", "count": 5, "identities_sha256": "ff6404cd4d27782334dd844223be3ec86bb3260bef7f7e29c658fb6ad6e7938e"}, - {"rule": "test-only-callsite", "scope": "Packages/StrandAnalytics", "reason": "Exact declaration is reached only from tests in the conservative lexical call graph; dynamic, callback and external entry points are intentionally not inferred.", "provenance": "Exact identities emitted by parity_ledger rule test-only-callsite for current upstream scope Packages/StrandAnalytics; no wildcard matching.", "count": 44, "identities_sha256": "fff777acbc66e7dad92bd0a6c8b7644e5c7ab6ddc0a7106dfecc49319da6127b"}, - {"rule": "test-only-callsite", "scope": "Packages/StrandImport", "reason": "Exact declaration is reached only from tests in the conservative lexical call graph; dynamic, callback and external entry points are intentionally not inferred.", "provenance": "Exact identities emitted by parity_ledger rule test-only-callsite for current upstream scope Packages/StrandImport; no wildcard matching.", "count": 14, "identities_sha256": "1acdc71178e0a555f000c42980345e8c38495f9e3806a866f31b260865d3ac98"}, - {"rule": "test-only-callsite", "scope": "Packages/WhoopProtocol", "reason": "Exact declaration is reached only from tests in the conservative lexical call graph; dynamic, callback and external entry points are intentionally not inferred.", "provenance": "Exact identities emitted by parity_ledger rule test-only-callsite for current upstream scope Packages/WhoopProtocol; no wildcard matching.", "count": 30, "identities_sha256": "a9dc115534e8710217db9cd39bcceaa1419741737bfd3b733007dc0215cd5af2"}, - {"rule": "test-only-callsite", "scope": "Packages/WhoopStore", "reason": "Exact declaration is reached only from tests in the conservative lexical call graph; dynamic, callback and external entry points are intentionally not inferred.", "provenance": "Exact identities emitted by parity_ledger rule test-only-callsite for current upstream scope Packages/WhoopStore; no wildcard matching.", "count": 37, "identities_sha256": "2a62c1ece1c541b662dd407f0668d9a02782bb4d21f9bb181518272ff12f0e40"}, - {"rule": "test-only-callsite", "scope": "android/analytics", "reason": "Exact declaration is reached only from tests in the conservative lexical call graph; dynamic, callback and external entry points are intentionally not inferred.", "provenance": "Exact identities emitted by parity_ledger rule test-only-callsite for current upstream scope android/analytics; no wildcard matching.", "count": 70, "identities_sha256": "ba5baa3af54f2a05759f3ddaa3fe61cfa8f99864852c7fd804a25ef2efdc3b80"}, - {"rule": "test-only-callsite", "scope": "android/data", "reason": "Exact declaration is reached only from tests in the conservative lexical call graph; dynamic, callback and external entry points are intentionally not inferred.", "provenance": "Exact identities emitted by parity_ledger rule test-only-callsite for current upstream scope android/data; no wildcard matching.", "count": 14, "identities_sha256": "9bab871f5ddbc1d2634805bf525888125c3880d160ab09b3ee7f3d7af6c487cd"}, - {"rule": "test-only-callsite", "scope": "android/ingest", "reason": "Exact declaration is reached only from tests in the conservative lexical call graph; dynamic, callback and external entry points are intentionally not inferred.", "provenance": "Exact identities emitted by parity_ledger rule test-only-callsite for current upstream scope android/ingest; no wildcard matching.", "count": 12, "identities_sha256": "d4f45d66087768eab133b918af6658b06ec1d91305e38fb045f6ae1629c94678"}, - {"rule": "test-only-callsite", "scope": "android/oura", "reason": "Exact declaration is reached only from tests in the conservative lexical call graph; dynamic, callback and external entry points are intentionally not inferred.", "provenance": "Exact identities emitted by parity_ledger rule test-only-callsite for current upstream scope android/oura; no wildcard matching.", "count": 4, "identities_sha256": "9beaa9034af0a1021b9a9f43739971934f260147e0a9a5c462468548a7fd577d"}, - {"rule": "test-only-callsite", "scope": "android/protocol", "reason": "Exact declaration is reached only from tests in the conservative lexical call graph; dynamic, callback and external entry points are intentionally not inferred.", "provenance": "Exact identities emitted by parity_ledger rule test-only-callsite for current upstream scope android/protocol; no wildcard matching.", "count": 35, "identities_sha256": "22cba7a970620ff06a32a2f79a506c9b5bbd547e3a49d7bce475038e44fb740d"} - ], - "counters": {"day_string_implementations": 12, "resting_hr_paths": 1, "android_ui_pearson_implementations": 4} -} diff --git a/Tools/parity_ratchet.py b/Tools/parity_ratchet.py deleted file mode 100644 index c3bbc9416b..0000000000 --- a/Tools/parity_ratchet.py +++ /dev/null @@ -1,711 +0,0 @@ -#!/usr/bin/env python3 -"""Fail-closed governance ratchet for the checked-in parity inventory. - -This first layer deliberately governs metadata only. It prevents the lexical -inventory or its accepted debt from being weakened in the same change that -updates the baseline. Differential runners, corpora, execution coverage and -native orchestration are separate layers and are not dependencies of this -tool. -""" - -from __future__ import annotations - -import argparse -import io -import json -import re -import subprocess -import sys -import tarfile -import tempfile -from contextlib import contextmanager -from datetime import date, datetime -from pathlib import Path -from typing import Iterable - -import issue_ref -import parity_ledger - - -ROOT = Path(__file__).resolve().parent.parent -TWIN_MAP_PATH = "Tools/parity_twin_map.json" -LEDGER_BASELINE_PATH = "Tools/parity_ledger_baseline.json" -DISPOSITIONS_PATH = "Tools/parity_dispositions.json" -class RatchetError(ValueError): - """Raised when governance inputs cannot be interpreted safely.""" - - -def _git(root: Path, arguments: list[str]) -> str: - try: - return subprocess.check_output( - ["git", *arguments], cwd=root, text=True, stderr=subprocess.STDOUT - ).strip() - except (OSError, subprocess.CalledProcessError) as exc: - detail = exc.output.strip() if isinstance(exc, subprocess.CalledProcessError) and exc.output else str(exc) - raise RatchetError(f"git {' '.join(arguments)} failed: {detail}") from exc - - -def resolve_base(root: Path, base: str | None) -> str: - """Resolve the exact requested base, defaulting to current ``origin/main``.""" - return _git(root, ["rev-parse", "--verify", base or "origin/main"]) - - -def _read_current(root: Path, relative: str) -> dict: - path = root / relative - try: - value = json.loads(path.read_text(encoding="utf-8")) - issue_ref.validate_current_issue_fields(value, relative) - except (OSError, json.JSONDecodeError, issue_ref.IssueRefError) as exc: - raise RatchetError(f"cannot read current {relative}: {exc}") from exc - if not isinstance(value, dict): - raise RatchetError(f"{relative}: JSON root must be an object") - return value - - -def _read_current_dispositions(root: Path) -> dict: - path = root / DISPOSITIONS_PATH - if not path.exists(): - return {"schema_version": 1, "dispositions": []} - return _read_current(root, DISPOSITIONS_PATH) - - -def _read_base(root: Path, base: str, relative: str) -> dict | None: - try: - raw = subprocess.check_output( - ["git", "show", f"{base}:{relative}"], - cwd=root, - text=True, - stderr=subprocess.DEVNULL, - ) - except (OSError, subprocess.CalledProcessError) as exc: - try: - listed = subprocess.check_output( - ["git", "ls-tree", "--name-only", base, "--", relative], - cwd=root, - text=True, - stderr=subprocess.DEVNULL, - ).strip() - except (OSError, subprocess.CalledProcessError) as tree_exc: - raise RatchetError( - f"cannot inspect base {base}:{relative}: {tree_exc}" - ) from tree_exc - if not listed: - return None - detail = exc.output.strip() if isinstance(exc, subprocess.CalledProcessError) and exc.output else str(exc) - raise RatchetError(f"cannot read base {base}:{relative}: {detail}") from exc - try: - value = json.loads(raw) - except json.JSONDecodeError as exc: - raise RatchetError(f"{base}:{relative}: invalid JSON: {exc}") from exc - if not isinstance(value, dict): - raise RatchetError(f"{base}:{relative}: JSON root must be an object") - issue_ref.validate_current_issue_fields(value, f"{base}:{relative}") - return value - - -def _array(value: dict, key: str, location: str) -> list: - result = value.get(key) - if not isinstance(result, list): - raise RatchetError(f"{location}: {key} must be an array") - return result - - -def _validate_twin_map(value: dict, location: str) -> None: - if value.get("schema_version") != 3: - raise RatchetError(f"{location}: schema_version must be 3") - expected_keys = {"schema_version", "derivation", "scope", "authority"} - if set(value) != expected_keys: - raise RatchetError(f"{location}: v3 top-level keys must be exact") - if value.get("derivation") != "parity_ledger.build_twin_map/v3": - raise RatchetError(f"{location}: unsupported v3 derivation") - expected_scope = { - "swift_roots": [glob.split("/**", 1)[0] for glob in parity_ledger.SWIFT_GLOBS], - "kotlin_roots": [glob.split("/**", 1)[0] for glob in parity_ledger.KOTLIN_GLOBS], - } - if value.get("scope") != expected_scope: - raise RatchetError(f"{location}: scope must equal the exact derivation roots") - authority = value.get("authority") - required = set(parity_ledger.SEMANTIC_AUTHORITY_SETS) - if not isinstance(authority, dict) or set(authority) != required: - raise RatchetError(f"{location}: authority must contain every semantic set exactly") - for name, item in authority.items(): - if (not isinstance(item, dict) or set(item) != {"count", "sha256"} - or type(item.get("count")) is not int or item["count"] < 0 - or not isinstance(item.get("sha256"), str) - or re.fullmatch(r"[0-9a-f]{64}", item["sha256"]) is None): - raise RatchetError(f"{location}: authority.{name} needs count and lowercase SHA-256") - - -def _validate_dispositions(value: dict, location: str) -> None: - if value.get("schema_version") != 1 or set(value) != {"schema_version", "dispositions"}: - raise RatchetError(f"{location}: typed disposition registry keys must be exact") - dispositions = value.get("dispositions") - if not isinstance(dispositions, list): - raise RatchetError(f"{location}: dispositions must be an array") - seen_identities: set[str] = set() - seen_issues: set[issue_ref.IssueRef] = set() - forbidden = issue_ref.parse_current("bhelm/noop#17") - allowed_kinds = { - "add-unpaired-file", "add-unpaired-function", "add-unpaired-property", - "add-unpaired-constant", - } - for index, item in enumerate(dispositions): - prefix = f"{location}: dispositions[{index}]" - if not isinstance(item, dict): - raise RatchetError(f"{prefix} must be an object") - disposition_type = item.get("type") - common = {"type", "kind", "identity", "identity_sha256", "platform"} - expected = { - "experimental": common | {"issue", "reason", "expires_on"}, - "platform_specific": common | {"rationale"}, - }.get(disposition_type) - if expected is None or set(item) != expected: - missing = "expires_on" if disposition_type == "experimental" and "expires_on" not in item else "keys" - raise RatchetError(f"{prefix} {missing} must be exact for type {disposition_type!r}") - kind, identity = item["kind"], item["identity"] - reason = item["reason"] if disposition_type == "experimental" else item["rationale"] - if (not isinstance(kind, str) or not kind or not isinstance(identity, str) - or not identity or "*" in identity): - raise RatchetError(f"{prefix} needs exact kind and identity without globs") - if kind not in allowed_kinds: - raise RatchetError(f"{prefix} cannot waive shared parity or pair removal: {kind!r}") - platform = item.get("platform") - if platform not in {"swift", "kotlin"} or not identity.startswith(platform + "\0"): - raise RatchetError(f"{prefix} platform must match the exact identity") - if item["identity_sha256"] != parity_ledger._canonical_sha256(identity): - raise RatchetError(f"{prefix} identity hash mismatch") - if not isinstance(reason, str) or len(reason.strip()) < 20: - raise RatchetError(f"{prefix} needs a specific non-generic rationale") - issue = issue_ref.parse_current(item["issue"]) if disposition_type == "experimental" else None - if issue == forbidden: - raise RatchetError(f"{prefix}: umbrella issue bhelm/noop#17 is forbidden") - if disposition_type == "experimental": - try: - datetime.strptime(item["expires_on"], "%Y-%m-%d") - except (TypeError, ValueError) as exc: - raise RatchetError(f"{prefix} expires_on must be YYYY-MM-DD") from exc - if identity in seen_identities or (issue is not None and issue in seen_issues): - raise RatchetError(f"{location}: dispositions require unique exact identities and issues") - seen_identities.add(identity) - if issue is not None: - seen_issues.add(issue) - - -def _validate_baseline(value: dict, location: str) -> None: - if value.get("schema_version") != 3 or set(value) != {"schema_version", "accepted_findings", "counters"}: - raise RatchetError(f"{location}: compact baseline v3 keys must be exact") - groups = _array(value, "accepted_findings", location) - identities: set[tuple[str, str]] = set() - group_keys = {"rule", "scope", "reason", "provenance", "count", "identities_sha256"} - for index, group in enumerate(groups): - if not isinstance(group, dict) or set(group) != group_keys: - raise RatchetError(f"{location}: accepted_findings[{index}] keys must be exact") - for key in ("rule", "scope", "reason", "provenance"): - if not isinstance(group[key], str) or not group[key].strip(): - raise RatchetError(f"{location}: accepted_findings[{index}] needs non-empty {key}") - expected_reason = parity_ledger.BASELINE_REASONS.get(group["rule"]) - expected_provenance = parity_ledger.baseline_group_provenance( - group["rule"], group["scope"] - ) - if group["reason"] != expected_reason or group["provenance"] != expected_provenance: - raise RatchetError( - f"{location}: accepted_findings[{index}] must retain its canonical reviewed reason and provenance" - ) - identity = (group["rule"], group["scope"]) - if identity in identities: - raise RatchetError(f"{location}: duplicate accepted finding group {identity}") - identities.add(identity) - if (type(group["count"]) is not int or group["count"] < 1 - or not isinstance(group["identities_sha256"], str) - or re.fullmatch(r"[0-9a-f]{64}", group["identities_sha256"]) is None): - raise RatchetError(f"{location}: accepted_findings[{index}] needs count and identity SHA-256") - counters = value.get("counters") - if not isinstance(counters, dict): - raise RatchetError(f"{location}: counters must be an object") - for name, count in counters.items(): - if not isinstance(name, str) or type(count) is not int or count < 0: - raise RatchetError(f"{location}: counter {name!r} must be a non-negative integer") - - -def _issues(value: object) -> set[issue_ref.IssueRef]: - found: set[issue_ref.IssueRef] = set() - - def walk(node: object) -> None: - if isinstance(node, dict): - for key, child in node.items(): - if key == "issue": - found.add(issue_ref.parse_current(child)) - walk(child) - elif isinstance(node, list): - for child in node: - walk(child) - - walk(value) - return found - - -def _fetch_issue(issue: issue_ref.IssueRef) -> dict | None: - try: - response = subprocess.run( - ["gh", "api", f"repos/{issue.repo}/issues/{issue.number}"], - check=True, - capture_output=True, - text=True, - ) - except FileNotFoundError as exc: - raise RatchetError("gh is required for online issue validation") from exc - except (OSError, subprocess.CalledProcessError): - return None - try: - payload = json.loads(response.stdout) - except (TypeError, json.JSONDecodeError): - return None - return payload if isinstance(payload, dict) else None - - -def _issue_payload_matches(issue: issue_ref.IssueRef, payload: dict | None) -> bool: - """Validate both issue number and repository; pull requests do not qualify.""" - if payload is None or "pull_request" in payload: - return False - if type(payload.get("number")) is not int or payload["number"] != issue.number: - return False - proofs: list[bool] = [] - if "repository_url" in payload: - proofs.append(payload["repository_url"] == f"https://api.github.com/repos/{issue.repo}") - if "html_url" in payload: - proofs.append(payload["html_url"] == f"https://github.com/{issue.repo}/issues/{issue.number}") - return bool(proofs) and all(proofs) - - -def issue_exists(issue: issue_ref.IssueRef) -> bool: - return _issue_payload_matches(issue, _fetch_issue(issue)) - - -def _exemption_payload_is_bound( - issue: issue_ref.IssueRef, - payload: dict | None, - identity_sha256: str, - base_created_at: str, -) -> bool: - try: - created = datetime.fromisoformat(str(payload["created_at"]).replace("Z", "+00:00")) - base_created = datetime.fromisoformat(base_created_at.replace("Z", "+00:00")) - except (TypeError, KeyError, ValueError): - return False - return ( - _exemption_payload_has_identity(issue, payload, identity_sha256) - and payload.get("state") == "open" - and created > base_created - ) - - -def _exemption_payload_has_identity( - issue: issue_ref.IssueRef, payload: dict | None, identity_sha256: str -) -> bool: - marker = f"parity-governance-identity-sha256: {identity_sha256}" - return ( - _issue_payload_matches(issue, payload) - and isinstance(payload.get("body"), str) - and marker in payload["body"] - ) - - -def exemption_issue_is_bound( - issue: issue_ref.IssueRef, identity_sha256: str, base_created_at: str -) -> bool: - """Require a post-base issue whose body names the exact governed identity hash.""" - return _exemption_payload_is_bound( - issue, _fetch_issue(issue), identity_sha256, base_created_at - ) - - -@contextmanager -def _base_tree(root: Path, base: str): - """Materialize the exact base tree; unavailable/shallow bases fail closed.""" - try: - archive = subprocess.check_output( - ["git", "archive", "--format=tar", base], cwd=root, stderr=subprocess.STDOUT - ) - except (OSError, subprocess.CalledProcessError) as exc: - detail = exc.output.decode(errors="replace").strip() if isinstance(exc, subprocess.CalledProcessError) else str(exc) - raise RatchetError(f"cannot independently scan base {base}: {detail}") from exc - with tempfile.TemporaryDirectory() as directory: - target = Path(directory) - try: - with tarfile.open(fileobj=io.BytesIO(archive), mode="r:") as bundle: - bundle.extractall(target, filter="data") - except (tarfile.TarError, TypeError) as exc: - raise RatchetError(f"cannot materialize base {base}: {exc}") from exc - yield target - - -def _required_v3_exemptions( - base_sets: dict[str, list[str]], - current_sets: dict[str, list[str]], - base_findings: set[str], - current_findings: set[str], -) -> set[tuple[str, str]]: - required: set[tuple[str, str]] = set() - # Explicit singulars rather than name[:-1]: "unpaired_properties" stems to - # "unpaired-propertie", which _validate_dispositions does not accept, so a one-sided - # property could be REQUIRED to carry a disposition that could never be written. Every - # other set survives the naive strip, which is why this went unnoticed: no test had a - # one-sided property until Lift Log added two. - for name, singular in ( - ("unpaired_files", "unpaired-file"), - ("unpaired_functions", "unpaired-function"), - ("unpaired_properties", "unpaired-property"), - ("unpaired_constants", "unpaired-constant"), - ): - for identity in set(current_sets[name]) - set(base_sets[name]): - required.add((f"add-{singular}", identity)) - for name in ("function_pairs", "property_pairs", "constant_pairs"): - for identity in set(base_sets[name]) - set(current_sets[name]): - required.add((f"remove-{name[:-1].replace('_', '-')}", identity)) - for identity in current_findings - base_findings: - required.add(("add-finding", identity)) - return required - - -def _exemption_applies( - key: tuple[str, str], - current_sets: dict[str, list[str]], - current_findings: set[str], - current_counters: dict[str, int], - bootstrap_unpaired_debts: set[str], -) -> bool: - kind, identity = key - add_sets = { - "add-unpaired-file": "unpaired_files", - "add-unpaired-function": "unpaired_functions", - "add-unpaired-property": "unpaired_properties", - "add-unpaired-constant": "unpaired_constants", - } - remove_sets = { - "remove-function-pair": "function_pairs", - "remove-property-pair": "property_pairs", - "remove-constant-pair": "constant_pairs", - } - if kind in add_sets: - return identity in current_sets[add_sets[kind]] - if kind in remove_sets: - return identity not in current_sets[remove_sets[kind]] - if kind == "add-finding": - return identity in current_findings - if kind == "bootstrap-unpaired-function": - return identity in current_sets["unpaired_functions"] and identity in bootstrap_unpaired_debts - if kind == "add-counter": - parts = identity.split("\u0000") - try: - return len(parts) == 3 and current_counters.get(parts[0]) == int(parts[2]) - except ValueError: - return False - return False - - -def compare_metadata( - root: Path, - base: str, - *, - offline: bool, - repair_stale_base: bool = False, - migrate_authority: bool = False, - warnings: list[str] | None = None, -) -> list[str]: - """Compare current governance metadata with the exact requested base.""" - root = root.resolve() - errors: list[str] = [] - warnings = warnings if warnings is not None else [] - current_map = _read_current(root, TWIN_MAP_PATH) - current_baseline = _read_current(root, LEDGER_BASELINE_PATH) - current_registry = _read_current_dispositions(root) - _validate_twin_map(current_map, TWIN_MAP_PATH) - _validate_baseline(current_baseline, LEDGER_BASELINE_PATH) - _validate_dispositions(current_registry, DISPOSITIONS_PATH) - for disposition in current_registry["dispositions"]: - if (disposition["type"] == "experimental" - and date.fromisoformat(disposition["expires_on"]) < date.today()): - errors.append( - f"{DISPOSITIONS_PATH}: experimental disposition expired on {disposition['expires_on']}: {disposition['identity']}" - ) - current_sets = parity_ledger.semantic_authority(root) - current_manifest = parity_ledger.authority_manifest(current_sets) - current_scan_map = parity_ledger.build_compact_twin_map(root) - current_scan_map["exemptions"] = current_registry["dispositions"] - current_scan = parity_ledger.scan(root, current_scan_map) - - old_map = _read_base(root, base, TWIN_MAP_PATH) - old_baseline = _read_base(root, base, LEDGER_BASELINE_PATH) - old_registry = _read_base(root, base, DISPOSITIONS_PATH) - if old_registry is None: - old_registry = {"schema_version": 1, "dispositions": []} - _validate_dispositions(old_registry, f"{base}:{DISPOSITIONS_PATH}") - new_exemptions: list[dict] = [] - active_exemptions: list[dict] = [] - if old_map is not None: - _validate_twin_map(old_map, f"{base}:{TWIN_MAP_PATH}") - with _base_tree(root, base) as base_root: - base_sets = parity_ledger.semantic_authority(base_root) - base_manifest = parity_ledger.authority_manifest(base_sets) - base_scan_map = parity_ledger.build_compact_twin_map(base_root) - base_scan_map["exemptions"] = old_registry["dispositions"] - base_scan = parity_ledger.scan(base_root, base_scan_map) - base_authority_is_stale = old_map["authority"] != base_manifest - if base_authority_is_stale: - repair_mismatches: list[str] = [] - if current_sets != base_sets: - repair_mismatches.append("semantic authority differs from the exact base") - if ({item.identity for item in current_scan.findings} - != {item.identity for item in base_scan.findings}): - repair_mismatches.append("finding identities differ from the exact base") - if current_scan.counters != base_scan.counters: - repair_mismatches.append("counters differ from the exact base") - if current_registry != old_registry: - repair_mismatches.append("typed dispositions differ from the exact base") - if current_map["authority"] != current_manifest: - repair_mismatches.append("current authority is not exactly derived") - if current_baseline != parity_ledger.build_compact_baseline(current_scan): - repair_mismatches.append("current baseline is not exactly derived") - if migrate_authority: - # The base's checked-in authority cannot be reproduced by the current derivation, - # so there is no exact basis to compare against and `--repair-stale-base` cannot - # help: repair exists for a base whose GOVERNED STATE matches, and here it does not. - # - # Migration re-bases the comparison onto a freshly derived base authority. It - # deliberately waives only the reproducibility of the base's stored manifest. It - # waives NO semantic debt: `required` below is computed from the freshly derived - # base and current sets, so every new one-sided declaration still needs its own - # issue-bound disposition, and an undeclared one still fails. - if current_map["authority"] != current_manifest: - errors.append( - f"{TWIN_MAP_PATH}: authority migration requires an exactly derived current " - "authority; refresh the snapshots rather than hand-editing them" - ) - else: - warnings.append( - f"{TWIN_MAP_PATH}: base authority at {base} is not reproducible with the " - "current derivation; migrated onto a freshly derived base. New debt still " - "requires issue-bound dispositions." - ) - elif not repair_stale_base: - errors.append( - f"{TWIN_MAP_PATH}: base authority cannot be reproduced with the current derivation; " - "migration required (see --migrate-authority)" - ) - elif repair_mismatches: - errors.append( - f"{TWIN_MAP_PATH}: stale-base repair rejected because " - + "; ".join(repair_mismatches) - ) - else: - warnings.append( - f"{TWIN_MAP_PATH}: repaired stale metadata already present in the exact base; " - "no current-tree governance delta accepted" - ) - base_findings = {item.identity for item in base_scan.findings} - current_findings = {item.identity for item in current_scan.findings} - required = _required_v3_exemptions( - base_sets, - current_sets, - base_findings, - current_findings, - ) - if current_map["authority"] != current_manifest: - if current_map["authority"] == old_map["authority"]: - warnings.append( - f"{TWIN_MAP_PATH}: debt decreased; checked authority may be cleaned up later" - ) - else: - errors.append( - f"{TWIN_MAP_PATH}: authority matches neither the current tree nor the exact base" - ) - for name, value in current_scan.counters.items(): - old_value = base_scan.counters.get(name, 0) - if value > old_value: - required.add(("add-counter", f"{name}\u0000{old_value}\u0000{value}")) - current_by_key = { - (item["kind"], item["identity"]): item - for item in current_registry["dispositions"] - } - old_by_key = { - (item["kind"], item["identity"]): item - for item in old_registry["dispositions"] - } - inherited: set[tuple[str, str]] = set() - for key, old_item in old_by_key.items(): - applied_at_base = _exemption_applies( - key, base_sets, base_findings, base_scan.counters, - base_scan.bootstrap_unpaired_debts, - ) - applies_now = _exemption_applies( - key, current_sets, current_findings, current_scan.counters, - current_scan.bootstrap_unpaired_debts, - ) - current_item = current_by_key.get(key) - if applied_at_base and applies_now and current_item is None: - errors.append(f"{TWIN_MAP_PATH}: inherited exemption was removed {key[0]} {key[1]}") - elif applied_at_base and applies_now and current_item != old_item: - errors.append(f"{TWIN_MAP_PATH}: inherited exemption changed {key[0]} {key[1]}") - elif applied_at_base and applies_now: - inherited.add(key) - elif current_item is not None and not applies_now: - warnings.append( - f"{DISPOSITIONS_PATH}: obsolete disposition {key[0]} {key[1]}; cleanup is optional" - ) - elif current_item is not None and applies_now: - warnings.append( - f"{DISPOSITIONS_PATH}: obsolete disposition cannot authorize reintroduced debt {key[0]} {key[1]}" - ) - checked = { - key for key in current_by_key - if key not in old_by_key or key in inherited - } - for kind, identity in sorted(required - checked): - errors.append( - f"{TWIN_MAP_PATH}: derived inventory changed without an exact issue-bound authority change: {kind} {identity}" - ) - for kind, identity in sorted(checked - required - inherited - set(old_by_key)): - warnings.append( - f"{DISPOSITIONS_PATH}: obsolete disposition {kind} {identity}; cleanup is optional" - ) - new_exemptions = [ - item for key, item in current_by_key.items() - if key not in old_by_key and key in required - ] - active_exemptions = [ - item for key, item in current_by_key.items() - if key in inherited or key in required - ] - elif old_map is None: - if current_map["authority"] != current_manifest: - errors.append(f"{TWIN_MAP_PATH}: authority does not match independently derived current semantic sets") - dispositions = current_registry["dispositions"] - if dispositions: - errors.append( - f"{DISPOSITIONS_PATH}: bootstrap cannot introduce dispositions; " - "add issue-bound authority in a later reviewed change" - ) - - if old_baseline is not None: - _validate_baseline(old_baseline, f"{base}:{LEDGER_BASELINE_PATH}") - - if not offline: - issues = _issues(current_baseline) | _issues(active_exemptions) - payloads = {issue: _fetch_issue(issue) for issue in sorted(issues)} - for issue, payload in payloads.items(): - if not _issue_payload_matches(issue, payload): - errors.append(f"issue {issue} does not exist or is not accessible") - for disposition in active_exemptions: - if disposition["type"] != "experimental": - continue - issue = issue_ref.parse_current(disposition["issue"]) - if (payloads.get(issue) or {}).get("state") != "open": - errors.append(f"experimental disposition issue {issue} must remain open") - base_created_at = _git(root, ["show", "-s", "--format=%cI", base]) - for exemption in new_exemptions: - if exemption["type"] == "platform_specific": - continue - issue = issue_ref.parse_current(exemption["issue"]) - if not _exemption_payload_is_bound( - issue, payloads.get(issue), exemption["identity_sha256"], base_created_at - ): - errors.append( - f"issue {issue} is not fresh after the base or lacks the exact identity-hash marker" - ) - return errors - - -def repository_consistency_errors( - root: Path, *, warnings: list[str] | None = None -) -> list[str]: - """Reject current-tree regressions while allowing proven debt reductions.""" - root = root.resolve() - warnings = warnings if warnings is not None else [] - twin_map = _read_current(root, TWIN_MAP_PATH) - baseline = _read_current(root, LEDGER_BASELINE_PATH) - registry = _read_current_dispositions(root) - _validate_twin_map(twin_map, TWIN_MAP_PATH) - _validate_baseline(baseline, LEDGER_BASELINE_PATH) - _validate_dispositions(registry, DISPOSITIONS_PATH) - scan_map = parity_ledger.build_compact_twin_map(root) - scan_map["exemptions"] = registry["dispositions"] - result = parity_ledger.scan(root, scan_map) - errors = [finding.output() for finding in result.errors] - current = parity_ledger.build_compact_baseline(result) - checked_groups = { - (item["rule"], item["scope"]): item - for item in baseline["accepted_findings"] - } - current_groups = { - (item["rule"], item["scope"]): item - for item in current["accepted_findings"] - } - for key in sorted(set(checked_groups) | set(current_groups)): - old = checked_groups.get(key) - new = current_groups.get(key) - label = "|".join(key) - if old is None: - errors.append(f"{LEDGER_BASELINE_PATH}: new accepted finding group {label}") - elif new is None or new["count"] < old["count"]: - warnings.append(f"{LEDGER_BASELINE_PATH}: debt decreased in {label}; cleanup is optional") - elif new["count"] > old["count"] or new["identities_sha256"] != old["identities_sha256"]: - errors.append(f"{LEDGER_BASELINE_PATH}: accepted finding group regressed {label}") - for name, value in result.counters.items(): - old_value = baseline["counters"].get(name, 0) - if value > old_value: - errors.append(f"{LEDGER_BASELINE_PATH}: counter {name} increased from {old_value} to {value}") - elif value < old_value: - warnings.append(f"{LEDGER_BASELINE_PATH}: counter {name} decreased from {old_value} to {value}; cleanup is optional") - return errors - - -def _print_errors(errors: Iterable[str]) -> int: - errors = list(errors) - for error in errors: - print(f"ERROR: {error}", file=sys.stderr) - print(f"parity governance ratchet: errors={len(errors)}") - return 1 if errors else 0 - - -def main(argv: list[str] | None = None) -> int: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--root", type=Path, default=ROOT) - parser.add_argument("--base", help="base ref; defaults durably to origin/main") - parser.add_argument("--offline", action="store_true", help="skip GitHub issue existence checks") - parser.add_argument( - "--repair-stale-base", - action="store_true", - help="adopt exactly derived metadata only when governed state is unchanged from an already-stale base", - ) - parser.add_argument( - "--migrate-authority", - action="store_true", - help="re-base onto a freshly derived base authority when the base's stored one cannot be " - "reproduced; new debt still requires issue-bound dispositions", - ) - args = parser.parse_args(argv) - if args.repair_stale_base and args.migrate_authority: - parser.error("--repair-stale-base and --migrate-authority are different remedies; use one") - root = args.root.resolve() - try: - base = resolve_base(root, args.base) - warnings: list[str] = [] - errors = repository_consistency_errors(root, warnings=warnings) - errors.extend(compare_metadata( - root, - base, - offline=args.offline, - repair_stale_base=args.repair_stale_base, - migrate_authority=args.migrate_authority, - warnings=warnings, - )) - for warning in warnings: - print(f"WARNING: {warning}", file=sys.stderr) - return _print_errors(errors) - except (RatchetError, issue_ref.IssueRefError) as exc: - print(f"ERROR: {exc}", file=sys.stderr) - return 2 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/Tools/parity_twin_map.json b/Tools/parity_twin_map.json deleted file mode 100644 index 04df8ad57d..0000000000 --- a/Tools/parity_twin_map.json +++ /dev/null @@ -1,34 +0,0 @@ -{ - "schema_version": 3, - "derivation": "parity_ledger.build_twin_map/v3", - "scope": { - "swift_roots": [ - "Packages/StrandAnalytics/Sources", - "Packages/StrandImport/Sources", - "Packages/WhoopStore/Sources", - "Packages/WhoopProtocol/Sources", - "Packages/OuraProtocol/Sources" - ], - "kotlin_roots": [ - "android/app/src/main/java/com/noop/analytics", - "android/app/src/main/java/com/noop/ingest", - "android/app/src/main/java/com/noop/data", - "android/app/src/main/java/com/noop/protocol", - "android/app/src/main/java/com/noop/oura" - ] - }, - "authority": { - "files": {"count": 500, "sha256": "515000556406ef31edcca8c0c5a5aae1c519e69b998e1920a30921b1d87cea57"}, - "functions": {"count": 4449, "sha256": "1adfe3587fad29217a74fcfdd41629780744e18c8334684a7382411c3a55c6c6"}, - "properties": {"count": 458, "sha256": "d654302949fe0cb34f6e43a757e3a47fe485758e8a902c247285c0e288964d45"}, - "constants": {"count": 1943, "sha256": "28aef07e38b51398fa1f0b85c1d90415a12cf5d9923a582a38c0753cd2d451fd"}, - "file_pairs": {"count": 66, "sha256": "54ce5acd351bb1d2ce6bea06495cbfb7181975294ef7a94811e649be938f7f2a"}, - "function_pairs": {"count": 173, "sha256": "3bf3599416d9adc1e339cf20fdb915b9d9f8ce955e0fb8329834af88105959ca"}, - "property_pairs": {"count": 148, "sha256": "6fa59982fca9e8e306562b9d027676457c9fa4cc6766c1d6171e86084f3b2c54"}, - "constant_pairs": {"count": 676, "sha256": "e1e9dc35e152e5a439f1033e04a41ee28f5b6ea30cd8163746b67362ba0cdbed"}, - "unpaired_files": {"count": 386, "sha256": "19bf9fb79a6000964912b38eeeb61515ec0dfe7b8632e1d539dc29e537d61cc6"}, - "unpaired_functions": {"count": 4109, "sha256": "a1c5691444fb06aa29ff47405d6fc673187efcc66acf6cbe31fbf318727b382f"}, - "unpaired_properties": {"count": 162, "sha256": "eede893a804203b080f27bb05ca6a09ee9ac9d708ade984e2855813f35b6ae22"}, - "unpaired_constants": {"count": 591, "sha256": "c20942c8756be7c7c0bd38862eec53b1f0eb4ab25fbb68535d76d88d7b639514"} - } -} diff --git a/Tools/test_i18n_audit.py b/Tools/test_i18n_audit.py index 2cbf8ef24a..ac53f1a612 100644 --- a/Tools/test_i18n_audit.py +++ b/Tools/test_i18n_audit.py @@ -426,10 +426,15 @@ def test_android_positional_specifiers_are_stripped(self): # "%1$d%%" is pure format + literal percent — nothing to translate. self.assertFalse(ia._has_translatable_words("%1$d%%")) - def test_two_word_brand_is_flagged(self): - # Deliberately True: "Apple Health" IS caught, and the ratchet baseline absorbs it as an allowed - # legitimate echo — the gate's job is to block GROWTH, not to pre-judge every identical string. - self.assertTrue(ia._has_translatable_words("Apple Health")) + def test_two_word_brand_phrase_is_not(self): + # 0c18441e4 added "Apple Health" to BRAND_PHRASES: a multi-word brand is the same case as the + # single-word one above, one size up — an untranslated brand name is not a translation gap. + self.assertFalse(ia._has_translatable_words("Apple Health")) + + def test_brand_phrase_with_a_real_word_is_still_flagged(self): + # Only a string that is ENTIRELY brand is exempted — "Apple Health sync" still has "sync" to + # translate, so an identical copy in another locale is a genuine echo, not a legitimate term. + self.assertTrue(ia._has_translatable_words("Apple Health sync")) if __name__ == "__main__": diff --git a/Tools/test_steps_i18n.py b/Tools/test_steps_i18n.py index aede3f0fec..5e8ce4ea8b 100644 --- a/Tools/test_steps_i18n.py +++ b/Tools/test_steps_i18n.py @@ -1,12 +1,11 @@ -"""Regression coverage for every shipped locale of the steps feature.""" +"""Regression coverage for every shipped locale of the steps feature (iOS — this fork ships no Android +tree; see docs/FORK_GUIDE.md).""" import json import re import unittest -import xml.etree.ElementTree as ET from pathlib import Path ROOT = Path(__file__).resolve().parents[1] -RES = ROOT / "android/app/src/main/res" IOS_KEYS = [ "3 weeks", "sparse, widened to %@", @@ -57,33 +56,6 @@ def signature(value): return result class StepsTranslationsTests(unittest.TestCase): - def test_android_all_shipped_locales(self): - source = ET.parse(RES / "values/steps_view.xml").getroot() - keys = {entry.attrib["name"]: entry for entry in source} - directories = [RES / "values", *sorted(RES.glob("values-*"))] - for directory in directories: - if directory != RES / "values" and not (directory / "strings.xml").exists(): - continue - with self.subTest(locale=directory.name): - entries = {} - for path in directory.glob("*.xml"): - for entry in ET.parse(path).getroot(): - name = entry.get("name") - if name not in keys: - continue - self.assertNotIn(name, entries, f"Duplicate {name} in {directory}") - entries[name] = entry - self.assertEqual(set(entries), set(keys)) - for name, original in keys.items(): - translated = entries[name] - self.assertEqual(original.tag, translated.tag) - if original.tag == "string-array": - self.assertEqual(len(original), len(translated)) - self.assertTrue(all(item.text for item in translated)) - else: - self.assertTrue(translated.text) - self.assertEqual(signature(original.text), signature(translated.text), name) - def test_ios_all_shipped_locales(self): catalog = json.loads( (ROOT / "Strand/Resources/Localizable.xcstrings").read_text(), diff --git a/Tools/tests/test_parity_arity.py b/Tools/tests/test_parity_arity.py deleted file mode 100644 index c1605987b1..0000000000 --- a/Tools/tests/test_parity_arity.py +++ /dev/null @@ -1,137 +0,0 @@ -"""An argument the arity walk cannot parse must not make the whole call disappear.""" - -from __future__ import annotations - -import re -import sys -import unittest -from pathlib import Path - - -TOOLS = Path(__file__).resolve().parents[1] -REPOSITORY = TOOLS.parent -sys.path.insert(0, str(TOOLS)) - -import parity_ledger # noqa: E402 - - -def arity_of(source: str) -> int | None: - """Arity of the first call in `source`, as `_call_sites` would compute it.""" - return parity_ledger._arity(source, source.index("(")) - - -class HalfOpenRangeTests(unittest.TestCase): - """Why this exists: `_arity` returns None when it cannot balance the brackets, and every caller - responds with `if arity is None: continue`. An unparseable argument therefore does not degrade - the callsite, it ERASES it -- the scan reports a declaration nobody calls. - - Swift's half-open range operator ends in a `<` whose next character starts the upper bound, so it - presents exactly like `Array`. The walk pushed a bracket nothing ever closed, ran to the end - of the file and returned None. `Interpreter.hexString/1` is called once, on the line after its own - declaration, passing `frame[max(0, off)..(), key)"), 2) - - def test_nested_generic_argument_still_balances(self): - self.assertEqual(arity_of("store(Dictionary>(), key)"), 2) - - def test_less_than_comparison_is_not_an_opening_bracket(self): - self.assertEqual(arity_of("assert(a < b)"), 1) - - def test_less_than_or_equal_comparison_is_not_an_opening_bracket(self): - self.assertEqual(arity_of("assert(a <= b, message)"), 2) - - def test_empty_argument_list(self): - self.assertEqual(arity_of("reset()"), 0) - - -class OperatorAngleBracketTests(unittest.TestCase): - """Why this exists: `<` is the only genuinely ambiguous character in the walk. A generic argument - list needs it treated as a bracket; `a << b` and `a> 8)"), 1) - - def test_comparison_without_surrounding_space(self): - self.assertEqual(arity_of("assert(i(), flags << 2)"), 2) - - def test_retry_only_runs_when_the_strict_walk_fails(self): - """The retry is lossy for a generic carrying a comma, because demoting the angle brackets - exposes that comma as a separator. This is tolerable ONLY because a call that parses - strictly never reaches the retry. Pin that ordering, since losing it would silently - re-arity every generic call in the repository.""" - source = "store(Dictionary(), key)" - self.assertEqual(arity_of(source), 2) - self.assertEqual( - parity_ledger._arity(source, source.index("("), angles_are_brackets=False), - 3, - "retry is expected to over-count here; the strict walk must therefore win", - ) - - def test_half_open_range_is_kept_off_the_lossy_retry(self): - """`..<` is exempted in the strict walk rather than left to the retry, so a call that pairs - it with a comma-carrying generic still parses accurately.""" - self.assertEqual(arity_of("f(Dictionary(), x.. set[str]: - current = dict(self.EMPTY) - identity = "swift" + chr(0) + "X.swift::a/1#1" - for name in ("unpaired_files", "unpaired_functions", - "unpaired_properties", "unpaired_constants"): - current[name] = [identity] - required = parity_ratchet._required_v3_exemptions( - self.EMPTY, current, set(), set() - ) - return {kind for kind, _ in required} - - def test_every_required_add_kind_is_declarable(self) -> None: - identity = "swift" + chr(0) + "X.swift::a/1#1" - for kind in sorted(self._required_kinds()): - doc = { - "schema_version": 1, - "dispositions": [{ - "type": "platform_specific", - "kind": kind, - "identity": identity, - "identity_sha256": parity_ledger._canonical_sha256(identity), - "platform": "swift", - "rationale": "Declared one-sided on purpose for this vocabulary test.", - }], - } - with self.subTest(kind=kind): - parity_ratchet._validate_dispositions(doc, "test") - - def test_property_kind_is_singular(self) -> None: - self.assertIn("add-unpaired-property", self._required_kinds()) - self.assertNotIn("add-unpaired-propertie", self._required_kinds()) - - -if __name__ == "__main__": - unittest.main() diff --git a/Tools/tests/test_parity_governance_acceptance.py b/Tools/tests/test_parity_governance_acceptance.py deleted file mode 100644 index f4ffff3e76..0000000000 --- a/Tools/tests/test_parity_governance_acceptance.py +++ /dev/null @@ -1,1134 +0,0 @@ -"""Acceptance tests for the product-free parity governance foundation.""" - -from __future__ import annotations - -import io -import json -import subprocess -import sys -import tempfile -import unittest -from fnmatch import fnmatchcase -from pathlib import Path -from unittest import mock - - -TOOLS = Path(__file__).resolve().parents[1] -REPOSITORY = TOOLS.parent -sys.path.insert(0, str(TOOLS)) - -import issue_ref # noqa: E402 -import parity_ledger # noqa: E402 -import parity_ratchet # noqa: E402 - - -class IssueReferenceTests(unittest.TestCase): - def test_current_metadata_requires_repository_qualified_references(self) -> None: - reference = issue_ref.parse_current("bhelm/noop#17") - self.assertEqual(("bhelm/noop", 17), (reference.repo, reference.number)) - for invalid in (17, True, "#17", " bhelm/noop#17", "bhelm/noop#0"): - with self.subTest(invalid=invalid), self.assertRaises(issue_ref.IssueRefError): - issue_ref.parse_current(invalid) - - def test_repository_is_part_of_issue_identity(self) -> None: - self.assertNotEqual( - issue_ref.parse_current("bhelm/noop#17"), - issue_ref.parse_current("other/noop#17"), - ) - - -class RepositoryBaselineTests(unittest.TestCase): - @classmethod - def setUpClass(cls) -> None: - # These acceptance tests all inspect one immutable checkout. Build the - # repository evidence once for the class instead of paying for the same - # full-tree parse in every assertion. This is test-local state, not a - # persistent production cache; normal scanner calls still read fresh. - cls.compact_map = parity_ledger._load_json(TOOLS / "parity_twin_map.json", {}) - cls.baseline = parity_ledger._load_json(TOOLS / "parity_ledger_baseline.json", {}) - cls.snapshot = parity_ledger._SourceSnapshot() - cls.inventory = parity_ledger._inventory(REPOSITORY, cls.snapshot) - ( - cls.swift_files, - cls.kotlin_files, - cls.swift_functions, - cls.kotlin_functions, - _swift_properties, - _kotlin_properties, - _swift_constants, - _kotlin_constants, - ) = cls.inventory - _reference_files, declarations = parity_ledger._reference_declarations( - REPOSITORY, - cls.inventory[2:6], - cls.snapshot, - ) - cls.repo_swift_functions = [ - item for item in declarations - if item.language == "swift" and item.kind == "function" - ] - cls.repo_kotlin_functions = [ - item for item in declarations - if item.language == "kotlin" and item.kind == "function" - ] - cls.references = ( - parity_ledger.parse_twin_references( - REPOSITORY, cls.swift_files, "swift", cls.swift_functions, cls.snapshot - ) - + parity_ledger.parse_twin_references( - REPOSITORY, cls.kotlin_files, "kotlin", cls.kotlin_functions, cls.snapshot - ) - ) - cls.resolutions = parity_ledger.attached_function_resolutions( - cls.references, cls.repo_swift_functions, cls.repo_kotlin_functions - ) - cls.expanded_map = parity_ledger.build_twin_map( - REPOSITORY, cls.inventory, cls.snapshot - ) - cls.authority = parity_ledger.authority_manifest( - parity_ledger.semantic_authority( - REPOSITORY, - expanded=cls.expanded_map, - inventory=cls.inventory, - snapshot=cls.snapshot, - ) - ) - cls.result = parity_ledger.scan(REPOSITORY, cls.compact_map) - - def test_repository_has_one_typed_manual_disposition_registry(self) -> None: - registry = parity_ledger._load_json(TOOLS / "parity_dispositions.json", {}) - parity_ratchet._validate_dispositions(registry, "fixture") - self.assertEqual(1, registry["schema_version"]) - self.assertNotIn("exemptions", parity_ledger._load_json(TOOLS / "parity_twin_map.json", {})) - - def test_core_tools_filter_covers_every_governance_tool_path(self) -> None: - core = (REPOSITORY / ".github/workflows/tools-python.yml").read_text( - encoding="utf-8" - ) - governance = (REPOSITORY / ".github/workflows/parity-governance.yml").read_text( - encoding="utf-8" - ) - self.assertIn("pull_request:\n branches: [main]\n paths:", core) - core_paths = [ - line.strip()[3:-1] - for line in core.splitlines() - if line.startswith(" - '") - ] - self.assertEqual( - ["Tools/**", ".github/workflows/tools-python.yml"] * 2, - core_paths, - ) - self.assertNotIn("unittest discover -s tests", core) - self.assertIn("pull_request:\n branches: [main]\n paths:", governance) - governance_paths = [ - line.strip()[3:-1] - for line in governance.splitlines() - if line.startswith(" - '") - ] - self.assertEqual( - [ - "Tools/issue_ref.py", - "Tools/parity_*.py", - "Tools/parity_*.json", - "Tools/tests/test_parity_*.py", - "Tools/tests/test_rr_legacy_preservation_contract.py", - ".github/workflows/parity-governance.yml", - ] * 2, - governance_paths, - ) - self.assertIn("Tools/**", core_paths) - self.assertTrue( - all( - path.startswith("Tools/") - for path in governance_paths - if not path.startswith(".github/") - ) - ) - self.assertNotIn("'Packages/**/*.swift'", governance) - self.assertNotIn("'android/**/*.kt'", governance) - self.assertNotIn("Tools/tests/**", governance) - self.assertNotIn("test_german_today_localization", governance) - self.assertIn("tests.test_parity_ledger", governance) - self.assertIn("tests.test_parity_governance_acceptance", governance) - self.assertIn("tests.test_rr_legacy_preservation_contract", governance) - pull_request_paths = governance_paths[:6] - self.assertFalse(any( - fnmatchcase("Tools/tests/test_german_today_localization.py", pattern) - for pattern in pull_request_paths - )) - self.assertTrue(any( - fnmatchcase("Tools/tests/test_parity_ledger.py", pattern) - for pattern in pull_request_paths - )) - self.assertTrue(any( - fnmatchcase("Tools/tests/test_rr_legacy_preservation_contract.py", pattern) - for pattern in pull_request_paths - )) - - def test_checked_in_inventory_and_baseline_match_current_sources(self) -> None: - self.assertEqual([], self.result.errors) - self.assertEqual([], parity_ledger.compact_baseline_drift(self.result, self.baseline)) - self.assertEqual(self.result.counters, self.baseline["counters"]) - - def test_checked_metadata_is_compact_v3_and_expands_losslessly(self) -> None: - self.assertEqual(3, self.compact_map["schema_version"]) - self.assertEqual(self.compact_map["authority"], self.authority) - self.assertNotIn("unpaired_functions", self.compact_map) - self.assertNotIn("constant_pairs", self.compact_map) - - self.assertEqual(3, self.baseline["schema_version"]) - self.assertNotIn("findings", self.baseline) - self.assertTrue(self.baseline["accepted_findings"]) - for group in self.baseline["accepted_findings"]: - self.assertTrue(group["reason"].strip()) - self.assertTrue(group["provenance"].strip()) - self.assertGreater(group["count"], 0) - self.assertRegex(group["identities_sha256"], r"^[0-9a-f]{64}$") - - def test_v3_authority_schema_is_exact_and_canonically_ordered(self) -> None: - compact = self.compact_map - self.assertEqual( - list(parity_ledger.SEMANTIC_AUTHORITY_SETS), list(compact["authority"]) - ) - parity_ratchet._validate_twin_map(compact, "fixture") - - malformed = json.loads(json.dumps(compact)) - malformed["authority"]["unknown"] = {"count": 0, "sha256": "0" * 64} - with self.assertRaisesRegex(parity_ratchet.RatchetError, "every semantic set exactly"): - parity_ratchet._validate_twin_map(malformed, "fixture") - - unknown = json.loads(json.dumps(compact)) - unknown["ignored_override"] = True - with self.assertRaisesRegex(parity_ratchet.RatchetError, "top-level keys"): - parity_ratchet._validate_twin_map(unknown, "fixture") - - misleading = json.loads(json.dumps(compact)) - misleading["scope"]["swift_roots"] = ["Elsewhere"] - with self.assertRaisesRegex(parity_ratchet.RatchetError, "exact derivation roots"): - parity_ratchet._validate_twin_map(misleading, "fixture") - - tampered = json.loads(json.dumps(self.baseline)) - tampered["accepted_findings"][0]["reason"] = "arbitrary non-empty replacement" - with self.assertRaisesRegex(parity_ratchet.RatchetError, "canonical reviewed reason"): - parity_ratchet._validate_baseline(tampered, "fixture") - - def test_repository_metadata_uses_invariants_not_frozen_counts_or_commits(self) -> None: - for relative in ("parity_twin_map.json", "parity_ledger_baseline.json"): - value = json.loads((TOOLS / relative).read_text(encoding="utf-8")) - issue_ref.validate_current_issue_fields(value, relative) - self.assertNotIn("expected_count", json.dumps(value)) - self.assertNotIn("source_commit", json.dumps(value)) - - def test_checked_function_pairs_equal_current_attached_source_claims(self) -> None: - declared = set( - parity_ledger.resolved_attached_function_pairs( - self.references, self.repo_swift_functions, self.repo_kotlin_functions - ) - ) - checked = { - (item["swift"], item["kotlin"]) - for item in self.expanded_map["function_pairs"] - } - self.assertEqual(declared, checked) - declared_files = parity_ledger.resolved_file_pairs( - declared, self.repo_swift_functions, self.repo_kotlin_functions - ) - checked_files = { - (item["swift"], item["kotlin"]) - for item in self.expanded_map["file_pairs"] - } - self.assertEqual(declared_files, checked_files) - - def test_every_nonunique_attached_claim_has_one_explicit_finding(self) -> None: - unresolved_sites = { - (reference.path, reference.line) - for reference, candidates in self.resolutions.items() - if len(candidates) != 1 - } - finding_sites = { - (finding.path, finding.line) - for finding in self.result.findings - if finding.rule in { - "unresolved-attached-function-claim", - "ambiguous-attached-function-claim", - } - } - self.assertEqual(unresolved_sites, finding_sites) - - def test_repository_has_only_correct_collapse_pair_and_no_rank_waiver(self) -> None: - pairs = { - (item["swift"], item["kotlin"]) - for item in self.expanded_map["function_pairs"] - } - wrong = ( - "Packages/StrandAnalytics/Sources/StrandAnalytics/HRVAnalyzer.swift::collapseOverCount/4#1", - "android/app/src/main/java/com/noop/analytics/HrvAnalyzer.kt::collapsedCoverage/3#1", - ) - correct = ( - "Packages/StrandAnalytics/Sources/StrandAnalytics/HRVAnalyzer.swift::collapseOverCount/4#1", - "android/app/src/main/java/com/noop/analytics/HrvAnalyzer.kt::collapseOverCount/4#1", - ) - self.assertNotIn(wrong, pairs) - self.assertIn(correct, pairs) - kotlin_targets = [kotlin for _swift, kotlin in pairs] - self.assertEqual(len(kotlin_targets), len(set(kotlin_targets))) - - self.assertFalse(any( - "TestCentreLayout.swift::rank/1#1" in item.identity - for item in self.result.findings - )) - - def test_checked_constant_pairs_equal_current_dynamic_pairs(self) -> None: - swift_constants, kotlin_constants = self.inventory[6:8] - file_pairs = { - (item["swift"], item["kotlin"]) - for item in self.expanded_map["file_pairs"] - } - dynamic, _ambiguous = parity_ledger._constant_pairing( - swift_constants, kotlin_constants, file_pairs - ) - resolved = {(swift.key, kotlin.key) for swift, kotlin in dynamic} - checked = { - (item["swift"], item["kotlin"]) - for item in self.expanded_map["constant_pairs"] - } - self.assertEqual(resolved, checked) - - -class GovernanceRatchetTests(unittest.TestCase): - def setUp(self) -> None: - self.temp = tempfile.TemporaryDirectory() - self.root = Path(self.temp.name) - subprocess.run(["git", "init", "-q"], cwd=self.root, check=True) - subprocess.run(["git", "config", "user.email", "tests@example.invalid"], cwd=self.root, check=True) - subprocess.run(["git", "config", "user.name", "Tests"], cwd=self.root, check=True) - - def tearDown(self) -> None: - self.temp.cleanup() - - def write(self, relative: str, value: object) -> None: - path = self.root / relative - path.parent.mkdir(parents=True, exist_ok=True) - path.write_text(json.dumps(value), encoding="utf-8") - - def commit(self) -> str: - subprocess.run(["git", "add", "."], cwd=self.root, check=True) - subprocess.run(["git", "commit", "-qm", "base"], cwd=self.root, check=True) - return subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=self.root, text=True).strip() - - def experimental_registry(self, identity: str, issue: str = "bhelm/noop#78") -> dict: - return {"schema_version": 1, "dispositions": [{ - "type": "experimental", "kind": "add-unpaired-function", - "identity": identity, "platform": identity.split("\0", 1)[0], - "identity_sha256": parity_ledger._canonical_sha256(identity), - "issue": issue, "reason": "Time-boxed synthetic experiment awaiting its parity implementation.", - "expires_on": "2026-12-31", - }]} - - def stale_base_repair_fixture(self) -> tuple[Path, str]: - swift = self.root / "Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift" - kotlin = self.root / "android/app/src/main/java/com/noop/analytics/Engine.kt" - test = self.root / "Packages/StrandAnalytics/Tests/StrandAnalyticsTests/EngineTests.swift" - swift.parent.mkdir(parents=True, exist_ok=True) - kotlin.parent.mkdir(parents=True, exist_ok=True) - test.parent.mkdir(parents=True, exist_ok=True) - swift.write_text("enum Engine {}\n", encoding="utf-8") - kotlin.write_text("object Engine {}\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", compact) - self.write( - "Tools/parity_ledger_baseline.json", - parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)), - ) - self.write("Tools/parity_dispositions.json", {"schema_version": 1, "dispositions": []}) - self.commit() - - swift.write_text("enum Engine { static func alreadyOnMain() {} }\n", encoding="utf-8") - test.write_text("func testOnly() { Engine.alreadyOnMain() }\n", encoding="utf-8") - base = self.commit() - refreshed = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", refreshed) - self.write( - "Tools/parity_ledger_baseline.json", - parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, refreshed)), - ) - return swift, base - - def test_typed_dispositions_require_explicit_platform_and_lifecycle_fields(self) -> None: - experimental = { - "schema_version": 1, - "dispositions": [{ - "type": "experimental", "kind": "add-unpaired-function", - "identity": "swift\0Engine.swift::trial/0#1", "platform": "swift", - "identity_sha256": parity_ledger._canonical_sha256("swift\0Engine.swift::trial/0#1"), - "issue": "bhelm/noop#78", "reason": "Time-boxed experiment awaiting parity decision.", - "expires_on": "2026-12-31", - }], - } - parity_ratchet._validate_dispositions(experimental, "fixture") - malformed = json.loads(json.dumps(experimental)) - del malformed["dispositions"][0]["expires_on"] - with self.assertRaisesRegex(parity_ratchet.RatchetError, "expires_on"): - parity_ratchet._validate_dispositions(malformed, "fixture") - - platform_specific = { - "schema_version": 1, - "dispositions": [{ - "type": "platform_specific", "kind": "add-unpaired-function", - "identity": "kotlin\0Engine.kt::androidOnly/0#1", "platform": "kotlin", - "identity_sha256": parity_ledger._canonical_sha256("kotlin\0Engine.kt::androidOnly/0#1"), - "rationale": "Uses an Android-only operating-system capability with no iOS equivalent.", - }], - } - parity_ratchet._validate_dispositions(platform_specific, "fixture") - - def test_bootstrap_refuses_to_overwrite_existing_authority(self) -> None: - tools = self.root / "Tools" - tools.mkdir() - map_path = tools / "parity_twin_map.json" - baseline_path = tools / "parity_ledger_baseline.json" - map_path.write_bytes(b"map-old-bytes\n") - baseline_path.write_bytes(b"baseline-old-bytes\n") - output = io.StringIO() - with mock.patch("sys.stdout", output): - code = parity_ledger.main([ - "--root", str(self.root), "--bootstrap-map", "--write-baseline", - ]) - self.assertEqual(2, code) - self.assertIn("initial authority creation only", output.getvalue()) - self.assertEqual(b"map-old-bytes\n", map_path.read_bytes()) - self.assertEqual(b"baseline-old-bytes\n", baseline_path.read_bytes()) - - def test_bootstrap_and_baseline_flags_are_inseparable(self) -> None: - for flag in ("--bootstrap-map", "--write-baseline"): - with self.subTest(flag=flag): - output = io.StringIO() - with mock.patch("sys.stdout", output): - code = parity_ledger.main(["--root", str(self.root), flag]) - self.assertEqual(2, code) - self.assertIn("must be used together", output.getvalue()) - self.assertFalse((self.root / "Tools/parity_twin_map.json").exists()) - self.assertFalse((self.root / "Tools/parity_ledger_baseline.json").exists()) - - def test_bootstrap_scan_exception_preserves_prior_absence(self) -> None: - with mock.patch.object(parity_ledger, "scan", side_effect=RuntimeError("synthetic scan failure")): - with self.assertRaisesRegex(RuntimeError, "synthetic scan failure"): - parity_ledger.main([ - "--root", str(self.root), "--bootstrap-map", "--write-baseline", - ]) - self.assertFalse((self.root / "Tools/parity_twin_map.json").exists()) - self.assertFalse((self.root / "Tools/parity_ledger_baseline.json").exists()) - - def test_bootstrap_scan_error_preserves_prior_absence(self) -> None: - finding = mock.Mock() - finding.output.return_value = "synthetic scan error" - result = mock.Mock(errors=[finding]) - output = io.StringIO() - with mock.patch.object(parity_ledger, "scan", return_value=result), mock.patch("sys.stdout", output): - code = parity_ledger.main([ - "--root", str(self.root), "--bootstrap-map", "--write-baseline", - ]) - self.assertEqual(1, code) - self.assertIn("snapshots unchanged", output.getvalue()) - self.assertFalse((self.root / "Tools/parity_twin_map.json").exists()) - self.assertFalse((self.root / "Tools/parity_ledger_baseline.json").exists()) - - def test_bootstrap_publication_failure_rolls_back_both_files(self) -> None: - real_replace = parity_ledger.os.replace - calls = 0 - - def fail_second(source, destination): - nonlocal calls - calls += 1 - if calls == 2: - raise OSError("synthetic second replace failure") - return real_replace(source, destination) - - with mock.patch.object(parity_ledger.os, "replace", side_effect=fail_second): - with self.assertRaisesRegex(OSError, "synthetic second replace failure"): - parity_ledger.main([ - "--root", str(self.root), "--bootstrap-map", "--write-baseline", - ]) - self.assertFalse((self.root / "Tools/parity_twin_map.json").exists()) - self.assertFalse((self.root / "Tools/parity_ledger_baseline.json").exists()) - - def test_refresh_restores_snapshots_when_new_debt_is_undisposed(self) -> None: - swift = self.root / "Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift" - kotlin = self.root / "android/app/src/main/java/com/noop/analytics/Engine.kt" - swift.parent.mkdir(parents=True, exist_ok=True) - kotlin.parent.mkdir(parents=True, exist_ok=True) - swift.write_text("enum Engine {}\n", encoding="utf-8") - kotlin.write_text("object Engine {}\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - baseline = parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", baseline) - self.write("Tools/parity_dispositions.json", {"schema_version": 1, "dispositions": []}) - base = self.commit() - before_map = (self.root / "Tools/parity_twin_map.json").read_bytes() - before_baseline = (self.root / "Tools/parity_ledger_baseline.json").read_bytes() - before_dispositions = (self.root / "Tools/parity_dispositions.json").read_bytes() - - swift.write_text("enum Engine { static func accidentalDrift() {} }\n", encoding="utf-8") - output = io.StringIO() - with mock.patch("sys.stdout", output): - code = parity_ledger.main([ - "--root", str(self.root), "--refresh-derived", "--base", base, - ]) - self.assertEqual(1, code) - self.assertIn("snapshots restored", output.getvalue()) - self.assertEqual(before_map, (self.root / "Tools/parity_twin_map.json").read_bytes()) - self.assertEqual(before_baseline, (self.root / "Tools/parity_ledger_baseline.json").read_bytes()) - self.assertEqual(before_dispositions, (self.root / "Tools/parity_dispositions.json").read_bytes()) - - def test_stale_base_repair_accepts_only_governance_state_already_on_base(self) -> None: - _swift, base = self.stale_base_repair_fixture() - - ordinary = parity_ratchet.compare_metadata(self.root, base, offline=True) - repaired = parity_ratchet.compare_metadata( - self.root, base, offline=True, repair_stale_base=True - ) - - self.assertTrue(any("migration required" in error for error in ordinary), ordinary) - self.assertEqual([], repaired) - self.assertEqual( - {"schema_version": 1, "dispositions": []}, - parity_ledger._load_json(self.root / "Tools/parity_dispositions.json", {}), - ) - - def test_stale_base_repair_rejects_branch_added_governance_debt(self) -> None: - swift, base = self.stale_base_repair_fixture() - swift.write_text( - "enum Engine { static func alreadyOnMain() {}; static func addedOnBranch() {} }\n", - encoding="utf-8", - ) - refreshed = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", refreshed) - self.write( - "Tools/parity_ledger_baseline.json", - parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, refreshed)), - ) - - errors = parity_ratchet.compare_metadata( - self.root, base, offline=True, repair_stale_base=True - ) - - self.assertTrue(any("stale-base repair rejected" in error for error in errors), errors) - self.assertTrue(any("addedOnBranch" in error for error in errors), errors) - - def test_stale_base_repair_rejects_disposition_changes(self) -> None: - _swift, base = self.stale_base_repair_fixture() - identity = next( - item for item in parity_ledger.semantic_authority(self.root)["unpaired_functions"] - if "alreadyOnMain" in item - ) - self.write("Tools/parity_dispositions.json", { - "schema_version": 1, - "dispositions": [{ - "type": "platform_specific", - "kind": "add-unpaired-function", - "identity": identity, - "platform": "swift", - "identity_sha256": parity_ledger._canonical_sha256(identity), - "rationale": "Synthetic platform-only decision must not ride a stale-base repair.", - }], - }) - - errors = parity_ratchet.compare_metadata( - self.root, base, offline=True, repair_stale_base=True - ) - - self.assertTrue(any( - "stale-base repair rejected" in error - and "typed dispositions differ from the exact base" in error - for error in errors - ), errors) - - def test_stale_base_repair_rejects_nonexact_map(self) -> None: - _swift, base = self.stale_base_repair_fixture() - compact = parity_ledger._load_json(self.root / "Tools/parity_twin_map.json", {}) - compact["authority"]["functions"]["count"] += 1 - self.write("Tools/parity_twin_map.json", compact) - - errors = parity_ratchet.compare_metadata( - self.root, base, offline=True, repair_stale_base=True - ) - - self.assertTrue(any( - "stale-base repair rejected" in error - and "current authority is not exactly derived" in error - for error in errors - ), errors) - - def test_stale_base_repair_rejects_nonexact_baseline(self) -> None: - _swift, base = self.stale_base_repair_fixture() - baseline = parity_ledger._load_json( - self.root / "Tools/parity_ledger_baseline.json", {} - ) - baseline["accepted_findings"][0]["count"] += 1 - self.write("Tools/parity_ledger_baseline.json", baseline) - - errors = parity_ratchet.compare_metadata( - self.root, base, offline=True, repair_stale_base=True - ) - - self.assertTrue(any( - "stale-base repair rejected" in error - and "current baseline is not exactly derived" in error - for error in errors - ), errors) - - def test_authority_migration_accepts_a_stale_base_when_new_debt_is_disposed(self) -> None: - """The remedy #2229 asked for: a base whose stored authority cannot be reproduced. - - `--repair-stale-base` cannot help here, because repair is for a base whose GOVERNED STATE - matches and this one carries genuinely new drift. Migration re-bases onto a freshly derived - base authority; the new declaration still has to be declared to pass. - """ - swift, base = self.stale_base_repair_fixture() - swift.write_text( - "enum Engine { static func alreadyOnMain() {}; static func addedOnBranch() {} }\n", - encoding="utf-8", - ) - refreshed = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", refreshed) - self.write( - "Tools/parity_ledger_baseline.json", - parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, refreshed)), - ) - identity = next( - item for item in parity_ledger.semantic_authority(self.root)["unpaired_functions"] - if "addedOnBranch" in item - ) - self.write("Tools/parity_dispositions.json", { - "schema_version": 1, - "dispositions": [{ - "type": "platform_specific", - "kind": "add-unpaired-function", - "identity": identity, - "identity_sha256": parity_ledger._canonical_sha256(identity), - "platform": "swift", - "rationale": "Swift-only by design for this fixture.", - }], - }) - - warnings: list[str] = [] - errors = parity_ratchet.compare_metadata( - self.root, base, offline=True, migrate_authority=True, warnings=warnings, - ) - - self.assertEqual([], errors) - self.assertTrue(any("migrated onto a freshly derived base" in w for w in warnings), warnings) - - def test_authority_migration_still_rejects_undeclared_new_debt(self) -> None: - """Migration waives the base manifest's reproducibility and nothing else. - - This is the guard worth pinning: if migration ever started waiving semantic debt too, the - flag would become a way to launder undeclared one-sided declarations onto main. - """ - swift, base = self.stale_base_repair_fixture() - swift.write_text( - "enum Engine { static func alreadyOnMain() {}; static func addedOnBranch() {} }\n", - encoding="utf-8", - ) - refreshed = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", refreshed) - self.write( - "Tools/parity_ledger_baseline.json", - parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, refreshed)), - ) - - errors = parity_ratchet.compare_metadata( - self.root, base, offline=True, migrate_authority=True, - ) - - self.assertTrue(any("addedOnBranch" in error for error in errors), errors) - self.assertTrue( - any("issue-bound authority change" in error for error in errors), errors - ) - - def test_authority_migration_rejects_a_hand_edited_current_authority(self) -> None: - _swift, base = self.stale_base_repair_fixture() - tampered = parity_ledger._load_json(self.root / "Tools/parity_twin_map.json", {}) - tampered["authority"]["unpaired_functions"]["count"] += 1 - self.write("Tools/parity_twin_map.json", tampered) - - errors = parity_ratchet.compare_metadata( - self.root, base, offline=True, migrate_authority=True, - ) - - self.assertTrue( - any("requires an exactly derived current authority" in error for error in errors), - errors, - ) - - def test_authority_migration_rejects_a_current_authority_equal_to_the_stale_base(self) -> None: - """The sharp case, found by mutating the guard rather than by reading it. - - The general protection against a non-exact current authority only ERRORS when it matches - neither the tree nor the base; when it matches the stale base exactly it merely warns - "debt decreased". Under migration that shape would otherwise sail through and adopt an - unrefreshed authority, which is the one thing migration must not do. - """ - _swift, base = self.stale_base_repair_fixture() - stale = parity_ratchet._read_base(self.root, base, "Tools/parity_twin_map.json") - self.write("Tools/parity_twin_map.json", stale) - - warnings: list[str] = [] - errors = parity_ratchet.compare_metadata( - self.root, base, offline=True, migrate_authority=True, warnings=warnings, - ) - - self.assertTrue( - any("requires an exactly derived current authority" in error for error in errors), - errors, - ) - self.assertFalse( - any("migrated onto a freshly derived base" in w for w in warnings), - "a stale current authority must not be reported as a completed migration", - ) - - def test_migrate_authority_flag_requires_guarded_refresh(self) -> None: - output = io.StringIO() - with mock.patch("sys.stdout", output): - code = parity_ledger.main([ - "--root", str(self.root), "--migrate-authority", - ]) - self.assertEqual(2, code) - self.assertIn("requires --refresh-derived", output.getvalue()) - - def test_migrate_authority_and_repair_are_mutually_exclusive(self) -> None: - output = io.StringIO() - with mock.patch("sys.stdout", output): - code = parity_ledger.main([ - "--root", str(self.root), "--refresh-derived", - "--repair-stale-base", "--migrate-authority", - ]) - self.assertEqual(2, code) - self.assertIn("different remedies", output.getvalue()) - - def test_ordinary_refusal_names_the_migration_flag(self) -> None: - """A dead end that does not name its exit is what made #2229 take a day.""" - _swift, base = self.stale_base_repair_fixture() - - errors = parity_ratchet.compare_metadata(self.root, base, offline=True) - - self.assertTrue(any("migration required" in error for error in errors), errors) - self.assertTrue(any("--migrate-authority" in error for error in errors), errors) - - def test_repair_stale_base_flag_requires_guarded_refresh(self) -> None: - output = io.StringIO() - with mock.patch("sys.stdout", output): - code = parity_ledger.main([ - "--root", str(self.root), "--repair-stale-base", - ]) - self.assertEqual(2, code) - self.assertIn("requires --refresh-derived", output.getvalue()) - - def test_expired_experimental_disposition_blocks(self) -> None: - marker = self.root / "README" - marker.write_text("base\n", encoding="utf-8") - base = self.commit() - compact = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact))) - identity = "swift\0Example.swift::trial/0#1" - registry = self.experimental_registry(identity) - registry["dispositions"][0]["expires_on"] = "2020-01-01" - self.write("Tools/parity_dispositions.json", registry) - errors = parity_ratchet.compare_metadata(self.root, base, offline=True) - self.assertTrue(any("experimental disposition expired" in error for error in errors), errors) - - def test_unreadable_base_blob_is_not_treated_as_absent_bootstrap_metadata(self) -> None: - self.write("Tools/parity_twin_map.json", {"schema_version": 3}) - base = self.commit() - real_check_output = parity_ratchet.subprocess.check_output - - def fail_only_show(arguments, **kwargs): - if arguments[:2] == ["git", "show"]: - raise subprocess.CalledProcessError(128, arguments, output="missing blob") - return real_check_output(arguments, **kwargs) - - with mock.patch.object( - parity_ratchet.subprocess, "check_output", side_effect=fail_only_show - ): - with self.assertRaisesRegex(parity_ratchet.RatchetError, "cannot read base"): - parity_ratchet._read_base( - self.root, base, "Tools/parity_twin_map.json" - ) - - def test_regenerating_compact_map_and_baseline_cannot_accept_new_unpaired_source(self) -> None: - swift = self.root / "Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift" - kotlin = self.root / "android/app/src/main/java/com/noop/analytics/Engine.kt" - swift.parent.mkdir(parents=True, exist_ok=True) - kotlin.parent.mkdir(parents=True, exist_ok=True) - swift.write_text("enum Engine {}\n", encoding="utf-8") - kotlin.write_text("object Engine {}\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - baseline = parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", baseline) - base = self.commit() - - swift.write_text("enum Engine { static func newlyUnpaired() {} }\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - baseline = parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", baseline) - - errors = parity_ratchet.compare_metadata(self.root, base, offline=True) - - self.assertTrue( - any("derived inventory changed without an exact issue-bound authority change" in error for error in errors), - errors, - ) - - def test_regenerating_compact_metadata_cannot_accept_one_sided_constant(self) -> None: - swift = self.root / "Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift" - kotlin = self.root / "android/app/src/main/java/com/noop/analytics/Engine.kt" - swift.parent.mkdir(parents=True, exist_ok=True) - kotlin.parent.mkdir(parents=True, exist_ok=True) - swift.write_text("enum Engine {}\n", encoding="utf-8") - kotlin.write_text("object Engine {}\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - baseline = parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", baseline) - base = self.commit() - - swift.write_text("enum Engine { static let swiftOnlyLimit = 7 }\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - baseline = parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", baseline) - - errors = parity_ratchet.compare_metadata(self.root, base, offline=True) - self.assertTrue( - any("add-unpaired-constant" in error and "swiftOnlyLimit" in error for error in errors), - errors, - ) - - def test_exact_compact_exemption_allows_only_its_current_delta(self) -> None: - swift = self.root / "Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift" - kotlin = self.root / "android/app/src/main/java/com/noop/analytics/Engine.kt" - swift.parent.mkdir(parents=True, exist_ok=True) - kotlin.parent.mkdir(parents=True, exist_ok=True) - swift.write_text("enum Engine {}\n", encoding="utf-8") - kotlin.write_text("object Engine {}\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - baseline = parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", baseline) - base = self.commit() - - swift.write_text("enum Engine { static func newlyUnpaired() {} }\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - identity = next( - item for item in parity_ledger.semantic_authority(self.root)["unpaired_functions"] - if "newlyUnpaired" in item - ) - registry = self.experimental_registry(identity) - self.write("Tools/parity_dispositions.json", registry) - baseline = parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", baseline) - self.assertEqual([], parity_ratchet.compare_metadata(self.root, base, offline=True)) - - swift.write_text("enum Engine {}\n", encoding="utf-8") - refreshed = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", refreshed) - self.write( - "Tools/parity_ledger_baseline.json", - parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, refreshed)), - ) - warnings: list[str] = [] - errors = parity_ratchet.compare_metadata( - self.root, base, offline=True, warnings=warnings - ) - self.assertEqual([], errors) - self.assertTrue(any("obsolete disposition" in warning for warning in warnings), warnings) - with mock.patch.object(parity_ratchet, "_fetch_issue") as fetched: - online_warnings: list[str] = [] - self.assertEqual( - [], - parity_ratchet.compare_metadata( - self.root, base, offline=False, warnings=online_warnings - ), - ) - fetched.assert_not_called() - - def test_platform_specific_disposition_allows_only_exact_one_sided_identity(self) -> None: - swift = self.root / "Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift" - kotlin = self.root / "android/app/src/main/java/com/noop/analytics/Engine.kt" - swift.parent.mkdir(parents=True, exist_ok=True) - kotlin.parent.mkdir(parents=True, exist_ok=True) - swift.write_text("enum Engine {}\n", encoding="utf-8") - kotlin.write_text("object Engine {}\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact))) - base = self.commit() - - kotlin.write_text("object Engine { fun androidOnly() = Unit }\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - identity = next(item for item in parity_ledger.semantic_authority(self.root)["unpaired_functions"] if "androidOnly" in item) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact))) - self.write("Tools/parity_dispositions.json", {"schema_version": 1, "dispositions": [{ - "type": "platform_specific", "kind": "add-unpaired-function", - "identity": identity, "platform": "kotlin", - "identity_sha256": parity_ledger._canonical_sha256(identity), - "rationale": "Uses an Android-only operating-system capability with no iOS equivalent.", - }]}) - self.assertEqual([], parity_ratchet.compare_metadata(self.root, base, offline=True)) - - def test_debt_decrease_needs_no_metadata_rewrite_and_warns(self) -> None: - swift = self.root / "Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift" - kotlin = self.root / "android/app/src/main/java/com/noop/analytics/Engine.kt" - swift.parent.mkdir(parents=True, exist_ok=True) - kotlin.parent.mkdir(parents=True, exist_ok=True) - swift.write_text("enum Engine { static func oldDebt() {} }\n", encoding="utf-8") - kotlin.write_text("object Engine {}\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - baseline = parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", baseline) - base = self.commit() - - swift.write_text("enum Engine {}\n", encoding="utf-8") - warnings: list[str] = [] - self.assertEqual( - [], - parity_ratchet.compare_metadata( - self.root, base, offline=True, warnings=warnings - ), - ) - self.assertTrue(any("debt decreased" in warning for warning in warnings), warnings) - - def test_obsolete_inherited_exemption_cannot_authorize_reintroduction(self) -> None: - swift = self.root / "Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift" - kotlin = self.root / "android/app/src/main/java/com/noop/analytics/Engine.kt" - swift.parent.mkdir(parents=True, exist_ok=True) - kotlin.parent.mkdir(parents=True, exist_ok=True) - swift.write_text("enum Engine {}\n", encoding="utf-8") - kotlin.write_text("object Engine {}\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - identity = "swift\0Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift::oldDebt/0#1" - self.write("Tools/parity_dispositions.json", self.experimental_registry(identity)) - baseline = parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", baseline) - base = self.commit() - - swift.write_text("enum Engine { static func oldDebt() {} }\n", encoding="utf-8") - refreshed = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", refreshed) - self.write( - "Tools/parity_ledger_baseline.json", - parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, refreshed)), - ) - errors = parity_ratchet.compare_metadata(self.root, base, offline=True) - self.assertTrue(any("add-unpaired-function" in error and "oldDebt" in error for error in errors), errors) - - def test_removing_a_real_twin_claim_is_not_treated_as_debt_reduction(self) -> None: - swift = self.root / "Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift" - kotlin = self.root / "android/app/src/main/java/com/noop/analytics/Engine.kt" - swift.parent.mkdir(parents=True, exist_ok=True) - kotlin.parent.mkdir(parents=True, exist_ok=True) - swift.write_text( - "enum Engine {\n /// Kotlin twin: `Engine.score`.\n static func score(_ value: Int) -> Int { value }\n}\n", - encoding="utf-8", - ) - kotlin.write_text( - "object Engine { fun score(value: Int): Int = value }\n", encoding="utf-8" - ) - compact = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", compact) - self.write( - "Tools/parity_ledger_baseline.json", - parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)), - ) - base = self.commit() - - swift.write_text( - "enum Engine { static func score(_ value: Int) -> Int { value } }\n", - encoding="utf-8", - ) - refreshed = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", refreshed) - self.write( - "Tools/parity_ledger_baseline.json", - parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, refreshed)), - ) - errors = parity_ratchet.compare_metadata(self.root, base, offline=True) - self.assertTrue(any("remove-function-pair" in error for error in errors), errors) - - def test_lower_debt_count_cannot_mask_replacement_identity_in_ratchet(self) -> None: - swift = self.root / "Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift" - kotlin = self.root / "android/app/src/main/java/com/noop/analytics/Engine.kt" - swift.parent.mkdir(parents=True, exist_ok=True) - kotlin.parent.mkdir(parents=True, exist_ok=True) - swift.write_text( - "enum Engine { static func oldOne() {}\n static func oldTwo() {} }\n", - encoding="utf-8", - ) - kotlin.write_text("object Engine {}\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", compact) - self.write( - "Tools/parity_ledger_baseline.json", - parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)), - ) - base = self.commit() - - swift.write_text("enum Engine { static func replacement() {} }\n", encoding="utf-8") - refreshed = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", refreshed) - self.write( - "Tools/parity_ledger_baseline.json", - parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, refreshed)), - ) - errors = parity_ratchet.compare_metadata(self.root, base, offline=True) - self.assertTrue( - any("add-unpaired-function" in error and "replacement" in error for error in errors), - errors, - ) - - def test_unchanged_inherited_exemption_stays_valid_without_refresh(self) -> None: - swift = self.root / "Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift" - kotlin = self.root / "android/app/src/main/java/com/noop/analytics/Engine.kt" - swift.parent.mkdir(parents=True, exist_ok=True) - kotlin.parent.mkdir(parents=True, exist_ok=True) - swift.write_text("enum Engine { static func inheritedDebt() {} }\n", encoding="utf-8") - kotlin.write_text("object Engine {}\n", encoding="utf-8") - compact = parity_ledger.build_compact_twin_map(self.root) - identity = next( - item for item in parity_ledger.semantic_authority(self.root)["unpaired_functions"] - if "inheritedDebt" in item - ) - self.write("Tools/parity_dispositions.json", self.experimental_registry(identity)) - baseline = parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", baseline) - base = self.commit() - - self.assertEqual([], parity_ratchet.compare_metadata(self.root, base, offline=True)) - payload = { - "number": 78, - "repository_url": "https://api.github.com/repos/bhelm/noop", - "html_url": "https://github.com/bhelm/noop/issues/78", - "state": "open", - } - with mock.patch.object(parity_ratchet, "_fetch_issue", return_value=payload) as fetched: - self.assertEqual([], parity_ratchet.compare_metadata(self.root, base, offline=False)) - fetched.assert_called_once() - - def test_bootstrap_on_base_without_governance_files_is_allowed(self) -> None: - marker = self.root / "README" - marker.write_text("base\n", encoding="utf-8") - base = self.commit() - compact = parity_ledger.build_compact_twin_map(self.root) - self.write("Tools/parity_twin_map.json", compact) - self.write("Tools/parity_ledger_baseline.json", parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact))) - self.assertEqual([], parity_ratchet.compare_metadata(self.root, base, offline=True)) - - def test_bootstrap_cannot_introduce_dispositions(self) -> None: - marker = self.root / "README" - marker.write_text("base\n", encoding="utf-8") - base = self.commit() - compact = parity_ledger.build_compact_twin_map(self.root) - identity = "swift\0Example.swift::invented/0#1" - self.write("Tools/parity_dispositions.json", self.experimental_registry(identity, "bhelm/noop#18")) - self.write("Tools/parity_twin_map.json", compact) - self.write( - "Tools/parity_ledger_baseline.json", - parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, compact)), - ) - - errors = parity_ratchet.compare_metadata(self.root, base, offline=True) - - self.assertTrue(any("bootstrap cannot introduce dispositions" in error for error in errors), errors) - - def test_same_issue_number_in_wrong_repository_fails_closed(self) -> None: - ref = issue_ref.parse_current("bhelm/noop#77") - response = subprocess.CompletedProcess( - [], 0, - json.dumps({ - "number": 77, - "repository_url": "https://api.github.com/repos/other/noop", - "html_url": "https://github.com/other/noop/issues/77", - }), - "", - ) - with mock.patch.object(parity_ratchet.subprocess, "run", return_value=response): - self.assertFalse(parity_ratchet.issue_exists(ref)) - - def test_pull_request_and_ambiguous_response_fail_closed(self) -> None: - ref = issue_ref.parse_current("bhelm/noop#77") - for payload in ( - {"number": 77, "pull_request": {}, "html_url": "https://github.com/bhelm/noop/issues/77"}, - {"number": 77}, - {"number": True, "repository_url": "https://api.github.com/repos/bhelm/noop"}, - ): - with self.subTest(payload=payload), mock.patch.object( - parity_ratchet.subprocess, - "run", - return_value=subprocess.CompletedProcess([], 0, json.dumps(payload), ""), - ): - self.assertFalse(parity_ratchet.issue_exists(ref)) - - def test_exemption_issue_must_be_fresh_and_bind_exact_identity_hash(self) -> None: - ref = issue_ref.parse_current("bhelm/noop#78") - payload = { - "number": 78, - "repository_url": "https://api.github.com/repos/bhelm/noop", - "html_url": "https://github.com/bhelm/noop/issues/78", - "created_at": "2026-08-21T12:00:00Z", - "state": "open", - "body": "parity-governance-identity-sha256: " + "a" * 64, - } - response = subprocess.CompletedProcess([], 0, json.dumps(payload), "") - with mock.patch.object(parity_ratchet.subprocess, "run", return_value=response): - self.assertTrue( - parity_ratchet.exemption_issue_is_bound( - ref, "a" * 64, "2026-08-21T11:00:00+00:00" - ) - ) - self.assertFalse( - parity_ratchet.exemption_issue_is_bound( - ref, "b" * 64, "2026-08-21T11:00:00+00:00" - ) - ) - self.assertFalse( - parity_ratchet.exemption_issue_is_bound( - ref, "a" * 64, "2026-08-21T13:00:00+00:00" - ) - ) - payload["state"] = "closed" - response = subprocess.CompletedProcess([], 0, json.dumps(payload), "") - with mock.patch.object(parity_ratchet.subprocess, "run", return_value=response): - self.assertFalse( - parity_ratchet.exemption_issue_is_bound( - ref, "a" * 64, "2026-08-21T11:00:00+00:00" - ) - ) - - def test_default_base_is_exact_current_origin_main_not_an_old_merge_base(self) -> None: - marker = self.root / "marker" - marker.write_text("common\n", encoding="utf-8") - common = self.commit() - subprocess.run(["git", "checkout", "-qb", "candidate"], cwd=self.root, check=True) - marker.write_text("candidate\n", encoding="utf-8") - subprocess.run(["git", "commit", "-qam", "candidate"], cwd=self.root, check=True) - subprocess.run(["git", "checkout", "-qb", "upstream", common], cwd=self.root, check=True) - marker.write_text("upstream\n", encoding="utf-8") - subprocess.run(["git", "commit", "-qam", "upstream"], cwd=self.root, check=True) - upstream = subprocess.check_output(["git", "rev-parse", "HEAD"], cwd=self.root, text=True).strip() - subprocess.run(["git", "update-ref", "refs/remotes/origin/main", upstream], cwd=self.root, check=True) - subprocess.run(["git", "checkout", "-q", "candidate"], cwd=self.root, check=True) - - self.assertEqual(upstream, parity_ratchet.resolve_base(self.root, None)) - - -if __name__ == "__main__": - unittest.main() diff --git a/Tools/tests/test_parity_ledger.py b/Tools/tests/test_parity_ledger.py deleted file mode 100644 index 00b609719a..0000000000 --- a/Tools/tests/test_parity_ledger.py +++ /dev/null @@ -1,1279 +0,0 @@ -"""Acceptance tests for the cross-language parity ledger.""" - -from __future__ import annotations - -import contextlib -import io -import json -import os -import subprocess -import sys -import tempfile -import unittest -from decimal import Decimal -from pathlib import Path -from unittest import mock - -TOOLS = Path(__file__).resolve().parents[1] -sys.path.insert(0, str(TOOLS)) - -import parity_ledger # noqa: E402 - - -class ParityLedgerTests(unittest.TestCase): - def setUp(self) -> None: - self.temp = tempfile.TemporaryDirectory() - self.root = Path(self.temp.name) - self.swift = self.root / "Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift" - self.kotlin = self.root / "android/app/src/main/java/com/noop/analytics/Engine.kt" - self.swift.parent.mkdir(parents=True) - self.kotlin.parent.mkdir(parents=True) - - def tearDown(self) -> None: - self.temp.cleanup() - - def write_clean_tree(self) -> None: - self.swift.write_text( - """public enum Engine { - /// Kotlin twin: `Engine.score`. - public static func score(_ value: Int) -> Int { value } - public static let sampleLimit = 3 -} -""" - ) - self.kotlin.write_text( - """object Engine { - /** Swift twin: `Engine.score`. */ - fun score(value: Int): Int = value - const val SAMPLE_LIMIT = 3 -} -""" - ) - - def findings(self, twin_map: dict | None = None) -> list[parity_ledger.Finding]: - if twin_map is None: - twin_map = parity_ledger.build_twin_map(self.root) - return parity_ledger.scan(self.root, twin_map).findings - - def run_cli( - self, - twin_map: dict, - baseline: dict | None = None, - *, - no_baseline: bool = False, - base: str | None = None, - ) -> tuple[int, str]: - map_path = self.root / "map.json" - baseline_path = self.root / "baseline.json" - map_path.write_text(json.dumps(twin_map)) - args = ["--root", str(self.root), "--map", str(map_path), "--baseline", str(baseline_path)] - if baseline is not None: - baseline_path.write_text(json.dumps(baseline)) - if no_baseline: - args.append("--no-baseline") - if base is not None: - args.extend(["--base", base]) - output = io.StringIO() - with contextlib.redirect_stdout(output): - code = parity_ledger.main(args) - return code, output.getvalue() - - def exit_code(self, twin_map: dict) -> int: - return self.run_cli(twin_map, no_baseline=True)[0] - - def baseline_for(self, twin_map: dict) -> dict: - return parity_ledger.build_compact_baseline(parity_ledger.scan(self.root, twin_map)) - - def mark_current_tree_as_origin_main(self) -> None: - subprocess.run(["git", "init", "-q"], cwd=self.root, check=True) - subprocess.run(["git", "add", "."], cwd=self.root, check=True) - subprocess.run( - [ - "git", "-c", "user.name=Parity Test", "-c", - "user.email=parity@example.invalid", "commit", "-qm", "base", - ], - cwd=self.root, - check=True, - ) - subprocess.run(["git", "branch", "origin/main", "HEAD"], cwd=self.root, check=True) - - def test_base_semantic_state_survives_a_symlinked_temp_root(self) -> None: - # #2143: on macOS the temp dir is /var/folders/..., a symlink to /private/var/.... The base - # checkout built its inventory from the unresolved root, then build_twin_map resolved the - # root, so relative_to saw two spellings of one directory and raised. Pointing tempfile at a - # symlink reproduces that on any OS, including the Linux runner. - self.write_clean_tree() - self.mark_current_tree_as_origin_main() - with tempfile.TemporaryDirectory() as holder: - real = Path(holder) / "real" - real.mkdir() - link = Path(holder) / "link" - link.symlink_to(real, target_is_directory=True) - with mock.patch.object(tempfile, "tempdir", str(link)): - state = parity_ledger._base_semantic_state(self.root) - self.assertIsNotNone(state) - - def test_clean_synthetic_tree_has_no_findings(self) -> None: - self.write_clean_tree() - twin_map = parity_ledger.build_twin_map(self.root) - self.assertEqual([], self.findings(twin_map)) - self.assertEqual(0, self.exit_code(twin_map)) - - def test_compact_map_expands_losslessly_and_detects_source_drift(self) -> None: - self.write_clean_tree() - compact = parity_ledger.build_compact_twin_map(self.root) - expanded, drift = parity_ledger.expand_twin_map(self.root, compact) - self.assertEqual(3, compact["schema_version"]) - self.assertEqual([], drift) - self.assertEqual(parity_ledger.build_twin_map(self.root), expanded) - - self.swift.write_text(self.swift.read_text() + "\npublic func addedAfterFreeze() {}\n") - result = parity_ledger.scan(self.root, compact) - self.assertIn("twin-map-authority-drift", {item.rule for item in result.findings}) - - def test_expand_twin_map_accepts_a_symlinked_root(self) -> None: - # expand_twin_map built its inventory from the root as given while build_twin_map resolved it, - # so a root reached through a symlink (macOS /var -> /private/var) raised in relative_to. A - # symlinked root reproduces that on any OS, including the Linux runner. - self.write_clean_tree() - compact = parity_ledger.build_compact_twin_map(self.root) - with tempfile.TemporaryDirectory() as holder: - link = Path(holder) / "link" - link.symlink_to(self.root, target_is_directory=True) - expanded, drift = parity_ledger.expand_twin_map(link, compact) - self.assertEqual([], drift) - self.assertEqual(parity_ledger.build_twin_map(self.root), expanded) - - def test_compact_authority_drift_names_new_one_sided_declaration(self) -> None: - self.write_clean_tree() - subprocess.run(["git", "init", "-q"], cwd=self.root, check=True) - subprocess.run(["git", "add", "."], cwd=self.root, check=True) - subprocess.run( - [ - "git", "-c", "user.name=Parity Test", "-c", "user.email=parity@example.invalid", - "commit", "-qm", "fixture", - ], - cwd=self.root, - check=True, - ) - subprocess.run(["git", "branch", "origin/main", "HEAD"], cwd=self.root, check=True) - compact = parity_ledger.build_compact_twin_map(self.root) - baseline = self.baseline_for(compact) - self.kotlin.write_text( - self.kotlin.read_text() - + "\nfun aFreshlyInventedOneSidedHelper(value: Int): Int = value\n" - ) - subprocess.run(["git", "add", "."], cwd=self.root, check=True) - subprocess.run( - [ - "git", "-c", "user.name=Parity Test", "-c", "user.email=parity@example.invalid", - "commit", "-qm", "add one-sided helper", - ], - cwd=self.root, - check=True, - ) - - code, output = self.run_cli(compact, baseline) - - self.assertEqual(1, code) - self.assertIn("add-unpaired-function", output) - self.assertIn("kotlin", output) - self.assertIn("android/app/src/main/java/com/noop/analytics/Engine.kt", output) - self.assertIn("aFreshlyInventedOneSidedHelper/1#1", output) - - def test_compact_authority_drift_uses_declaration_inventory_for_new_functions(self) -> None: - self.write_clean_tree() - self.swift.write_text( - self.swift.read_text().replace( - " public static let sampleLimit = 3\n", - """ public static func existingOneSided(_ value: Int) -> Int { - value - } - public static let sampleLimit = 3 -""", - ) - ) - self.mark_current_tree_as_origin_main() - compact = parity_ledger.build_compact_twin_map(self.root) - baseline = self.baseline_for(compact) - - self.swift.write_text( - self.swift.read_text().replace( - """ public static func existingOneSided(_ value: Int) -> Int { - value - } -""", - " public static func existingOneSided(_ value: Int) -> Int { value }\n", - ).replace( - " public static let sampleLimit = 3\n", - """ /// Kotlin twin: `Engine.claimedHelper`. - public static func claimedHelper(_ value: Int) -> Int { value } - public static let sampleLimit = 3 -""", - ) - ) - self.kotlin.write_text( - self.kotlin.read_text().replace( - " const val SAMPLE_LIMIT = 3\n", - """ fun claimedHelper(value: Int): Int = value - const val SAMPLE_LIMIT = 3 -""", - ) - + "\nfun genuinelyNewOneSided(value: Int): Int = value\n" - ) - - code, output = self.run_cli(compact, baseline) - - self.assertEqual(1, code) - self.assertIn("add-unpaired-function", output) - self.assertIn("genuinelyNewOneSided/1#1", output) - self.assertNotIn("add-unpaired-function: new one-sided swift function", output) - self.assertNotIn("existingOneSided/1#1", output) - - def test_protocol_and_oura_source_pairs_are_in_inventory_scope(self) -> None: - self.assertIn("Packages/WhoopProtocol/Sources/**/*.swift", parity_ledger.SWIFT_GLOBS) - self.assertIn("Packages/OuraProtocol/Sources/**/*.swift", parity_ledger.SWIFT_GLOBS) - self.assertIn("android/app/src/main/java/com/noop/protocol/**/*.kt", parity_ledger.KOTLIN_GLOBS) - self.assertIn("android/app/src/main/java/com/noop/oura/**/*.kt", parity_ledger.KOTLIN_GLOBS) - self.write_clean_tree() - scope = parity_ledger.build_compact_twin_map(self.root)["scope"] - self.assertIn("Packages/StrandAnalytics/Sources", scope["swift_roots"]) - self.assertIn("android/app/src/main/java/com/noop/analytics", scope["kotlin_roots"]) - - def test_repo_wide_line_comment_reference_is_checked_outside_inventory(self) -> None: - self.write_clean_tree() - strand = self.root / "Strand/Outside.swift" - strand.parent.mkdir(parents=True) - strand.write_text("// Kotlin twin: MissingOwner.missing\nfunc outside() {}\n") - rules = {item.rule for item in self.findings() if item.path == "Strand/Outside.swift"} - self.assertIn("dead-twin-reference", rules) - - def test_attached_claim_resolves_exact_counterpart_outside_authority_roots(self) -> None: - self.swift.write_text("enum Engine {}\n") - self.kotlin.write_text( - """object Engine { - /** Swift twin: `ExternalEngine.compute`. */ - fun compute(value: Int): Int = value -} -""" - ) - strand = self.root / "Strand/ExternalEngine.swift" - strand.parent.mkdir(parents=True, exist_ok=True) - strand.write_text( - "enum ExternalEngine { static func compute(_ value: Int) -> Int { value } }\n" - ) - twin_map = parity_ledger.build_twin_map(self.root) - result = parity_ledger.scan(self.root, twin_map) - self.assertEqual([], result.errors) - self.assertTrue( - any("Strand/ExternalEngine.swift::compute/1#1" == item["swift"] for item in twin_map["function_pairs"]) - ) - - def test_swift_package_module_qualifies_top_level_twin(self) -> None: - swift = self.root / "Packages/StrandAnalytics/Sources/StrandAnalytics/Summary.swift" - swift.parent.mkdir(parents=True, exist_ok=True) - swift.write_text("public func summary(_ value: Int) -> Int { value }\n") - self.kotlin.write_text( - "/** Swift twin: `StrandAnalytics.summary`. */\n" - "fun summary(value: Int): Int = value\n" - ) - - result = parity_ledger.scan(self.root, parity_ledger.build_twin_map(self.root)) - self.assertFalse(any(item.rule == "dead-twin-reference" for item in result.findings)) - self.assertEqual([], result.errors) - - def test_kotlin_constructor_property_resolves_owned_twin_reference(self) -> None: - self.kotlin.write_text("data class LiveState(val historyReady: Boolean)\n") - self.swift.write_text( - "final class LiveState {\n" - " /// Kotlin twin: `LiveState.historyReady`.\n" - " var historyReady = false\n" - "}\n" - ) - - result = parity_ledger.scan(self.root, parity_ledger.build_twin_map(self.root)) - self.assertFalse(any(item.rule == "dead-twin-reference" for item in result.findings)) - - def test_android_test_reference_resolves_swift_strand_test_symbol(self) -> None: - self.write_clean_tree() - swift_test = self.root / "StrandTests/WorkoutSourceTests.swift" - swift_test.parent.mkdir(parents=True) - swift_test.write_text( - "final class WorkoutSourceTests { func testPreservingCapturedCarriesStepsFromOld() {} }\n" - ) - kotlin_test = self.root / "android/app/src/test/java/com/noop/ui/WorkoutEditingTest.kt" - kotlin_test.parent.mkdir(parents=True) - kotlin_test.write_text( - "/** Twin of Swift `testPreservingCapturedCarriesStepsFromOld`. */\n" - "fun preservingCapturedCarriesStepsFromOld() {}\n" - ) - - findings = [ - item for item in self.findings() - if item.rule == "dead-twin-reference" and item.path.endswith("WorkoutEditingTest.kt") - ] - self.assertEqual([], findings) - - def test_new_one_sided_function_is_rejected(self) -> None: - self.write_clean_tree() - twin_map = parity_ledger.build_twin_map(self.root) - self.swift.write_text(self.swift.read_text() + "\npublic func newlyAddedOnlyOnSwift(_ value: Int) -> Int { value }\n") - rules = {finding.rule for finding in self.findings(twin_map)} - self.assertIn("unmapped-function", rules) - self.assertEqual(1, self.exit_code(twin_map)) - - def test_trailing_comma_does_not_add_a_parameter(self) -> None: - self.kotlin.write_text("fun score(first: Int, second: Int,) = first + second\n") - declarations = parity_ledger.parse_functions(self.root, self.kotlin, "kotlin") - self.assertEqual([("score", 2)], [(item.name, item.arity) for item in declarations]) - - def test_generic_kotlin_extension_receivers_are_inventoried(self) -> None: - self.kotlin.write_text( - """fun Map.cell(vararg keys: String) = "" -fun Map.double(vararg keys: String) = 0.0 -fun Map.bool(vararg keys: String) = false -""" - ) - declarations = parity_ledger.parse_functions(self.root, self.kotlin, "kotlin") - self.assertEqual(["cell", "double", "bool"], [item.name for item in declarations]) - - def test_computed_swift_and_kotlin_properties_are_paired(self) -> None: - self.swift.write_text( - """enum SleepStageTotals { struct Minutes { - var asleep: Double { 1 } - var inBed: Double { asleep + 1 } -} } -""" - ) - self.kotlin.write_text( - """object SleepStageTotals { data class Minutes(val awake: Double) { - val asleep: Double get() = 1.0 - val inBed: Double - get() { return asleep + 1.0 } -} } -""" - ) - twin_map = parity_ledger.build_twin_map(self.root) - self.assertEqual(2, len(twin_map["property_pairs"])) - self.assertFalse(any(item.rule == "unmapped-property" for item in self.findings(twin_map))) - - def test_dead_twin_reference_is_rejected(self) -> None: - self.write_clean_tree() - self.swift.write_text(self.swift.read_text() + "\n/// Kotlin twin: `Engine.missingTarget`.\npublic func claimsMissingTwin() {}\n") - twin_map = parity_ledger.build_twin_map(self.root) - rules = {finding.rule for finding in self.findings(twin_map)} - self.assertIn("dead-twin-reference", rules) - self.assertEqual(1, self.exit_code(twin_map)) - - def test_dead_and_unresolved_claims_cannot_be_generated_into_a_green_baseline(self) -> None: - self.swift.write_text( - """enum Engine { - /// Kotlin twin: `Engine.missingTarget`. - static func unresolvedClaim() {} -} -""" - ) - self.kotlin.write_text("object Engine {}\n") - compact = parity_ledger.build_compact_twin_map(self.root) - result = parity_ledger.scan(self.root, compact) - baseline = parity_ledger.build_compact_baseline(result) - self.assertIn("dead-twin-reference", {item.rule for item in result.errors}) - self.assertIn("unresolved-attached-function-claim", {item.rule for item in result.errors}) - self.assertEqual(1, self.run_cli(compact, baseline)[0]) - - def test_repeated_dead_reference_has_one_identity_per_claim_site(self) -> None: - self.swift.write_text( - """/// Kotlin twin: `MissingOwner.missing`. -public enum Engine { - /// Kotlin twin: `MissingOwner.missing`. - public static func score(_ value: Int) -> Int { value } -} -""" - ) - self.kotlin.write_text("object Engine {}\n") - - # A baseline identity represents one claim site, not merely one target - # string. Otherwise removing either claim is invisible to the ratchet. - findings = [item for item in self.findings() if item.rule == "dead-twin-reference"] - self.assertEqual(2, len(findings)) - self.assertEqual(2, len({item.identity for item in findings})) - - def test_only_nearest_function_twin_reference_attaches_to_declaration(self) -> None: - self.swift.write_text( - """enum Engine { - /// Kotlin twin is `Engine.wrong`. - /// Mirrors Kotlin `Engine.right`. - static func claim(_ value: Int) -> Int { value } -} -""" - ) - self.kotlin.write_text( - """object Engine { - fun wrong(value: Int): Int = value - fun right(value: Int): Int = value -} -""" - ) - - twin_map = parity_ledger.build_twin_map(self.root) - pairs = { - (item["swift"], item["kotlin"]) - for item in twin_map["function_pairs"] - } - - self.assertIn( - ( - "Packages/StrandAnalytics/Sources/StrandAnalytics/Engine.swift::claim/1#1", - "android/app/src/main/java/com/noop/analytics/Engine.kt::right/1#1", - ), - pairs, - ) - self.assertFalse(any("::wrong/1#" in kotlin for _swift, kotlin in pairs)) - - def test_file_twin_reference_is_not_attached_to_nearby_function(self) -> None: - self.swift.write_text( - """/// The Kotlin twin is Engine.kt and is covered by parity tests. -enum Engine { - static func rank(_ value: Int) -> Int { value } -} -""" - ) - self.kotlin.write_text("object Engine { fun rank(value: Int): Int = value }\n") - - twin_map = parity_ledger.build_twin_map(self.root) - rules = {item.rule for item in self.findings(twin_map)} - - self.assertEqual([], twin_map["function_pairs"]) - self.assertNotIn("unresolved-attached-function-claim", rules) - - def test_swift_selector_label_disambiguates_same_arity_overload(self) -> None: - self.swift.write_text( - """enum Engine { - static func from(hardwareId: String) -> Int { 1 } - static func from(model: String) -> Int { 2 } -} -""" - ) - self.kotlin.write_text( - """object Engine { - /** Twin of Swift `Engine.from(hardwareId:)`. */ - fun fromHardwareId(value: String): Int = 1 -} -""" - ) - twin_map = parity_ledger.build_twin_map(self.root) - result = parity_ledger.scan(self.root, twin_map) - self.assertNotIn( - "ambiguous-attached-function-claim", {item.rule for item in result.findings} - ) - self.assertTrue(any("::from/1#1" in item["swift"] for item in twin_map["function_pairs"])) - - def test_attached_claim_allows_exact_endpoints_with_different_arities(self) -> None: - self.swift.write_text( - """enum Engine { - /// Kotlin twin: `Engine.score`. - static func score(a: Int, b: Int, c: Int, d: Int) -> Int { a + b + c + d } -} -""" - ) - self.kotlin.write_text( - "object Engine { fun score(a: Int, b: Int, c: Int): Int = a + b + c }\n" - ) - twin_map = parity_ledger.build_twin_map(self.root) - result = parity_ledger.scan(self.root, twin_map) - self.assertNotIn("unresolved-attached-function-claim", {item.rule for item in result.errors}) - self.assertEqual(1, len(twin_map["function_pairs"])) - self.assertIn("::score/4#1", twin_map["function_pairs"][0]["swift"]) - self.assertIn("::score/3#1", twin_map["function_pairs"][0]["kotlin"]) - - def test_attached_claim_rejects_stale_explicit_selector(self) -> None: - self.swift.write_text( - "enum Engine { static func score(actual: Int) -> Int { actual } }\n" - ) - self.kotlin.write_text( - """object Engine { - /** Swift twin: `Engine.score(stale:)`. */ - fun score(value: Int): Int = value -} -""" - ) - twin_map = parity_ledger.build_twin_map(self.root) - result = parity_ledger.scan(self.root, twin_map) - self.assertIn("unresolved-attached-function-claim", {item.rule for item in result.errors}) - self.assertEqual([], twin_map["function_pairs"]) - - def test_ambiguous_attached_claim_is_an_unbaselinable_scan_error(self) -> None: - self.swift.write_text( - """enum Engine { - static func score(first: Int) -> Int { first } - static func score(second: Int) -> Int { second } -} -""" - ) - self.kotlin.write_text( - """object Engine { - /** Swift twin: `Engine.score`. */ - fun score(value: Int): Int = value -} -""" - ) - twin_map = parity_ledger.build_twin_map(self.root) - result = parity_ledger.scan(self.root, twin_map) - self.assertIn("ambiguous-attached-function-claim", {item.rule for item in result.errors}) - - def test_normal_block_comment_reference_is_checked(self) -> None: - self.write_clean_tree() - self.kotlin.write_text(self.kotlin.read_text() + "\n/* Swift twin: MissingOwner.nope */\nfun claimant() = 1\n") - self.assertTrue(any(item.rule == "dead-twin-reference" for item in self.findings())) - - def test_attached_reference_retarget_to_existing_function_invalidates_frozen_map(self) -> None: - self.swift.write_text( - """public enum Engine { - /// Kotlin twin: `Engine.score`. - public static func score(_ value: Int) -> Int { value } -} -""" - ) - self.kotlin.write_text( - """object Engine { - fun score(value: Int): Int = value - fun other(value: Int): Int = value -} -""" - ) - frozen_map = parity_ledger.build_twin_map(self.root) - self.swift.write_text(self.swift.read_text().replace("Engine.score", "Engine.other")) - - rules = {item.rule for item in self.findings(frozen_map)} - - self.assertIn("unmapped-declared-function-pair", rules) - self.assertIn("stale-declared-function-pair", rules) - - def test_same_metadata_retarget_is_rescanned_from_current_content(self) -> None: - self.swift.write_text( - """public enum Engine { - /// Kotlin twin: `Engine.score`. - public static func score(_ value: Int) -> Int { value } -} -""" - ) - self.kotlin.write_text( - """object Engine { - fun score(value: Int): Int = value - fun other(value: Int): Int = value -} -""" - ) - frozen_map = parity_ledger.build_twin_map(self.root) - original_stat = self.swift.stat() - original_size = original_stat.st_size - - self.swift.write_text(self.swift.read_text().replace("Engine.score", "Engine.other")) - self.assertEqual(original_size, self.swift.stat().st_size) - os.utime(self.swift, ns=(original_stat.st_atime_ns, original_stat.st_mtime_ns)) - - rules = {item.rule for item in self.findings(frozen_map)} - self.assertIn("unmapped-declared-function-pair", rules) - self.assertIn("stale-declared-function-pair", rules) - - def test_scan_uses_one_immutable_source_snapshot_per_file(self) -> None: - self.write_clean_tree() - twin_map = parity_ledger.build_twin_map(self.root) - original_read = parity_ledger._read - reads: list[Path] = [] - - def recording_read(path: Path) -> str: - reads.append(path) - return original_read(path) - - with mock.patch.object(parity_ledger, "_read", side_effect=recording_read): - parity_ledger.scan(self.root, twin_map) - - # The ledger reads through resolved paths; compare resolved so a symlinked temp dir (macOS - # /var -> /private/var) does not fail a test about read counts. - self.assertEqual(sorted([self.swift.resolve(), self.kotlin.resolve()]), sorted(reads)) - - def test_retargeted_claims_cannot_hide_behind_stale_file_and_constant_pairs(self) -> None: - swift_one = self.swift.with_name("One.swift") - swift_two = self.swift.with_name("Two.swift") - kotlin_one = self.kotlin.with_name("One.kt") - kotlin_two = self.kotlin.with_name("Two.kt") - swift_one.write_text( - """enum Engine { - static let limit = 1 -} -enum SwiftOne { - /// Kotlin twin: `KotlinOne.score`. - static func score(_ value: Int) -> Int { value } -} -""" - ) - swift_two.write_text( - """enum Engine { - static let limit = 2 -} -enum SwiftTwo { - /// Kotlin twin: `KotlinTwo.score`. - static func score(_ value: Int) -> Int { value } -} -""" - ) - kotlin_one.write_text( - "object Engine { const val LIMIT = 1 }; object KotlinOne { fun score(value: Int): Int = value }\n" - ) - kotlin_two.write_text( - "object Engine { const val LIMIT = 2 }; object KotlinTwo { fun score(value: Int): Int = value }\n" - ) - frozen_map = parity_ledger.build_twin_map(self.root) - - swift_one.write_text(swift_one.read_text().replace("KotlinOne.score", "KotlinTwo.score")) - swift_two.write_text(swift_two.read_text().replace("KotlinTwo.score", "KotlinOne.score")) - refreshed_map = parity_ledger.build_twin_map(self.root) - stale_authority = json.loads(json.dumps(frozen_map)) - stale_authority["function_pairs"] = refreshed_map["function_pairs"] - stale_authority["unpaired_functions"] = refreshed_map["unpaired_functions"] - - rules = {item.rule for item in self.findings(stale_authority)} - - self.assertIn("unmapped-declared-file-pair", rules) - self.assertIn("stale-declared-file-pair", rules) - self.assertTrue( - {"unmapped-constant-pair", "stale-constant-pair", "constant-value-mismatch"} - & rules, - rules, - ) - - def test_attached_reference_removal_invalidates_frozen_map(self) -> None: - self.swift.write_text( - """public enum Engine { - /// Kotlin twin: `Engine.score`. - public static func score(_ value: Int) -> Int { value } -} -""" - ) - self.kotlin.write_text("object Engine { fun score(value: Int): Int = value }\n") - frozen_map = parity_ledger.build_twin_map(self.root) - self.swift.write_text(self.swift.read_text().replace(" /// Kotlin twin: `Engine.score`.\n", "")) - - self.assertTrue( - any(item.rule == "stale-declared-function-pair" for item in self.findings(frozen_map)) - ) - - def test_new_attached_reference_requires_corresponding_map_pair(self) -> None: - self.swift.write_text("public enum Engine { public static func score(_ value: Int) -> Int { value } }\n") - self.kotlin.write_text("object Engine { fun score(value: Int): Int = value }\n") - frozen_map = parity_ledger.build_twin_map(self.root) - self.swift.write_text( - """public enum Engine { - /// Kotlin twin: `Engine.score`. - public static func score(_ value: Int) -> Int { value } -} -""" - ) - - self.assertTrue( - any(item.rule == "unmapped-declared-function-pair" for item in self.findings(frozen_map)) - ) - - def test_stale_attached_reference_to_missing_target_remains_red(self) -> None: - self.swift.write_text( - """public enum Engine { - /// Kotlin twin: `Engine.missing`. - public static func score(_ value: Int) -> Int { value } -} -""" - ) - self.kotlin.write_text("object Engine { fun score(value: Int): Int = value }\n") - - rules = {item.rule for item in self.findings()} - self.assertIn("dead-twin-reference", rules) - self.assertIn("unresolved-attached-function-claim", rules) - - def test_ambiguous_attached_reference_is_an_explicit_finding(self) -> None: - self.swift.write_text( - """public enum Engine { - /// Kotlin twin: `Engine.score`. - public static func claim(_ value: Int) -> Int { value } -} -""" - ) - self.kotlin.write_text( - """object Engine { - fun score(value: Int): Int = value - fun score(value: Int, extra: Int): Int = value + extra -} -""" - ) - twin_map = parity_ledger.build_twin_map(self.root) - - self.assertTrue( - any( - item.rule == "ambiguous-attached-function-claim" - for item in self.findings(twin_map) - ) - ) - - def test_qualified_reference_requires_the_actual_owner(self) -> None: - self.swift.write_text("enum Bar { static func existingName() {} }\n") - self.kotlin.write_text("// Swift twin: Foo.existingName\nfun claim() = 1\n") - findings = self.findings() - self.assertTrue(any(item.rule == "dead-twin-reference" for item in findings)) - - def test_constant_expression_is_fully_evaluated(self) -> None: - self.swift.write_text("enum Engine { static let hours = 48 * 3_600 }\n") - self.kotlin.write_text("object Engine { const val HOURS = 48L * 3_600L }\n") - twin_map = parity_ledger.build_twin_map(self.root) - self.assertFalse(any(item.rule.startswith("constant-") for item in self.findings(twin_map))) - self.kotlin.write_text("object Engine { const val HOURS = 48L * 3_601L }\n") - self.assertTrue(any(item.rule == "constant-value-mismatch" for item in self.findings(twin_map))) - - def test_new_equal_mirrored_constant_pair_must_be_added_to_map(self) -> None: - self.write_clean_tree() - frozen_map = parity_ledger.build_twin_map(self.root) - self.swift.write_text( - self.swift.read_text().replace( - "sampleLimit = 3", "sampleLimit = 3\n public static let freshLimit = 7" - ) - ) - self.kotlin.write_text( - self.kotlin.read_text().replace( - "SAMPLE_LIMIT = 3", "SAMPLE_LIMIT = 3\n const val FRESH_LIMIT = 7" - ) - ) - - findings = self.findings(frozen_map) - - self.assertTrue(any(item.rule == "unmapped-constant-pair" for item in findings), findings) - refreshed_map = parity_ledger.build_twin_map(self.root) - self.assertFalse( - any(item.rule == "unmapped-constant-pair" for item in self.findings(refreshed_map)) - ) - - def test_new_one_sided_constant_is_not_assumed_to_be_a_twin(self) -> None: - self.write_clean_tree() - frozen_map = parity_ledger.build_twin_map(self.root) - self.swift.write_text( - self.swift.read_text().replace( - "sampleLimit = 3", "sampleLimit = 3\n public static let swiftOnlyLimit = 7" - ) - ) - - self.assertFalse( - any(item.rule == "unmapped-constant-pair" for item in self.findings(frozen_map)) - ) - - def test_numeric_expression_parser_consumes_every_supported_operator(self) -> None: - parsed = parity_ledger._literal("-(2 - 50) * (7_200 / 2) + 0x10 - 0b1") - self.assertEqual("number:172815", parsed[0]) - self.assertIsNone(parity_ledger._literal("48 * 3_600 trailing")) - - def test_hex_digits_f_and_d_are_not_stripped_as_suffixes(self) -> None: - cases = { - "0xffff_ffff": "4294967295", - "0xffff_ffffL": "4294967295", - "0xFFFFFFFF": "4294967295", - "0xABCD": "43981", - "0x2F": "47", - "0b1L": "1", - "1.5f": "1.5", - "2.0d": "2", - "10L": "10", - } - for raw, expected in cases.items(): - with self.subTest(raw=raw): - canonical = parity_ledger._literal(raw)[0] - self.assertEqual(f"number:{Decimal(expected).normalize()}", canonical) - - def test_hex_constant_ending_in_f_is_paired_by_its_full_value(self) -> None: - self.swift.write_text("enum Engine { static let mask: UInt32 = 0xffff_ffff }\n") - self.kotlin.write_text("object Engine { const val MASK = 0xffff_ffffL }\n") - twin_map = parity_ledger.build_twin_map(self.root) - self.assertEqual(1, len(twin_map["constant_pairs"])) - self.assertFalse(any(item.rule.startswith("constant-") for item in self.findings(twin_map))) - self.kotlin.write_text("object Engine { const val MASK = 0xffff_fffdL }\n") - self.assertTrue(any(item.rule == "constant-value-mismatch" for item in self.findings(twin_map))) - - def test_unparseable_mapped_constant_is_reported(self) -> None: - self.swift.write_text("enum Engine { static let limit = makeLimit() }\n") - self.kotlin.write_text("object Engine { const val LIMIT = 3 }\n") - twin_map = parity_ledger.build_twin_map(self.root) - self.assertTrue(any(item.rule == "constant-unverifiable" for item in self.findings(twin_map))) - - def test_stale_constant_pair_is_reported_after_one_sided_rename(self) -> None: - self.write_clean_tree() - twin_map = parity_ledger.build_twin_map(self.root) - self.kotlin.write_text(self.kotlin.read_text().replace("SAMPLE_LIMIT", "RENAMED_LIMIT")) - - stale = [item for item in self.findings(twin_map) if item.rule == "stale-constant-pair"] - - self.assertEqual(1, len(stale)) - self.assertIn("SAMPLE_LIMIT", stale[0].text) - - def test_resolvable_constant_pair_has_no_stale_finding(self) -> None: - self.write_clean_tree() - twin_map = parity_ledger.build_twin_map(self.root) - - self.assertFalse(any(item.rule == "stale-constant-pair" for item in self.findings(twin_map))) - - def test_constant_owner_disambiguates_same_normalized_name(self) -> None: - self.swift.write_text("enum SedentaryDetector { static let defaultSmoothWindowS = 240.0 }\n") - self.kotlin = self.kotlin.with_name("SedentaryDetector.kt") - self.kotlin.write_text("object SedentaryDetector { const val DEFAULT_SMOOTH_WINDOW_S = 240.0 }\n") - self.kotlin.with_name("NapDetector.kt").write_text( - "object NapDetector { const val DEFAULT_SMOOTH_WINDOW_S = 120.0 }\n" - ) - twin_map = parity_ledger.build_twin_map(self.root) - pairs = twin_map["constant_pairs"] - self.assertEqual(1, len(pairs)) - self.assertIn("SedentaryDetector", pairs[0]["kotlin"]) - self.assertFalse(any(item.rule == "constant-ambiguous" for item in self.findings(twin_map))) - - def test_remaining_constant_ambiguity_is_reported(self) -> None: - self.swift.write_text("enum Engine { static let limit = 3 }\n") - self.kotlin.write_text("object First { const val LIMIT = 3 }\n") - self.kotlin.with_name("Second.kt").write_text("object Second { const val LIMIT = 3 }\n") - twin_map = parity_ledger.build_twin_map(self.root) - self.assertTrue(any(item.rule == "constant-ambiguous" for item in self.findings(twin_map))) - - def test_constant_value_mismatch_is_rejected(self) -> None: - self.write_clean_tree() - self.kotlin.write_text(self.kotlin.read_text().replace("SAMPLE_LIMIT = 3", "SAMPLE_LIMIT = 4")) - twin_map = parity_ledger.build_twin_map(self.root) - rules = {finding.rule for finding in self.findings(twin_map)} - self.assertIn("constant-value-mismatch", rules) - self.assertEqual(1, self.exit_code(twin_map)) - - def test_same_metadata_constant_change_is_rescanned_from_current_content(self) -> None: - self.write_clean_tree() - twin_map = parity_ledger.build_twin_map(self.root) - original_stat = self.kotlin.stat() - original_size = original_stat.st_size - - self.kotlin.write_text(self.kotlin.read_text().replace("SAMPLE_LIMIT = 3", "SAMPLE_LIMIT = 4")) - self.assertEqual(original_size, self.kotlin.stat().st_size) - os.utime(self.kotlin, ns=(original_stat.st_atime_ns, original_stat.st_mtime_ns)) - - rules = {finding.rule for finding in self.findings(twin_map)} - self.assertIn("constant-value-mismatch", rules) - - def test_platform_database_schema_versions_are_not_parity_twins(self) -> None: - swift = self.root / "Packages/WhoopStore/Sources/WhoopStore/WhoopStore.swift" - kotlin = self.root / "android/app/src/main/java/com/noop/data/WhoopDatabase.kt" - swift.parent.mkdir(parents=True) - kotlin.parent.mkdir(parents=True) - swift.write_text("public enum WhoopStore { public static let schemaVersion = 18 }\n") - kotlin.write_text("object WhoopDatabase { const val SCHEMA_VERSION = 31 }\n") - - twin_map = parity_ledger.build_twin_map(self.root) - pairs = {(item["swift"], item["kotlin"]) for item in twin_map["constant_pairs"]} - self.assertNotIn( - ( - "Packages/WhoopStore/Sources/WhoopStore/WhoopStore.swift::schemaVersion", - "android/app/src/main/java/com/noop/data/WhoopDatabase.kt::SCHEMA_VERSION", - ), - pairs, - ) - constant_findings = [item for item in self.findings(twin_map) if item.rule.startswith("constant-")] - self.assertEqual([], constant_findings) - - def test_test_only_wiring_is_rejected(self) -> None: - self.write_clean_tree() - self.swift.write_text(self.swift.read_text() + "\npublic func testOnlyHelper(_ value: Int) -> Int { value }\n") - test_path = self.root / "Packages/StrandAnalytics/Tests/StrandAnalyticsTests/EngineTests.swift" - test_path.parent.mkdir(parents=True) - test_path.write_text("func testHelper() { _ = testOnlyHelper(1) }\n") - twin_map = parity_ledger.build_twin_map(self.root) - rules = {finding.rule for finding in self.findings(twin_map)} - self.assertIn("test-only-callsite", rules) - self.assertEqual(1, self.exit_code(twin_map)) - - def test_test_only_calls_use_owner_and_arity_and_ignore_extension_declaration(self) -> None: - self.kotlin.write_text( - """object Alpha { fun collide(first: Int, second: Int) = first + second } -object Beta { fun collide(value: Int) = value } -fun Map.extensionOnly(value: Int) = value -fun production() = Alpha.collide(1, 2) -""" - ) - test_path = self.root / "android/app/src/test/java/com/noop/analytics/EngineTest.kt" - test_path.parent.mkdir(parents=True) - test_path.write_text("fun testIt() { Beta.collide(1); Map.extensionOnly(1) }\n") - findings = [item for item in self.findings() if item.rule == "test-only-callsite"] - texts = "\n".join(item.text for item in findings) - self.assertIn("Beta.collide/1", texts) - self.assertIn("Map.extensionOnly/1", texts) - self.assertNotIn("Alpha.collide/1", texts) - - def test_call_omitting_defaults_counts_as_production_callsite(self) -> None: - self.kotlin.write_text( - """object Roller { fun roll(rr: Int, windowSec: Int = 90, stepSec: Int = 0) = rr + windowSec + stepSec } -fun production() = Roller.roll(1) -""" - ) - test_path = self.root / "android/app/src/test/java/com/noop/analytics/RollTest.kt" - test_path.parent.mkdir(parents=True) - test_path.write_text("fun testIt() { Roller.roll(1, 2, 3) }\n") - texts = "\n".join( - item.text for item in self.findings() if item.rule == "test-only-callsite" - ) - self.assertNotIn("Roller.roll/3", texts) - - def test_call_cannot_omit_required_parameters(self) -> None: - self.kotlin.write_text( - """object Roller { fun roll(rr: Int, windowSec: Int, stepSec: Int) = rr + windowSec + stepSec } -fun unrelated() = Roller.roll(1) -""" - ) - test_path = self.root / "android/app/src/test/java/com/noop/analytics/RollTest.kt" - test_path.parent.mkdir(parents=True) - test_path.write_text("fun testIt() { Roller.roll(1, 2, 3) }\n") - texts = "\n".join( - item.text for item in self.findings() if item.rule == "test-only-callsite" - ) - self.assertIn("Roller.roll/3", texts) - - def test_unqualified_same_file_call_counts_despite_sibling_owner(self) -> None: - self.kotlin.write_text( - """object Verdict { fun verdict(value: Int) = value - fun production() = verdict(1) } -""" - ) - sibling = self.kotlin.parent / "Sibling.kt" - sibling.write_text("object Sibling { fun verdict(first: Int, second: Int) = first + second }\n") - test_path = self.root / "android/app/src/test/java/com/noop/analytics/VerdictTest.kt" - test_path.parent.mkdir(parents=True) - test_path.write_text("fun testIt() { Verdict.verdict(1); Sibling.verdict(1, 2) }\n") - texts = "\n".join( - item.text for item in self.findings() if item.rule == "test-only-callsite" - ) - self.assertNotIn("Verdict.verdict/1", texts) - self.assertIn("Sibling.verdict/2", texts) - - def test_exact_arity_match_wins_over_relaxed_overload(self) -> None: - self.kotlin.write_text( - """object Over { fun pick(value: Int) = value - fun pick(value: Int, extra: Int = 0) = value + extra } -fun production() = Over.pick(1) -""" - ) - test_path = self.root / "android/app/src/test/java/com/noop/analytics/OverTest.kt" - test_path.parent.mkdir(parents=True) - test_path.write_text("fun testIt() { Over.pick(1, 2) }\n") - texts = "\n".join( - item.text for item in self.findings() if item.rule == "test-only-callsite" - ) - self.assertIn("Over.pick/2", texts) - - def test_lowercase_instance_receiver_resolves_like_unqualified(self) -> None: - self.kotlin.write_text( - """object Burst { fun codesWithTimes(first: Int, second: Int, extra: Int = 0) = first + second + extra } -object Assembler { val burst = Burst - fun production(): Int { return burst.codesWithTimes(1, 2) } } -""" - ) - test_path = self.root / "android/app/src/test/java/com/noop/analytics/BurstTest.kt" - test_path.parent.mkdir(parents=True) - test_path.write_text("fun testIt() { Burst.codesWithTimes(1, 2, 3) }\n") - texts = "\n".join( - item.text for item in self.findings() if item.rule == "test-only-callsite" - ) - self.assertNotIn("Burst.codesWithTimes/3", texts) - - def test_same_file_resolution_prefers_the_lexical_owner(self) -> None: - self.kotlin.write_text( - """object First { fun add(value: Int) = value - fun production() = add(1) } -object Second { fun add(value: Int) = value } -""" - ) - test_path = self.root / "android/app/src/test/java/com/noop/analytics/AddTest.kt" - test_path.parent.mkdir(parents=True) - test_path.write_text("fun testIt() { Second.add(1) }\n") - texts = "\n".join( - item.text for item in self.findings() if item.rule == "test-only-callsite" - ) - self.assertNotIn("First.add/1", texts) - self.assertIn("Second.add/1", texts) - - def test_kotlin_call_inside_string_template_counts_as_production(self) -> None: - self.kotlin.write_text( - '''object Trace { fun suffix(value: Int) = value } -fun production(value: Int) = "trace=${Trace.suffix(value)}" -''' - ) - test_path = self.root / "android/app/src/test/java/com/noop/analytics/TraceTest.kt" - test_path.parent.mkdir(parents=True) - test_path.write_text("fun testIt() { Trace.suffix(1) }\n") - - texts = "\n".join( - item.text for item in self.findings() if item.rule == "test-only-callsite" - ) - - self.assertNotIn("Trace.suffix/1", texts) - - def test_kotlin_nested_templates_preserve_code_and_mask_nested_string_text(self) -> None: - self.kotlin.write_text( - r'''object Trace { - fun suffix(value: Int) = value - fun literalOnly(value: Int) = value -} -fun production(value: Int) = - "outer ${run { if (value > 0) { "inner ${Trace.suffix(value)} noise=Trace.literalOnly(value) tab=\t dollar=\$" } else "none" }}" -''' - ) - test_path = self.root / "android/app/src/test/java/com/noop/analytics/TraceTest.kt" - test_path.parent.mkdir(parents=True) - test_path.write_text( - "fun testIt() { Trace.suffix(1); Trace.literalOnly(1) }\n" - ) - - texts = "\n".join( - item.text for item in self.findings() if item.rule == "test-only-callsite" - ) - - self.assertNotIn("Trace.suffix/1", texts) - self.assertIn("Trace.literalOnly/1", texts) - - def test_kotlin_string_arguments_keep_overload_arity_and_argument_positions(self) -> None: - self.kotlin.write_text( - r'''object Target { - fun emit(value: Int) = value - fun emit(value: Int, text: String) = text - fun first(text: String, middle: Int, last: Int) = text - fun middle(first: Int, text: String, last: Int) = text - fun last(first: Int, middle: Int, text: String) = text - fun nested(value: Int) = value - fun literalOnly(value: Int) = value -} -fun production(value: String) { - Target.emit(1, "outer ${value.ifEmpty { "fallback" }}") - Target.first("plain literal", 2, 3) - Target.middle(1, "plain literal", 3) - Target.last(1, 2, "outer ${run { "inner ${Target.nested(4)}" }}") - val noise = "Target.literalOnly(5)" -} -''' - ) - test_path = self.root / "android/app/src/test/java/com/noop/analytics/TargetTest.kt" - test_path.parent.mkdir(parents=True) - test_path.write_text( - """fun testIt(text: String) { - Target.emit(1, text) - Target.first(text, 2, 3) - Target.middle(1, text, 3) - Target.last(1, 2, text) - Target.nested(4) - Target.literalOnly(5) -} -""" - ) - - texts = "\n".join( - item.text for item in self.findings() if item.rule == "test-only-callsite" - ) - - for signature in ("Target.emit/2", "Target.first/3", "Target.middle/3", "Target.last/3", - "Target.nested/1"): - self.assertNotIn(signature, texts) - self.assertIn("Target.literalOnly/1", texts) - - def test_kotlin_literal_comment_and_plain_malformed_strings_stay_masked(self) -> None: - self.kotlin.write_text( - r'''object Trace { - fun literalOnly(value: Int) = value - fun malformed(value: Int) = value -} -fun literal(value: Int): String { - val plain = "Trace.literalOnly(value)" - val escaped = "\${Trace.literalOnly(value)}" - // ${Trace.literalOnly(value) - /* nested /* ${Trace.literalOnly(value) */ comment */ - return plain + escaped -} -fun broken(value: Int) = "broken Trace.malformed(value) -''' - ) - test_path = self.root / "android/app/src/test/java/com/noop/analytics/TraceTest.kt" - test_path.parent.mkdir(parents=True) - test_path.write_text( - "fun testIt() { Trace.literalOnly(1); Trace.malformed(1) }\n" - ) - - result = parity_ledger.scan(self.root, parity_ledger.build_twin_map(self.root)) - texts = "\n".join( - item.text for item in result.findings if item.rule == "test-only-callsite" - ) - - self.assertEqual([], result.errors) - self.assertIn("Trace.literalOnly/1", texts) - self.assertIn("Trace.malformed/1", texts) - - def test_unterminated_kotlin_template_is_a_stable_scan_error_and_cli_failure(self) -> None: - self.kotlin.write_text( - '''object Trace { fun suffix(value: Int) = value } -fun broken(value: Int) = "broken ${run { Trace.suffix(value) } -''' - ) - twin_map = parity_ledger.build_twin_map(self.root) - - result = parity_ledger.scan(self.root, twin_map) - errors = [error.output() for error in result.errors] - - self.assertEqual( - [ - "android/app/src/main/java/com/noop/analytics/Engine.kt:2: " - "malformed-kotlin-template: unterminated Kotlin string template" - ], - errors, - ) - code, output = self.run_cli(twin_map, parity_ledger.build_compact_baseline(result)) - self.assertEqual(1, code) - self.assertIn("FAIL 1 parity ledger scan error(s):", output) - self.assertIn(errors[0], output) - self.assertTrue( - output.rstrip().endswith("Baseline not evaluated: 1 scan error."), - output, - ) - - def test_invalid_utf8_is_a_stable_hard_error(self) -> None: - self.write_clean_tree() - twin_map = parity_ledger.build_twin_map(self.root) - self.swift.write_bytes(b"enum Engine {\n\xff\n}\n") - result = parity_ledger.scan(self.root, twin_map) - self.assertEqual(["invalid-utf8"], [item.rule for item in result.errors]) - code, output = self.run_cli(twin_map, no_baseline=True) - self.assertEqual(1, code) - self.assertIn("invalid-utf8", output) - - def test_artificial_duplicate_is_rejected(self) -> None: - self.write_clean_tree() - extra = self.swift.with_name("Other.swift") - extra.write_text("public func dayString(_ value: Int) -> String { \"x\" }\n") - self.swift.write_text(self.swift.read_text() + "\npublic func dayString(_ value: Int, offset: Int) -> String { \"x\" }\n") - twin_map = parity_ledger.build_twin_map(self.root) - rules = {finding.rule for finding in self.findings(twin_map)} - self.assertIn("duplicate-implementation", rules) - self.assertEqual(1, self.exit_code(twin_map)) - - def test_baseline_suppresses_known_finding_but_rejects_new_finding(self) -> None: - self.write_clean_tree() - self.kotlin.write_text(self.kotlin.read_text().replace("SAMPLE_LIMIT = 3", "SAMPLE_LIMIT = 4")) - twin_map = parity_ledger.build_twin_map(self.root) - baseline = self.baseline_for(twin_map) - self.assertEqual(0, self.run_cli(twin_map, baseline)[0]) - self.swift.write_text(self.swift.read_text() + "\nfunc newRegression() {}\n") - self.assertEqual(1, self.run_cli(twin_map, baseline)[0]) - - def test_disappeared_baseline_finding_is_warning_only(self) -> None: - self.write_clean_tree() - self.kotlin.write_text(self.kotlin.read_text().replace("SAMPLE_LIMIT = 3", "SAMPLE_LIMIT = 4")) - twin_map = parity_ledger.build_twin_map(self.root) - baseline = self.baseline_for(twin_map) - self.mark_current_tree_as_origin_main() - self.kotlin.write_text(self.kotlin.read_text().replace("SAMPLE_LIMIT = 4", "SAMPLE_LIMIT = 3")) - code, output = self.run_cli(twin_map, baseline) - self.assertEqual(0, code) - self.assertIn("WARNING debt decreased", output) - - def test_lower_count_with_replacement_finding_still_blocks(self) -> None: - self.write_clean_tree() - self.swift.write_text( - self.swift.read_text().replace( - "public static let sampleLimit = 3", - "public static let sampleLimit = 3\n public static let windowLimit = 5", - ) - ) - self.kotlin.write_text( - self.kotlin.read_text().replace( - "const val SAMPLE_LIMIT = 3", - "const val SAMPLE_LIMIT = 4\n const val WINDOW_LIMIT = 6", - ) - ) - twin_map = parity_ledger.build_twin_map(self.root) - baseline = self.baseline_for(twin_map) - self.mark_current_tree_as_origin_main() - - self.swift.write_text( - self.swift.read_text().replace( - "public static let windowLimit = 5", - "public static let windowLimit = 5\n public static let newLimit = 8", - ) - ) - self.kotlin.write_text( - self.kotlin.read_text() - .replace("const val SAMPLE_LIMIT = 4", "const val SAMPLE_LIMIT = 3") - .replace("const val WINDOW_LIMIT = 6", "const val WINDOW_LIMIT = 5\n const val NEW_LIMIT = 9") - ) - current_map = parity_ledger.build_twin_map(self.root) - code, output = self.run_cli(current_map, baseline) - self.assertEqual(1, code) - self.assertIn("replacement findings are new", output) - self.assertIn("newLimit", output) - - def test_improvement_base_selection_and_missing_ref_are_fail_closed(self) -> None: - self.write_clean_tree() - self.kotlin.write_text(self.kotlin.read_text().replace("SAMPLE_LIMIT = 3", "SAMPLE_LIMIT = 4")) - twin_map = parity_ledger.build_twin_map(self.root) - baseline = self.baseline_for(twin_map) - self.mark_current_tree_as_origin_main() - base_sha = subprocess.check_output( - ["git", "rev-parse", "origin/main"], cwd=self.root, text=True - ).strip() - self.kotlin.write_text(self.kotlin.read_text().replace("SAMPLE_LIMIT = 4", "SAMPLE_LIMIT = 3")) - - real = parity_ledger.finding_identities_at_git_ref - with mock.patch.object(parity_ledger, "finding_identities_at_git_ref", wraps=real) as scanned: - self.assertEqual(0, self.run_cli(twin_map, baseline)[0]) - scanned.assert_called_once_with(self.root.resolve(), "origin/main") - with mock.patch.object(parity_ledger, "finding_identities_at_git_ref", wraps=real) as scanned: - self.assertEqual(0, self.run_cli(twin_map, baseline, base=base_sha)[0]) - scanned.assert_called_once_with(self.root.resolve(), base_sha) - code, output = self.run_cli(twin_map, baseline, base="missing/shallow-base") - self.assertEqual(1, code) - self.assertIn("cannot scan exact base", output) - - def test_base_ref_is_resolved_once_before_archive(self) -> None: - self.write_clean_tree() - self.mark_current_tree_as_origin_main() - expected = subprocess.check_output( - ["git", "rev-parse", "origin/main"], cwd=self.root, text=True - ).strip() - real = subprocess.check_output - calls: list[list[str]] = [] - - def recording(arguments, **kwargs): - calls.append(arguments) - return real(arguments, **kwargs) - - with mock.patch.object(parity_ledger.subprocess, "check_output", side_effect=recording): - parity_ledger.finding_identities_at_git_ref(self.root, "origin/main") - - self.assertEqual( - [["git", "rev-parse", "--verify", "origin/main^{commit}"], - ["git", "archive", "--format=tar", expected]], - calls, - ) - - def test_counter_increase_beyond_baseline_is_rejected(self) -> None: - self.write_clean_tree() - old_map = parity_ledger.build_twin_map(self.root) - baseline = self.baseline_for(old_map) - self.swift.write_text(self.swift.read_text() + "\nfunc dayString(_ value: Int) -> String { \"x\" }\n") - new_map = parity_ledger.build_twin_map(self.root) - code, output = self.run_cli(new_map, baseline) - self.assertEqual(1, code) - self.assertIn("compact baseline drift", output) - - def test_changed_value_of_baselined_mismatch_is_a_new_finding(self) -> None: - self.write_clean_tree() - self.kotlin.write_text(self.kotlin.read_text().replace("SAMPLE_LIMIT = 3", "SAMPLE_LIMIT = 4")) - twin_map = parity_ledger.build_twin_map(self.root) - baseline = self.baseline_for(twin_map) - self.kotlin.write_text(self.kotlin.read_text().replace("SAMPLE_LIMIT = 4", "SAMPLE_LIMIT = 5")) - code, output = self.run_cli(twin_map, baseline) - self.assertEqual(1, code) - self.assertIn("constant-value-mismatch", output) - - -if __name__ == "__main__": - unittest.main() diff --git a/Tools/tests/test_rr_legacy_preservation_contract.py b/Tools/tests/test_rr_legacy_preservation_contract.py deleted file mode 100644 index 8e8be1d6a4..0000000000 --- a/Tools/tests/test_rr_legacy_preservation_contract.py +++ /dev/null @@ -1,58 +0,0 @@ -"""Source contracts for cross-platform legacy WHOOP 5 score preservation.""" - -from __future__ import annotations - -import unittest -from pathlib import Path - - -ROOT = Path(__file__).resolve().parents[2] -SWIFT_ENGINE = ROOT / "Strand/Data/IntelligenceEngine.swift" -ANDROID_PERSISTENCE = ( - ROOT - / "android/app/src/main/java/com/noop/analytics/IntelligencePersistence.kt" -) - - -class LegacyScorePreservationContractTests(unittest.TestCase): - def test_swift_protects_only_the_persisted_and_displayed_copy(self) -> None: - source = SWIFT_ENGINE.read_text() - protection_start = source.index("// Apply the exact snapshot only after") - persistence_end = source.index("markPostLoopPhase(\"persist\")", protection_start) - persistence = source[protection_start:persistence_end] - - self.assertIn("var persistedDailies = dailies", persistence) - self.assertIn("for index in persistedDailies.indices", persistence) - self.assertIn("let fresh = persistedDailies[index]", persistence) - self.assertIn("persistedDailies[index] = fresh.with(avgHrv: snapshot.avgHrv", persistence) - self.assertIn("recovery: snapshot.recovery", persistence) - self.assertIn("respRateBpm: snapshot.respRateBpm", persistence) - self.assertIn("avgSdnn: snapshot.avgSdnn", persistence) - self.assertIn("for daily in persistedDailies", persistence) - self.assertIn("dailyMetrics: persistedDailies", persistence) - self.assertNotIn("dailies[index] = fresh.with(avgHrv:", persistence) - - derivations = source[persistence_end:] - self.assertIn("let fa7 = dailies.sorted", derivations) - self.assertIn("for d in dailies { faGateByDay[d.day] = d }", derivations) - self.assertIn("let vHRVs = fa7.compactMap { $0.avgHrv }", derivations) - self.assertNotIn("persistedDailies", derivations) - - def test_android_also_protects_a_separate_persistence_copy(self) -> None: - source = ANDROID_PERSISTENCE.read_text() - preparation_start = source.index("suspend fun prepareComputedWindow") - preparation_end = source.index("fun scoreProvenance", preparation_start) - preparation = source[preparation_start:preparation_end] - - self.assertIn("val mutableDailies = dailies.toMutableList()", preparation) - self.assertIn( - "repo, computedId, from, to, mutableDailies, ownerByDay,", - preparation, - ) - self.assertIn("dailies = mutableDailies", preparation) - self.assertIn("respRateBpm = existing.respRateBpm", source) - self.assertIn("avgSdnn = existing.avgSdnn", source) - - -if __name__ == "__main__": - unittest.main()