Repository navigation
feat(legal-status): global append-only store for public legal status … - #14
Merged
Merged
Conversation
…(ADR 0006) Legal status of a published patent is public and the same for every organization, so it is stored once in two global tables without organization_id or tenant RLS. Which organization looks at which patent is sensitive, so no requester, case or user field is allowed in these tables; tenant references will live in tenant-scoped (FORCE RLS) tables. - 0018_legal_status_store: legal_status_source_records (each lookup, including not_found, with request_ref, retrieved_at, the source's own data time, the rule-relevant extract, and the SHA-256, size and storage key of the exact response bytes) and legal_status_assessments (status, term, countries, evidence, review reasons, rules_version, content_sha256; unique per record/as_of/rules_version). Runtime roles: worker SELECT+INSERT, app SELECT, platform nothing; no UPDATE/DELETE, so both are append-only. - Raw responses are content-addressed in object storage (public/legal-status/<source>/<sha256>.json). The DB keeps only fields the rules read. Raw retention involves public personal data and needs compliance sign-off (ADR 0006). - Worker LegalStatusRefresher: fetch -> archive exact bytes -> record -> assess from the stored extract -> record. A source failure writes nothing; not_found records the lookup but no status. - Clients now expose the exact response bytes (SourceResponse) so the stored hash proves what the source sent. - RULES_VERSION and canonical serialization in modules/legal_status. Tests: 8 Postgres integration tests (exact grants, no tenant columns, app cannot write, worker cannot UPDATE/DELETE, found rows must carry extract and hash, US/EP end-to-end with each stored assessment reproduced from its stored extract to the same content hash, not_found, failure writes nothing) plus 3 unit tests. test-postgres.sh now runs the new file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
corlin
added a commit
that referenced
this pull request
Oct 4, 2026
…5) (#16) ## What Stage 1 of ADR 0005: the candidate set of US/EP patents that the stage-2 legal-status check (#11, #14) runs over. The design is in **ADR 0007**, and every scope parameter there is backed by a measurement. | Decision | Basis (measured 2026-10-04) | |---|---| | US `B1/B2/E1`, EP `B1/B2/B3`, granted, H01M/H02J by **CPC OR IPC** | Reissues (`E1`, 201) replace the original patent. CPC-only filtering misses IPC-only documents (see the spike report) | | **US: filed ≥ as_of − 20y − 5y** | Patent term adjustment is common: 48/59 sampled patents had it, up to **1,220 days**. Stage 2 computes the exact expiry | | **EP: filed ≥ as_of − 20y, no margin** | The EPC has no term adjustment. BigQuery's EP filing date matched EPO's for **57/57** samples | | Cost of the margin | +11% rows (US 140,344 → 156,306; EP 48,173 → 53,452) | ADR 0007 also defines a **calibration rule** for the US margin: after the first full legal-status run, check whether any US patent whose term started 20–25 years ago is still presumed in force. ## Storage (ADR 0006 pattern) - Global, append-only tables `patent_corpus_loads` and `patent_corpus_members`. Worker: SELECT+INSERT; app: SELECT; no UPDATE/DELETE; no tenant columns. - Each load records its scope and cutoffs, the **exact query and its SHA-256**, bytes billed, the source table's last-modified time and the row count, so an FTO analysis can name the corpus it used. - A load is written **atomically** (tested: a failure part-way leaves nothing). The row count is known up front because the tables can't be updated. - Inventors are not loaded. Assignee names are, since they're needed for ownership; some may be individuals' names, which are public bibliographic data. ## Loader `scripts/load-patent-corpus.py` is run by an operator; it's not part of any image. - It dry-runs by default (no cost), and the load query dry-runs at **58.88 GB**. - `--execute` requires `--max-gb`, and the script refuses if the scan exceeds it (both guards tested: exit 2 and exit 1). - The BigQuery client is an optional `ingest` extra. **It's absent from the API image** (verified on the rebuilt image). - Credentials come from the operator's Google Cloud ADC; nothing is stored in the repo. ## Not done yet - **No data has been loaded.** The first full load runs after this is reviewed (≈59 GB scan; this month's usage so far is about 333 GB of the 1 TB free tier). - Legal-status support for US reissues (`RE`-prefixed numbers) isn't there yet; they're flagged. - Next steps: batch legal-status job, then the watch tier (published applications), then margin calibration. ## Validation - `./scripts/test-postgres.sh`: **103 passed** (99 + 4 new), including the downgrade/upgrade cycle through `0019` - `.venv/bin/pytest apps/api/tests/unit/`: 382 passed (6 new) - `pnpm test:web`: 62 passed; `pnpm build:web`: ok - `verify-source-lock`, `verify-scaffold`, `verify-p0-02`, `verify-p0-e2e`, `docker compose config`: ok - Loader dry run against BigQuery: 58.88 GB; guard refusals verified 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…(ADR 0006)
Legal status of a published patent is public and the same for every organization, so it is stored once in two global tables without organization_id or tenant RLS. Which organization looks at which patent is sensitive, so no requester, case or user field is allowed in these tables; tenant references will live in tenant-scoped (FORCE RLS) tables.
assess from the stored extract -> record. A source failure writes nothing; not_found records the lookup but no status.
Tests: 8 Postgres integration tests (exact grants, no tenant columns, app cannot write, worker cannot UPDATE/DELETE, found rows must carry extract and hash, US/EP end-to-end with each stored assessment reproduced from its stored extract to the same content hash, not_found, failure writes nothing) plus 3 unit tests. test-postgres.sh now runs the new file.