Add runnable examples, a working quickstart, metadata and the 0.1.0 changeset

.changeset/README.md

@@ -24,5 +24,14 @@ Add a changeset for every meaningful change:
 pnpm changeset
 ```
 
-During pre-alpha, pick **patch**: that keeps the package on the `0.0.x` ladder until its first
-alpha. See the cosyte version ladder in the meta-repo's `documentation/conventions.md`.
+Pick the bump type from what the change does to the published package. On the `0.1.x` ladder:
+
+- **minor** for anything a consumer gains: a newly exported symbol, a new option, a new stable
+  code, or a new published artifact they can use. It takes `0.1.x` to `0.2.0`.
+- **patch** for a fix, for documentation of a surface that already ships, and for contributor-only
+  tooling. It takes `0.1.0` to `0.1.1`.
+- **major** only when you mean to declare `1.0.0`. Before that, a breaking change is a **minor**,
+  and the break has to be spelled out in the summary, because the number alone tells a consumer on
+  a `0.x` release nothing.
+
+See the cosyte version ladder in the meta-repo's `documentation/conventions.md`.

.changeset/what-0-1-means.md

@@ -0,0 +1,13 @@
+---
+"@cosyte/deid": minor
+---
+
+`0.1.0` is the first release of `@cosyte/deid` whose public API we ask you to build on.
+
+**What is covered, and what you can depend on.** Applying a de-identification policy to a structurally located model of a healthcare document and getting back the transformed model plus a value-free manifest. That means the built-in HIPAA Safe Harbor policy across all 18 identifier categories, `defineDeidPolicy`, the two named profiles and `defineDeidProfile` (which can only tighten a profile), the five transforms (redact, generalize, keyed date-shift, keyed pseudonymize, keyed hash), the adapters for HL7 v2, C-CDA, FHIR R4, X12, NCPDP Telecom and DICOM metadata at their own subpaths, the corpus registry for consistent keyed surrogates and date shifts across documents, the free-text redactor interface, and the Expert Determination support report. Failing closed is part of the contract: an uncertain value is blocked, free text is blocked unless your redactor handles it, and a keyed transform with no key is a fatal `DEID_NO_KEY`. The stable `DEID_*` codes are public API, so renaming or removing one is a breaking change. Node.js 22 and 24, ESM and CommonJS, with type declarations for both, and no third-party runtime dependency.
+
+**What the version promises.** Until 1.0, a breaking change raises the minor version (0.1 to 0.2), and the changelog entry says what broke and what to change. A patch release (0.1.x) does not break you, so a `^0.1.0` range takes the patches and stops before 0.2.0.
+
+**What it is not, and what is not covered yet.** A result is "Safe-Harbor-transformed per the configured policy". It is not a certification that data is de-identified and not a compliance claim: the actual-knowledge condition of Safe Harbor stays with you, and an Expert Determination is supported, never rendered. Not covered yet: NCPDP SCRIPT, which is refused with `DEID_FORMAT_UNSUPPORTED`; DICOM pixel data, where burned-in text is flagged and never cleaned; a built-in free-text scrubber; and value-bearing positions no rule names, which are counted as unexamined residuals rather than cleaned. The documentation at https://docs.cosyte.com/deid lists the known limitations in full.
+
+The repository now carries runnable examples under `examples/`, including the fail-closed cases, run against the built package on every change.

.github/workflows/ci.yml

@@ -64,3 +64,29 @@ jobs:
     uses: cosyte/.github/.github/workflows/ci.yml@main
     with:
       run-phi-scan: true
+
+  # The programs under examples/, run against the BUILT package the way a consumer imports it:
+  # `pnpm examples` (scripts/run-examples.ts) fails when any example exits non-zero, and each
+  # example checks its own output, the fail-closed cases included. Added on top of the caller above,
+  # never instead of it. Its contexts, `examples (22)` and `examples (24)`, are new and deliberately
+  # NOT in the `ci-required-checks` ruleset yet: let them run on `main` first, then read the real
+  # names off a live check run before requiring them.
+  examples:
+    strategy:
+      fail-fast: false
+      matrix:
+        node: ["22", "24"]
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    steps:
+      - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
+      - uses: pnpm/action-setup@0ebf47130e4866e96fce0953f49152a61190b271 # v6.0.9
+      - uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
+        with:
+          node-version: ${{ matrix.node }}
+          cache: pnpm
+      - run: pnpm install --frozen-lockfile
+      - name: Build (the examples import the built package)
+        run: pnpm build
+      - name: Run every example
+        run: pnpm examples

README.md

@@ -23,15 +23,17 @@ but **inverts the reflex**: where a parser is liberal on input, a de-identifier
 > (§164.514(b)(1)) is supported, never rendered or certified.** The certification is
 > always the consumer's.
 
-> **Status:** pre-alpha (`0.0.x`), published on npm. This release ships the **format-agnostic
-> core** plus six format bindings: the **HL7 v2 adapter** (`@cosyte/deid/hl7`), the **C-CDA adapter**
+> **Status:** `0.1`, published on npm. Below 1.0, a breaking change raises the minor version and the
+> changelog says what broke. This release ships the **format-agnostic core** plus six format
+> bindings: the **HL7 v2 adapter** (`@cosyte/deid/hl7`), the **C-CDA adapter**
 > (`@cosyte/deid/ccda`), the **FHIR R4 adapter** (`@cosyte/deid/fhir`), the **X12 EDI adapter**
-> (`@cosyte/deid/x12`), the **NCPDP Telecom adapter** (`@cosyte/deid/ncpdp`), and the **DICOM adapter**
-> (`@cosyte/deid/dicom`), plus the **longitudinal layer**, the corpus registry (`createDeidRegistry`)
-> for cross-document consistency and the formalized key contract, and the **Expert-Determination support
-> report** (`buildExpertDeterminationSupportReport`) that structures the manifest for a statistician
-> **without ever rendering a determination**. NCPDP SCRIPT is **not** supported, and an entry point
-> handed one refuses it outright rather than returning a partial pass.
+> (`@cosyte/deid/x12`), the **NCPDP Telecom adapter** (`@cosyte/deid/ncpdp`), and the **DICOM
+> adapter** (`@cosyte/deid/dicom`), plus the **longitudinal layer**, the corpus registry
+> (`createDeidRegistry`) for cross-document consistency and the formalized key contract, and the
+> **Expert-Determination support report** (`buildExpertDeterminationSupportReport`) that structures
+> the manifest for a statistician **without ever rendering a determination**. NCPDP SCRIPT is
+> **not** supported, and an entry point handed one refuses it outright rather than returning a
+> partial pass.
 
 ## Install
 
@@ -44,6 +46,10 @@ npm install @cosyte/deid
 ```ts
 import { deidentify, SAFE_HARBOR_CATEGORIES } from "@cosyte/deid";
 
+// Synthetic inputs: placeholders in the shape of an HL7 v2 name and date of birth, not a person.
+const name = "ZZFAMILY^ZZGIVEN";
+const dob = "19900215";
+
 const { document, manifest } = deidentify(
   {
     loci: [
@@ -55,12 +61,20 @@ const { document, manifest } = deidentify(
   {},
 );
 
-// document.loci[0].value === null   (name removed)
-// document.loci[1].value === "<year>" (date generalized)
-// document.loci[2].value === "5.4 mmol/L" (clinical value retained, the over-scrub guard)
-// manifest records each category + locus + disposition, never a value.
+console.log(document.loci.map((locus) => locus.value));
+console.log(manifest.map((entry) => `${entry.locus} ${entry.category} ${entry.disposition}`));
+```
+
+```text
+[ null, '1990', '5.4 mmol/L' ]
+[ 'PID-5 NAMES removed', 'PID-7 DATES transformed' ]
 ```
 
+The name is removed, the date keeps only its year, and the clinical value is retained untouched (the
+over-scrub guard). The manifest records the category, locus and disposition of each decision, never
+the value. [`examples/`](https://github.com/cosyte/deid/tree/main/examples) has runnable programs for
+the HL7 v2 adapter, the fail-closed cases and the Expert Determination support report.
+
 ## Keyed transforms
 
 Pseudonymization and keyed hashing use a **keyed HMAC-SHA-256**; the key is the consumer's and never

docs-content/installation.md

@@ -11,7 +11,7 @@ sidebar_position: 1
 builds with per-condition type declarations, so it works from either module system without
 configuration.
 
-> **Status:** pre-alpha (`0.0.x`), published to npm. The command below installs the current release.
+> **Status:** `0.1`, published to npm. The command below installs the current release.
 > `npm view @cosyte/deid version` is the authority on which one that is.
 
 ## Prerequisites

docs-content/intro.md

@@ -22,9 +22,10 @@ through as safe.
 > Determination (§164.514(b)(1)) is *supported*, never *rendered* or certified. The
 > certification is always the consumer's.
 
-> **Status:** pre-alpha (`0.0.x`), published to npm. This release ships the **format-agnostic
-> core** (policy engine, five transforms, 18-category Safe Harbor model, fail-closed rule, value-free
-> manifest) **plus six format bindings**: HL7 v2 (`@cosyte/deid/hl7`), C-CDA (`@cosyte/deid/ccda`),
+> **Status:** `0.1`, published to npm. Below 1.0, a breaking change raises the minor version and
+> the changelog says what broke. This release ships the **format-agnostic core** (policy engine,
+> five transforms, 18-category Safe Harbor model, fail-closed rule, value-free manifest) **plus six
+> format bindings**: HL7 v2 (`@cosyte/deid/hl7`), C-CDA (`@cosyte/deid/ccda`),
 > FHIR R4 (`@cosyte/deid/fhir`), X12 EDI (`@cosyte/deid/x12`), NCPDP Telecom (`@cosyte/deid/ncpdp`) and
 > DICOM (`@cosyte/deid/dicom`). NCPDP SCRIPT is **not** supported, and an entry point handed one
 > refuses it outright rather than returning a partial pass.

eslint.config.js

@@ -1,3 +1,15 @@
 import cosyte from "@cosyte/eslint-config";
 
-export default cosyte(import.meta.dirname);
+export default [
+  ...cosyte(import.meta.dirname, {
+    files: ["src/**/*.ts", "test/**/*.ts", "scripts/**/*.ts", "examples/**/*.ts", "*.config.ts"],
+  }),
+
+  // The examples are programs a reader runs: printing what they show is their job.
+  {
+    files: ["examples/**/*.ts"],
+    rules: {
+      "no-console": "off",
+    },
+  },
+];

examples/README.md

@@ -0,0 +1,36 @@
+# Examples
+
+Four small programs that show what `@cosyte/deid` does, each runnable as it stands. Every example
+imports `@cosyte/deid` by its published name, which resolves through this package's own `exports`
+to the built `dist/`, so build first. Each one checks its own output and exits non-zero on a
+mismatch.
+
+```bash
+pnpm install
+pnpm build
+pnpm examples                                # runs all four
+pnpm tsx examples/deidentify-a-model.ts      # or one at a time
+```
+
+| Example                                                                | What it shows                                                                                                                                                                                                           |
+| ---------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| [`deidentify-a-model.ts`](./deidentify-a-model.ts)                     | The core engine on a generic locus model: a name removed, a date kept to its year, a ZIP kept to three digits, a clinical value retained, free text and an unknown value blocked, and a manifest that carries no value. |
+| [`deidentify-an-hl7-message.ts`](./deidentify-an-hl7-message.ts)       | The `@cosyte/deid/hl7` adapter on a parsed ORU^R01: identifiers located by structure, structured results kept, string and narrative results and the NTE comment blocked, and no input identifier left in the output.    |
+| [`fail-closed.ts`](./fail-closed.ts)                                   | What happens when the library cannot be sure: free text blocked with no redactor, and again when your redactor throws or returns nothing; a keyed transform with no key refused; a mislabelled policy refused.          |
+| [`expert-determination-support.ts`](./expert-determination-support.ts) | The Expert Determination support report built from a manifest. `determination` is always `null`: the library structures the facts for a qualified expert and never renders a determination or a risk score.             |
+
+**What these examples do not show, because the library does not do it.** A result is
+"Safe-Harbor-transformed per the configured policy". It is not a certification that data is
+de-identified, and nothing here makes the actual-knowledge judgment Safe Harbor also requires, or an
+Expert Determination. Those stay with you. See the
+[known limitations](../docs-content/limitations.md) before relying on this for data that leaves your
+control.
+
+Every input is synthetic, in the shape of the repository's own fixtures under `test/fixtures/`: `ZZ`
+placeholder names and identifiers, reserved `555` phone numbers, and `900`-range numbers that are
+never issued. `@cosyte/hl7` is an optional peer dependency of the package and a development
+dependency here, which is what the HL7 examples parse with.
+
+`tsconfig.json` in this folder maps `@cosyte/deid` to the source, so `pnpm typecheck` and
+`pnpm lint` check the examples before anything is built. At run time nothing maps the name: Node
+resolves it through `exports`, as it does in your project.

examples/deidentify-a-model.ts

@@ -0,0 +1,87 @@
+/**
+ * Apply the built-in Safe Harbor policy to a generic locus model and read the value-free manifest.
+ *
+ * The core works on a flat list of structurally located values; the per-format adapters build that
+ * list for you. Identifiers are removed, a date keeps its year, a ZIP keeps a safe three-digit
+ * prefix, a clinical value is kept untouched, and anything uncertain (free text, a value with no
+ * known category) is blocked. The result is "Safe-Harbor-transformed per the configured policy":
+ * a record of what the code did, not a certification that the output is de-identified.
+ *
+ * Run it after `pnpm build`:
+ *
+ *     pnpm tsx examples/deidentify-a-model.ts
+ */
+
+import assert from "node:assert/strict";
+
+import { deidentify, OUTPUT_LABEL, SAFE_HARBOR_CATEGORIES } from "@cosyte/deid";
+
+// Synthetic values in the shape of the repository's own fixtures: none of them is a person.
+const syntheticName = "ZZLABFAMILY^ZZLABGIVEN";
+const syntheticDateOfBirth = "19850302";
+const syntheticZip = "90210";
+const narrative = "Reported by ZZLABFAMILY on behalf of the ordering provider.";
+
+const { document, manifest } = deidentify(
+  {
+    loci: [
+      {
+        path: "PID-5",
+        kind: "identifier",
+        category: SAFE_HARBOR_CATEGORIES.NAMES,
+        value: syntheticName,
+      },
+      {
+        path: "PID-7",
+        kind: "date",
+        category: SAFE_HARBOR_CATEGORIES.DATES,
+        value: syntheticDateOfBirth,
+      },
+      {
+        path: "PID-11",
+        kind: "zip",
+        category: SAFE_HARBOR_CATEGORIES.GEOGRAPHIC,
+        value: syntheticZip,
+      },
+      { path: "OBX-5", kind: "clinical", value: "140 mmol/L" },
+      { path: "NTE-3", kind: "freetext", value: narrative },
+      { path: "ZPI-2", kind: "unknown", value: "ZZSECRETNOTE" },
+    ],
+  },
+  {},
+);
+
+for (const locus of document.loci) {
+  console.log(
+    `${locus.path.padEnd(7)} ${locus.disposition.padEnd(12)} ${JSON.stringify(locus.value)}`,
+  );
+}
+
+const byPath = new Map(document.loci.map((locus) => [locus.path, locus]));
+assert.equal(byPath.get("PID-5")?.value, null, "the name is removed");
+assert.equal(byPath.get("PID-7")?.value, "1985", "the date of birth keeps its year only");
+assert.equal(byPath.get("PID-11")?.value, "902", "the ZIP keeps a safe three-digit prefix");
+assert.equal(byPath.get("OBX-5")?.value, "140 mmol/L", "the clinical value is retained untouched");
+assert.equal(byPath.get("NTE-3")?.disposition, "blocked", "free text fails closed");
+assert.equal(byPath.get("ZPI-2")?.disposition, "blocked", "a value of unknown kind fails closed");
+
+// The manifest records category, transform, locus, count, disposition and code: never a value.
+console.log("manifest:");
+for (const entry of manifest) {
+  console.log(
+    `  ${entry.locus.padEnd(7)} ${entry.category.padEnd(16)} ${entry.transform.padEnd(11)} ${entry.code}`,
+  );
+}
+const recorded = JSON.stringify(manifest);
+for (const input of [
+  syntheticName,
+  syntheticDateOfBirth,
+  syntheticZip,
+  narrative,
+  "ZZSECRETNOTE",
+]) {
+  assert.ok(!recorded.includes(input), "the manifest carries no input value");
+}
+
+console.log(`label: ${OUTPUT_LABEL}`);
+assert.equal(OUTPUT_LABEL, "Safe-Harbor-transformed per the configured policy");

examples/deidentify-an-hl7-message.ts

@@ -0,0 +1,81 @@
+/**
+ * De-identify a parsed HL7 v2 ORU^R01 with the `@cosyte/deid/hl7` adapter.
+ *
+ * The adapter locates identifiers by the message's structure (PID-5 is the patient's name because
+ * HL7 v2 says so), never by a regular expression over the bytes, and returns a transformed
+ * `Hl7Message` plus the value-free manifest. `@cosyte/hl7` is an optional peer dependency: install
+ * it beside `@cosyte/deid` to use this subpath. The built-in Safe Harbor policy uses no keyed
+ * transform, so no key is needed here.
+ *
+ * Run it after `pnpm build`:
+ *
+ *     pnpm tsx examples/deidentify-an-hl7-message.ts
+ */
+
+import assert from "node:assert/strict";
+
+import { parseHL7 } from "@cosyte/hl7";
+
+import { deidentifyHl7 } from "@cosyte/deid/hl7";
+
+// The repository's synthetic fixture test/fixtures/hl7/oru-r01.hl7: every identifier is a ZZ
+// placeholder, a reserved 555 phone number or a 900-range number that is never issued.
+const message = [
+  "MSH|^~\\&|LABAPP|LABFAC|EHRAPP|EHRFAC|20200102080000||ORU^R01|SYNTHMSG002|P|2.5",
+  "PID|1||ZZMRN002^^^HOSP^MR||ZZLABFAMILY^ZZLABGIVEN||19850302|F|||ZZLABSTREET^^ZZLABCITY^MA^90210|049|5550000010|||||ZZACCT200|900000010",
+  "OBR|1|PLACER001|FILLER001|2951-2^Sodium^LN|||20200102080000",
+  "OBX|1|NM|2951-2^Sodium^LN|1|140|mmol/L|135-145|N|||F",
+  "OBX|2|NM|2823-3^Potassium^LN|1|4.2|mmol/L|3.5-5.1|N|||F",
+  "OBX|3|CWE|883-9^ABO group^LN|1|O^O positive^L||||||F",
+  "OBX|4|ST|11557-6^Interpretation^LN|1|Within normal limits|||N|||F",
+  "OBX|5|TX|1234-5^Pathology note^LN|1|Specimen from ZZLABFAMILY reviewed; contact 5550000010 for questions|||||F",
+  "NTE|1|L|Reported by ZZLABFAMILY on behalf of the ordering provider.",
+].join("\r");
+
+const { document, manifest, unexaminedResiduals } = deidentifyHl7(parseHL7(message));
+const output = document.toString();
+console.log(output.split("\r").join("\n"));
+
+const segments = output.split("\r").map((segment) => segment.split("|"));
+const pid = segments.find((fields) => fields[0] === "PID") ?? [];
+const obx = segments.filter((fields) => fields[0] === "OBX");
+
+assert.equal(pid[5], "", "PID-5, the patient's name, is removed");
+assert.equal(pid[7], "1985", "PID-7, the date of birth, keeps its year only");
+assert.equal(pid[11], "^^^^902", "PID-11 keeps only a safe three-digit ZIP");
+assert.deepEqual(
+  obx.map((fields) => fields[5]),
+  ["140", "4.2", "O^O positive^L", "", ""],
+  "structured clinical results survive; string and narrative results are blocked",
+);
+
+// Not one identifier from the input survives anywhere in the output, or in the manifest.
+const identifiers = [
+  "ZZMRN002",
+  "ZZLABFAMILY",
+  "ZZLABGIVEN",
+  "19850302",
+  "ZZLABSTREET",
+  "ZZLABCITY",
+  "5550000010",
+  "ZZACCT200",
+  "900000010",
+];
+const recorded = JSON.stringify(manifest);
+for (const identifier of identifiers) {
+  assert.ok(!output.includes(identifier), "no input identifier survives in the output");
+  assert.ok(!recorded.includes(identifier), "the manifest carries no input value");
+}
+
+const removed = manifest
+  .filter((entry) => entry.disposition === "removed")
+  .map((entry) => entry.locus);
+const blocked = manifest
+  .filter((entry) => entry.disposition === "blocked")
+  .map((entry) => entry.locus);
+console.log(`removed: ${removed.join(", ")}`);
+console.log(`blocked (fail closed): ${blocked.join(", ")}`);
+console.log(
+  `positions no rule examined, counted and never cleaned: ${String(unexaminedResiduals.length)}`,
+);
+assert.ok(blocked.includes("NTE-3"), "the NTE comment is blocked");