diff --git a/.changeset/brave-pandas-remove.md b/.changeset/brave-pandas-remove.md new file mode 100644 index 0000000..ac59587 --- /dev/null +++ b/.changeset/brave-pandas-remove.md @@ -0,0 +1,48 @@ +--- +"@cosyte/dicom": minor +--- + +Remove a retained private value this run did not enumerate, instead of shipping it under +`(0012,0062) = YES`. + +`RetainSafePrivate` plus a `Profile` used to write private values into de-identified output that the +run never looked inside. `0.0.19` disclosed that (`DICOM_DEIDENT_PRIVATE_CARRIER_NOT_AUDITABLE` plus +a `report.unauditableSequences` entry stamped as kept) and said outright that the disclosure was not +a fix. It is fixed now: PS3.15 2026c §E.3.10 retains Private Attributes "known by the de-identifier +to be safe from identity leakage" and sends "all other Private Attributes" to removal or to the +`(0008,0307)` action this library does not implement, so a value nothing enumerated is removed. + +**The over-redaction is the point rather than a corner case, and this is the size of it.** Three +classes of private value reach the output on something this run enumerated: one the run **walked as +Data Elements** and put through the Annex E action table (a private `SQ` whose items the parser +materialized), a **Private Creator `(gggg,00EE)` whose whole decoded value is a member of your +profile's private dictionary**, and a **zero-length** value. Decoding a value under the VR your +profile declares for it is not enumeration, and neither is the embedded-attribute scanner's silence, +so **an ordinary vendor scalar under an ordinary string VR that the file's own declaration does not +name safe is removed**. If you carried opaque vendor values through `RetainSafePrivate`, they are +gone unless your senders declare them: getting them back otherwise needs the content test that +separates a nested Data Set from a legitimate binary blob, which is an open product question. The +declaration route that keeps the fourth class is the other entry in this release, and what it +retains rests on the sender's assertion rather than on anything this run examined. + +**Three things to act on if you consume the audit surfaces.** + +1. **New report surface.** `report.unenumerablePrivateRemovals` records each removal per instance: + the tag, the Data Set it lived in, `applied: "removed"` and `reason: "unenumerable"`. It is + complete and **never capped** at any input size, which is what lets you separate an unenumerable + removal from an Annex E one and from an emptied value; the matching warnings stay bounded. + `report.removedPrivateTags` names these removals too, unchanged in meaning. +2. **`report.unauditableSequences` no longer produces its retired `kept` outcome** for a retained + private value. Its `applied` field was `"emptied" | "kept"` and is `"emptied"` alone now, so an + entry there means content is not in your output again. A comparison against the retired outcome + stops compiling, which is the intended way to find out. +3. **`DICOM_DEIDENT_PRIVATE_CARRIER_NOT_AUDITABLE` changes meaning**, from "this value was shipped + unexamined" to "this attribute was removed unexamined". **No published warning code is renamed or + retired**, so a consumer narrowing on that code name keeps compiling and must re-read what it now + means rather than re-type it. This work adds no code; the code this release does add, + `DICOM_DEIDENT_PRIVATE_DECLARATION_NOT_RESOLVED`, belongs to the declaration entry beside it. + +The measured matrix in `test/integration/deident-private-reservation.test.ts` flips its twenty +leaking cells from kept-verbatim to removed, on every combination of profile-declared VR and on-wire +VR it enumerates and on the VR-less Implicit VR LE encoding, and gains a mutation control that reds +if the retention decision is reverted. diff --git a/.changeset/coded-method-sequence.md b/.changeset/coded-method-sequence.md new file mode 100644 index 0000000..460826d --- /dev/null +++ b/.changeset/coded-method-sequence.md @@ -0,0 +1,40 @@ +--- +"@cosyte/dicom": patch +--- + +`deidentify()` now writes `(0012,0064)` De-identification Method Code Sequence on every run, beside +the `(0012,0063)` text, so a consumer can read which Profile and Options produced an object from +codes instead of parsing English, and can tell the two PS3.15 §E.3.6 temporal branches apart by +`113106` and `113107`. + +PS3.15 2026c §E.1.1 asks for "one or more codes from CID 7050 "De-identification Method" +corresponding to the Profile and Options used ... and/or a text string". The text already satisfied +the "and/or", so this is the machine-readable half rather than a conformance fix. + +- **What is written.** A top-level `SQ` with an Item for each code from PS3.16 2026d CID 7050 + (context group version `20170914`, UID `1.2.840.10008.6.1.925`): `113100` Basic Application + Confidentiality Profile first, then an Item for each active Option in `DEIDENTIFY_OPTIONS` order, + whatever order `retain` names them in. Each Item carries exactly Code Value, Coding Scheme + Designator `DCM` and Code Meaning. +- **Added to, never replaced.** A prior `(0012,0064)` the source carried keeps every Item, in order + and byte for byte, and this run's Items follow. A code a prior Item already records under the same + designator and value is not added again, so de-identifying an output again with the same options + is a fixed point. +- **New public export** `DEIDENTIFICATION_METHOD_CODES` (with the `DeidentificationMethodCode` type) + carries the CID 7050 rows, the context group UID and version, and the code each Option writes. +- **Two new stable warning codes on `report.warnings`**, never on `Dataset.warnings`: + `DICOM_DEIDENT_METHOD_CODES_PRIOR_RETAINED` when prior Items were kept (neither `(0012,0064)` nor + the code attributes in its Items has a Table E.1-1 row, so what a sender wrote there reaches the + output un-inspected, and no prior code is checked against CID 7050), and + `DICOM_DEIDENT_METHOD_CODES_PRIOR_REPLACED` when a prior `(0012,0064)` was not a Sequence of Items + this run could read and was replaced by this run's codes. Neither message carries the prior's bytes + or VR. `DICOM_DEIDENT_METHOD_PRIOR_RETAINED` keeps its published meaning, `(0012,0063)` only. + +**Limits, stated with the capability.** `113101` Clean Pixel Data Option and `113102` Clean +Recognizable Visual Features Option are never written, because this layer does not touch pixels. +`113107` says the modified-dates column was resolved, not that any date was shifted: the same limit +`(0028,0303) = MODIFIED` carries, and `DICOM_DEIDENT_DATES_NOT_TRANSFORMED` still rides with it. An +Option's code means the Option was active, whether or not the object carried an attribute it acts +on, and a caller's `deidentificationMethod` text never changes the codes. Only the top-level +attribute is written or read. `(0012,0062)`, `(0012,0063)` and `(0028,0303)` are written exactly as +before. diff --git a/.changeset/dicomdir-record-tree.md b/.changeset/dicomdir-record-tree.md new file mode 100644 index 0000000..f018339 --- /dev/null +++ b/.changeset/dicomdir-record-tree.md @@ -0,0 +1,45 @@ +--- +"@cosyte/dicom": patch +--- + +A DICOMDIR's Directory Record tree is now read, and `serializeDicom()` writes every record offset as +the byte offset of the record it named, so a DICOMDIR this library rewrites or de-identifies still +points each record at the right bytes. Until now a DICOMDIR parsed as an ordinary Data Set and every +write carried the source's offsets into a file whose layout had moved (a rebuilt File Meta group, +Data Sets put in ascending order, Items `deidentify()` re-encoded), so a viewer could step from one +patient's record into another's images. + +- **New `Dataset.directory`** (types `DicomDirectory`, `DirectoryRecord`): for an object whose + `(0002,0002)` is `1.2.840.10008.1.3.10`, every Item of the Directory Record Sequence `(0004,1220)` + as a record with its `(0004,1430)` type, its `(0004,1500)` Referenced File ID components (verbatim) + and its lower-level records, plus the root entity `(0004,1200)` names. `undefined` for anything + else. Read under Implicit VR LE, Explicit VR LE and Explicit VR BE. +- **New `Item.fileOffset`**: where the parser found an Item's `(FFFE,E000)` tag, counted from the + first byte of the File Preamble, as PS3.3's Basic Directory IOD counts DICOMDIR offsets (a file read + without a preamble still counts the 132 bytes it lacks). `deidentify()` carries it to the Item it + rebuilds; it is the identity the writer ties each offset to. An offset names a record only when it + equals a Directory Record Sequence Item's `fileOffset`, never an Item tag found by scanning. +- **New Tier-2 codes, each raised only by a non-conformant DICOMDIR, with registry-only messages:** + `DICOM_DIRECTORY_OFFSET_UNRESOLVED` (an offset that is not a record's Item tag: inside a record, on + an Item of a Sequence nested in a record, on the Sequence header, past the end), + `DICOM_DIRECTORY_OFFSET_MALFORMED` (a Value Length other than 4), + `DICOM_DIRECTORY_RECORD_REVISITED` (an offset that would reach a record already reached; it is in + the tree once and the parse finishes) and `DICOM_DIRECTORY_OFFSET_DEFLATED` (a Deflated DICOMDIR + with a non-zero offset, which PS3.10 does not allow, since it requires a DICOMDIR File to use + Explicit VR Little Endian; no offset resolves). No + offset value, record index or record key reaches a message. `{ strict: true }` escalates them. +- **New `SERIALIZE_ERROR_CODES` members `DIRECTORY_OFFSET_UNRESOLVED` and + `DIRECTORY_OFFSET_DEFLATED`.** The writer refuses, and returns no bytes for, a DICOMDIR carrying an + offset it cannot tie to a record the Dataset holds (one that did not resolve on read, one left after + the Directory Record Sequence was removed or emptied, one on a hand-built Dataset with no records, + a value that is not one 32-bit unsigned integer) and a Deflated DICOMDIR with a non-zero offset. It + never writes the stale value or zero. Both messages are fixed strings with no digit in them. +- **`DICOM_DEIDENT_DICOMDIR_FILE_SET_NOT_DISCHARGED` keeps its code and its trigger** (every run on a + DICOMDIR); its message now names only the two File-set clauses of PS3.15 §E.1.1 this run does not + discharge, a DICOMDIR created from the de-identified files and the removal of the non-de-identified + DICOMDIR from the File-set, and no longer says the directory records were not de-identified or that + DICOMDIR is not modelled. +- **Limits, stated beside the capability in `docs-content/limitations.md`:** Referenced File IDs are + kept verbatim; no File-set view; record keys, `(0004,1202)` against the end of the root chain, the + File-set Consistency Flag and Private Record UIDs are not checked; the retired MRDR offset + `(0004,1504)` is written as read. diff --git a/.changeset/dose-calculation-model-name.md b/.changeset/dose-calculation-model-name.md new file mode 100644 index 0000000..1af63ac --- /dev/null +++ b/.changeset/dose-calculation-model-name.md @@ -0,0 +1,18 @@ +--- +"@cosyte/dicom": patch +--- + +The data dictionary and the de-identification action table are now generated from the **PS3.6 and +PS3.15 2026d** DocBook, re-pinned together under `vendor/nema/` by SHA-256. PS3.5 stays at 2026c. + +- **`deidentify()` now removes Dose Calculation Model Name `(3004,007F)` under the Basic Profile + (`X`), and cleans it under `CleanDescriptors` (`C`)**, at the top level and inside Sequence Items + alike, and the report audits it. 2026d adds that attribute to Table E.1-1; against the 2026c table + it had no row, so it was kept verbatim under every Option with a report that said nothing about + it. No other Option changes its treatment. +- **`Dictionary.lookup("3004007F")`** now returns `DoseCalculationModelName`, VR `LO`, VM `1`, so an + Implicit VR Little Endian object reads that element as `LO` rather than `UN`. +- **The report name for `(0018,11BB)` changes case**, from "Acquisition Field Of View Label" to + "Acquisition Field of View Label", as 2026d prints it. Its action codes (`D`, and `C` under + `CleanDescriptors`) do not change. A caller matching `DeidentifiedAttribute.keyword` for that tag + on the old spelling needs the new one. diff --git a/.changeset/encapsulated-serialize.md b/.changeset/encapsulated-serialize.md new file mode 100644 index 0000000..069001b --- /dev/null +++ b/.changeset/encapsulated-serialize.md @@ -0,0 +1,35 @@ +--- +"@cosyte/dicom": patch +--- + +`serializeDicom()` now writes objects under every Transfer Syntax PS3.5 2026c section A.4 names +for encapsulated Pixel Data (JPEG, JPEG-LS, JPEG 2000, HTJ2K, RLE, MPEG, HEVC, JPEG XL, Deflated +Image Frame and Encapsulated Uncompressed), the same generated list `parseDicom()` reads. It used to +refuse each one with `UNSUPPORTED_TRANSFER_SYNTAX`, so a JPEG or JPEG 2000 object run through +`deidentify()` could not be saved. The File Meta Transfer Syntax UID is kept, nothing is +transcoded, and no pixel is decoded. + +- **The Data Set is written as Explicit VR Little Endian**, as section A.4 requires: the bytes after + the File Meta group are the bytes the same Data Set writes under Explicit VR LE. +- **Top-level Pixel Data `(7FE0,0010)` is written as `OB` of undefined length**: the input's Basic + Offset Table Item and every fragment Item byte for byte and in order (Item tag, Item Length and + value), then a Sequence Delimitation Item of length zero. The header is written this way whatever + VR and reserved bytes the input span carried. No offset table is interpreted or rebuilt and no + frame is assembled. +- **New `SERIALIZE_ERROR_CODES` member `INVALID_ENCAPSULATED_PIXEL_DATA`.** Under a section A.4 + syntax the writer refuses, rather than repairs, a top-level Data Set the section does not allow: + no top-level Pixel Data; Float or Double Float Pixel Data `(7FE0,0008)` / `(7FE0,0009)`; Pixel Data + with a defined Value Length; a fragment stream not ended by its Sequence Delimitation Item + (including one `parseDicom()` read with `DICOM_PIXEL_DATA_FRAGMENTS_NOT_DELIMITED`, since a + delimiter appended to a stream that may be short would be a well-formed file missing frames); + anything other than an Item of defined length before the delimiter; an Item of odd length; a + fragment Item of length zero after the Basic Offset Table; no Basic Offset Table Item or no + fragment Item; and bytes after the delimiter. Nothing is returned on a refusal, and the message is a + fixed string that carries no length, byte or UID from the Dataset. +- **Unchanged:** the four native syntaxes serialize exactly as before; every other UID (JPIP + Referenced, SMPTE ST 2110, every retired UID) still throws `UNSUPPORTED_TRANSFER_SYNTAX`, and a + missing one `MISSING_TRANSFER_SYNTAX`. Pixel Data nested in a Sequence Item, such as an Icon Image + Sequence, is written as read and not checked. `parseDicom()` and `deidentify()` are untouched, and + burned-in annotation is still reported by `DICOM_BURNED_IN_ANNOTATION_NOT_REMOVED`, never removed. +- The `UNSUPPORTED_TRANSFER_SYNTAX` message now says the writer also supports every section A.4 + syntax. The code is unchanged. diff --git a/.changeset/encapsulated-transfer-syntaxes.md b/.changeset/encapsulated-transfer-syntaxes.md new file mode 100644 index 0000000..1101214 --- /dev/null +++ b/.changeset/encapsulated-transfer-syntaxes.md @@ -0,0 +1,38 @@ +--- +"@cosyte/dicom": patch +--- + +`parseDicom()` now reads objects under every encapsulated Pixel Data Transfer Syntax that PS3.5 +2026c section A.4 names. + +Those are JPEG Baseline, Extended, Lossless and Lossless First-Order Prediction; RLE Lossless; +JPEG-LS Lossless and Near-Lossless; JPEG 2000 and HTJ2K; MPEG2; MPEG-4 AVC/H.264; HEVC/H.265 Main +and Main 10; JPEG XL; Deflated Image Frame Compression; and Encapsulated Uncompressed Explicit VR +LE. Such an object used to throw `UNSUPPORTED_TRANSFER_SYNTAX` and lose its +metadata with it. Section A.4 makes the whole Data Set Explicit VR Little Endian with only Pixel Data +`(7FE0,0010)` encapsulated, so it is read by the Explicit VR LE reader and its patient, study, series +and image views read exactly as the same Data Set does under Explicit VR LE. The list is read out of +the vendored, SHA-pinned PS3.5 by a new generator that joins `gen:all` and the byte-identical regen +gate, and each UID is checked against the PS3.6 registry as a non-retired Transfer Syntax. + +- **New: `readPixelDataFragments(ds)`** returns a `PixelDataFragments`: the Basic Offset Table's + Item Value and every fragment's Item Value as raw bytes, in file order, with no Item header in any + of them. An empty Basic Offset Table reads as zero bytes. It returns `undefined` when Pixel Data is + absent or native. The buffers follow `copyValues` like `Element.rawBytes`. +- **No pixel is ever decoded.** Nothing is decompressed, no frame is assembled from fragments, and + neither the Basic Offset Table nor an Extended Offset Table is interpreted. +- **`serializeDicom()` refuses every one of these syntaxes** with `UNSUPPORTED_TRANSFER_SYNTAX`, a + de-identified object included, so a compressed object is read and not written. +- **An empty Basic Offset Table no longer raises `DICOM_EMPTY_ITEM_IN_SEQUENCE`.** Section A.4 makes + it the conformant form decoders must accept. A later empty fragment still raises it. A caller that + switched on that code for a conformant encapsulated file under Explicit VR LE no longer sees it. +- **New Tier-2 code `DICOM_PIXEL_DATA_FRAGMENTS_NOT_DELIMITED`** for a fragment stream that ends + with the input before its Sequence Delimitation Item. The metadata still parses and the fragment + list may be short; `{ strict: true }` refuses it. Its message carries no fragment byte, length or + count. +- **Still refused:** the JPIP Referenced syntaxes (their Pixel Data is a reference, not fragments), + the SMPTE ST 2110 syntaxes, and every retired UID, the retired JPEG processes included. The fatal + names the syntax by its PS3.6 registry name, never by the UID, and its message now summarises the + supported set instead of listing four UIDs. +- **Not diagnosed:** an encapsulation-syntax object with no top-level Pixel Data, or with native + top-level Pixel Data, parses without a code, as the same bytes do under Explicit VR LE. diff --git a/.changeset/first-minor-release.md b/.changeset/first-minor-release.md new file mode 100644 index 0000000..30ac79d --- /dev/null +++ b/.changeset/first-minor-release.md @@ -0,0 +1,30 @@ +--- +"@cosyte/dicom": minor +--- + +**This is 0.1.0, the first release whose public API we treat as settled.** + +What is covered, and what you can build against: + +- Reading DICOM Part 10 metadata: the four typed views (`patient`, `study`, `series`, `image`), + access to any element by tag with the Part 6 data dictionary compiled in, and lazy typed decode of + all 34 value representations, honoring Specific Character Set. +- The four native transfer syntaxes and every PS3.5 2026c section A.4 encapsulation syntax are read; + encapsulated pixel data comes back as raw fragments, and a DICOMDIR reads as a record tree. +- Lenient parsing: only four structural conditions are fatal, and every tolerated deviation is a + stable warning code with a byte offset. Vendor profiles (`ge`, `siemens`, `philips`, `strict`, + `lenient`, or your own through `defineProfile`) only ever tighten or annotate a parse. +- A spec-clean serializer that writes the source transfer syntax back, and metadata + de-identification with the PS3.15 Annex E Basic Profile and the metadata-affecting options, + returning a fresh dataset and an audit report. + +What the version promises. The exported names, options, return shapes and warning codes are the +surface we keep stable. While the package is below 1.0, a breaking change bumps the minor version +(0.1 to 0.2) and is called out in this changelog with its migration; a fix that changes no public +value ships as a patch. + +What is not covered yet. Pixel data is never decoded, in any transfer syntax. A de-identified output +is metadata-de-identified only: burned-in annotation is warned about, not removed, and the +residuals the README lists under known limitations are disclosed rather than closed. There is no +transcoding between transfer syntaxes, no DIMSE networking and no DICOMweb, and coded values are +surfaced but not validated against a terminology. diff --git a/.changeset/first-use-examples-executed.md b/.changeset/first-use-examples-executed.md new file mode 100644 index 0000000..156cdeb --- /dev/null +++ b/.changeset/first-use-examples-executed.md @@ -0,0 +1,7 @@ +--- +"@cosyte/dicom": patch +--- + +Docs: the README's first usage example and the quickstart's first example are now executed by the test suite, read straight out of the page they are printed on. + +The README example claims its values as checked results now, against a synthetic CT object saved without its preamble, and one claim moved in the process: the study instance UID it shows is the one that object actually carries, not a placeholder. The quickstart's object is byte-identical to a fixture the test corpus builds, so the PHI scan reads it with the rest of the fixtures. Both examples are also compiled with the settings `tsc --init` writes for a new TypeScript project, so an example that does not compile fails the suite too. A changed value in either example fails the suite, and every install command the README and the installation page print is checked against the package's own name. diff --git a/.changeset/great-blocks-listen.md b/.changeset/great-blocks-listen.md new file mode 100644 index 0000000..efabb8c --- /dev/null +++ b/.changeset/great-blocks-listen.md @@ -0,0 +1,28 @@ +--- +"@cosyte/dicom": patch +--- + +Model CommonMark's HTML blocks in the PHI gate's override log, so a comment cannot exempt a scan +target. + +`scripts/phi-scan.ts` refuses a `--allow-fixture ` bypass unless `phi-scan-overrides.md` +carries a `### ` heading, and its block structure was fenced code blocks and nothing else. A +fenced block shows its contents to a reviewer; an HTML comment shows nothing at all, so a heading +written inside `` was a live allow entry that exempted its target at exit 0 while the +rendered log looked empty. `overrideLogPaths` now models CommonMark 0.31.2 section 4.6's HTML +blocks, start conditions 1 to 6 and their end conditions, and ignores fences and headings inside +one exactly as section 4.6 requires. + +Start condition 7 is deliberately out of scope, and its cost is measured in both directions rather +than assumed: it needs a complete tag AND the knowledge that the line does not interrupt a +paragraph, which is paragraph state this parser does not have. A heading under `` on its own +line is still a live entry here where CommonMark hides it; after a paragraph line, where condition +7 cannot fire, the two agree. Both arms are pinned by tests. + +No direction is claimed. A block boundary is parity, and on a log carrying an odd number of fence +delimiters inside a comment the two readings' entry sets are disjoint and both non-empty, each +exempting at exit 0 a target the other refuses at exit 2. The tag lists section 4.6 closes over are +read out of the vendored spec and driven name by name through the `--allow-fixture` membership +oracle, so the tables are checked against the document rather than against whoever typed them, and +no count of either is written anywhere. `scripts/measure-phi-scan-html-blocks.ts` ships beside the +change so the figures can be re-run against any tree. diff --git a/.changeset/iso2022-code-extensions.md b/.changeset/iso2022-code-extensions.md new file mode 100644 index 0000000..bd03375 --- /dev/null +++ b/.changeset/iso2022-code-extensions.md @@ -0,0 +1,32 @@ +--- +"@cosyte/dicom": patch +--- + +`PN`, `LO`, `SH`, `UC`, `ST`, `LT` and `UT` values now decode ISO 2022 code extensions when +`(0008,0005)` Specific Character Set has more than one Value (and Value 1 is not `ISO_IR 192`, +`GB18030` or `GBK`). Every escape sequence in PS3.3's code-extension tables switches G0 or G1, so a +Japanese, Korean or Chinese Person Name comes back as the characters its sender encoded rather than +with escape bytes inside it or JIS bytes read as ASCII. PS3.5 2026d's worked examples H.3-2, I.2-1 +and K.2-1 used to come back wrong with no warning; all four examples now decode to the names the +standard prints. + +- **Reset points follow PS3.5 section 6.1.2.5.3.** Value 1's designations come back at the start of + the value, after CR, LF or FF, after the `\` of a multi-valued VR and after the `^` and `=` of + `PN`. A delimiter counts only as a single byte, never as half of a two-byte character. +- **`decodeText(bytes, terms)`** returns the same characters for a multi-valued term list, with the + text reset rules (CR, LF and FF). +- **New Tier-2 code `DICOM_CHARSET_ESCAPE_UNDECLARED`**: an escape sequence switches to a set + `(0008,0005)` does not declare. The bytes are decoded in the set it names. +- **New Tier-2 code `DICOM_CHARSET_BYTES_UNDECODABLE`**: bytes no designated set decodes (an + unrecognized escape sequence and what follows it, a GR byte with no G1 set, a code the set does + not define, or a set this runtime cannot decode) read as U+FFFD, never as another set's reading. + `Element.rawBytes` keeps them. +- **New Tier-2 code `DICOM_CHARSET_EXTENSION_NOT_RESET`**: a line, value or `PN` component ends + while G0 still holds a set other than Value 1's. The characters are kept and what follows decodes + from Value 1. +- The three codes ride on `Element.value.warnings`, at most once each per value, and never reach + `Dataset.warnings` or `onWarning`, so `{ strict: true }` does not refuse such a file. Their + messages carry the tag and VR only. +- **Unchanged:** a single-valued `(0008,0005)`, a multi-valued one whose Value 1 is `ISO_IR 192`, + `GB18030` or `GBK`, `resolveDecoderLabel`, `serializeDicom` and `deidentify()`. ISO-IR 14's + `0x5C` and `0x7E` still read as `\` and `~`. diff --git a/.changeset/lemon-badgers-declare.md b/.changeset/lemon-badgers-declare.md new file mode 100644 index 0000000..7cefb4d --- /dev/null +++ b/.changeset/lemon-badgers-declare.md @@ -0,0 +1,58 @@ +--- +"@cosyte/dicom": patch +--- + +Read the file's own safe-private declaration, so a caller with a DICOM file from a vendor they hold +no profile for can de-identify it and still keep the private attributes that file itself declares +non-identifying. + +PS3.15 2026c §E.3.10 lists four ways a Private Attribute may be "known by the de-identifier to be +safe from identity leakage", and names the in-file declaration **first**: "its presence in a block of +Private Data Elements with a Value of \"SAFE\" in Block Identifying Information Status (0008,0303) or +individually listed in Nonidentifying Private Elements (gggg,0004) (within Private Data Element +Characteristics Sequence (0008,0300)". This package resolved safety only through a caller `Profile`, +so `RetainSafePrivate` kept nothing without one and the technique values a quantitative read depends +on - §E.3.10's own examples are CT helical span pitch and PET SUV rescale factors - were removed from +every file whose vendor nobody had written a profile for. + +- **Both branches of that bullet are read.** A block whose `(0008,0303)` is `SAFE` is known safe + whole; a block declared `MIXED` is known safe for exactly the elements its `(0008,0304)` list + names, identified by the lowest 8 bits of the tag within the block (PS3.3 2026c §C.12.1). The + Private Creator that reserves such a block is retained with it, which is what §E.3.10's "together + with the Private Creator IDs that are required to fully define the retained Private Attributes" + asks for. A block declared `UNSAFE` retains nothing, its creator included. +- **No profile is needed, and passing one only ever adds.** A profile retention survives a block the + declaration calls `UNSAFE` or says nothing about, and nothing outside the odd-group private block + an Item describes is retained on account of that Item. +- **A block is keyed by its Private Creator VALUE and never by a block number**, which is what + §C.12.1's own Note requires ("since instances may be modified and numeric block numbers + reassigned"). The block number an element sits in is resolved separately, against the reservations + of the Data Set that element lives in, because PS3.5 2026c §7.8.1 scopes a reservation to one Data + Set and Items do not inherit an enclosing one. +- **A declared-safe private Sequence is still parsed in its entirety**, with each nested attribute + handled on its own merits, exactly as §E.3.10's Sequence paragraph requires. The declaration + decides the carrier and never the Data Sets below it. +- **An Item that does not resolve retains nothing and is disclosed** under a new stable warning code, + `DICOM_DEIDENT_PRIVATE_DECLARATION_NOT_RESOLVED`, raised once per Item on `report.warnings` and + capped per run. An Item is unresolvable when a Type 1 Value is absent or empty, when `(0008,0303)` + carries a Value outside the three §C.12.1 enumerates, when `(0008,0301)` is not an odd group, when + a `MIXED` Item carries no usable `(0008,0304)` list, or when `(0008,0302)` names a creator no block + in that Data Set reserves. The message carries no token of the Item: the creator, the group and an + unenumerated status are all the sender's own values, and there is no string parameter for one to + travel through. + +🩺 **What this retains rests on the SENDER's assertion, and that is the cost rather than a footnote.** +The system that wrote the file said the block carries no identifying information; nothing here +re-derives it, so an opaque vendor value in a block declared `SAFE` reaches de-identified output +unexamined - the class removal closed for the profile route. §E.3.10 offers exactly one mitigation +and it is the caller's: if you do not trust the sender, do not pass `RetainSafePrivate`, and every +Private Attribute is removed ("When this Option is not specified, all Private Attributes shall be +removed"). The Option is off by default and this changes nothing about a run that leaves it off. + +Everything that guarded the profile route still acts first and is untouched: a Data Set whose +reservations the file does not settle, an element outside the settled run, an on-wire VR that is not +one of the 34, and a value the embedded-attribute scanner reads as a swallowed Data Element run. +Deidentification Action (0008,0307), Deidentification Action Sequence (0008,0305) and Identifying +Private Elements (0008,0306) are **not** implemented and are not implemented by this: §E.3.10's +second branch is "removed **or** processed in the element-specific manner recommended by +Deidentification Action (0008,0307)", and removal stays the branch this library takes. diff --git a/.changeset/lucky-tigers-report.md b/.changeset/lucky-tigers-report.md new file mode 100644 index 0000000..7cafdc1 --- /dev/null +++ b/.changeset/lucky-tigers-report.md @@ -0,0 +1,27 @@ +--- +"@cosyte/dicom": patch +--- + +Gate the public surface against internal project bookkeeping, and sweep the tree it would have failed. + +`no-internal-refs.yml` and `scripts/check-no-internal-refs.ts` enforce the founder directive of +2026-07-27 on every surface a consumer of `@cosyte/dicom` reads: the published markdown at the +repository root, `docs-content/`, the npm `description` and `keywords`, and all of `src/`. Work-item +and plan ids, phase and wave language, roadmap and plan citations, ADR numbers, meta-repo paths and +commentary about how the work is being run are findings; `CHANGELOG.md`, `.changeset/`, commit +messages, pull request text, `test/`, `.github/` and this repository's agent-context docs are +excluded, because those are the surfaces the identifiers belong on. + +The gate self-tests before it reports: every rule has to match its own positive sample, a DICOM +reference corpus (`PS3.6`, `(0010,0010)`, `PN`, `UI`, `DICOM-SR`, `1.2.840.10008.1.2.1`, +`ICD-10-CM`, `CP-246`, `MRN-42` and the rest) has to survive every rule untouched, and the scope +function has to classify a fixed list of paths the way the rule says. It refuses to print a result +from a scan that read nothing, from a declared surface that selects no tracked file, or from a file +in scope it could not read or could not decode as UTF-8. There is no list of excused occurrences and +there must not be one. + +The sweep is the other half, and it is why this is a `patch` rather than a chore: 530 findings, 526 +of them in `src/` comment blocks that `tsup` copies verbatim into the published `dist/index.d.ts` and +`dist/index.d.cts`, against 4 in markdown anyone reviews. Comment and documentation TEXT is rewritten +and nothing else: no exported symbol, signature, message string or runtime behaviour changes, and the +build, the type declarations' shape, the lint ladder and all 1,586 tests are unchanged. diff --git a/.changeset/metal-things-greet.md b/.changeset/metal-things-greet.md new file mode 100644 index 0000000..c5e8f0a --- /dev/null +++ b/.changeset/metal-things-greet.md @@ -0,0 +1,7 @@ +--- +"@cosyte/dicom": patch +--- + +Correct the troubleshooting remedy for `DICOM_DEIDENT_PRIVATE_CARRIER_NOT_AUDITABLE`: a value the file's own `(0008,0300)` declaration names safe is retained, not removed. + +The remedy said that, beyond the three cases the package enumerates, every private value a `Profile` vouched for under `RetainSafePrivate` is removed, an ordinary scalar under an ordinary string VR included. That was false whenever the file's own Private Data Element Characteristics Sequence `(0008,0300)` names the value safe, in a block whose status is `SAFE` or listed individually in a `MIXED` one: `deidentify()` retains that value on the sender's word, and `report.unenumerablePrivateRemovals` does not record it. The guide now names that route and keeps the removal statement for everything outside it. No behaviour changes. diff --git a/.changeset/olive-clocks-convert.md b/.changeset/olive-clocks-convert.md new file mode 100644 index 0000000..fcb343c --- /dev/null +++ b/.changeset/olive-clocks-convert.md @@ -0,0 +1,58 @@ +--- +"@cosyte/dicom": minor +--- + +Add `toObject`, `toISO` and `toDate`, one conversion surface over the decoded `DA`, `TM` and `DT` +values, so reading a timestamp out of an object no longer means writing a per-type projection at +every call site. + +`parseDate`, `parseTime` and `parseDateTime` decode three separate wire syntaxes into three separate +shapes. The three new functions take any of them and answer in one shape: the calendar components, +an ISO-8601 string, or an absolute-instant `Date`. `DateParts` and `ToDateOptions` are exported +beside them. Nothing existing moves: the decoders, their `legacy` and `nonstandardOffset` flags and +`DicomDate` / `DicomTime` / `DicomDateTime` are all unchanged, and the package still takes no +dependency of any kind. + +The same three names, with the same meanings, are exported by every `@cosyte/*` parser that decodes +a date, so they collide on import by design. A file that reads two of them aliases +(`import { toISO as dicomToISO } from "@cosyte/dicom"`) or namespace-imports; `README.md` and the +typed-values page both show the pattern. + +Four decisions in the surface are worth knowing before you depend on it. + +- **The key set is the precision.** A component the value did not state is absent from `DateParts` + rather than present and `undefined`, and nothing is zero-filled, so `Object.keys()` recovers what + the sender wrote. There is no `precision` key because the key set is one, and no `raw` or `valid` + key because parse bookkeeping is not a calendar component. `toISO` truncates the same way: a `DT` + that stated only an hour renders `2024-01-15T13`, never a padded-out `T13:00:00`. +- **The keys are singular where the decoded types are plural.** `DicomTime` and `DicomDateTime` + spell the time fields `hours` / `minutes` / `seconds`; `DateParts` spells them `hour` / `minute` / + `second`, and `month` is 1 to 12 rather than the JS `Date` 0 to 11. That is the shape + `Temporal.PlainDateTime.from` and luxon's `DateTime.fromObject` accept, so deleting + `offsetMinutes` leaves an object either constructor takes with no key rename. +- **`toDate` never guesses a zone.** A stated `&ZZXX` offset converts exactly and beats any + `assumeOffsetMinutes` the caller passes. With no stated offset, the caller's assumption is the + only route to an instant, an explicit `0` meaning "read this naive value as UTC". With neither, + the answer is `undefined`: the host machine's zone is never read and UTC is never assumed. A + non-finite `assumeOffsetMinutes` names no zone either, so it answers `undefined` rather than an + `Invalid Date`, and so does a finite one so large that applying it leaves the range a `Date` + represents. A `TM` states no year, so it is never an instant at all. +- **An impossible date converts to nothing, never to the day after it.** The decoders range-check + each component on its own, so a `DA` of `18700230` decodes with `valid: true` and `day: 30`. The + conversion surface reads them together and refuses the whole value: a month outside 1 to 12, a day + outside the one its month really has (full 4/100/400 leap rule), an out-of-range hour, minute or + second, or a component that is not a whole number. `second: 60` is refused with them, even though + `TM` and `DT` permit it for a leap second, because there is no ISO string for it a reader does not + move and no instant to build. The refusal is in the projection only: `parseDate`, `parseTime` and + `parseDateTime` are unchanged and still report what the sender wrote. + +`millisecond` is derived from the digits in `raw`, taken verbatim and right-padded (`"5"` is 500, +`"0500"` is 50, `"123456"` is 123), and never from `fractionalSeconds`, which is a binary float. +`toISO` renders those digits exactly as written, neither padded to three nor rounded. A stated zero +offset renders `Z`, so `toISO` is deliberately not a byte round-trip of the wire value; +`serializeDicom` remains the route that reproduces the original bytes. + +The retired dotted `YYYY.MM.DD` date converts identically to the canonical form, and the `legacy` +flag the decoder reports beside the value never reaches a result. All three functions answer +`undefined` for a value the decoders marked `valid: false`, for `null` and `undefined`, and for a +value that stated no component at all, and none of them throws for any input. diff --git a/.changeset/olive-dates-resolve.md b/.changeset/olive-dates-resolve.md new file mode 100644 index 0000000..ebdeb76 --- /dev/null +++ b/.changeset/olive-dates-resolve.md @@ -0,0 +1,40 @@ +--- +"@cosyte/dicom": patch +--- + +Add `RetainLongitudinalTemporalModifiedDates`, the second PS3.15 §E.3.6 Retain Longitudinal Temporal +Information Option, so a caller running a longitudinal study can keep the temporal relationships +their analysis needs without keeping real dates. + +§E.3.6 defines **two** mutually exclusive Options and Table E.1-1 gives them separate columns. This +package published one name, carrying the full-dates column, so the modified-dates column was +unreachable and `(0028,0303) Longitudinal Temporal Information Modified` could never say `MODIFIED`. + +- **`RetainLongitudinalTemporalModifiedDates`** resolves Table E.1-1's `Rtn. Long. Modif. Dates Opt.` + column, which is the more protective of the two on every attribute where they differ: full dates + keeps the real value, modified dates cleans it. An attribute the modified-dates column publishes no + action for takes the Basic Profile action, never the full-dates one and never a keep. +- **`(0028,0303)` takes the value `MODIFIED`** on that branch, replacing whatever the source carried + there, exactly as `REMOVED` and `UNMODIFIED` do. +- **A Value naming the Option is added to `(0012,0063)` De-identification Method**, inside `LO`'s + 64-character per-Value maximum like every other name this library writes there. +- **The two temporal Options are mutually exclusive.** A call naming both is rejected with a + `DeidentifyError` carrying the existing `INVALID_OPTIONS` code. Either one alone is accepted, and + `RetainLongitudinalTemporal` is unchanged in both respects: it still resolves the full-dates column + and still writes `UNMODIFIED`. + +**This library performs no date transformation, and says so at run time.** §E.3.6 has two halves: the +Table E.1-1 column, and a requirement that the dates themselves be modified with the manner of +modification described in a Conformance Statement. A library can deliver the first and neither of the +second, because PS3.2 Annex N scopes a Conformance Statement to a named product and version. So +**you** perform the transformation, and a `MODIFIED` on an object whose dates nobody shifted is a +defect this package cannot detect. Every run under the new Option raises the new stable warning code +`DICOM_DEIDENT_DATES_NOT_TRANSFORMED` on `report.warnings`, stating the half of §E.3.6 the run did +not discharge. Nothing here claims PS3.15 Annex E conformance for the Retain Longitudinal Temporal +Information With Modified Dates Option, which §E.1.1 makes all-or-nothing. + +`DICOM_DEIDENT_DATES_NOT_TRANSFORMED` is added to `WARNING_CODES` and is raised by `deidentify()` +only: it is a statement about the run's options rather than about the file, so it never reaches +`Dataset.warnings` and a `{ strict: true }` parse of the same input is unaffected. No published code +is renamed or retired, no coded Item is added to `(0012,0064)`, and what the Basic Profile retains +with neither temporal Option active is unchanged. diff --git a/.changeset/olive-fences-return.md b/.changeset/olive-fences-return.md new file mode 100644 index 0000000..84b8e59 --- /dev/null +++ b/.changeset/olive-fences-return.md @@ -0,0 +1,32 @@ +--- +"@cosyte/dicom": patch +--- + +Claim the PHI gate's `CRLF` line split, and delete the disclosure that said it could not be claimed. + +`#113` shipped `scripts/phi-scan.ts`'s `splitLines` with a disclosure that its `CRLF` half was +unobservable through either caller and claimed by no test, because `loadAllowList` trims and +`tripleHashValue` trims. Both premises are true and the conclusion is false: `overrideLogPaths` +hands the RAW line to `fenceRun` before anything trims it, and a closing fence run is `bare` only +when it is followed by spaces or tabs (CommonMark 0.31.2 section 4.5). On an override log written +with `CRLF`, a `CR`-blind split therefore leaves a `CR` after the closing run, the run reads as an +info string rather than as a close, the block never ends, and every entry below it is dropped. + +Measured on one log carrying a fenced template and two live entries below it: the shipped script +exits 0 with both entries honoured, a `CR`-blind mutant exits 2 having dropped both, and with the +same log written `LF` the two agree. Both `--allow-fixture` directions are asked on each arm, so a +parser that made everything below an entry fails it too. + +No direction is claimed for that mutant. A draft called it fail-closed, on the grounds that a `CR` +can only prevent a close and never cause one; the gate falsified that with an input, and it is +deleted rather than narrowed. On a log whose lines are `CRLF` except one opening fence the two +entry sets are disjoint rather than nested, and the mutant exempts at exit 0 a target the shipped +script refuses at exit 2. Fence state is parity, so a wrong answer moves entries in both directions. + +The behaviour was already correct and is unchanged: comment-stripped, `scripts/phi-scan.ts` is +byte-identical on base and head. What changed is that four carriers of the false disclosure inside +this package are corrected. + +The mutation figure that stood here is DELETED rather than restated: a later change in this same +release moved `overrideLogPaths` onto its own CommonMark splitter, so the `CR`-blind mutant of +`splitLines` no longer reds anything, and both changesets land in one generated `CHANGELOG.md`. diff --git a/.changeset/olive-headings-listen.md b/.changeset/olive-headings-listen.md new file mode 100644 index 0000000..0242711 --- /dev/null +++ b/.changeset/olive-headings-listen.md @@ -0,0 +1,31 @@ +--- +"@cosyte/dicom": patch +--- + +Refuse a `###` run an invisible character separates from the path, in the PHI gate's override log. + +`scripts/phi-scan.ts` refuses a `--allow-fixture ` bypass unless `phi-scan-overrides.md` +carries a `### ` heading. `tripleHashValue` separated the `###` run from the path with the +whole of `\s`, where CommonMark 0.31.2 section 4.2 says the opening run "must be followed by spaces +or tabs, or by the end of line". So a line whose separator was an invisible character was a live +allow entry where the document renders a paragraph, and that target was exempted at exit 0. Measured +live on three trees before the change, each against the same bytes without the flag, which exit 1. + +It is one conjunct on the pattern the function replaced, so for every line the parser returns either +that pattern's answer or nothing: no line that was not already an entry becomes one, and every +difference is a refusal at exit 2. That is a property of the source rather than one a test can +prove, and it is left as one: the evidence beside it is the comment-stripped diff, a single added +early return, and the per-log relation the shipped instrument prints against a base, which reads as +a subset on the one log the two trees differ on. + +Section 4.2's strip is deliberately NOT taken, and that is measured rather than assumed: the prose +says leading and trailing spaces or tabs, and `commonmark@0.31.2`, the reference implementation of +the pinned document version, strips with `String.prototype.trim`, which is the whole of `\s` and is +what this parser does. A draft that took the prose named `path` where this parser names +`path`, which exempted at exit 0 a target it refuses at exit 2, and it was deleted. The strip, +section 4.2's optional closing sequence and its inline parsing are each pinned as they are, with +their cost stated in both directions. + +`scripts/measure-phi-scan-atx-heading.ts` ships beside the change so the figures can be re-run +against any tree. It carries a control the sibling instruments do not: an entry must be shown to +exempt a real hit, not merely to be accepted. diff --git a/.changeset/olive-otters-describe.md b/.changeset/olive-otters-describe.md new file mode 100644 index 0000000..fe3ddbb --- /dev/null +++ b/.changeset/olive-otters-describe.md @@ -0,0 +1,49 @@ +--- +"@cosyte/dicom": minor +--- + +Stop the de-identified header naming the sending site: replace the File Meta group and remove group +0004. + +Two of PS3.15 2026c §E.1.1's unconditional bullets are about the parts of a DICOM File that are not +the Data Set, and neither was discharged. A file this library de-identified still carried the sending +site's `(0002,0016)` Source Application Entity Title, the source vendor's implementation identity, +every non-modeled `(0002,xxxx)` element the source stuffed into the group - Sending and Receiving AE +Title, Private Information Creator UID, Private Information - and every `(0004,xxxx)` Data Element, +all under a report that said the run succeeded. `rebuildFileMeta` changed exactly one field, the +Media Storage SOP Instance UID, and nothing anywhere filtered on group 0004. + +**The File Meta group of de-identified output now describes this de-identifying application.** +§E.1.1: the File Meta Information "shall be replaced with a description of the de-identifying +application", because otherwise "identity information may leak through unmodified File Meta +Information or preamble ... includ[ing] information regarding Application Entity Titles, Presentation +Addresses, implementation information, and private information". So `(0002,0016)` is gone, +`(0002,0012)`/`(0002,0013)` name `@cosyte/dicom`, and every non-modeled `(0002,xxxx)` element is +dropped rather than re-emitted. What survives identifies the **object**: File Meta Information +Version, Media Storage SOP Class UID, Transfer Syntax UID, and the Media Storage SOP Instance UID +with its existing `RetainUIDs` behaviour unchanged. The replacement is a construction rather than an +edit, so it is unconditional on how well the source group parsed and on which Options are active. + +**Group 0004 is removed at every depth**, as §E.1.1 requires "from any SOP Instance or DICOM File +other than a DICOMDIR File" - including inside Sequence Items, and including a `(0004,xxxx)` Sequence +with everything in it. No Annex E Option qualifies either rule. + +**Two things to act on.** + +1. **The byte-for-byte File Meta round trip no longer holds for `deidentify()` output**, and that is + the point rather than a side effect. It still holds for parse-then-serialize, which is guarded by + its own test. If you relied on a non-modeled `(0002,xxxx)` element surviving a de-identify call, + read it off the parsed dataset before the call; it is not recoverable from the output. +2. **New report surface and three new warning codes.** `report.fileMetaElementsDropped` and + `report.group0004Removals` record what went, each capped per run in the style of + `MAX_UNAUDITABLE_SEQUENCE_FINDINGS`, each beside an uncapped + `...Count` so a caller can count the loss at any input size and tell the two rules apart. + `DICOM_DEIDENT_FILE_META_REPLACED` and `DICOM_DEIDENT_GROUP_0004_REMOVED` are raised once per run, + not once per element, so neither string is multiplied by an element count the input chooses. + +**A DICOMDIR keeps its `(0004,xxxx)` elements and is told what was not discharged.** An object whose +`(0002,0002)` is `1.2.840.10008.1.3.10` takes §E.1.1's carve-out. The rest of that bullet needs a +DICOMDIR model this library does not have, so every such run raises +`DICOM_DEIDENT_DICOMDIR_FILE_SET_NOT_DISCHARGED` - including one where the object carried no +`(0004,xxxx)` element at all, because what was not discharged has nothing to do with what the object +happened to carry. Do not treat that output as a conformant de-identified DICOMDIR. diff --git a/.changeset/olive-scanners-withhold.md b/.changeset/olive-scanners-withhold.md new file mode 100644 index 0000000..564ad4c --- /dev/null +++ b/.changeset/olive-scanners-withhold.md @@ -0,0 +1,34 @@ +--- +"@cosyte/dicom": patch +--- + +Withhold the PHI gate's verdict over a target it enumerated and never opened, and put the install +hardening the estate baseline asks for actually in force. + +`scripts/phi-scan.ts` filters an `--allow-fixture` target out of the run after enumerating it, and +until now the run then reported on whatever was left as if that were the whole corpus. The shape +that costs the most is not the obvious one: withdraw the ONLY violator and every surviving target is +clean, so the same argv that means "excuse this one fixture" produced a run indistinguishable from a +clean sweep. The scanner now tracks the paths a run declared against the paths it read, and a +non-empty difference exits **3** with a `WITHHELD:` line naming each path, after printing the hit +detail for everything it did read. + +3 rather than 1 or 2, and each is a different sentence. 1 says "PHI is here", and overloading it +would make a bypass look like a finding and a finding look like a bypass. 2 says "the scan never +ran" and is what an UNLOGGED `--allow-fixture` already returns, having opened nothing; a withheld +run opened every other target and its hit lines are real. The rule is computed off the two sets +rather than off the flag, so the boundary is "enumerated and not read": a bypass naming a path this +run never enumerated withdraws nothing and still exits 0. + +Every path with no bypass on it is byte-for-byte unchanged, which is the property the pre-commit +hook and CI depend on: `pnpm measure:phi-scan-unread` against the pre-change scanner reports +`violations: 0` over 195 cells, identical exit codes and an identical hit-line multiset. The new +suite coverage carries a mutation control that removes the rule from a planted copy and asserts the +graded run falls back to the hits code, and refuses as vacuous if the line it removes is ever +reworded. + +`package.json` gains the `js-yaml@>=4.0.0 <4.3.0` override the shared baseline requires, ordered so +the stronger `<4.3.1` pin this repo already carried is the one that applies; no resolved version +moves. `pnpm-workspace.yaml` declares `minimumReleaseAge: 1440` and `trustPolicy: no-downgrade`, and +`packageManager` moves to `pnpm@10.34.5` because a pnpm below `10.21.0` ignores those keys entirely, +which is a settings file that decorates rather than defends. diff --git a/.changeset/ordered-data-sets.md b/.changeset/ordered-data-sets.md new file mode 100644 index 0000000..974b917 --- /dev/null +++ b/.changeset/ordered-data-sets.md @@ -0,0 +1,40 @@ +--- +"@cosyte/dicom": patch +--- + +`serializeDicom()` now writes every Data Set in ascending tag order, so a file this library writes +satisfies the element-ordering rule a strict receiver checks even when the file it was read from +did not. It used to write elements in the order the `Dataset` held them (parse order, plus anything +your code or `deidentify()` added afterwards), and to copy every Sequence through as read, so a +round trip propagated an out-of-order source instead of normalizing it. + +PS3.5 2026c §7.1: "The Data Elements in a Data Set shall be ordered by increasing Data Element Tag +Number", and §7.5.1 says the same of the Data Set inside every Item. + +- **What is ordered.** The root Data Set, and the Data Set of every Item of every Sequence the + writer can walk on the wire, at every depth up to `NESTING_DEPTH_LIMIT`, under all four supported + transfer syntaxes. That includes the elements `deidentify()` inserts, such as `(0012,0062)` and + `(0012,0063)`. +- **What does not change.** Items keep their order (§7.5). Only whole elements move, so no value + byte changes and every Item and Sequence length stays valid. The Sequence is walked from its own + bytes, never re-encoded from the parsed `items`, so nothing its bytes did not carry is written, + and it is re-ordered only where those `items` match its bytes, so the output reads back as the + source did. The input `Dataset` is not modified, and `Dataset.elements()` still returns parse + order. +- **What is still not ordered.** A tag repeated inside an Item is kept twice, in source order, so + that output still breaks the "at most once" of PS3.5 2026c §7.1. A Sequence whose Item stream + cannot be walked to exactly its end, one nested past the bound, one whose parsed `items` do not + match its bytes (a Sequence the parser did not descend, `DICOM_SQ_NOT_DESCENDED`, for one), and any + `UN`-carried Sequence (under Implicit VR LE that includes a private Sequence inside an Item, even + one a `Profile` resolved to `SQ`, since a default read resolves its tag to `UN`) are written as + read, unordered. An element the parser relocated because a length field lied is ordered where the + parser placed it. +- **What is written last, whatever its tag.** An element whose own bytes do not show where a reader + ends it: an undefined-length `UN` the parser could not read as a Sequence, or a Sequence, `UN` or + Pixel Data value missing its Sequence Delimitation Item. A reader takes whatever follows such a + value into it, so it stays after the ascending rest of its Data Set, at the root and inside an + Item. A Sequence nested past the bound also goes after the ascending rest of its Item, unless it + is a defined-length Sequence under Implicit VR LE, since seeing where a reader ends any other + would take the walk past the bound. +- **A source that was already in order** is written back with the same bytes after the File Meta + group, as before. diff --git a/.changeset/quiet-lines-agree.md b/.changeset/quiet-lines-agree.md new file mode 100644 index 0000000..ccf6684 --- /dev/null +++ b/.changeset/quiet-lines-agree.md @@ -0,0 +1,14 @@ +--- +"@cosyte/dicom": patch +--- + +Parse the PHI gate's override log with CommonMark's line ending, and pin the spec that says so. + +A lone `CR` in `phi-scan-overrides.md` hid a fence OPENER from the gate's line split, so a +`### ` a human sees inside a rendered code block was a live allow entry and `--allow-fixture` +exempted that PHI scan target at exit 0. `overrideLogPaths` now splits per CommonMark 0.31.2 section +2.1; the allow list keeps `/\r?\n/` deliberately, because `scripts/phi-allow-list.txt` is not a +markdown document. No direction is claimed for either split, on the override log or on the allow +list: each has a measured input whose two entry sets are disjoint, so neither is the conservative +one. The spec is vendored under `vendor/commonmark/` and re-hashed +as a precondition, so the section numbers the gate cites are derived rather than asserted. diff --git a/.changeset/shaggy-pumas-invent.md b/.changeset/shaggy-pumas-invent.md new file mode 100644 index 0000000..f369fe6 --- /dev/null +++ b/.changeset/shaggy-pumas-invent.md @@ -0,0 +1,38 @@ +--- +"@cosyte/dicom": minor +--- + +Disclose a retained private value this run kept without enumerating it, instead of stamping +`Patient Identity Removed = YES` in silence over it. + +`RetainSafePrivate` plus a `Profile` is the route this entry is about, and this release adds a +second one beside it: the file's own Private Data Element Characteristics Sequence `(0008,0300)`, +covered by the declaration entry. PS3.15 2026c §E.3.10 licenses that retention for a Private +_Attribute_ +"known by the de-identifier to be safe"; it says nothing about a Data Set the sender nested inside +that attribute's value, which PS3.5 §7.5.1 makes Data Elements and PS3.15 §E.1.1 still covers +"whether contained in the top level Data Set or embedded in an Item of a Sequence of Items". Where +the run neither walked that value nor emptied it, the obligation went undischarged and nothing said +so: on a **fully conformant** file, on 20 of the 30 cells of the matrix in +`test/integration/deident-private-reservation.test.ts`, a nested `(0010,0010)` reached the output +byte-verbatim with `ds.warnings` empty and `report.unauditableSequences` empty. Every one of those +cells now raises `DICOM_DEIDENT_PRIVATE_CARRIER_NOT_AUDITABLE` on `report.warnings` and adds a +`report.unauditableSequences` entry. + +**The value is still kept, and that is a product decision rather than an unfinished one.** Emptying +these carriers needs a content test on exactly the VRs arbitrary bytes are for, so it also empties +conformant binary values on legitimate files; the leak stays a documented limit, and what changed is +that it is a disclosed one. Nothing about how any file is read or written changes. + +`UnauditableSequenceFinding` gains `applied: "emptied" | "kept"`, and it is the field to read first: +that array had one meaning for its whole life - this content is _not_ in your output - and the new +class means the opposite. The two are budgeted on separate counters against the same cap, so a +crafted file carrying tens of thousands of retained private attributes cannot spend the budget that +reports dropped content and silence it. The new message names no byte count, because +`Element.rawBytes.length` is the Value Length off a header an under-declared length upstream may +have composed out of somebody's value - the bound `#91` took out of the two sibling factories - and +the private tag it does name renders `` through `renderTag`'s membership test. + +A Private Creator is deliberately not reported: it is retained only when its whole decoded value is +a member of the profile's private dictionary, which is an enumeration, and one carrying anything +else fails that lookup and is removed. diff --git a/.changeset/spicy-regions-rescue.md b/.changeset/spicy-regions-rescue.md new file mode 100644 index 0000000..1d23817 --- /dev/null +++ b/.changeset/spicy-regions-rescue.md @@ -0,0 +1,31 @@ +--- +"@cosyte/dicom": patch +--- + +`parseDicom()` now reads objects under the four JPIP Referenced Transfer Syntaxes of PS3.5 2026c: +JPIP Referenced `1.2.840.10008.1.2.4.94` (section A.6), JPIP Referenced Deflate `…1.2.4.95` +(section A.7), JPIP HTJ2K Referenced `…1.2.4.204` (section A.11) and JPIP HTJ2K Referenced Deflate +`…1.2.4.205` (section A.12). Such an object used to throw `UNSUPPORTED_TRANSFER_SYNTAX` and lose its +metadata with it. Sections A.6 and A.11 make the whole Data Set Explicit VR Little Endian, so it is +read by the Explicit VR LE reader; sections A.7 and A.12 are that Data Set compressed per RFC 1951, +so it is inflated by the Deflated reader first, under the same decompression cap and inflate fatals +as Deflated Explicit VR LE. Its patient, study, series and image views read exactly as the same Data +Set does under Explicit VR LE. + +- **The Pixel Data Provider URL is surfaced and never fetched.** A JPIP object carries no Pixel + Data; `ds.get("00287FE0")` returns its Pixel Data Provider URL as the `UR` element the file + carries, its `value` the URL text with only its trailing padding removed. Nothing in this package + fetches, resolves or validates it. +- **`serializeDicom()` refuses all four** with `UNSUPPORTED_TRANSFER_SYNTAX`: a JPIP object is read, + not written. +- **New `DEIDENTIFY_ERROR_CODES.UNSUPPORTED_TRANSFER_SYNTAX`.** `deidentify()` throws a + `DeidentifyError` with this code, and a fixed message, for a `Dataset` whose File Meta Transfer + Syntax UID is any of the four, whatever the options, and returns no dataset or report. The URL + has no PS3.15 Table E.1-1 row, so a de-identified copy would otherwise keep it by omission. + `DeidentifyError` is no longer thrown only for author-time misconfiguration: switch on `code`. + The refusal keys on the File Meta alone, so a non-JPIP object that carries `(0028,7FE0)` is + de-identified as before and keeps it. +- **Not diagnosed:** a JPIP object with no `(0028,7FE0)`, with a top-level Pixel Data, or with a + Photometric Interpretation outside the four Values section A.6 allows parses without a code. +- **Still refused:** the SMPTE ST 2110 syntaxes and every retired UID. The + `UNSUPPORTED_TRANSFER_SYNTAX` fatal's message now names the JPIP sections among the supported set. diff --git a/.changeset/tidy-moons-declare.md b/.changeset/tidy-moons-declare.md new file mode 100644 index 0000000..6658ec5 --- /dev/null +++ b/.changeset/tidy-moons-declare.md @@ -0,0 +1,38 @@ +--- +"@cosyte/dicom": minor +--- + +Write `(0028,0303) Longitudinal Temporal Information Modified` on every `deidentify()` run, so a +de-identified object says what happened to its dates instead of leaving a recipient to guess. + +PS3.15 2026c §E.2 requires the attribute with a Value of `REMOVED` "if none of the Retain +Longitudinal Temporal Information Options is applied", and §E.3.6 requires `UNMODIFIED` under the +Full Dates Option. This package wrote it in neither state through `0.0.19`, so a receiver reading +dates could not tell real ones from scrubbed ones - and guessing wrong hurts in both directions: +treating real dates as scrubbed under-protects the patient, and treating scrubbed dates as real +corrupts a longitudinal analysis. Both states are written now. + +- **`REMOVED`** on a run with no Retain Longitudinal Temporal Information Option active, which is the + default. +- **`UNMODIFIED`** on a run with `RetainLongitudinalTemporal` active, which is this package's name + for §E.3.6's Full Dates Option. + +**It is REPLACED rather than added to.** The attribute is `VM 1` and both clauses say the value +"shall be added to the Data Set with a Value of" one named state, so a `(0028,0303)` the source file +already carried is discarded - including one under another VR or holding a value the standard does +not define. That is the opposite of what `(0012,0063)` does with a prior method text, and the +asymmetry is the standard's: joining two temporal states into one single-valued attribute would leave +a recipient reading a state no run produced. Re-de-identifying an object always leaves exactly one +`(0028,0303)`, holding the latest run's state. + +**The third state, `MODIFIED`, arrives in the same release**, under §E.3.6's With Modified Dates +Option. See the entry for `RetainLongitudinalTemporalModifiedDates`, which adds that Option's Table +E.1-1 column, the option name that selects it, and the disclosure that this library resolves the +column and performs no date transformation. + +Nothing else moves. `(0012,0062)` and `(0012,0063)` are unchanged, the published warning-code set is +unchanged, no warning code is added or retired, and `DeidentifyReport` keeps its shape - like +`(0012,0062)`, this is a statement the object makes about itself and it is not reported there. The +declaration is written into the **top-level** Data Set only, which is where §E.2 and §E.3.6 put it; +`(0028,0303)` has no row in Table E.1-1, so a copy nested in a Sequence Item is retained by omission +like every other unlisted attribute and still says whatever the sender wrote. diff --git a/.changeset/unregistered-standard-elements.md b/.changeset/unregistered-standard-elements.md new file mode 100644 index 0000000..7905c24 --- /dev/null +++ b/.changeset/unregistered-standard-elements.md @@ -0,0 +1,29 @@ +--- +"@cosyte/dicom": patch +--- + +**`deidentify()` now removes a non-private attribute that neither this build's PS3.6 2026d data +dictionary nor PS3.15 2026d Table E.1-1 carries**, at the top level and inside every Sequence Item +it walks, whatever its VR. Until now a Table E.1-1 miss meant "keep", so a Standard Attribute from a +later edition, or one a sender invented, went into de-identified output verbatim with a report that +said nothing about it. The notes to Table E.1-1 name "new Standard Attributes" as a place identifying +information may be. + +- **"Registered" means a literal PS3.6 row or a masked row the tag matches.** The `(50xx,....)` and + `(60xx,....)` families count only across the even groups PS3.5 2026c section 7.6 bounds them to, + so `(6002,0010)` Overlay Rows is kept and `(6020,0010)` is removed; every other masked row, such as + `(0028,04x0)`, is read as PS3.6 prints it. A registered attribute Table E.1-1 does not list is kept + exactly as before, `UN` VR included. +- **Private attributes, `(0004,xxxx)`, `(0002,xxxx)`, group lengths `(gggg,0000)` and every listed + attribute are unchanged.** An unregistered element whose on-wire VR is outside the 34 PS3.5 + section 6.2 defines is now removed rather than emptied, and is no longer listed on + `report.undefinedVrElements`; a registered one still is. An unregistered Sequence is removed whole + and nothing inside it is walked or reported. +- **New report fields:** `report.unregisteredElementRemovals`, a list of the new exported + `UnregisteredElementRemoval` (a byte offset, plus the context path when nested, and deliberately + no tag or VR, since such a tag may be four bytes of some element's value), capped per run, and + `report.unregisteredElementRemovalCount`, the complete total. +- **New warning code:** `DICOM_DEIDENT_UNREGISTERED_ELEMENT_REMOVED`, raised once per run that + removed anything, on `report.warnings` only, with no tag, VR or count in its message. +- **The cost:** a conformant attribute from a PS3.6 edition newer than this build is removed too, and + there is no option to keep it in this release. diff --git a/CHANGELOG.md b/CHANGELOG.md index 3a1e399..accb3d0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,764 +1,5 @@ # Changelog -## 0.1.0 - -### Minor Changes - -- 372bacd: Remove a retained private value this run did not enumerate, instead of shipping it under - `(0012,0062) = YES`. - - `RetainSafePrivate` plus a `Profile` used to write private values into de-identified output that the - run never looked inside. `0.0.19` disclosed that (`DICOM_DEIDENT_PRIVATE_CARRIER_NOT_AUDITABLE` plus - a `report.unauditableSequences` entry stamped as kept) and said outright that the disclosure was not - a fix. It is fixed now: PS3.15 2026c §E.3.10 retains Private Attributes "known by the de-identifier - to be safe from identity leakage" and sends "all other Private Attributes" to removal or to the - `(0008,0307)` action this library does not implement, so a value nothing enumerated is removed. - - **The over-redaction is the point rather than a corner case, and this is the size of it.** Three - classes of private value reach the output on something this run enumerated: one the run **walked as - Data Elements** and put through the Annex E action table (a private `SQ` whose items the parser - materialized), a **Private Creator `(gggg,00EE)` whose whole decoded value is a member of your - profile's private dictionary**, and a **zero-length** value. Decoding a value under the VR your - profile declares for it is not enumeration, and neither is the embedded-attribute scanner's silence, - so **an ordinary vendor scalar under an ordinary string VR that the file's own declaration does not - name safe is removed**. If you carried opaque vendor values through `RetainSafePrivate`, they are - gone unless your senders declare them: getting them back otherwise needs the content test that - separates a nested Data Set from a legitimate binary blob, which is an open product question. The - declaration route that keeps the fourth class is the other entry in this release, and what it - retains rests on the sender's assertion rather than on anything this run examined. - - **Three things to act on if you consume the audit surfaces.** - 1. **New report surface.** `report.unenumerablePrivateRemovals` records each removal per instance: - the tag, the Data Set it lived in, `applied: "removed"` and `reason: "unenumerable"`. It is - complete and **never capped** at any input size, which is what lets you separate an unenumerable - removal from an Annex E one and from an emptied value; the matching warnings stay bounded. - `report.removedPrivateTags` names these removals too, unchanged in meaning. - 2. **`report.unauditableSequences` no longer produces its retired `kept` outcome** for a retained - private value. Its `applied` field was `"emptied" | "kept"` and is `"emptied"` alone now, so an - entry there means content is not in your output again. A comparison against the retired outcome - stops compiling, which is the intended way to find out. - 3. **`DICOM_DEIDENT_PRIVATE_CARRIER_NOT_AUDITABLE` changes meaning**, from "this value was shipped - unexamined" to "this attribute was removed unexamined". **No published warning code is renamed or - retired**, so a consumer narrowing on that code name keeps compiling and must re-read what it now - means rather than re-type it. This work adds no code; the code this release does add, - `DICOM_DEIDENT_PRIVATE_DECLARATION_NOT_RESOLVED`, belongs to the declaration entry beside it. - - The measured matrix in `test/integration/deident-private-reservation.test.ts` flips its twenty - leaking cells from kept-verbatim to removed, on every combination of profile-declared VR and on-wire - VR it enumerates and on the VR-less Implicit VR LE encoding, and gains a mutation control that reds - if the retention decision is reverted. - -- bb9a7f0: **This is 0.1.0, the first release whose public API we treat as settled.** - - What is covered, and what you can build against: - - Reading DICOM Part 10 metadata: the four typed views (`patient`, `study`, `series`, `image`), - access to any element by tag with the Part 6 data dictionary compiled in, and lazy typed decode of - all 34 value representations, honoring Specific Character Set. - - The four native transfer syntaxes and every PS3.5 2026c section A.4 encapsulation syntax are read; - encapsulated pixel data comes back as raw fragments, and a DICOMDIR reads as a record tree. - - Lenient parsing: only four structural conditions are fatal, and every tolerated deviation is a - stable warning code with a byte offset. Vendor profiles (`ge`, `siemens`, `philips`, `strict`, - `lenient`, or your own through `defineProfile`) only ever tighten or annotate a parse. - - A spec-clean serializer that writes the source transfer syntax back, and metadata - de-identification with the PS3.15 Annex E Basic Profile and the metadata-affecting options, - returning a fresh dataset and an audit report. - - What the version promises. The exported names, options, return shapes and warning codes are the - surface we keep stable. While the package is below 1.0, a breaking change bumps the minor version - (0.1 to 0.2) and is called out in this changelog with its migration; a fix that changes no public - value ships as a patch. - - What is not covered yet. Pixel data is never decoded, in any transfer syntax. A de-identified output - is metadata-de-identified only: burned-in annotation is warned about, not removed, and the - residuals the README lists under known limitations are disclosed rather than closed. There is no - transcoding between transfer syntaxes, no DIMSE networking and no DICOMweb, and coded values are - surfaced but not validated against a terminology. - -- 460d9f1: Add `toObject`, `toISO` and `toDate`, one conversion surface over the decoded `DA`, `TM` and `DT` - values, so reading a timestamp out of an object no longer means writing a per-type projection at - every call site. - - `parseDate`, `parseTime` and `parseDateTime` decode three separate wire syntaxes into three separate - shapes. The three new functions take any of them and answer in one shape: the calendar components, - an ISO-8601 string, or an absolute-instant `Date`. `DateParts` and `ToDateOptions` are exported - beside them. Nothing existing moves: the decoders, their `legacy` and `nonstandardOffset` flags and - `DicomDate` / `DicomTime` / `DicomDateTime` are all unchanged, and the package still takes no - dependency of any kind. - - The same three names, with the same meanings, are exported by every `@cosyte/*` parser that decodes - a date, so they collide on import by design. A file that reads two of them aliases - (`import { toISO as dicomToISO } from "@cosyte/dicom"`) or namespace-imports; `README.md` and the - typed-values page both show the pattern. - - Four decisions in the surface are worth knowing before you depend on it. - - **The key set is the precision.** A component the value did not state is absent from `DateParts` - rather than present and `undefined`, and nothing is zero-filled, so `Object.keys()` recovers what - the sender wrote. There is no `precision` key because the key set is one, and no `raw` or `valid` - key because parse bookkeeping is not a calendar component. `toISO` truncates the same way: a `DT` - that stated only an hour renders `2024-01-15T13`, never a padded-out `T13:00:00`. - - **The keys are singular where the decoded types are plural.** `DicomTime` and `DicomDateTime` - spell the time fields `hours` / `minutes` / `seconds`; `DateParts` spells them `hour` / `minute` / - `second`, and `month` is 1 to 12 rather than the JS `Date` 0 to 11. That is the shape - `Temporal.PlainDateTime.from` and luxon's `DateTime.fromObject` accept, so deleting - `offsetMinutes` leaves an object either constructor takes with no key rename. - - **`toDate` never guesses a zone.** A stated `&ZZXX` offset converts exactly and beats any - `assumeOffsetMinutes` the caller passes. With no stated offset, the caller's assumption is the - only route to an instant, an explicit `0` meaning "read this naive value as UTC". With neither, - the answer is `undefined`: the host machine's zone is never read and UTC is never assumed. A - non-finite `assumeOffsetMinutes` names no zone either, so it answers `undefined` rather than an - `Invalid Date`, and so does a finite one so large that applying it leaves the range a `Date` - represents. A `TM` states no year, so it is never an instant at all. - - **An impossible date converts to nothing, never to the day after it.** The decoders range-check - each component on its own, so a `DA` of `18700230` decodes with `valid: true` and `day: 30`. The - conversion surface reads them together and refuses the whole value: a month outside 1 to 12, a day - outside the one its month really has (full 4/100/400 leap rule), an out-of-range hour, minute or - second, or a component that is not a whole number. `second: 60` is refused with them, even though - `TM` and `DT` permit it for a leap second, because there is no ISO string for it a reader does not - move and no instant to build. The refusal is in the projection only: `parseDate`, `parseTime` and - `parseDateTime` are unchanged and still report what the sender wrote. - - `millisecond` is derived from the digits in `raw`, taken verbatim and right-padded (`"5"` is 500, - `"0500"` is 50, `"123456"` is 123), and never from `fractionalSeconds`, which is a binary float. - `toISO` renders those digits exactly as written, neither padded to three nor rounded. A stated zero - offset renders `Z`, so `toISO` is deliberately not a byte round-trip of the wire value; - `serializeDicom` remains the route that reproduces the original bytes. - - The retired dotted `YYYY.MM.DD` date converts identically to the canonical form, and the `legacy` - flag the decoder reports beside the value never reaches a result. All three functions answer - `undefined` for a value the decoders marked `valid: false`, for `null` and `undefined`, and for a - value that stated no component at all, and none of them throws for any input. - -- 15c4fea: Stop the de-identified header naming the sending site: replace the File Meta group and remove group 0004. - - Two of PS3.15 2026c §E.1.1's unconditional bullets are about the parts of a DICOM File that are not - the Data Set, and neither was discharged. A file this library de-identified still carried the sending - site's `(0002,0016)` Source Application Entity Title, the source vendor's implementation identity, - every non-modeled `(0002,xxxx)` element the source stuffed into the group - Sending and Receiving AE - Title, Private Information Creator UID, Private Information - and every `(0004,xxxx)` Data Element, - all under a report that said the run succeeded. `rebuildFileMeta` changed exactly one field, the - Media Storage SOP Instance UID, and nothing anywhere filtered on group 0004. - - **The File Meta group of de-identified output now describes this de-identifying application.** - §E.1.1: the File Meta Information "shall be replaced with a description of the de-identifying - application", because otherwise "identity information may leak through unmodified File Meta - Information or preamble ... includ[ing] information regarding Application Entity Titles, Presentation - Addresses, implementation information, and private information". So `(0002,0016)` is gone, - `(0002,0012)`/`(0002,0013)` name `@cosyte/dicom`, and every non-modeled `(0002,xxxx)` element is - dropped rather than re-emitted. What survives identifies the **object**: File Meta Information - Version, Media Storage SOP Class UID, Transfer Syntax UID, and the Media Storage SOP Instance UID - with its existing `RetainUIDs` behaviour unchanged. The replacement is a construction rather than an - edit, so it is unconditional on how well the source group parsed and on which Options are active. - - **Group 0004 is removed at every depth**, as §E.1.1 requires "from any SOP Instance or DICOM File - other than a DICOMDIR File" - including inside Sequence Items, and including a `(0004,xxxx)` Sequence - with everything in it. No Annex E Option qualifies either rule. - - **Two things to act on.** - 1. **The byte-for-byte File Meta round trip no longer holds for `deidentify()` output**, and that is - the point rather than a side effect. It still holds for parse-then-serialize, which is guarded by - its own test. If you relied on a non-modeled `(0002,xxxx)` element surviving a de-identify call, - read it off the parsed dataset before the call; it is not recoverable from the output. - 2. **New report surface and three new warning codes.** `report.fileMetaElementsDropped` and - `report.group0004Removals` record what went, each capped per run in the style of - `MAX_UNAUDITABLE_SEQUENCE_FINDINGS`, each beside an uncapped - `...Count` so a caller can count the loss at any input size and tell the two rules apart. - `DICOM_DEIDENT_FILE_META_REPLACED` and `DICOM_DEIDENT_GROUP_0004_REMOVED` are raised once per run, - not once per element, so neither string is multiplied by an element count the input chooses. - - **A DICOMDIR keeps its `(0004,xxxx)` elements and is told what was not discharged.** An object whose - `(0002,0002)` is `1.2.840.10008.1.3.10` takes §E.1.1's carve-out. The rest of that bullet needs a - DICOMDIR model this library does not have, so every such run raises - `DICOM_DEIDENT_DICOMDIR_FILE_SET_NOT_DISCHARGED` - including one where the object carried no - `(0004,xxxx)` element at all, because what was not discharged has nothing to do with what the object - happened to carry. Do not treat that output as a conformant de-identified DICOMDIR. - -- b8fd5ae: Disclose a retained private value this run kept without enumerating it, instead of stamping - `Patient Identity Removed = YES` in silence over it. - - `RetainSafePrivate` plus a `Profile` is the route this entry is about, and this release adds a - second one beside it: the file's own Private Data Element Characteristics Sequence `(0008,0300)`, - covered by the declaration entry. PS3.15 2026c §E.3.10 licenses that retention for a Private - _Attribute_ - "known by the de-identifier to be safe"; it says nothing about a Data Set the sender nested inside - that attribute's value, which PS3.5 §7.5.1 makes Data Elements and PS3.15 §E.1.1 still covers - "whether contained in the top level Data Set or embedded in an Item of a Sequence of Items". Where - the run neither walked that value nor emptied it, the obligation went undischarged and nothing said - so: on a **fully conformant** file, on 20 of the 30 cells of the matrix in - `test/integration/deident-private-reservation.test.ts`, a nested `(0010,0010)` reached the output - byte-verbatim with `ds.warnings` empty and `report.unauditableSequences` empty. Every one of those - cells now raises `DICOM_DEIDENT_PRIVATE_CARRIER_NOT_AUDITABLE` on `report.warnings` and adds a - `report.unauditableSequences` entry. - - **The value is still kept, and that is a product decision rather than an unfinished one.** Emptying - these carriers needs a content test on exactly the VRs arbitrary bytes are for, so it also empties - conformant binary values on legitimate files; the leak stays a documented limit, and what changed is - that it is a disclosed one. Nothing about how any file is read or written changes. - - `UnauditableSequenceFinding` gains `applied: "emptied" | "kept"`, and it is the field to read first: - that array had one meaning for its whole life - this content is _not_ in your output - and the new - class means the opposite. The two are budgeted on separate counters against the same cap, so a - crafted file carrying tens of thousands of retained private attributes cannot spend the budget that - reports dropped content and silence it. The new message names no byte count, because - `Element.rawBytes.length` is the Value Length off a header an under-declared length upstream may - have composed out of somebody's value - the bound `#91` took out of the two sibling factories - and - the private tag it does name renders `` through `renderTag`'s membership test. - - A Private Creator is deliberately not reported: it is retained only when its whole decoded value is - a member of the profile's private dictionary, which is an enumeration, and one carrying anything - else fails that lookup and is removed. - -- 9defcde: Write `(0028,0303) Longitudinal Temporal Information Modified` on every `deidentify()` run, so a - de-identified object says what happened to its dates instead of leaving a recipient to guess. - - PS3.15 2026c §E.2 requires the attribute with a Value of `REMOVED` "if none of the Retain - Longitudinal Temporal Information Options is applied", and §E.3.6 requires `UNMODIFIED` under the - Full Dates Option. This package wrote it in neither state through `0.0.19`, so a receiver reading - dates could not tell real ones from scrubbed ones - and guessing wrong hurts in both directions: - treating real dates as scrubbed under-protects the patient, and treating scrubbed dates as real - corrupts a longitudinal analysis. Both states are written now. - - **`REMOVED`** on a run with no Retain Longitudinal Temporal Information Option active, which is the - default. - - **`UNMODIFIED`** on a run with `RetainLongitudinalTemporal` active, which is this package's name - for §E.3.6's Full Dates Option. - - **It is REPLACED rather than added to.** The attribute is `VM 1` and both clauses say the value - "shall be added to the Data Set with a Value of" one named state, so a `(0028,0303)` the source file - already carried is discarded - including one under another VR or holding a value the standard does - not define. That is the opposite of what `(0012,0063)` does with a prior method text, and the - asymmetry is the standard's: joining two temporal states into one single-valued attribute would leave - a recipient reading a state no run produced. Re-de-identifying an object always leaves exactly one - `(0028,0303)`, holding the latest run's state. - - **The third state, `MODIFIED`, arrives in the same release**, under §E.3.6's With Modified Dates - Option. See the entry for `RetainLongitudinalTemporalModifiedDates`, which adds that Option's Table - E.1-1 column, the option name that selects it, and the disclosure that this library resolves the - column and performs no date transformation. - - Nothing else moves. `(0012,0062)` and `(0012,0063)` are unchanged, the published warning-code set is - unchanged, no warning code is added or retired, and `DeidentifyReport` keeps its shape - like - `(0012,0062)`, this is a statement the object makes about itself and it is not reported there. The - declaration is written into the **top-level** Data Set only, which is where §E.2 and §E.3.6 put it; - `(0028,0303)` has no row in Table E.1-1, so a copy nested in a Sequence Item is retained by omission - like every other unlisted attribute and still says whatever the sender wrote. - -### Patch Changes - -- 47604f6: `deidentify()` now writes `(0012,0064)` De-identification Method Code Sequence on every run, beside - the `(0012,0063)` text, so a consumer can read which Profile and Options produced an object from - codes instead of parsing English, and can tell the two PS3.15 §E.3.6 temporal branches apart by - `113106` and `113107`. - - PS3.15 2026c §E.1.1 asks for "one or more codes from CID 7050 "De-identification Method" - corresponding to the Profile and Options used ... and/or a text string". The text already satisfied - the "and/or", so this is the machine-readable half rather than a conformance fix. - - **What is written.** A top-level `SQ` with an Item for each code from PS3.16 2026d CID 7050 - (context group version `20170914`, UID `1.2.840.10008.6.1.925`): `113100` Basic Application - Confidentiality Profile first, then an Item for each active Option in `DEIDENTIFY_OPTIONS` order, - whatever order `retain` names them in. Each Item carries exactly Code Value, Coding Scheme - Designator `DCM` and Code Meaning. - - **Added to, never replaced.** A prior `(0012,0064)` the source carried keeps every Item, in order - and byte for byte, and this run's Items follow. A code a prior Item already records under the same - designator and value is not added again, so de-identifying an output again with the same options - is a fixed point. - - **New public export** `DEIDENTIFICATION_METHOD_CODES` (with the `DeidentificationMethodCode` type) - carries the CID 7050 rows, the context group UID and version, and the code each Option writes. - - **Two new stable warning codes on `report.warnings`**, never on `Dataset.warnings`: - `DICOM_DEIDENT_METHOD_CODES_PRIOR_RETAINED` when prior Items were kept (neither `(0012,0064)` nor - the code attributes in its Items has a Table E.1-1 row, so what a sender wrote there reaches the - output un-inspected, and no prior code is checked against CID 7050), and - `DICOM_DEIDENT_METHOD_CODES_PRIOR_REPLACED` when a prior `(0012,0064)` was not a Sequence of Items - this run could read and was replaced by this run's codes. Neither message carries the prior's bytes - or VR. `DICOM_DEIDENT_METHOD_PRIOR_RETAINED` keeps its published meaning, `(0012,0063)` only. - - **Limits, stated with the capability.** `113101` Clean Pixel Data Option and `113102` Clean - Recognizable Visual Features Option are never written, because this layer does not touch pixels. - `113107` says the modified-dates column was resolved, not that any date was shifted: the same limit - `(0028,0303) = MODIFIED` carries, and `DICOM_DEIDENT_DATES_NOT_TRANSFORMED` still rides with it. An - Option's code means the Option was active, whether or not the object carried an attribute it acts - on, and a caller's `deidentificationMethod` text never changes the codes. Only the top-level - attribute is written or read. `(0012,0062)`, `(0012,0063)` and `(0028,0303)` are written exactly as - before. - -- 755e408: A DICOMDIR's Directory Record tree is now read, and `serializeDicom()` writes every record offset as - the byte offset of the record it named, so a DICOMDIR this library rewrites or de-identifies still - points each record at the right bytes. Until now a DICOMDIR parsed as an ordinary Data Set and every - write carried the source's offsets into a file whose layout had moved (a rebuilt File Meta group, - Data Sets put in ascending order, Items `deidentify()` re-encoded), so a viewer could step from one - patient's record into another's images. - - **New `Dataset.directory`** (types `DicomDirectory`, `DirectoryRecord`): for an object whose - `(0002,0002)` is `1.2.840.10008.1.3.10`, every Item of the Directory Record Sequence `(0004,1220)` - as a record with its `(0004,1430)` type, its `(0004,1500)` Referenced File ID components (verbatim) - and its lower-level records, plus the root entity `(0004,1200)` names. `undefined` for anything - else. Read under Implicit VR LE, Explicit VR LE and Explicit VR BE. - - **New `Item.fileOffset`**: where the parser found an Item's `(FFFE,E000)` tag, counted from the - first byte of the File Preamble, as PS3.3's Basic Directory IOD counts DICOMDIR offsets (a file read - without a preamble still counts the 132 bytes it lacks). `deidentify()` carries it to the Item it - rebuilds; it is the identity the writer ties each offset to. An offset names a record only when it - equals a Directory Record Sequence Item's `fileOffset`, never an Item tag found by scanning. - - **New Tier-2 codes, each raised only by a non-conformant DICOMDIR, with registry-only messages:** - `DICOM_DIRECTORY_OFFSET_UNRESOLVED` (an offset that is not a record's Item tag: inside a record, on - an Item of a Sequence nested in a record, on the Sequence header, past the end), - `DICOM_DIRECTORY_OFFSET_MALFORMED` (a Value Length other than 4), - `DICOM_DIRECTORY_RECORD_REVISITED` (an offset that would reach a record already reached; it is in - the tree once and the parse finishes) and `DICOM_DIRECTORY_OFFSET_DEFLATED` (a Deflated DICOMDIR - with a non-zero offset, which PS3.10 does not allow, since it requires a DICOMDIR File to use - Explicit VR Little Endian; no offset resolves). No - offset value, record index or record key reaches a message. `{ strict: true }` escalates them. - - **New `SERIALIZE_ERROR_CODES` members `DIRECTORY_OFFSET_UNRESOLVED` and - `DIRECTORY_OFFSET_DEFLATED`.** The writer refuses, and returns no bytes for, a DICOMDIR carrying an - offset it cannot tie to a record the Dataset holds (one that did not resolve on read, one left after - the Directory Record Sequence was removed or emptied, one on a hand-built Dataset with no records, - a value that is not one 32-bit unsigned integer) and a Deflated DICOMDIR with a non-zero offset. It - never writes the stale value or zero. Both messages are fixed strings with no digit in them. - - **`DICOM_DEIDENT_DICOMDIR_FILE_SET_NOT_DISCHARGED` keeps its code and its trigger** (every run on a - DICOMDIR); its message now names only the two File-set clauses of PS3.15 §E.1.1 this run does not - discharge, a DICOMDIR created from the de-identified files and the removal of the non-de-identified - DICOMDIR from the File-set, and no longer says the directory records were not de-identified or that - DICOMDIR is not modelled. - - **Limits, stated beside the capability in `docs-content/limitations.md`:** Referenced File IDs are - kept verbatim; no File-set view; record keys, `(0004,1202)` against the end of the root chain, the - File-set Consistency Flag and Private Record UIDs are not checked; the retired MRDR offset - `(0004,1504)` is written as read. - -- 401909f: The data dictionary and the de-identification action table are now generated from the **PS3.6 and - PS3.15 2026d** DocBook, re-pinned together under `vendor/nema/` by SHA-256. PS3.5 stays at 2026c. - - **`deidentify()` now removes Dose Calculation Model Name `(3004,007F)` under the Basic Profile - (`X`), and cleans it under `CleanDescriptors` (`C`)**, at the top level and inside Sequence Items - alike, and the report audits it. 2026d adds that attribute to Table E.1-1; against the 2026c table - it had no row, so it was kept verbatim under every Option with a report that said nothing about - it. No other Option changes its treatment. - - **`Dictionary.lookup("3004007F")`** now returns `DoseCalculationModelName`, VR `LO`, VM `1`, so an - Implicit VR Little Endian object reads that element as `LO` rather than `UN`. - - **The report name for `(0018,11BB)` changes case**, from "Acquisition Field Of View Label" to - "Acquisition Field of View Label", as 2026d prints it. Its action codes (`D`, and `C` under - `CleanDescriptors`) do not change. A caller matching `DeidentifiedAttribute.keyword` for that tag - on the old spelling needs the new one. - -- 6473903: `serializeDicom()` now writes objects under every Transfer Syntax PS3.5 2026c section A.4 names - for encapsulated Pixel Data (JPEG, JPEG-LS, JPEG 2000, HTJ2K, RLE, MPEG, HEVC, JPEG XL, Deflated - Image Frame and Encapsulated Uncompressed), the same generated list `parseDicom()` reads. It used to - refuse each one with `UNSUPPORTED_TRANSFER_SYNTAX`, so a JPEG or JPEG 2000 object run through - `deidentify()` could not be saved. The File Meta Transfer Syntax UID is kept, nothing is - transcoded, and no pixel is decoded. - - **The Data Set is written as Explicit VR Little Endian**, as section A.4 requires: the bytes after - the File Meta group are the bytes the same Data Set writes under Explicit VR LE. - - **Top-level Pixel Data `(7FE0,0010)` is written as `OB` of undefined length**: the input's Basic - Offset Table Item and every fragment Item byte for byte and in order (Item tag, Item Length and - value), then a Sequence Delimitation Item of length zero. The header is written this way whatever - VR and reserved bytes the input span carried. No offset table is interpreted or rebuilt and no - frame is assembled. - - **New `SERIALIZE_ERROR_CODES` member `INVALID_ENCAPSULATED_PIXEL_DATA`.** Under a section A.4 - syntax the writer refuses, rather than repairs, a top-level Data Set the section does not allow: - no top-level Pixel Data; Float or Double Float Pixel Data `(7FE0,0008)` / `(7FE0,0009)`; Pixel Data - with a defined Value Length; a fragment stream not ended by its Sequence Delimitation Item - (including one `parseDicom()` read with `DICOM_PIXEL_DATA_FRAGMENTS_NOT_DELIMITED`, since a - delimiter appended to a stream that may be short would be a well-formed file missing frames); - anything other than an Item of defined length before the delimiter; an Item of odd length; a - fragment Item of length zero after the Basic Offset Table; no Basic Offset Table Item or no - fragment Item; and bytes after the delimiter. Nothing is returned on a refusal, and the message is a - fixed string that carries no length, byte or UID from the Dataset. - - **Unchanged:** the four native syntaxes serialize exactly as before; every other UID (JPIP - Referenced, SMPTE ST 2110, every retired UID) still throws `UNSUPPORTED_TRANSFER_SYNTAX`, and a - missing one `MISSING_TRANSFER_SYNTAX`. Pixel Data nested in a Sequence Item, such as an Icon Image - Sequence, is written as read and not checked. `parseDicom()` and `deidentify()` are untouched, and - burned-in annotation is still reported by `DICOM_BURNED_IN_ANNOTATION_NOT_REMOVED`, never removed. - - The `UNSUPPORTED_TRANSFER_SYNTAX` message now says the writer also supports every section A.4 - syntax. The code is unchanged. - -- 0d21249: `parseDicom()` now reads objects under every Transfer Syntax PS3.5 2026c section A.4 names for - encapsulated Pixel Data: JPEG Baseline, Extended, Lossless and Lossless First-Order Prediction; RLE - Lossless; JPEG-LS Lossless and Near-Lossless; JPEG 2000 and HTJ2K; MPEG2; MPEG-4 AVC/H.264; - HEVC/H.265 Main and Main 10; JPEG XL; Deflated Image Frame Compression; and Encapsulated - Uncompressed Explicit VR LE. Such an object used to throw `UNSUPPORTED_TRANSFER_SYNTAX` and lose its - metadata with it. Section A.4 makes the whole Data Set Explicit VR Little Endian with only Pixel Data - `(7FE0,0010)` encapsulated, so it is read by the Explicit VR LE reader and its patient, study, series - and image views read exactly as the same Data Set does under Explicit VR LE. The list is read out of - the vendored, SHA-pinned PS3.5 by a new generator that joins `gen:all` and the byte-identical regen - gate, and each UID is checked against the PS3.6 registry as a non-retired Transfer Syntax. - - **New: `readPixelDataFragments(ds)`** returns a `PixelDataFragments`: the Basic Offset Table's - Item Value and every fragment's Item Value as raw bytes, in file order, with no Item header in any - of them. An empty Basic Offset Table reads as zero bytes. It returns `undefined` when Pixel Data is - absent or native. The buffers follow `copyValues` like `Element.rawBytes`. - - **No pixel is ever decoded.** Nothing is decompressed, no frame is assembled from fragments, and - neither the Basic Offset Table nor an Extended Offset Table is interpreted. - - **`serializeDicom()` refuses every one of these syntaxes** with `UNSUPPORTED_TRANSFER_SYNTAX`, a - de-identified object included, so a compressed object is read and not written. - - **An empty Basic Offset Table no longer raises `DICOM_EMPTY_ITEM_IN_SEQUENCE`.** Section A.4 makes - it the conformant form decoders must accept. A later empty fragment still raises it. A caller that - switched on that code for a conformant encapsulated file under Explicit VR LE no longer sees it. - - **New Tier-2 code `DICOM_PIXEL_DATA_FRAGMENTS_NOT_DELIMITED`** for a fragment stream that ends - with the input before its Sequence Delimitation Item. The metadata still parses and the fragment - list may be short; `{ strict: true }` refuses it. Its message carries no fragment byte, length or - count. - - **Still refused:** the JPIP Referenced syntaxes (their Pixel Data is a reference, not fragments), - the SMPTE ST 2110 syntaxes, and every retired UID, the retired JPEG processes included. The fatal - names the syntax by its PS3.6 registry name, never by the UID, and its message now summarises the - supported set instead of listing four UIDs. - - **Not diagnosed:** an encapsulation-syntax object with no top-level Pixel Data, or with native - top-level Pixel Data, parses without a code, as the same bytes do under Explicit VR LE. - -- c99016b: Docs: the README's first usage example and the quickstart's first example are now executed by the test suite, read straight out of the page they are printed on. - - The README example claims its values as checked results now, against a synthetic CT object saved without its preamble, and one claim moved in the process: the study instance UID it shows is the one that object actually carries, not a placeholder. The quickstart's object is byte-identical to a fixture the test corpus builds, so the PHI scan reads it with the rest of the fixtures. Both examples are also compiled with the settings `tsc --init` writes for a new TypeScript project, so an example that does not compile fails the suite too. A changed value in either example fails the suite, and every install command the README and the installation page print is checked against the package's own name. - -- 94069e8: Model CommonMark's HTML blocks in the PHI gate's override log, so a comment cannot exempt a scan - target. - - `scripts/phi-scan.ts` refuses a `--allow-fixture ` bypass unless `phi-scan-overrides.md` - carries a `### ` heading, and its block structure was fenced code blocks and nothing else. A - fenced block shows its contents to a reviewer; an HTML comment shows nothing at all, so a heading - written inside `` was a live allow entry that exempted its target at exit 0 while the - rendered log looked empty. `overrideLogPaths` now models CommonMark 0.31.2 section 4.6's HTML - blocks, start conditions 1 to 6 and their end conditions, and ignores fences and headings inside - one exactly as section 4.6 requires. - - Start condition 7 is deliberately out of scope, and its cost is measured in both directions rather - than assumed: it needs a complete tag AND the knowledge that the line does not interrupt a - paragraph, which is paragraph state this parser does not have. A heading under `` on its own - line is still a live entry here where CommonMark hides it; after a paragraph line, where condition - 7 cannot fire, the two agree. Both arms are pinned by tests. - - No direction is claimed. A block boundary is parity, and on a log carrying an odd number of fence - delimiters inside a comment the two readings' entry sets are disjoint and both non-empty, each - exempting at exit 0 a target the other refuses at exit 2. The tag lists section 4.6 closes over are - read out of the vendored spec and driven name by name through the `--allow-fixture` membership - oracle, so the tables are checked against the document rather than against whoever typed them, and - no count of either is written anywhere. `scripts/measure-phi-scan-html-blocks.ts` ships beside the - change so the figures can be re-run against any tree. - -- 185c0ed: `PN`, `LO`, `SH`, `UC`, `ST`, `LT` and `UT` values now decode ISO 2022 code extensions when - `(0008,0005)` Specific Character Set has more than one Value (and Value 1 is not `ISO_IR 192`, - `GB18030` or `GBK`). Every escape sequence in PS3.3's code-extension tables switches G0 or G1, so a - Japanese, Korean or Chinese Person Name comes back as the characters its sender encoded rather than - with escape bytes inside it or JIS bytes read as ASCII. PS3.5 2026d's worked examples H.3-2, I.2-1 - and K.2-1 used to come back wrong with no warning; all four examples now decode to the names the - standard prints. - - **Reset points follow PS3.5 section 6.1.2.5.3.** Value 1's designations come back at the start of - the value, after CR, LF or FF, after the `\` of a multi-valued VR and after the `^` and `=` of - `PN`. A delimiter counts only as a single byte, never as half of a two-byte character. - - **`decodeText(bytes, terms)`** returns the same characters for a multi-valued term list, with the - text reset rules (CR, LF and FF). - - **New Tier-2 code `DICOM_CHARSET_ESCAPE_UNDECLARED`**: an escape sequence switches to a set - `(0008,0005)` does not declare. The bytes are decoded in the set it names. - - **New Tier-2 code `DICOM_CHARSET_BYTES_UNDECODABLE`**: bytes no designated set decodes (an - unrecognized escape sequence and what follows it, a GR byte with no G1 set, a code the set does - not define, or a set this runtime cannot decode) read as U+FFFD, never as another set's reading. - `Element.rawBytes` keeps them. - - **New Tier-2 code `DICOM_CHARSET_EXTENSION_NOT_RESET`**: a line, value or `PN` component ends - while G0 still holds a set other than Value 1's. The characters are kept and what follows decodes - from Value 1. - - The three codes ride on `Element.value.warnings`, at most once each per value, and never reach - `Dataset.warnings` or `onWarning`, so `{ strict: true }` does not refuse such a file. Their - messages carry the tag and VR only. - - **Unchanged:** a single-valued `(0008,0005)`, a multi-valued one whose Value 1 is `ISO_IR 192`, - `GB18030` or `GBK`, `resolveDecoderLabel`, `serializeDicom` and `deidentify()`. ISO-IR 14's - `0x5C` and `0x7E` still read as `\` and `~`. - -- 8f35150: Read the file's own safe-private declaration, so a caller with a DICOM file from a vendor they hold - no profile for can de-identify it and still keep the private attributes that file itself declares - non-identifying. - - PS3.15 2026c §E.3.10 lists four ways a Private Attribute may be "known by the de-identifier to be - safe from identity leakage", and names the in-file declaration **first**: "its presence in a block of - Private Data Elements with a Value of \"SAFE\" in Block Identifying Information Status (0008,0303) or - individually listed in Nonidentifying Private Elements (gggg,0004) (within Private Data Element - Characteristics Sequence (0008,0300)". This package resolved safety only through a caller `Profile`, - so `RetainSafePrivate` kept nothing without one and the technique values a quantitative read depends - on - §E.3.10's own examples are CT helical span pitch and PET SUV rescale factors - were removed from - every file whose vendor nobody had written a profile for. - - **Both branches of that bullet are read.** A block whose `(0008,0303)` is `SAFE` is known safe - whole; a block declared `MIXED` is known safe for exactly the elements its `(0008,0304)` list - names, identified by the lowest 8 bits of the tag within the block (PS3.3 2026c §C.12.1). The - Private Creator that reserves such a block is retained with it, which is what §E.3.10's "together - with the Private Creator IDs that are required to fully define the retained Private Attributes" - asks for. A block declared `UNSAFE` retains nothing, its creator included. - - **No profile is needed, and passing one only ever adds.** A profile retention survives a block the - declaration calls `UNSAFE` or says nothing about, and nothing outside the odd-group private block - an Item describes is retained on account of that Item. - - **A block is keyed by its Private Creator VALUE and never by a block number**, which is what - §C.12.1's own Note requires ("since instances may be modified and numeric block numbers - reassigned"). The block number an element sits in is resolved separately, against the reservations - of the Data Set that element lives in, because PS3.5 2026c §7.8.1 scopes a reservation to one Data - Set and Items do not inherit an enclosing one. - - **A declared-safe private Sequence is still parsed in its entirety**, with each nested attribute - handled on its own merits, exactly as §E.3.10's Sequence paragraph requires. The declaration - decides the carrier and never the Data Sets below it. - - **An Item that does not resolve retains nothing and is disclosed** under a new stable warning code, - `DICOM_DEIDENT_PRIVATE_DECLARATION_NOT_RESOLVED`, raised once per Item on `report.warnings` and - capped per run. An Item is unresolvable when a Type 1 Value is absent or empty, when `(0008,0303)` - carries a Value outside the three §C.12.1 enumerates, when `(0008,0301)` is not an odd group, when - a `MIXED` Item carries no usable `(0008,0304)` list, or when `(0008,0302)` names a creator no block - in that Data Set reserves. The message carries no token of the Item: the creator, the group and an - unenumerated status are all the sender's own values, and there is no string parameter for one to - travel through. - - 🩺 **What this retains rests on the SENDER's assertion, and that is the cost rather than a footnote.** - The system that wrote the file said the block carries no identifying information; nothing here - re-derives it, so an opaque vendor value in a block declared `SAFE` reaches de-identified output - unexamined - the class removal closed for the profile route. §E.3.10 offers exactly one mitigation - and it is the caller's: if you do not trust the sender, do not pass `RetainSafePrivate`, and every - Private Attribute is removed ("When this Option is not specified, all Private Attributes shall be - removed"). The Option is off by default and this changes nothing about a run that leaves it off. - - Everything that guarded the profile route still acts first and is untouched: a Data Set whose - reservations the file does not settle, an element outside the settled run, an on-wire VR that is not - one of the 34, and a value the embedded-attribute scanner reads as a swallowed Data Element run. - Deidentification Action (0008,0307), Deidentification Action Sequence (0008,0305) and Identifying - Private Elements (0008,0306) are **not** implemented and are not implemented by this: §E.3.10's - second branch is "removed **or** processed in the element-specific manner recommended by - Deidentification Action (0008,0307)", and removal stays the branch this library takes. - -- c3f5484: Gate the public surface against internal project bookkeeping, and sweep the tree it would have failed. - - `no-internal-refs.yml` and `scripts/check-no-internal-refs.ts` enforce the founder directive of - 2026-07-27 on every surface a consumer of `@cosyte/dicom` reads: the published markdown at the - repository root, `docs-content/`, the npm `description` and `keywords`, and all of `src/`. Work-item - and plan ids, phase and wave language, roadmap and plan citations, ADR numbers, meta-repo paths and - commentary about how the work is being run are findings; `CHANGELOG.md`, `.changeset/`, commit - messages, pull request text, `test/`, `.github/` and this repository's agent-context docs are - excluded, because those are the surfaces the identifiers belong on. - - The gate self-tests before it reports: every rule has to match its own positive sample, a DICOM - reference corpus (`PS3.6`, `(0010,0010)`, `PN`, `UI`, `DICOM-SR`, `1.2.840.10008.1.2.1`, - `ICD-10-CM`, `CP-246`, `MRN-42` and the rest) has to survive every rule untouched, and the scope - function has to classify a fixed list of paths the way the rule says. It refuses to print a result - from a scan that read nothing, from a declared surface that selects no tracked file, or from a file - in scope it could not read or could not decode as UTF-8. There is no list of excused occurrences and - there must not be one. - - The sweep is the other half, and it is why this is a `patch` rather than a chore: 530 findings, 526 - of them in `src/` comment blocks that `tsup` copies verbatim into the published `dist/index.d.ts` and - `dist/index.d.cts`, against 4 in markdown anyone reviews. Comment and documentation TEXT is rewritten - and nothing else: no exported symbol, signature, message string or runtime behaviour changes, and the - build, the type declarations' shape, the lint ladder and all 1,586 tests are unchanged. - -- 0bc40ae: Correct the troubleshooting remedy for `DICOM_DEIDENT_PRIVATE_CARRIER_NOT_AUDITABLE`: a value the file's own `(0008,0300)` declaration names safe is retained, not removed. - - The remedy said that, beyond the three cases the package enumerates, every private value a `Profile` vouched for under `RetainSafePrivate` is removed, an ordinary scalar under an ordinary string VR included. That was false whenever the file's own Private Data Element Characteristics Sequence `(0008,0300)` names the value safe, in a block whose status is `SAFE` or listed individually in a `MIXED` one: `deidentify()` retains that value on the sender's word, and `report.unenumerablePrivateRemovals` does not record it. The guide now names that route and keeps the removal statement for everything outside it. No behaviour changes. - -- 339892a: Add `RetainLongitudinalTemporalModifiedDates`, the second PS3.15 §E.3.6 Retain Longitudinal Temporal - Information Option, so a caller running a longitudinal study can keep the temporal relationships - their analysis needs without keeping real dates. - - §E.3.6 defines **two** mutually exclusive Options and Table E.1-1 gives them separate columns. This - package published one name, carrying the full-dates column, so the modified-dates column was - unreachable and `(0028,0303) Longitudinal Temporal Information Modified` could never say `MODIFIED`. - - **`RetainLongitudinalTemporalModifiedDates`** resolves Table E.1-1's `Rtn. Long. Modif. Dates Opt.` - column, which is the more protective of the two on every attribute where they differ: full dates - keeps the real value, modified dates cleans it. An attribute the modified-dates column publishes no - action for takes the Basic Profile action, never the full-dates one and never a keep. - - **`(0028,0303)` takes the value `MODIFIED`** on that branch, replacing whatever the source carried - there, exactly as `REMOVED` and `UNMODIFIED` do. - - **A Value naming the Option is added to `(0012,0063)` De-identification Method**, inside `LO`'s - 64-character per-Value maximum like every other name this library writes there. - - **The two temporal Options are mutually exclusive.** A call naming both is rejected with a - `DeidentifyError` carrying the existing `INVALID_OPTIONS` code. Either one alone is accepted, and - `RetainLongitudinalTemporal` is unchanged in both respects: it still resolves the full-dates column - and still writes `UNMODIFIED`. - - **This library performs no date transformation, and says so at run time.** §E.3.6 has two halves: the - Table E.1-1 column, and a requirement that the dates themselves be modified with the manner of - modification described in a Conformance Statement. A library can deliver the first and neither of the - second, because PS3.2 Annex N scopes a Conformance Statement to a named product and version. So - **you** perform the transformation, and a `MODIFIED` on an object whose dates nobody shifted is a - defect this package cannot detect. Every run under the new Option raises the new stable warning code - `DICOM_DEIDENT_DATES_NOT_TRANSFORMED` on `report.warnings`, stating the half of §E.3.6 the run did - not discharge. Nothing here claims PS3.15 Annex E conformance for the Retain Longitudinal Temporal - Information With Modified Dates Option, which §E.1.1 makes all-or-nothing. - - `DICOM_DEIDENT_DATES_NOT_TRANSFORMED` is added to `WARNING_CODES` and is raised by `deidentify()` - only: it is a statement about the run's options rather than about the file, so it never reaches - `Dataset.warnings` and a `{ strict: true }` parse of the same input is unaffected. No published code - is renamed or retired, no coded Item is added to `(0012,0064)`, and what the Basic Profile retains - with neither temporal Option active is unchanged. - -- 734736f: Claim the PHI gate's `CRLF` line split, and delete the disclosure that said it could not be claimed. - - `#113` shipped `scripts/phi-scan.ts`'s `splitLines` with a disclosure that its `CRLF` half was - unobservable through either caller and claimed by no test, because `loadAllowList` trims and - `tripleHashValue` trims. Both premises are true and the conclusion is false: `overrideLogPaths` - hands the RAW line to `fenceRun` before anything trims it, and a closing fence run is `bare` only - when it is followed by spaces or tabs (CommonMark 0.31.2 section 4.5). On an override log written - with `CRLF`, a `CR`-blind split therefore leaves a `CR` after the closing run, the run reads as an - info string rather than as a close, the block never ends, and every entry below it is dropped. - - Measured on one log carrying a fenced template and two live entries below it: the shipped script - exits 0 with both entries honoured, a `CR`-blind mutant exits 2 having dropped both, and with the - same log written `LF` the two agree. Both `--allow-fixture` directions are asked on each arm, so a - parser that made everything below an entry fails it too. - - No direction is claimed for that mutant. A draft called it fail-closed, on the grounds that a `CR` - can only prevent a close and never cause one; the gate falsified that with an input, and it is - deleted rather than narrowed. On a log whose lines are `CRLF` except one opening fence the two - entry sets are disjoint rather than nested, and the mutant exempts at exit 0 a target the shipped - script refuses at exit 2. Fence state is parity, so a wrong answer moves entries in both directions. - - The behaviour was already correct and is unchanged: comment-stripped, `scripts/phi-scan.ts` is - byte-identical on base and head. What changed is that four carriers of the false disclosure inside - this package are corrected. - - The mutation figure that stood here is DELETED rather than restated: a later change in this same - release moved `overrideLogPaths` onto its own CommonMark splitter, so the `CR`-blind mutant of - `splitLines` no longer reds anything, and both changesets land in one generated `CHANGELOG.md`. - -- 09de547: Refuse a `###` run an invisible character separates from the path, in the PHI gate's override log. - - `scripts/phi-scan.ts` refuses a `--allow-fixture ` bypass unless `phi-scan-overrides.md` - carries a `### ` heading. `tripleHashValue` separated the `###` run from the path with the - whole of `\s`, where CommonMark 0.31.2 section 4.2 says the opening run "must be followed by spaces - or tabs, or by the end of line". So a line whose separator was an invisible character was a live - allow entry where the document renders a paragraph, and that target was exempted at exit 0. Measured - live on three trees before the change, each against the same bytes without the flag, which exit 1. - - It is one conjunct on the pattern the function replaced, so for every line the parser returns either - that pattern's answer or nothing: no line that was not already an entry becomes one, and every - difference is a refusal at exit 2. That is a property of the source rather than one a test can - prove, and it is left as one: the evidence beside it is the comment-stripped diff, a single added - early return, and the per-log relation the shipped instrument prints against a base, which reads as - a subset on the one log the two trees differ on. - - Section 4.2's strip is deliberately NOT taken, and that is measured rather than assumed: the prose - says leading and trailing spaces or tabs, and `commonmark@0.31.2`, the reference implementation of - the pinned document version, strips with `String.prototype.trim`, which is the whole of `\s` and is - what this parser does. A draft that took the prose named `path` where this parser names - `path`, which exempted at exit 0 a target it refuses at exit 2, and it was deleted. The strip, - section 4.2's optional closing sequence and its inline parsing are each pinned as they are, with - their cost stated in both directions. - - `scripts/measure-phi-scan-atx-heading.ts` ships beside the change so the figures can be re-run - against any tree. It carries a control the sibling instruments do not: an entry must be shown to - exempt a real hit, not merely to be accepted. - -- 5e7a823: Withhold the PHI gate's verdict over a target it enumerated and never opened, and put the install - hardening the estate baseline asks for actually in force. - - `scripts/phi-scan.ts` filters an `--allow-fixture` target out of the run after enumerating it, and - until now the run then reported on whatever was left as if that were the whole corpus. The shape - that costs the most is not the obvious one: withdraw the ONLY violator and every surviving target is - clean, so the same argv that means "excuse this one fixture" produced a run indistinguishable from a - clean sweep. The scanner now tracks the paths a run declared against the paths it read, and a - non-empty difference exits **3** with a `WITHHELD:` line naming each path, after printing the hit - detail for everything it did read. - - 3 rather than 1 or 2, and each is a different sentence. 1 says "PHI is here", and overloading it - would make a bypass look like a finding and a finding look like a bypass. 2 says "the scan never - ran" and is what an UNLOGGED `--allow-fixture` already returns, having opened nothing; a withheld - run opened every other target and its hit lines are real. The rule is computed off the two sets - rather than off the flag, so the boundary is "enumerated and not read": a bypass naming a path this - run never enumerated withdraws nothing and still exits 0. - - Every path with no bypass on it is byte-for-byte unchanged, which is the property the pre-commit - hook and CI depend on: `pnpm measure:phi-scan-unread` against the pre-change scanner reports - `violations: 0` over 195 cells, identical exit codes and an identical hit-line multiset. The new - suite coverage carries a mutation control that removes the rule from a planted copy and asserts the - graded run falls back to the hits code, and refuses as vacuous if the line it removes is ever - reworded. - - `package.json` gains the `js-yaml@>=4.0.0 <4.3.0` override the shared baseline requires, ordered so - the stronger `<4.3.1` pin this repo already carried is the one that applies; no resolved version - moves. `pnpm-workspace.yaml` declares `minimumReleaseAge: 1440` and `trustPolicy: no-downgrade`, and - `packageManager` moves to `pnpm@10.34.5` because a pnpm below `10.21.0` ignores those keys entirely, - which is a settings file that decorates rather than defends. - -- 9de3bf8: `serializeDicom()` now writes every Data Set in ascending tag order, so a file this library writes - satisfies the element-ordering rule a strict receiver checks even when the file it was read from - did not. It used to write elements in the order the `Dataset` held them (parse order, plus anything - your code or `deidentify()` added afterwards), and to copy every Sequence through as read, so a - round trip propagated an out-of-order source instead of normalizing it. - - PS3.5 2026c §7.1: "The Data Elements in a Data Set shall be ordered by increasing Data Element Tag - Number", and §7.5.1 says the same of the Data Set inside every Item. - - **What is ordered.** The root Data Set, and the Data Set of every Item of every Sequence the - writer can walk on the wire, at every depth up to `NESTING_DEPTH_LIMIT`, under all four supported - transfer syntaxes. That includes the elements `deidentify()` inserts, such as `(0012,0062)` and - `(0012,0063)`. - - **What does not change.** Items keep their order (§7.5). Only whole elements move, so no value - byte changes and every Item and Sequence length stays valid. The Sequence is walked from its own - bytes, never re-encoded from the parsed `items`, so nothing its bytes did not carry is written, - and it is re-ordered only where those `items` match its bytes, so the output reads back as the - source did. The input `Dataset` is not modified, and `Dataset.elements()` still returns parse - order. - - **What is still not ordered.** A tag repeated inside an Item is kept twice, in source order, so - that output still breaks the "at most once" of PS3.5 2026c §7.1. A Sequence whose Item stream - cannot be walked to exactly its end, one nested past the bound, one whose parsed `items` do not - match its bytes (a Sequence the parser did not descend, `DICOM_SQ_NOT_DESCENDED`, for one), and any - `UN`-carried Sequence (under Implicit VR LE that includes a private Sequence inside an Item, even - one a `Profile` resolved to `SQ`, since a default read resolves its tag to `UN`) are written as - read, unordered. An element the parser relocated because a length field lied is ordered where the - parser placed it. - - **What is written last, whatever its tag.** An element whose own bytes do not show where a reader - ends it: an undefined-length `UN` the parser could not read as a Sequence, or a Sequence, `UN` or - Pixel Data value missing its Sequence Delimitation Item. A reader takes whatever follows such a - value into it, so it stays after the ascending rest of its Data Set, at the root and inside an - Item. A Sequence nested past the bound also goes after the ascending rest of its Item, unless it - is a defined-length Sequence under Implicit VR LE, since seeing where a reader ends any other - would take the walk past the bound. - - **A source that was already in order** is written back with the same bytes after the File Meta - group, as before. - -- 8139687: Parse the PHI gate's override log with CommonMark's line ending, and pin the spec that says so. - - A lone `CR` in `phi-scan-overrides.md` hid a fence OPENER from the gate's line split, so a - `### ` a human sees inside a rendered code block was a live allow entry and `--allow-fixture` - exempted that PHI scan target at exit 0. `overrideLogPaths` now splits per CommonMark 0.31.2 section - 2.1; the allow list keeps `/\r?\n/` deliberately, because `scripts/phi-allow-list.txt` is not a - markdown document. No direction is claimed for either split, on the override log or on the allow - list: each has a measured input whose two entry sets are disjoint, so neither is the conservative - one. The spec is vendored under `vendor/commonmark/` and re-hashed - as a precondition, so the section numbers the gate cites are derived rather than asserted. - -- 94e28cc: `parseDicom()` now reads objects under the four JPIP Referenced Transfer Syntaxes of PS3.5 2026c: - JPIP Referenced `1.2.840.10008.1.2.4.94` (section A.6), JPIP Referenced Deflate `…1.2.4.95` - (section A.7), JPIP HTJ2K Referenced `…1.2.4.204` (section A.11) and JPIP HTJ2K Referenced Deflate - `…1.2.4.205` (section A.12). Such an object used to throw `UNSUPPORTED_TRANSFER_SYNTAX` and lose its - metadata with it. Sections A.6 and A.11 make the whole Data Set Explicit VR Little Endian, so it is - read by the Explicit VR LE reader; sections A.7 and A.12 are that Data Set compressed per RFC 1951, - so it is inflated by the Deflated reader first, under the same decompression cap and inflate fatals - as Deflated Explicit VR LE. Its patient, study, series and image views read exactly as the same Data - Set does under Explicit VR LE. - - **The Pixel Data Provider URL is surfaced and never fetched.** A JPIP object carries no Pixel - Data; `ds.get("00287FE0")` returns its Pixel Data Provider URL as the `UR` element the file - carries, its `value` the URL text with only its trailing padding removed. Nothing in this package - fetches, resolves or validates it. - - **`serializeDicom()` refuses all four** with `UNSUPPORTED_TRANSFER_SYNTAX`: a JPIP object is read, - not written. - - **New `DEIDENTIFY_ERROR_CODES.UNSUPPORTED_TRANSFER_SYNTAX`.** `deidentify()` throws a - `DeidentifyError` with this code, and a fixed message, for a `Dataset` whose File Meta Transfer - Syntax UID is any of the four, whatever the options, and returns no dataset or report. The URL - has no PS3.15 Table E.1-1 row, so a de-identified copy would otherwise keep it by omission. - `DeidentifyError` is no longer thrown only for author-time misconfiguration: switch on `code`. - The refusal keys on the File Meta alone, so a non-JPIP object that carries `(0028,7FE0)` is - de-identified as before and keeps it. - - **Not diagnosed:** a JPIP object with no `(0028,7FE0)`, with a top-level Pixel Data, or with a - Photometric Interpretation outside the four Values section A.6 allows parses without a code. - - **Still refused:** the SMPTE ST 2110 syntaxes and every retired UID. The - `UNSUPPORTED_TRANSFER_SYNTAX` fatal's message now names the JPIP sections among the supported set. - -- fad31fb: **`deidentify()` now removes a non-private attribute that neither this build's PS3.6 2026d data - dictionary nor PS3.15 2026d Table E.1-1 carries**, at the top level and inside every Sequence Item - it walks, whatever its VR. Until now a Table E.1-1 miss meant "keep", so a Standard Attribute from a - later edition, or one a sender invented, went into de-identified output verbatim with a report that - said nothing about it. The notes to Table E.1-1 name "new Standard Attributes" as a place identifying - information may be. - - **"Registered" means a literal PS3.6 row or a masked row the tag matches.** The `(50xx,....)` and - `(60xx,....)` families count only across the even groups PS3.5 2026c section 7.6 bounds them to, - so `(6002,0010)` Overlay Rows is kept and `(6020,0010)` is removed; every other masked row, such as - `(0028,04x0)`, is read as PS3.6 prints it. A registered attribute Table E.1-1 does not list is kept - exactly as before, `UN` VR included. - - **Private attributes, `(0004,xxxx)`, `(0002,xxxx)`, group lengths `(gggg,0000)` and every listed - attribute are unchanged.** An unregistered element whose on-wire VR is outside the 34 PS3.5 - section 6.2 defines is now removed rather than emptied, and is no longer listed on - `report.undefinedVrElements`; a registered one still is. An unregistered Sequence is removed whole - and nothing inside it is walked or reported. - - **New report fields:** `report.unregisteredElementRemovals`, a list of the new exported - `UnregisteredElementRemoval` (a byte offset, plus the context path when nested, and deliberately - no tag or VR, since such a tag may be four bytes of some element's value), capped per run, and - `report.unregisteredElementRemovalCount`, the complete total. - - **New warning code:** `DICOM_DEIDENT_UNREGISTERED_ELEMENT_REMOVED`, raised once per run that - removed anything, on `report.warnings` only, with no tag, VR or count in its message. - - **The cost:** a conformant attribute from a PS3.6 edition newer than this build is removed too, and - there is no option to keep it in this release. - ## 0.0.19 ### Patch Changes diff --git a/package.json b/package.json index 29be9e6..77df121 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@cosyte/dicom", - "version": "0.1.0", + "version": "0.0.19", "description": "DICOM Part 10 parser + utility library for Node.js and TypeScript: metadata-first, vendor-quirky-tolerant, dual ESM/CJS.", "keywords": [ "dicom", diff --git a/src/version.ts b/src/version.ts index 58ceaf5..da6c9c3 100644 --- a/src/version.ts +++ b/src/version.ts @@ -10,4 +10,4 @@ * import { VERSION } from "@cosyte/dicom"; * console.log(`@cosyte/dicom v${VERSION}`); */ -export const VERSION: string = "0.1.0"; +export const VERSION: string = "0.0.19";