Skip to content

Report a gain map claim with no payload as orphaned - #1

Merged
Booyaka101 merged 1 commit into
mainfrom
fix-orphaned-false-negative
Sep 22, 2026
Merged

Booyaka101 merged 1 commit into
mainfrom
fix-orphaned-false-negative

Conversation

@Booyaka101

Copy link
Copy Markdown
Owner

The bug

A JPEG whose primary XMP declares hdrgm:Version was classified ultrahdr even when the file held no gain map at all. Pillow, and anything else that re-encodes the primary image while carrying the XMP across, produces exactly that file. So gmaudit diff reported those round trips as clean, which is the one thing the tool exists to catch.

Before, on a real Pillow export whose gain map is gone:

state: ultrahdr | rules: ['ultrahdr']
gain_map: {'source': 'gcontainer', 'offset': None, 'length': 545133, ...}

offset: None was the tell. Nothing had ever located a payload.

The fix

The Ultra HDR rule now needs a payload it can point at: an MPF entry with an image really parsed behind it, or an appended image found by the SOI scan walk() falls back to. Two more cases fall out of the same rule:

  • a file truncated after its primary image keeps its MPF index, so the entry survives and the bytes do not
  • an entry MPF labels a thumbnail was never the gain map the XMP refers to

When nothing locates it, the file reports orphaned. gain_map.source now names what located the payload (mpf, appended, or those prefixed gcontainer+); the bare gcontainer source is gone, since a container directory entry on its own never located anything.

Two existing tests asserted the old behaviour on single-image fixtures. Those fixtures were the bug, so they now build real two-image pairs, and the single-image shape is asserted as orphaned.

How it was found

lab/, added here, is a real-file environment CI does not run: around fifty samples from libavif, Awesome-Gain-Maps and libultrahdr, pushed through Pillow, OpenCV and ffmpeg, plus fresh files encoded by ultrahdr_app, then audited and cross-checked against libultrahdr's own decoder.

libultrahdr disagreed with us on seven of eight real exports before this change:

7 disagreements before the fix
  export_xmp_kept\gain_mapped-photo-tokyo.jpg -- gmaudit says ultrahdr, ultrahdr_app.exe says no-gainmap
  ...

and none after. lab/audit.py reports 102 real files, 0 problems on this branch, and 21 problems with detect.py reverted, so it is not a vacuous green. It needs network access and third-party encoders, which is why it stays out of tests/ and out of the sdist.

Also here

--verify-with-ultrahdr now uses ultrahdr_app's probe mode (-P), which reads the gain map metadata without decoding and writes nothing. The old path dumped a 786 KB outrgb.raw per file into a scratch directory. Builds older than libultrahdr 1.5.0 have no -P and report it by name, so those fall back to the full decode as before, still in the scratch directory.

Testing

127 tests, ruff check clean, twine check passed. The four new detection tests each fail with detect.py reverted. The real-bytes regression test cuts the gain map off a genuine corpus file rather than committing a derived image, so tests/corpus/ stays purely upstream output.

🤖 Generated with Claude Code

A JPEG whose primary XMP declares hdrgm:Version was classified ultrahdr even
when the file held no gain map. Pillow, and anything else that re-encodes the
primary while carrying the XMP across, produces exactly that file, so
`gmaudit diff` called those round trips clean. That is the one thing the tool
exists to catch.

The Ultra HDR rule now needs a payload it can point at: an MPF entry with an
image really parsed behind it, or an appended image found by the SOI scan. An
MPF entry whose bytes are gone does not count, and neither does an entry MPF
labels a thumbnail. When nothing locates the gain map the XMP names, the file
reports orphaned, which is what that state is for.

Found by lab/, a real-file environment added here: fifty samples from libavif,
Awesome-Gain-Maps and libultrahdr, pushed through Pillow, OpenCV and ffmpeg,
plus fresh files encoded by ultrahdr_app, all cross-checked against
libultrahdr's own decoder. It disagreed with us on seven of eight real exports
before this change and none after. CI does not run it; it needs network access
and third-party encoders.

Also switches --verify-with-ultrahdr to ultrahdr_app's probe mode (-P), which
reads the gain map metadata without decoding and writes nothing. Builds older
than libultrahdr 1.5.0 have no -P and report it by name, so those fall back to
the full decode in a scratch directory as before.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Booyaka101
Booyaka101 merged commit 0da560a into main Sep 22, 2026
12 checks passed
@Booyaka101
Booyaka101 deleted the fix-orphaned-false-negative branch September 22, 2026 05:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant