Skip to content

PR3_bulb-blend-capture.md - #1609

Merged
marcinz606 merged 70 commits into
marcinz606:mainfrom
cimelab:feature/bulb-blend-capture
Nov 5, 2026
Merged

marcinz606 merged 70 commits into
marcinz606:mainfrom
cimelab:feature/bulb-blend-capture

Conversation

@cimelab

@cimelab cimelab commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Blend capture: R, G and B in turn inside one Bulb exposure

Refs #1375. Builds on #1569 (and so on #1565): the commits of those PRs come first, and the diff shrinks as they merge. The Blend work is three commits: af134ed, d900f33 and b6b4d36.

What this does

Blend is a third capture mode beside Trichrome and Single Exposure, as in Lento's Dock Connect. The camera holds Bulb and the light shows red, green and blue in turn, each for its Trichrome calibration time, so a frame is one RAW.

  • Shutter release: USB bulb through gphoto2, or Cable through the Scanlight's 3.5 mm jack (H2D_SHUTTER_PULSE, at most 2.55 s).
  • Calibration: a Blend calibration saves a sensor crosstalk profile by default, and the frame's roll takes it with Linear RAW.
  • Whole strip: a strip holds Bulb from frame to frame and moves the film while the camera writes the file.
  • Light timing: the Lento Dock switches the light without waiting for its reply, so each channel gets its time. Waited writes made a channel's light last a varying extra time. The dock's button poll pauses during a timed light, so no poll comes between its writes.
  • Shutter lead: the light stays dark 0.35 s after the Bulb press, because the A7 IV opens its shutter by 0.3 s. scripts/bulb_lead_probe.py measures another body.
  • Film Strip: the preset list names each preset's capture mode, and the Film Strip marks Blend and Single Exposure frames.
  • Reopening a frame: a Blend or Single Exposure frame with no saved edit opens with the settings its capture gave it (C-41, the narrowband decode and the Pakon print stock). Before, a frame first opened in a later session fell back to autodetect, which reads narrowband RGB as B&W. A trichrome has no such gap, because its triplet decides its decode.

docs/USER_GUIDE.md, docs/CAMERA_SCANNING.md, docs/PIPELINE.md and all four translations are updated.

Testing

  • Unit tests for the Bulb capture (gphoto2 and simulated), the capture service and worker (Blend frames, strips, flat), presets, calibration, capture-mode records and the fresh-frame settings.
  • Tested on hardware: Lento Dock 35 (firmware 1.0.238) and a Scanlight Big with a Sony A7 IV over USB. Blend whole strips scan and eject, and Blend frames from earlier sessions reopen correctly.
  • On ten frames of one strip, Blend with its sensor profile matched Trichrome within 1 to 2.6% RMS in R/G and B/G after a per-channel gain. Without the profile, the error was two to three times larger.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CiaS7mdcgupjy3zYtMa4Su

cimelab and others added 30 commits October 30, 2026 13:29
The scan waited for "SHOT done", the end of the dock's own 3 s exposure.
It now starts at the STATE st=EXPOSE that begins it. On the dock, CONTRGB
is accepted during that exposure, and TW does not shorten it. A film move
or eject waits until the exposure ends.

Refs marcinz606#1375

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiaS7mdcgupjy3zYtMa4Su
…old around the density stage

A camera RAW applied its whole matrix before the log, so the crosstalk
unmix, Fade Restoration and Remove Stain acted on mixed working-space
channels and not on the film's own dye channels.

The camera matrix now splits into two parts (camera_matrix_parts). The
white-balance gains apply before the log. The primaries apply after the
density matrix, in linear light, on the Positive and Transfer paths.
The primaries rows sum to 1, so a border that Remove Stain makes neutral
in white-balanced camera channels stays neutral. With the whole matrix
after the stain offset, a synthetic border (WB 2:1:1.5, white 0.8) came
out at log (+0.201, -0.100, +0.076). It now comes out at -0.1 in each
channel. A frame with no density matrix keeps its bits. The print path
does not change.

Fade Estimate now reads a camera RAW in white-balanced camera channels,
which are the channels the fade acts on.

The slide characterization goldens move by at most 0.0074 because they
use a crosstalk matrix. Bumps THUMBNAIL_RENDER_VERSION to 32.
BOUNDS_VERSION does not change: only the transfer meters move, and the
app does not save them as bounds.
…ity matrix offset

The camera white-balance gains are a per-channel log offset. They now go
into the last column of the density matrix (density_stage), so neither
engine multiplies the full image by a diagonal matrix before the log.
The GPU cam rows are identity in this case. The GPU meter grids reuse the
log grid they already have.

Identity primaries (an Input ICC, wb_only_cam_xyz) apply nothing after
the density matrix.

The Fade Estimate grid (transfer_grid) is made only when
PipelineContext.wants_fade_grid is set. fade_estimate_grid sets it.
Before, every CPU readback render of a camera RAW without a density
matrix did a second full-image pass for that grid.

density_stage returns one DensityStage (pre, unmix, post). The
white-balance fold is decided in one place, and both engines build
their meter grid with density_stage_grid.

The Remove Stain test now has a colored patch. It fails if the camera
matrix applies before the log.
Fade Restoration on camera RAWs gave casts that changed from frame to
frame. Two faults caused them.

The restoration scaled density from the decoder's white. A JPEG or
scanner TIFF has its white there, but a camera captures clear film
about 2 stops lower, so the restoration scaled that offset into a
cast. The cast moved with exposure and white balance: on a faded NEF,
+-1 stop moved the white's B/G from 0.35 to 0.81 and the Estimate's
Blue Survival from 0.75 to 0.63. A camera RAW now renders M (L - w) + w,
so density above the white is restored and the white stays. Remove
Stain only maps the white to neutral. Estimate reads the grays above
the white and stores it with Remove Stain off too. The Estimate and
the Positive render now do not change with exposure.

The stored white could come from another color space than the render.
Two Positive NEFs held a white read without the camera matrix, 0.28 D
off in red, which rendered magenta under Remove Stain. The white now
carries fade_white_space, a hash of the camera matrix and as-shot white
balance it was read through. A render in another space ignores it, and
the desktop measures it again on the open frame, with no roll lock.
JPEG and TIFF sources keep the empty tag and render as before.

Bumps THUMBNAIL_RENDER_VERSION to 32.
On top of the camera-matrix split, a camera RAW's fade white is read in
the white-balanced camera channels, not in working space. The
fade_white_space tag hashed only the camera matrix and white balance,
so a white read in working space on an earlier build matched the tag
and applied in the wrong channels. The hash now also holds a channel
marker. A white with an older tag reads as stale, and the desktop
measures it again on the open frame.
…white

A slide's rebate is black, so a frame with no clear film in the picture
has no true white. The measured white is then the brightest picture
color in each channel, for example a blue sky. With Remove Stain off,
the render added that white back with its color, while Estimate had
made the grays neutral relative to it. Every gray took the white's
color: on a synthetic faded slide the cast was +0.33 D in R-G and
+0.50 D in B-G.

With Remove Stain off, a camera RAW now scales density about the
neutral level of the white's cleanest channel (density_anchor), and
Estimate reads the grays against the same level. The grays stay within
0.03 D of neutral at any exposure, with or without clear film, on
Positive and Transfer. Remove Stain on does not change.

The automatic re-measure of a stale white records no undo step. Before,
an undo back to the stale white measured it again and dropped the redo
branch.
Highlight Reconstruction decodes a blown highlight as neutral at full
scale. On a camera RAW the fade scales density about the frame's white,
which sits below full scale, so the blown highlight sat above the
anchor and the fade scaled it per channel. It turned pink with Remove
Stain off and yellow with Remove Stain on. On a synthetic faded slide
the highlight came out (1.00, 0.83, 1.00) and (1.00, 0.97, 0.73).

Dyed film is not brighter than its own white, so a pixel above the
anchor in every channel is light past the film. The fade now acts on
each channel capped at the anchor (fade_anchor), and the amount the
pixel is above the anchor in every channel is added after it, equal in
each channel. The CPU uses the new negpy_core.unmix_log_anchored
kernel, and the GPU uses an anchor row in the normalization shader. At
Cast Removal 0, a reconstructed highlight now renders the same neutral
white with and without the fade. Pixels below the anchor do not change.

Estimate clears the stored white before it renders its grid, so the
anchor does not cap the grid that a new white is read from.
test_slide_separation requires that features/exposure does not name the
fade. The unmix_log_image docstring and the shader comment now call the
anchor the density anchor.
A scanner TIFF or JPEG with Remove Stain on capped each pixel above its
white and passed the excess unrestored. The white of such a file is a
percentile of its clear film, so the light above it is film. Now only a
camera RAW, whose white sits below the decoder's full scale, has a cap.
The fade card pushed one frame's white to every frame of the roll. A
camera RAW's white moves with each frame's exposure, so other frames
scaled their density about the wrong level and took a cast. A stale
roll white also came back on each open and was measured again.

The roll now drops a white tagged with a decode space when it is set,
and ignores one already stored. A scanner white still follows the roll.
The tag hashed the whole camera matrix, primaries included. The white
is read in the white-balanced camera channels, before the primaries, so
an Input ICC changed the tag and made a good white stale. The preview
then measured it again, and export and thumbnails ignored it.
The fade white is a bounds input. A white measured again on the open
frame kept the cached bounds, which were read with the old white.
Every other edit of the white clears them.
…lock

A failed Estimate on a camera RAW stored the white through the roll's
fade card, which locked the card away from the roll. The roll holds no
camera RAW white, so the white now goes on the frame alone.
Only the open frame in the preview measured a stale or missing white,
on the GUI thread. Export, thumbnails and batch renders skipped it and
rendered without Remove Stain or the anchor until the frame was opened.

Each render now measures the white on the Estimate grid before the
density stage (resolve_fade_white, fade_white_grid). This applies to
the CPU and GPU engines and to the tiled export's reference pass. The
CPU exposure stage reads it too, for the Remove Stain target. The
desktop stores the white from the render's metrics with no render, no
undo step and no roll lock.

With a crosstalk unmix U, the white is read after U, so the restoration
now scales about U^-1 w. It scaled about w, in the wrong channels.
Crosstalk alone takes no white: it keeps a neutral neutral, so a
crosstalk-only frame renders as before.

fade_density returns the matrix and the anchor from one pass, and the
white's tag is computed once per render.
The roll holds no camera RAW white, but the Fade card's lock check
needed every card field in the roll. Any fade edit on a RAW frame then
locked the card, even when its survivals matched the roll's.

A roll's scanner white (space "") also replaced a RAW frame's measured
white on each open. The render measured it again and the card locked
as diverged.

frame_roll_defaults now drops the roll's white for a frame that holds
its own. The overlay, the lock check, the diverged-card scan and
Reset to Roll use it, and the lock check does not need a white the
roll does not hold. A scanner roll keeps its shared white.
…talk

The desktop stored a white the render measured without moving the
render memo or the render identity, so the first navigate-back
rendered again. A half frame could also get its own settings row. The
white now joins the measured-bounds write, which handles all three.

The white is read after the crosstalk unmix, so its tag now names the
unmix (render_white_space). A crosstalk change makes the white stale,
and the render measures it again.

The GPU engine keeps the last white it measured for a frame, so drag
ticks before the white is stored do not measure it again.

fade_anchor had no caller left and is removed.
relog_matrix took 10^v in float64 one pixel at a time, which kept the
loop scalar. It now works in runs of 256 pixels, one channel at a time,
with the same arithmetic in the same order, so the pins do not move.

A 24 MP frame on 4 cores: 0.45 s to 0.12 s. Every camera RAW slide
with a density correction, and every Positive source profile, runs it.
…he image grid

A camera RAW's density matrix carries its white-balance gains as a log
offset, so every faded RAW frame took a full-image NumPy add after the
unmix. unmix_log_anchored now adds the offset in the same pass, with
the same float32 add, so the render keeps its bits.

A render that measures a stale white prefiltered the full image a
second time. The white and the meters now share one prefiltered grid.

The slide base stage of a 24 MP frame on 4 cores, against the base
branch: fade 0.44 s to 0.52 s, fade with Remove Stain 0.65 s to 0.57 s,
a stale white 0.59 s, an unfaded frame unchanged at 0.42 s.
…order-96c3ad

# Conflicts:
#	negpy/features/process/fade.py
…esets

The roll keeps a camera RAW's fade white on its frame, but Paste, the
Film Strip's Apply, presets and sticky settings copied the whole Fade
Restoration row. Two frames from one camera with the same white balance
have the same white tag, so the target took the source's white as its
own anchor and rendered with a cast.

apply_selected_fields now leaves a white with a camera tag on the
target, as rolls._without_decode_white does for a roll. The survivals
and Remove Stain still copy, and a scanner white still copies.
Highlight Reconstruction rebuilds the clipped pixels that a camera RAW's
white is read from. The tag named only the white-balance gains and the
crosstalk unmix, so a stored white stayed in use after the level changed.
The tag now hashes the effective highlight mode, and a white stored with
the old tag reads as stale. A negative now takes the empty tag.
The lock check ignored the white fields that the roll did not hold on
every frame. A scanner frame's own Remove Stain white then matched a roll
with no white, so the card stayed unlocked and its Roll button had nothing
to push. The check now ignores those fields only on a camera RAW frame,
whose white stays its own.
…Fade card

A push that held a camera RAW white deleted the roll's white fields. The
scanner frames on a mixed roll then lost the scanner white they share. The
push now leaves out only the RAW white. A RAW frame still ignores the
roll's scanner white, through its tag and frame_roll_defaults.

Updates test_a_camera_raw_fade_white_stays_on_its_frame in
tests/test_rolls.py, which asserted the deletion: the roll now keeps its
scanner white and takes only the pushed survivals.
The Estimate grid is fade_white_grid, which never applies the white. With
the survivals at 1.0 and Remove Stain off, no white acts in that render.
The zeroed white fields and their comment are removed.

test_estimate_reads_the_grid_without_the_stored_white in
tests/test_fade_white_space.py asserted the zeroed fields. It now renders
the grid with two different stored whites and expects equal grids.
Blend is a third capture mode beside Trichrome and Single Exposure, as
in Lento's Dock Connect. The camera holds Bulb and the light shows red,
green and blue in turn, each for its Trichrome calibration time, so a
frame is one RAW.

- Shutter release: USB bulb through gphoto2, or Cable through the
  Scanlight's 3.5mm jack (H2D_SHUTTER_PULSE, at most 2.55 s).
- A Blend calibration saves a sensor crosstalk profile by default, and
  the frame's roll takes it with Linear RAW.
- A strip holds Bulb from frame to frame and moves the film while the
  camera writes the file. On an A7 IV with the Lento Dock 35 a frame
  went from about 9 s to about 4.5 s.
- The Dock switches the light without waiting for its reply, so each
  channel gets its time. Waited writes made a 200 ms red last 312 to
  703 ms; the no-wait writes held it to within 0.5%.
- The light stays dark 0.35 s after the Bulb press: the A7 IV opens its
  shutter by 0.3 s. scripts/bulb_lead_probe.py measures another body.
- The Dock takes host control when NegPy connects, so it finds frame 0
  of a strip inserted after that. Under AUTO it reported idx=-1.
- The Dock keep-alive reaches the dock while a command waits for a
  slow move, and an unplugged dock logs one warning, not a traceback
  per second.
- The preset list names each preset's capture mode, and the Film Strip
  marks Blend and Single Exposure frames.

On ten frames of one strip, Blend with its sensor profile matched
Trichrome within 1 to 2.6% RMS in R/G and B/G after a per-channel gain;
without the profile the error was two to three times larger.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiaS7pstGdewkrwmf2fset
The avision backend gives its Minolta film scanners one source,
"Normal", which maps to no film mode. These scanners have no RGBI mode
and no negative-film invert option, so NegPy skipped them with "no
recognised film sources".

SANE reports the device type "film scanner" for these models. Film
detection now reads that type from get_devices() before the other
signals. It also works through saned, because the net backend passes
the type on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
avision has no depth option. Its depth is part of the mode name:
"Color" is 8-bit, and "12bit Color" and "16bit Color" both deliver
16-bit samples. Film models that follow the vendor driver offer no
"Color" mode, so setting it failed with "Invalid argument". On other
avision scanners, a 16-bit scan delivered 8-bit data.

When a backend has no depth option, the offered depths come from the
color mode names, and a color scan picks the mode for its depth.
Backends with a depth option keep "Color".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Changes from the DOCK Serial API reference (fw 1.0.239, contract 1.15.0):

- Scan with Next Frame or Whole Strip feeds a strip that waits at the
  entrance (st=IDLE, entrance=1) with INSERT. In SEMI the dock feeds a strip
  only on INSERT, and a port open resets it. The strip no longer has to go in
  between connect and Live View.
- A command the dock answers with BUSY goes again every 250 ms until the
  dock's own exposure ends.
- CONTRGB and CONTHOLD accept a bare OK, which the reference gives as their
  reply.
- FRAMEAT ERR ends the strip, like ROLLEND.

Refs marcinz606#1375

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiaS7mdcgupjy3zYtMa4Su
With Highlight Recovery off, green clips and the white-balance gains
lift red and blue past full scale in the same cells. The white read
that clip color as the film's, so Remove Stain turned the frame green
(white +0.104, 0, +0.102 on a clipped sky). Each channel now stops at
full scale, as the decode with Recovery does, and the white is the
same at both levels.
Film Strip Sort and the contact-sheet Order gain a Frame No. option. It orders by the metadata Roll, then the Frame No., the file name a tie. The name, date and scene orders are unchanged.

Adds the label to the four catalogs and updates the frames and contact_sheet guide panels (en, de, ja, ko, zh_CN).

Co-Authored-By: Claude Code <noreply@anthropic.com>
cimelab and others added 9 commits November 4, 2026 12:29
Brings in BTNMODE JOG while NegPy holds the dock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiaS7mdcgupjy3zYtMa4Su
After a whole strip ejected, the dock answered nothing three minutes
later: STATE? and CONTRGB timed out with no line from the dock, and NegPy
kept the dead link. The dock sleeps after 90 s without work (SLEEP,
default 90). While NegPy holds the dock it now sets SLEEP 0 and puts the
saved timeout back on release. A command that times out with no line from
the dock for that long marks the link silent, so the next use opens the
port again, which resets the dock.

Refs marcinz606#1375

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiaS7mdcgupjy3zYtMa4Su
Brings in SLEEP 0 while NegPy holds the dock, and the reconnect of a
silent dock. is_connected checks both the lost link and the silent dock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiaS7mdcgupjy3zYtMa4Su
describe() reported no white channel, so the Scan panel hid the
white-light presets and white mode for the dock. set_color already lights
R, G and B together at one level for white, so the dock now reports white.
It still reports no infrared.

Refs marcinz606#1375

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiaS7mdcgupjy3zYtMa4Su
Brings in the white-light presets for the dock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiaS7mdcgupjy3zYtMa4Su
Brings in the white-light presets for the dock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiaS7mdcgupjy3zYtMa4Su
…e-conflicts-81fec0

# Conflicts:
#	negpy/desktop/controller.py
Add Norwegian Bokmål (nb) as a UI language.

- nb.json catalog with all 3327 messages
- USER_GUIDE.nb.md with all 44 panel guides
- System Default resolves "no" and "nn" system languages to Bokmål
- Count forms keep "singular|plural" nouns, as for German
- Qt standard buttons and file dialogs use the Nynorsk catalog
  (qtbase_nn), because Qt ships no Bokmål one
- Norwegian search synonyms in the command palette
- Norwegian Bokmål in the README and the user guide language lists
- Tests for language resolution, count forms, the export ratio combo
  and the panel guides
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CiaS7mdcgupjy3zYtMa4Su
@cimelab cimelab changed the title Feature/bulb blend capture PR3_bulb-blend-capture.md Nov 4, 2026
jboneng and others added 19 commits November 4, 2026 22:21
A new frame took the last chosen flat field profile and the carried
distortion from any source, so a camera rig's corrections applied to a
film scanner roll. The Optics card showed only while a correction was
on, so clicking the bulb changed the picture and hid the card.

A new frame now drops the carried Optics when Scanning Setup says Film
Scanner or the file is not a camera RAW. Values that the frame's roll
holds still apply. A film scanner frame leaves the carried values alone
when it saves, so the next camera frame keeps them.

The card stays in view after a correction turns off, until another
frame opens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Recovery has no effect on a file that is not a camera RAW, but a carried
value kept the Raw Decode card in view on a film scanner frame, where a
Slide TIFF could not change it.

A new film scanner frame now drops the carried Recovery, and saving that
frame leaves the carried value for the next camera slide. The card stays
in view after Recovery turns off, until another frame opens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of the first two commits found gaps:

- A saved film scanner frame took the last chosen profile on reopen,
  and a legacy profile's distortion with it.
- A profile picked or cleared on a film scanner frame replaced the
  profile that new camera frames take.
- A frame locked against its roll on Optics lost the card, so it could
  not rejoin the roll.
- Every persisted edit parsed the stored sticky config.

_apply_sticky_settings now takes the frame's path and skips the camera
settings there, which replaces the strip after the carry. The fields
sit in one constant, _CAMERA_ONLY_FIELDS. The file check runs only when
a camera setting would carry.

Deleting the last chosen profile and Shoot Flat Field still set it from
any frame.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The memory limits in Preferences are fixed numbers. An 8 GB machine can set a
32 GB cache with no warning, and a 128 GB machine stops at a 32 GB cache and 16 HQ
buffers.

Each cache row now shows a recommendation of half of the machine's RAM, or how
many entries of that row's size fit in that half. The user can go past it, and the
hint then turns amber. A row stops at its old limit or all of RAM, whichever is
larger. GPU texture cap goes to 65536 px, past the long edge of a 400 MP frame, with
no recommendation. Default values and the default HQ cap do not change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nner-device-type

fix(scanners): make avision film scanners work over SANE
…sity-order

fix(fade): restore a camera RAW slide in its own channels, about its own white
…-optics

fix(optics): keep camera settings off film scanner frames
…e-memory-budgets

feat(perf): size the Preferences memory limits to the machine
…okmal

feat(i18n): add Norwegian Bokmål translation
Fill the Norwegian catalog after the memory-limit and optics merges
…e-no

feat(files): add Frame No. sort and contact-sheet order
Resolve conflicts in session.py imports, the Film Strip guide marks paragraph and the file browser tests. Fill the vi and nb catalogs and the nb scan_light guide panel for Blend.
@marcinz606
marcinz606 merged commit 4918d42 into marcinz606:main Nov 5, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants