imshow and subplots: images beside curves, for GAN-style training loops - #14
Open
davidquarel wants to merge 4 commits into
Open
davidquarel wants to merge 4 commits into
davidquarel wants to merge 4 commits into
Conversation
ARENA 0.5 shows GAN and VAE samples through `LiveImage`, an ipywidgets.Image in
an output area of its own -- a widget, which is the one thing liveplot exists to
avoid, and separate from the loss curves, so you cannot watch samples sharpen
next to the losses. This is the liveplot side of replacing it.
`LivePlot.subplots(1, 2)` mirrors `plt.subplots` and returns (plot, axes) with
matplotlib's squeeze rules. A panel holds curves via `ax.plot("lossD", "lossG")`
or a picture via `ax.imshow(tensor)`, which replaces whatever was there. That is
matplotlib's own `ax.plot("lossD", data=d)` form with the source implicit, and
`ax.twinx()` rather than a bespoke kwarg for the right-hand axis. `plot.imshow(x)`
alone is the whole of LiveImage in one call.
`_images.to_grid` does the shape reading, tiling and scaling as pure numpy, on
the calling thread: ~0.19 ms for ten 64x64 RGB samples, and it quarters what the
render process unpickles. It is forced anyway, since a CUDA tensor cannot be
pickled to another process. Torch is duck-typed, never imported.
Two findings shaped the design, both from checking matplotlib rather than
assuming:
- imshow refuses channels-first outright, so it has no (3, H, W) ambiguity to
solve. We must accept it, so we raise on the two shapes that are genuinely
ambiguous and name the fix. (1, H, W) is not one of them: both readings draw
the same pixels.
- imshow IGNORES vmin/vmax for RGB(A) data and merely clips floats to [0, 1] --
identical on 3.9.0 and 3.11.2. A generator ending in tanh outputs [-1, 1], so
44.8% of it would go to black with no way to say otherwise. So liveplot scales
values itself, which also makes vmin/vmax uniform across colour and grayscale.
One test is skipped on matplotlib 3.9.0-3.9.3, where any RGB draw from the
spawned render process dies inside the C resampler while the same array in the
same figure is fine in the parent, and which flips to passing when perturbed as
transparently as wrapping matplotlib.image._resample. That is memory corruption,
not logic; fixed in 3.9.4. See IMSHOW_REPORT.md.
58 passed on matplotlib 3.11.2; 57 passed, 1 skipped on 3.9.0. No existing test
changed. IMSHOW_REPORT.md is the handoff, including what is left undone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hz3Btx6kkABaf8LzKM7s9b
…, griddim - to_grid widens floating tensors before np.asarray: bfloat16 (autocast output) has no numpy dtype and raised TypeError. - padding=2 pixels of pad_value around every image, as torchvision's make_grid lays a grid out; a single image gets no border. - An image panel's title says the step it was drawn at, "samples (step 399)", since the picture only changes when imshow is called. - grid_size -> griddim=(rows, cols). - docs/imshow.gif: subplots(1, 2), loss beside a toy generator's samples. - IMSHOW_REPORT.md (the handoff note) leaves the branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ios, and a synthetic demo What splicing liveplot into ARENA [0.5]'s DCGANTrainer needed, in the names the code already uses: - plot.update(n) / plot.set_description(desc): tqdm's manual mode, for a loop that drives one bar for the whole run (progress_bar.update() in the trainer). The first update() opens the bar below the figure; finish() closes it with the final postfix. - ax.legend(**kwargs) -> Axes.legend, so a six-entry legend can move off the curves. - LivePlot.subplots(width_ratios=, height_ratios=), as plt.subplots; layout messages now carry the grid options, since subplots sets them after the render process has started. - examples/dcgan_synthetic.py: the trainer's shape with fake losses (lossD from ln 4 down, lossG from ln 2 up, D(x) / D(G(z)) on a right-hand axis, smoothed) and ten real CelebA faces coming out of noise as the fake generator's samples, fetched as 70 kB through the HF dataset viewer. Runs live in a notebook / interactive window, or records a GIF as a script. Checked in a real kernel: one image output holding both panels, and the tqdm bar underneath. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…, capped, image panels fitted - Tiles sit edge to edge by default (padding=0); padding= / pad_value= still give make_grid's gaps. - Default grid: near-square, rows first (3 -> 2x2, 5 -> 3x2, 8 -> 3x3, 64 -> 8x8), at most max_cols=8 wide; rows= / cols= / griddim= override, and are never capped. - max_images=64: a bigger batch shows its first 64, with a warning naming the override; max_images=None shows everything. - One-row figures with an image panel and no width_ratios: the image panel is sized to its picture at the row's height and the curve panels share the rest of figsize, so the samples fill their panel instead of floating in white space. A picture of a new shape re-fits. - The step goes in an image's title only once the plot is counting steps. - dcgan_synthetic: nine samples, a 3x3 grid by default. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013EWhdsyzPSUfHKMjcWif8D
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Images can now sit in the same live figure as the curves, replaced in place on each call. That's what the [0.5] DCGAN day needs:
lossD/lossGupdating every step, next tonetG(fixed_noise)every few hundred steps, in one output with the tqdm bar underneath. It replaces the chapter'sLiveImageipywidget.API, in matplotlib's and tqdm's names
LivePlot.subplots(nrows, ncols, figsize=, width_ratios=, height_ratios=)returns(plot, axes), with squeeze rules likeplt.subplots.ax.plot("lossD", "lossG")names the metrics on an axis,ax.twinx()gives the right-hand axis, andax.legend(**kwargs)passes through toAxes.legend.ax.imshow(x, rows=, cols=, griddim=, vmin=, vmax=, scale_each=, channels=, padding=2)takes numpy or torch, channels first or last, batched or not. A batch is tiled likemake_grid.(3, H, W)and(4, H, W)are ambiguous, so they raise and say whichchannels=to pass. liveplot does its own scaling because matplotlib ignoresvmin/vmaxfor colour images. The panel title shows the step the image was drawn at.plot.imshow(x)is the one-panel shorthand.plot.update(n)andplot.set_description(desc)are tqdm's manual mode, for a trainer that drives one bar across all epochs.Demo:
examples/dcgan_synthetic.pyhas the course trainer's shape, with fake losses and ten real CelebA faces coming out of noise. Run it cell by cell for the live view; run it as a script and it records a GIF. I checked it in a real Jupyter kernel: the cell has exactly one image output holding both panels, with the tqdm bar underneath.Caveat: matplotlib 3.9.0–3.9.3 has a bug that breaks drawing colour images in the spawned render process. The affected test is skipped on those versions; 3.9.4+ is fine.
62 tests pass.
🤖 Generated with Claude Code