Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
c571670
feat(animation): derive a GIF from a video and fix the clip palette
MAfarrag Aug 24, 2026
a3f6e0b
test(animation): cover the palette helpers and gif_from_video edge cases
MAfarrag Aug 24, 2026
438c135
docs(animation): add examples to the clip-palette helpers
MAfarrag Aug 24, 2026
91bdd2a
fix(animation): build the clip palette from its colours, not a downsa…
MAfarrag Aug 24, 2026
bd62e7a
perf(animation): stream gif_from_video instead of buffering the clip
MAfarrag Aug 24, 2026
1bdcb2d
fix(animation): validate writer inputs instead of failing deep in Pillow
MAfarrag Aug 24, 2026
d44ce04
fix(animation): resolve ffmpeg the same way for reading as for writing
MAfarrag Aug 24, 2026
4c42983
fix(animation): classify pixel formats by family, not by prefix
MAfarrag Aug 24, 2026
eba11a1
refactor(animation): annotate the new helpers and publish the palette…
MAfarrag Aug 24, 2026
5b9e874
test(animation): tighten the weak assertions in the new tests
MAfarrag Aug 24, 2026
d3a46b1
style(animation): drop reformatting of tests unrelated to this change
MAfarrag Aug 24, 2026
a6c2c2d
feat(animation): let callers choose the GIF palette strategy
MAfarrag Aug 24, 2026
e5f329e
docs(scope): admit re-encoding cleopatra's own animation output
MAfarrag Aug 24, 2026
dbdb553
docs(animation): show the palette strategy and say what median actual…
MAfarrag Aug 24, 2026
fcf0c7d
fix(animation): correct the colour census and the claims made for it
MAfarrag Aug 24, 2026
f5a2e2e
test(animation): decouple the video fixture and cover the corrected p…
MAfarrag Aug 24, 2026
8a34c6b
docs(animation): state the memory cost and what the strategies weight
MAfarrag Aug 24, 2026
56bdeae
fix(sonar): clear the quality-gate findings on the animation writers
MAfarrag Aug 24, 2026
197fbee
test(animation): build the empty stream outside the raises block
MAfarrag Aug 24, 2026
b82e673
Merge branch 'main' into feat/gif-palette-quality
MAfarrag Aug 24, 2026
b4be343
chore(deps): relock after raising the imageio-ffmpeg floor
MAfarrag Aug 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 35 additions & 8 deletions SCOPE.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,8 @@ colorbars, ticks, classification, and animation.
- `tiles` and `reference` are cleopatra's **only** networked features.
- `projection`: lightweight axes-frame / coordinate helpers.
- `animation`: turn a matplotlib `FuncAnimation` into a saved file, GIF bytes,
or an embeddable IPython image (via ffmpeg).
or an embeddable IPython image (via ffmpeg), and derive one output format from
another (`gif_from_video`) — see "Animation output".
- `config` (`Config`): opt-in matplotlib-backend selection; notebook detection.

### What new work generally belongs here
Expand All @@ -85,13 +86,14 @@ colorbars, ticks, classification, and animation.
- **Data I/O and formats:** reading/writing *user* GeoTIFF, NetCDF,
shapefiles, GeoJSON, CSV, databases. Users bring NumPy arrays already;
file/raster I/O of user data belongs in sibling packages (e.g. `pyramids`),
not here. The deliberate exception is the `tiles` / `reference` basemap
not here. Three deliberate exceptions: the `tiles` / `reference` basemap
helpers, which fetch a handful of *fixed public* reference datasets (never
user data) that cleopatra re-hosts as dependency-light artifacts — see
"Supporting utilities". Reading a **presentation asset** — a logo / watermark
image for `styling.watermark.stamp_mark` — is likewise allowed: it is
decoration on the rendered figure, not user data, and it loads via Pillow (an
existing dependency), never GDAL/geopandas.
"Supporting utilities"; reading a **presentation asset** — a logo / watermark
image for `styling.watermark.stamp_mark` — which is decoration on the rendered
figure, not user data, and loads via Pillow (an existing dependency), never
GDAL/geopandas; and re-encoding cleopatra's **own animation output** between
formats — see "Animation output" below.
- **GIS / geoprocessing:** reprojection of user data, clipping, resampling,
zonal stats, CRS management beyond what the optional `tiles` basemap needs.
- **Interactive / GUI apps:** dashboards, widget servers, event callbacks,
Expand All @@ -110,13 +112,38 @@ colorbars, ticks, classification, and animation.
than the `tiles` / `reference` basemap helpers, and general-purpose plotting
that matplotlib already does well without added value.

## Animation output

Rendering frames is by far the most expensive part of an animation — hours, for a
long scientific clip — while encoding them is cheap. Forcing every output format
to be produced from a live `FuncAnimation` therefore means re-rendering the same
frames once per format, which is the wrong trade at any real size.

So a helper may **read back a rendered video** and re-encode it to another
supported format, as `gif_from_video` does. The intended input is cleopatra's own
output, produced by `save_animation` moments earlier. Nothing in the code
enforces that — it decodes whatever FFmpeg can read, and a check would buy
nothing but a worse error message — so this is a statement of *purpose*, not a
guarantee about the argument.

What keeps it inside the line is what it does not do: it reads no user dataset in
an analytical format, opens no GIS format, exposes no transcoding matrix, and
adds no dependency — the FFmpeg it decodes with is the one `save_animation`
already encodes with. What it returns is an animation, not data.

The boundary that still holds: cleopatra does not become a general
media-conversion tool. A helper whose purpose was ingesting arbitrary user video,
or that grew a codec/container matrix, would be out of scope.

## Boundary heuristic for a feature request

Ask, in order:

1. **Input** — does it start from in-memory NumPy data (not a file/CRS/URL)?
(The `tiles` / `reference` basemap helpers are the deliberate exception:
they acquire fixed *public* reference data, never user files.)
(Three deliberate exceptions: the `tiles` / `reference` basemap helpers,
which acquire fixed *public* reference data, never user files; a presentation
asset such as a logo for `stamp_mark`; and the animation re-encoders, whose
input is cleopatra's own output — see "Animation output".)
2. **Output** — does it produce a matplotlib `Figure`/`Axes`/artist (or an
animation of one)?
3. **Reuse** — can it build on `Glyph` and the shared colour/colorbar/legend
Expand Down
63 changes: 63 additions & 0 deletions docs/reference/animation.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,69 @@ machinery as **glyph-independent** helpers. They operate on *any*
IPython is imported **lazily** (and is bundled with Jupyter, so any notebook already has
it); if it is absent, `embed_gif` raises a clear `ModuleNotFoundError` with a
`pip install ipython` hint — or use `to_gif` for raw bytes with no IPython dependency.
- `gif_from_video(src, path, fps=12, width=None, max_colors=254, ...)` derives a GIF from a
video **already on disk**, without re-rendering. Drawing is usually far more expensive than
encoding, so a long clip is best rendered once to MP4 and every other format derived from
that file.

## The GIF palette

Both GIF paths — `save_animation` and `gif_from_video` — quantise through one palette shared
by every frame, built by `build_clip_palette` from the colours the whole clip contains and
applied by `quantize_to_palette`. Both are public, so a downstream package writing its own
frames can reuse the same table rather than re-deriving one. Per-frame
palettes would make constant regions shimmer and let a colour drift between frames; two of
the 256 entries are pinned to pure black and white so single-colour overlays stay crisp.

The palette is chosen for colour **coverage**, not pixel population, over the set of colours the
clip contains. The distinction matters on exactly the clips this package produces: with a
population-weighted split (median cut) a large textured background claims nearly every palette
slot, and small saturated marks — overlay glyphs, thin paths, labels — collapse to the nearest
muddy neighbour. On the texture-heavy clip in `tests/test_animation.py` those marks landed 100–180 away (in RGB
distance) from the colours they were drawn in; selecting for coverage reproduces them exactly, and
`TestClipPaletteQuality` asserts it — so the claim is checked, not remembered.

Because coverage is computed over **distinct colours rather than pixels**, a mark survives no
matter how small it is: a one-pixel orbit path is kept as faithfully as a large glyph. Sampling
the frames spatially to build a cheaper palette source would undo that — an interpolating resize
blends a one-pixel mark into its background before the quantiser ever sees it.

The trade is a marginally coarser background, because palette entries now go to colours the clip
contains rather than to the colours it contains *most of*, and file size moves either way depending
on the clip. Both were measured while developing this and neither is asserted by a test, so treat
the direction as reliable and the magnitude as indicative.

`quantize_method` is the opt-out, on both `save_animation` and `gif_from_video`. It takes a key of
`QUANTIZE_METHODS` — `"coverage"` (the default), `"median"`, or `"octree"`. Reach for `"median"` on
a smooth photographic clip with no small marks at stake: it splits the colour cube by how densely
the clip populates it, so the crowded regions a photographic background occupies win the table.
Note none of these see pixel counts — the palette is built from each colour once, so they weight by
distinct colours, not by area:

```python
save_animation(anim, "clip.gif", fps=12, quantize_method="median")
```

!!! warning "Render the intermediate with `pix_fmt="yuv444p"` if a GIF will be derived from it"

`save_animation` writes `yuv420p` by default — the right choice for playback compatibility,
but it stores colour at half resolution in each direction. That loss happens *before* the
GIF palette ever runs, and no quantiser can undo it: on the same test clip a `yuv420p`
intermediate caps the derived GIF at ~50 RGB distance, against ~5 from a `yuv444p` one.
`gif_from_video` emits a `UserWarning` when it is handed a subsampled source.

!!! note "Memory"

`gif_from_video` decodes the source twice rather than holding it, so the decoded RGB frames are
never all resident. The quantised frames still are — Pillow's GIF encoder accumulates every
frame before writing its first byte. Expect roughly `width × height × frames` bytes at peak: a
third of what the RGB frames would cost, but still proportional to the clip's length. Use
`width` to bring a long master down.

```python
mp4 = save_animation(anim, "master.mp4", fps=12, crf=0, pix_fmt="yuv444p")
gif_from_video(mp4, "web.gif", fps=12, width=720)
```

`SUPPORTED_VIDEO_FORMAT` is `["gif", "mov", "avi", "mp4", "webp"]`. `Glyph.save_animation`
delegates to `save_animation`, so the writer/format logic has a single source of truth.
Expand Down
6 changes: 5 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,11 @@ dependencies = [
"numpy>=2.0.0",
"matplotlib>=3.9",
"pillow>=12.1.1",
"imageio-ffmpeg>=0.4.9",
# 0.6.0 is the floor the suite actually exercises. gif_from_video reads
# frames back through read_frames with output_params and relies on the
# reported size being post-filter, none of which the previous 0.4.9 floor
# was ever tested against.
"imageio-ffmpeg>=0.6.0",
"hpc-utils>=0.1.4",
]

Expand Down
Loading
Loading