Conversation
commit 0c2b3790c9611f35381d7159c6dea2295e8eb51e
Author: AJ Slater <aj@slater.net>
Date: Thu Jun 26 15:01:52 2025 -0700
internal mozjpeg works, remove external jpeg methods
commit 2a79892c44daf358901c5b1afdac7297e73cbe0e
Merge: 3a01d55 a793e0b
Author: AJ Slater <aj@slater.net>
Date: Thu Jun 26 14:22:38 2025 -0700
Merge branch 'develop' into mozjpeg-lossless-optimization
commit 3a01d55856896d2e4fe94d56cdb60dfd00ffe640
Author: AJ Slater <aj@slater.net>
Date: Wed Jun 11 15:14:40 2025 -0700
use internal mozjpeg. currently broken on mac
- plugins.pick_route_handler is the one routing decision: the factory uses it at runtime and config/handlers uses it to build the startup formats summary, replacing an 8-parameter re-derivation whose logic had already drifted from the factory's (it ignored empty-pipeline availability). The log can no longer disagree with the walk. - PathInfo's class docstring documents the mutable-identity contract: instances cross the process boundary and are mutated by the scheduler after hydration, with cache invalidation owned by rename()/set_data(). - computed.handler_stages is structurally typed dict[type, tuple]; the concrete Handler/Tool annotation is impossible because plugins.base imports settings and basedpyright's reportImportCycles counts even TYPE_CHECKING edges. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgZ7kxNgvEWuSEKeTJMJcH
probe() caches its ToolStatus per instance and concrete probes move to _probe(). Tools are module singletons, so each external tool spawns its --version subprocess once per process — a prerequisite for per-directory config files, whose confuse rebuilds rerun the pipeline probe loop. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgZ7kxNgvEWuSEKeTJMJcH
Ported from nudebomb's implementation (same confuse + treestamps stack). A .picopt.yaml placed in a target directory applies to that directory and everything beneath it. Files are discovered from each processed file's directory up to the CLI target root (never above it); deeper directories win over shallower ones, and env vars and CLI options still win over every directory file — implemented by rebuilding a fresh confuse Configuration per unique directory chain through the extracted _build_config layering core. DirConfig caches resolutions per directory with an identity fast path when no config applies, and isolates malformed files (logged once, recorded in stats, fall back to run-wide settings). Run-mode keys (dry_run, list_only, timestamps) are pinned to run-level values: a per-directory dry run with run-level timestamps would stamp files that were never optimized. Per-directory settings reach everything: guards (ignore/symlinks/ recurse) via per-directory skippers and the pre-count mirror, handlers via the factory's settings parameter (formats/convert_to/bigger/ keep_metadata/... travel into workers), archive members by inheriting their container's config (nesting inherits automatically), and repacks run under the handler's own config. .picopt.yaml files are skipped as optimization targets by basename. With -t, a sha256 fingerprint of every .picopt.yaml under the targets joins the treestamps program-config check, so editing, adding, or removing a directory config re-processes its tree (over-invalidation, never a wrong-skip). New write flags, all validate-before-persist with round-trip YAML merges that preserve comments and unrelated keys, written owner-only: -w/--write-config (user config), -W/--write-dir-config (a .picopt.yaml in each target directory), and --write-config-file PATH. Run-mode flags and target paths are never persisted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgZ7kxNgvEWuSEKeTJMJcH
- --fail-fast cancels every live top-level container and clears both scheduler queues on the first repack error; --fail-fast-container escalates an inner failure to its subtree root while leaving sibling top-level containers untouched; a leaf result arriving after its parent's cancellation is dropped without touching the parent's contents. - Optimization converges: a second pass may only shrink files (the multi-tool png pipeline legitimately shaves another byte when one tool's output enables another) and every file reaches its fixed point by the third pass. - Lossless paths are pixel-exact: png optimization, png-to-webp, and gif-to-png conversions decode to identical RGBA pixels. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgZ7kxNgvEWuSEKeTJMJcH
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MgZ7kxNgvEWuSEKeTJMJcH
Fill in five user-facing fixes that landed since v6.5.2 but were absent from the v6.6.0 entry: archive conversions recompress members by content, --preserve survives non-root chown failures, temp files and animated-WebP frame dirs are cleaned up on failure/cancellation, timezone-aware --after is honored, and -I without -i no longer aborts at startup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…yaml The verbose "Optimizing formats" banner is a run-wide summary, but it was emitted from set_format_handler_map(), which _build_config() re-runs for every per-directory .picopt.yaml resolution during the walk. With -W (or any planted directory config in the tree root), every directory finds a config in its ancestry, skips the "reuse run-wide settings" fast path, and rebuilds the config — reprinting the banner once per directory. Gate the banner behind a print_summary flag so only the top-level get_config() build emits it. Move the summary computation into _log_formats_summary() so the per-directory rebuilds skip it entirely. Add a regression test asserting the banner prints exactly once under -W. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
img2webp rejects any frame duration <= 0 ("Invalid negative duration"),
aborting the whole animated-WebP repack. Some sources (e.g. scraped
animated WebPs) carry 0ms frame durations, so clamp each duration up to
img2webp's 1ms minimum in Img2WebPAnimatedLossless.img2webp_args.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Use a real PicoptSettings (not SimpleNamespace) in the img2webp duration test so config assignment type-checks. - Mark the module-level pytest autouse fixtures with `# pyright: ignore[reportUnusedFunction]`; they are used by pytest, not referenced by name. - Disable reportImplicitStringConcatenation: ruff format canonicalizes adjacent string literals to implicit concatenation, so the check can never be satisfied. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
radon cc flagged four functions at rank C. Extract behavior-preserving helpers to bring each to rank A/B: - WalkSkipper._is_skippable (16): split into _skip_reason and the disk-only _skip_reason_fs. - Walk._count (12): extract the recursion-stop predicate _count_stops_here. - pick_route_handler (12): fold the convert gate into _pick_convert_handler and extract _can_pack_repack for the repack check. - Handler._cleanup_after_optimize (11): extract the _should_replace predicate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per-directory .picopt.yaml resolution rebuilds the confuse config for every directory in the tree, repeating the timestamps, ignore, after, and memory-limit log lines each time. Gate them on the existing print_summary flag, as set_format_handler_map already does. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The treestamps program config was recorded from the run-level CLI config, so identical options expressed via a tree root's .picopt.yaml instead of flags always mismatched and every stamp was silently discarded, re-optimizing whole trees. Each tree now records its own effective config: the new Grove builds one Treestamps per top path from DirConfig.get_tree_settings, which resolves the root's .picopt.yaml beneath CLI and env. timestamps: true at a tree root now enables stamping without -t (dry run and list still disable it, and subdirectory configs still can't toggle it mid-walk). The directory-config fingerprint is now per tree and value-based: it hashes the parsed picopt section of each config strictly below the root, so comment and formatting edits, -W rewrites, running with a different set of top paths, and file-vs-directory targets of the same tree no longer invalidate stamps — only real option changes do. The root's own config is excluded because its options are recorded as values. Stamps discarded on a config mismatch now warn with the differing keys (treestamps 4.1.1); stdlib logging is intercepted into loguru so the warning renders through the shared console, and log messages escape Rich markup so bracketed names like "Title [en]" display intact. Migration: existing stamp files mismatch once after this upgrade (now loudly); a single -N run preserves old stamps instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T7R5s6frL7jEykysAzjttq
PIL raises a plain OSError when it recognizes an image format but the contents are corrupt (e.g. a BMP with pixel depth 0), which escaped the detection probe and printed a full stack trace. Detection now wraps it in UnreadableImageError, reported as a one-line warning that is also counted and listed in the summary; the file is skipped untouched. The per-file exception guards now skip the stack trace for any PicoptError — extending the contract cli.main already had: expected conditions print their message, only unexpected exceptions dump stacks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01T7R5s6frL7jEykysAzjttq
- Drop the `cls: type[Any]` annotation on the RarDetector/ZipDetector `identify` classmethods; ty flagged the widened first parameter as an invalid override of `Detector.identify`. - Replace remark's `no-duplicate-headings` with `no-duplicate-headings-in-section`. A changelog necessarily repeats `### Fixes` under each version; the in-section rule still catches real duplicates. Also includes the in-progress dependency bumps and ruff rule additions that were already in the working tree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Add JPEG XL support Read and write JPEG XL via the pillow-jxl-plugin codec, as a new auto-discovered plugin. JXL is optimized by default and is now the preferred still conversion target ahead of WEBP and PNG. Only lossless JXL is re-encoded; lossy (XYB encoded) JXL is left alone, the same policy lossy WebP already has. The codec exposes no such signal, so picopt/pillow/jxl.py parses the codestream image header to read xyb_encoded, and refuses files carrying JPEG reconstruction data, whose re-encoding would silently destroy the ability to restore the source JPEG. New --convert-jpeg-to-jxl converts JPEG to JXL by lossless bitstream reconstruction, which is reversible byte for byte. It requires -c JXL as well: picopt otherwise never converts from a lossy source, so -c JXL alone must not sweep up a whole photo library. The codec only takes its reconstruction path when PIL knows the source filename, which it does only for path opens, so the tool opens by path and spills to a temp file for inputs that have none. That makes it work for archive members too. Three smaller things the codec needed: - It registers no save_all handler, and pil_save always passes save_all, which raised KeyError. The single-frame handler is registered for it; animated JXL is unsupported by the codec regardless. - Given no exif kwarg it reads EXIF off the opened image, which quietly defeated --strip-metadata. prepare_info now passes an empty exif. - It rejects palette and other modes, so ImageHandler gained a prepare_image hook, symmetric with prepare_info, to substitute one. Handlers can now declare CONFIG_ENABLED_KEY, a config flag that must be true for them to be probed at all. Failing it leaves the handler out of the probed handler map, which routing already reads as unavailable, so the startup conversion banner stays accurate for free. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Gate WebP to JXL conversion behind --convert-webp-to-jxl JPEG XL support in browsers and image tooling is still thin, so converting away from WebP, which every browser reads, should be a deliberate act rather than a side effect of asking for -c JXL. JPEG and WEBP are both formats picopt processes by default, which means the usual "name the source with -x" gate cannot express a choice about them: -x JPEG and -x WEBP are no-ops. Each therefore carries its own flag, and a plain -c JXL once again converts only GIF and PNG, as the README has always described. The previous commit had quietly made WEBP a third convert-by-default source. The flag hangs off the handler class, and JxlLossless also serves PNG, GIF and native JXL, which must stay ungated. So the WebP route now points at a JxlFromWebP subclass that exists only to carry CONFIG_ENABLED_KEY. Converting out of JXL into a more widely readable format stays ungated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * update deps * update deps * update deps --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Features
pillow-jxl-plugin module.
.jxlfiles that previous versions ignoredare re-encoded on a plain
picopt -rrun. Only lossless JXL is touched; lossy(XYB encoded) JXL is left alone, as lossy WebP is.
-c JXL, which now leads the preferenceorder for still images ahead of WEBP and PNG.
--convert-jpeg-to-jxlconverts JPEG to JXL losslessly and reversibly:the JXL stores the original JPEG bitstream and can restore it byte for byte.
Files carrying that reconstruction data are never re-encoded, so the original
JPEG stays recoverable.
--convert-webp-to-jxlconverts lossless WebP to JXL. Off by defaultbecause JXL support is still much thinner than WebP's.
-c JXLas well. JPEG and WEBP are formats picopt alreadyprocesses by default, so unlike BMP or TIFF they cannot be gated by naming
them with
-x. A plain-c JXLstill converts only GIF and PNG.