Optimize performance with parallel processing and faster compression settings - #2
Optimize performance with parallel processing and faster compression settings#2munsonbh wants to merge 2 commits into
Conversation
…settings - Add parallel image processing using threading (default: CPU core count) - Convert JPEG XL and WebP simultaneously for each image - Reduce JPEG XL effort from 9 to 7 (faster with minimal size difference) - Reduce WebP method from 6 to 4 (faster with minimal size difference) - Skip files already in optimized formats (JXL/WebP) - Add --workers flag to control parallelism - Update README with performance optimizations documentation Expected speed improvements: 4-8x faster overall processing time
The exc_info=True parameter only works within an active exception handler. Moved exception logging with full traceback to worker thread where exceptions occur, and removed exc_info=True from main thread where we only have error message strings.
|
Closing this PR to fix a critical issue with hang detection in the parallel processing path. Issue FoundMissing Hang Detection in Parallel Path The multi-threaded processing path (when Specific Problems:
Impact: Fix AppliedAdded Comprehensive Hang Detection:
The parallel path now has feature parity with the single-threaded path's hang detection, preventing indefinite blocking and properly alerting users when processing stalls. A new PR will be opened with this fix included. |
Summary
This PR implements several performance optimizations to significantly speed up image processing:
Changes
--workersflag to control parallelismPerformance Improvements
Testing
Documentation
--workersflag