Skip to content

Optional AI upscaling, for sharper results at 4x #14

Open

Description

@ftymoshchuk

Upscaled images (#9) resamples with Lanczos (src/pasteup/services/upscaler.py). Edges stay smooth, but no detail is added, so clip art upscaled 4x looks soft. AI upscalers such as Real-ESRGAN, which Upscayl is built on, redraw detail and do much better on illustrations.

Proposal: use Real-ESRGAN when it's installed, the way preview videos use ffmpeg, so everything still works without it.

  • Look for the realesrgan-ncnn-vulkan binary (builds exist for Windows, macOS and Linux) as videos look for ffmpeg, and mention at startup when it's missing, as pasteup does for ffmpeg.
  • In the Upscaled images tile, offer a choice of method only when the binary is found, keeping the built-in one as the fallback.
  • Run it per image, into the same Upscaled/PNG and Upscaled/JPG folders. Pasteup still flattens the JPG onto white and stamps the DPI.

To find out first:

  • It runs on the GPU through Vulkan. What does it do on a machine without a suitable one: fail, or fall back to the CPU? The result should say which.
  • Does it keep a transparent PNG's alpha channel, or must the alpha be upscaled separately, as Real-ESRGAN's Python version does?
  • Which of its models suits clip art best, and which sizes each offers natively; 2x and 3x may need its 4x output scaled down.
  • The licences of the binary and its models. Like ffmpeg, it would be installed by the user rather than shipped with Pasteup.

Tests can use a stand-in binary; the real one can't run in CI, which has no GPU.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions