diff --git a/CHANGELOG.md b/CHANGELOG.md index 2fd4a71..1bc71f6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,27 @@ All notable changes to Pasteup are recorded here. The format follows ## [Unreleased] +### Added +- **Browse…** beside the folder field opens a folder browser, so a folder can be + picked instead of typed. It opens at the folder in the field, or at your home + folder, says how many images the folder you're in holds, and scans the one + you choose. Typing or pasting a path still works. + (`POST /api/jobs/browse-folder`) +- Upscaled images, from a tile of their own in step 3: every image of a bundle, + or of each picked bundle, made 2x, 3x or 4x bigger and saved in an + `Upscaled` folder as a PNG, which keeps any transparency, and a JPG on + white. (`POST /api/jobs/upscale-images`, `POST /api/jobs/upscale-images-batch`) + +### Fixed +- Hidden files are no longer counted as images. A Mac writes a hidden `._` + file beside the files it copies to an exFAT or FAT32 drive or to some network + shares, so such a bundle could scan as twice its size, and making sheets then + reported the extra files as unreadable. +- Hidden folders, such as the `.Spotlight-V100` and `.Trashes` a Mac keeps on + those drives, and the `__MACOSX` folder left by unpacking a zip made on a + Mac, are no longer offered as bundles. Covers no longer take images from + them, nor from a `---Created---` folder inside the bundle. + ## [0.1.0] - 2026-10-16 The first release on PyPI. diff --git a/README.md b/README.md index fadf876..78aa313 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,8 @@ publishing: cut out by hand and pasted onto a layout board. main image. - **Preview video:** an MP4 slideshow of the sheets, exactly as long as you ask (Etsy's listing videos run 5 to 15 s). +- **Upscaled images:** every image made 2x, 3x or 4x bigger, saved in an + `Upscaled` folder as a PNG, which keeps any transparency, and a JPG on white. - **Clean-up:** transparent PNGs land on white instead of black, phone photos are turned upright, and near-white backgrounds can be removed. - **Batch mode:** process every bundle folder under a parent folder in one go. @@ -114,7 +116,8 @@ PyPI trusts `release.yml` running in this repository's `pypi` environment. Project layout: - `src/pasteup/main.py`: the FastAPI app - `src/pasteup/routes/`: the page and the `/api/jobs` endpoints -- `src/pasteup/services/`: scanning, page composition, collages, export, video +- `src/pasteup/services/`: scanning and browsing folders, page composition, + collages, export, video - `src/pasteup/models/`: request and response schemas - `src/pasteup/templates/` and `src/pasteup/static/`: the UI - `src/pasteup/cli.py`: the `pasteup` command; `shortcut.py` makes its launchers @@ -131,6 +134,12 @@ Input scan supports the following image extensions (case-insensitive): Notes: - Folder scan is **non-recursive** (only files directly inside the selected folder). +- Hidden files, whose names start with a dot, are skipped. Most are the `._` + files a Mac writes beside the files it copies to an exFAT or FAT32 drive or + to some network shares: they carry the image's extension but hold no image. +- Hidden folders, `__MACOSX` (left by unpacking a zip made on a Mac) and + `---Created---` are never offered as bundles, and covers don't look inside + them. - Export output format supports `PNG` and `JPG`. --- @@ -149,6 +158,7 @@ Default UI values: - `background_tolerance`: `240` - `background_feather`: `1` - `include_partial_page`: `false` +- upscaling `scale`: `2` (also `3` or `4`) Behavior defaults: - Scan sample size: first `5` files (`sample_files`) in sorted order. @@ -186,6 +196,34 @@ Validates a folder and scans for supported image files. - `estimated_pages` (int): `image_count // 12` (whole pages only). - `message` (str): summary or error detail. +### `POST /api/jobs/browse-folder` +Lists the folders inside a folder, for the folder browser behind **Browse…**. A +page can't read the path of a folder picked with the browser's own file +chooser, so the UI browses through this instead. + +**Request body** +```json +{ + "folder_path": "~/Pictures" +} +``` +A blank `folder_path` lists the home folder. + +**Response fields** +- `path` (str): the folder listed. One that does not exist or cannot be opened + gives way to the nearest folder above it that can, and `message` says why. +- `parent_path` (str | null): the folder above it; `null` at the top of a drive. +- `breadcrumbs` (list): `name` and `path` of each folder from the top of the + drive down to this one. +- `folders` (list): `name` and `path` of each folder inside, sorted, leaving + out hidden folders, `__MACOSX` and `---Created---`. +- `image_count` (int): supported images directly inside, counted as + `scan-folder` counts them. +- `places` (list): shortcuts: the home folder, its Desktop, Documents, + Downloads and Pictures folders when it has them, and each drive on Windows. +- `message` (str): empty unless a different folder was listed than the one + asked for. + ### `POST /api/jobs/scan-selected-batch-folders` Validates selected batch subfolders under a scanned parent folder and reports which selected folders are ready for generation. @@ -364,13 +402,79 @@ Makes one cover per bundle folder, as `/generate-all-folders` does for sheets. `pages_generated`, `total_images`, `output_path`, `output_files` and `message` - `message` (str) +### `POST /api/jobs/upscale-images` +Makes every image of a folder bigger, saving each as a PNG and as a JPG in an +`Upscaled` folder inside `output_path`: `Upscaled/PNG/.png` and +`Upscaled/JPG/.jpg`. The PNG keeps any transparency; the JPG is +flattened onto white. Images are resampled with Lanczos, which keeps edges +smooth but adds no detail. + +**Request body** +```json +{ + "folder_path": "/path/to/images", + "output_path": "/path/to/output", + "scale": 2, + "dpi": 300 +} +``` +- `scale` (`2` | `3` | `4`, default `2`): how many times bigger each side becomes. +- `dpi` (int, 1-1200, default `300`): stamped into every file. +- Each file keeps its image's name, so upscaling again replaces what the last + run made. +- Files sharing a name in different formats, such as `art.png` and `art.jpg`, + would be saved over each other, so one gives both formats: PNG first, then + WebP, then JPG. +- Upscaled images are capped at 100 megapixels; an image that would pass that + is left out and named in the response. + +**Response fields** +- `success` (bool): whether any image was upscaled +- `scale` (int) +- `total_images` (int): images upscaled +- `output_path` (str): the `Upscaled` folder +- `png_folder`, `jpg_folder` (str): the two folders inside it +- `output_files` (list[str]): every file written +- `images_skipped` (int): files that could not be read as images +- `too_large_files` (list[str]): images left out for passing the cap +- `same_name_files` (list[str]): files left out for sharing a name with the + image that was used +- `message` (str) + +### `POST /api/jobs/upscale-images-batch` +Upscales each selected bundle into `//Upscaled`. + +**Request body** +```json +{ + "parent_folder_path": "/path/to/parent-folder", + "created_output_root_path": "/path/to/created-output-root", + "selected_subfolders": ["/path/to/parent-folder/bundle-a", "/path/to/parent-folder/bundle-b"], + "scale": 4, + "dpi": 300 +} +``` + +**Response fields** +- `success` (bool) +- `created_output_root_path` (str) +- `processed_folders`, `selected_folders`, `total_detected_folders`, + `generated_folders` (int): as for `/generate-all-folders` +- `scale` (int) +- `bundle_results` (list): per bundle, `folder_name`, `success`, + `total_images`, `output_path`, `images_skipped`, `too_large_files`, + `same_name_files` and `message` +- `message` (str) + --- ## Using the app 1. Run `pasteup`; it opens in your browser. 2. **Pick your clip art.** Choose **One bundle** (a folder of images) or **Many - bundles** (a folder with one folder inside per bundle), enter the folder and - click **Scan folder**. Results go in a `---Created---` folder next to it. + bundles** (a folder with one folder inside per bundle), then click + **Browse…**: open folders until you're in the right one, and **Use this + folder** scans it. Or type or paste its path and click **Scan folder**. + Results go in a `---Created---` folder next to it. - One bundle: **Folder check** says how many images it found and how many full sheets of 12 they make. If the folder holds only other folders, pick one under **Choose folders**. @@ -378,8 +482,8 @@ Makes one cover per bundle folder, as `/generate-all-folders` does for sheets. Untick any you want to skip. A bundle that fails its check can be unticked, or fixed and checked again with **Check selected bundles**. 3. **Set up the page.** Size, units, DPI and format apply to sheets and covers - alike. **Clean up white backgrounds** can make near-white backgrounds pure - white or transparent. + alike, and DPI to upscaled images too. **Clean up white backgrounds** can + make near-white backgrounds pure white or transparent. 4. **Make your images.** - **Preview sheets:** every image, 12 to a page (`POST /api/jobs/generate`, or `POST /api/jobs/generate-all-folders` for many bundles), optionally with a @@ -388,6 +492,10 @@ Makes one cover per bundle folder, as `/generate-all-folders` does for sheets. (`POST /api/jobs/generate-cover-collage`, or `POST /api/jobs/generate-cover-collage-batch`). **Only use certain images** limits it to the files you name. + - **Upscaled images:** every image 2x, 3x or 4x bigger, as a PNG and a JPG + in an `Upscaled` folder (`POST /api/jobs/upscale-images`, or + `POST /api/jobs/upscale-images-batch`). It runs on its own, with no + sheets or cover needed. 5. **Your files** lists what was made and where it was saved; **Copy** puts a path on the clipboard. @@ -400,6 +508,9 @@ Makes one cover per bundle folder, as `/generate-all-folders` does for sheets. - Trailing images that do not fill a complete page are left out by default; the response reports how many. Set `include_partial_page: true` to render that final page with its unused cells left white. +- Upscaling resamples each image: edges stay smooth, but no detail is added, + so at 4x clip art looks soft. AI upscalers can invent detail; Pasteup has + none built in. --- @@ -470,7 +581,7 @@ Symptoms: Fixes: - Confirm files are directly in the selected folder (non-recursive scan). - Confirm extensions are supported. -- Check that files are not hidden/system placeholders. +- Check the files aren't hidden: names starting with a dot are skipped. ### Permission errors Symptoms: @@ -482,6 +593,12 @@ Fixes: - Verify write permission on output folder. - Try a user-owned directory (for example inside your home directory). +### "Pasteup isn't allowed to open" Desktop, Documents or Downloads on a Mac +macOS asks before a program reads these folders, and Pasteup reads them as the +terminal app it runs in (Terminal, for the `pasteup --shortcut` launcher). If +that was refused, turn the folder on for your terminal app under **System +Settings › Privacy & Security › Files and Folders**, then restart Pasteup. + --- ## How images are loaded diff --git a/src/pasteup/models/schemas.py b/src/pasteup/models/schemas.py index 074b361..eee91ce 100644 --- a/src/pasteup/models/schemas.py +++ b/src/pasteup/models/schemas.py @@ -3,6 +3,7 @@ from mullion import DEFAULT_TOLERANCE, BackgroundCleanMode from pasteup.services.cover_collage import CoverCollageFittingMode, CoverCollageTemplate +from pasteup.services.upscaler import UPSCALE_FACTORS from mullion import DEFAULT_DPI from pydantic import AliasChoices, BaseModel, Field, field_validator @@ -117,6 +118,25 @@ class ScanResponse(BaseModel): message: str +class BrowseFolderRequest(BaseModel): + folder_path: str = Field(default="", description="The folder to list. Blank lists the home folder.") + + +class FolderLink(BaseModel): + name: str + path: str + + +class BrowseFolderResponse(BaseModel): + path: str = Field(description="The folder listed: the one asked for, or the nearest that could be listed.") + parent_path: str | None = Field(description="The folder above it; null at the top of a drive.") + breadcrumbs: list[FolderLink] = Field(description="Each folder from the top of the drive down to this one.") + folders: list[FolderLink] = Field(description="The folders inside, without hidden ones, __MACOSX or ---Created---.") + image_count: int = Field(description="Supported images directly inside, as the scan counts them.") + places: list[FolderLink] = Field(description="Shortcuts: home, its standard folders, and drives on Windows.") + message: str = Field(description="Why another folder was listed than the one asked for; empty otherwise.") + + class SelectedBatchFolderValidationRequest(BaseModel): parent_folder_path: str selected_subfolders: list[str] = Field( @@ -318,6 +338,82 @@ class BatchCoverCollageGenerationResponse(BaseModel): message: str +class UpscaleOptions(BaseModel): + """How much bigger, and the resolution to stamp; shared by both upscale requests.""" + + scale: int = Field(default=2, description="How many times bigger each side becomes: 2, 3 or 4.") + dpi: int = Field(default=DEFAULT_DPI, gt=0, le=1200, description="Resolution stamped into the files.") + + @field_validator("scale") + @classmethod + def validate_scale(cls, value: int) -> int: + if value not in UPSCALE_FACTORS: + raise ValueError(f"scale must be one of {', '.join(map(str, UPSCALE_FACTORS))}.") + return value + + +class UpscaleRequest(UpscaleOptions): + folder_path: str + output_path: str = Field(description="The bundle's output folder; an Upscaled folder is made inside it.") + + +class UpscaleResponse(BaseModel): + success: bool + scale: int + total_images: int = Field(description="Images upscaled, each saved as PNG and as JPG.") + output_path: str = Field(description="The Upscaled folder, with a PNG and a JPG folder inside.") + png_folder: str + jpg_folder: str + output_files: list[str] + images_skipped: int = Field(default=0, description="Files that could not be read as images.") + too_large_files: list[str] = Field( + default_factory=list, description="Images left out because upscaled they would pass 100 megapixels." + ) + same_name_files: list[str] = Field( + default_factory=list, + description="Files left out because an image of the same name in another format gave both formats.", + ) + message: str + + +class BatchUpscaleRequest(UpscaleOptions): + parent_folder_path: str + created_output_root_path: str = Field( + validation_alias=AliasChoices("created_output_root_path", "output_root_path"), + ) + selected_subfolders: list[str] = Field( + validation_alias=AliasChoices("selected_subfolders", "selected_subfolder_paths"), + ) + + @field_validator("selected_subfolders", mode="before") + @classmethod + def normalize_selected_subfolders(cls, value: object) -> list[str] | None: + return _normalize_optional_string_list(value, field_name="selected_subfolders") + + +class BatchUpscaleResult(BaseModel): + folder_name: str + success: bool + total_images: int + output_path: str + images_skipped: int = 0 + too_large_files: list[str] = Field(default_factory=list) + same_name_files: list[str] = Field(default_factory=list) + message: str + + +class BatchUpscaleResponse(BaseModel): + success: bool + scale: int + processed_folders: int + generated_folders: int + total_detected_folders: int + selected_folders: int + bundle_results: list[BatchUpscaleResult] + created_output_root_path: str + message: str + + class BatchGenerationRequest(OutputQualityOptions): parent_folder_path: str created_output_root_path: str diff --git a/src/pasteup/routes/jobs.py b/src/pasteup/routes/jobs.py index 3514650..bd32c31 100644 --- a/src/pasteup/routes/jobs.py +++ b/src/pasteup/routes/jobs.py @@ -15,6 +15,11 @@ BatchCoverCollageResult, BatchGenerationRequest, BatchGenerationResponse, + BatchUpscaleRequest, + BatchUpscaleResponse, + BatchUpscaleResult, + BrowseFolderRequest, + BrowseFolderResponse, CoverCollageGenerationRequest, CoverCollageResponse, GenerationRequest, @@ -24,10 +29,13 @@ SelectedBatchFolderScanResult, SelectedBatchFolderValidationRequest, SelectedBatchFolderValidationResponse, + UpscaleRequest, + UpscaleResponse, ) from pasteup.services.cover_collage import compose_cover_collage from mullion import DEFAULT_TOLERANCE, BackgroundCleanMode from pasteup.services.exporter import export_pages +from pasteup.services.folder_browser import browse_folder from mullion import ( DEFAULT_DPI, PageGeometry, @@ -35,11 +43,13 @@ ) from pasteup.services.folder_scanner import ( is_generated_output_folder, + is_ignored_folder, list_supported_image_paths, resolve_folder_path, scan_folder, ) from pasteup.services.grid_composer import compose_image_grids +from pasteup.services.upscaler import UPSCALED_FOLDER_NAME, upscale_images from pasteup.services.video_exporter import build_preview_video router = APIRouter(prefix="/jobs", tags=["jobs"]) @@ -120,6 +130,11 @@ def scan_folder_job(payload: ScanRequest) -> ScanResponse: return ScanResponse(**scan_result) +@router.post("/browse-folder", response_model=BrowseFolderResponse) +def browse_folder_job(payload: BrowseFolderRequest) -> BrowseFolderResponse: + return BrowseFolderResponse(**browse_folder(payload.folder_path)) + + @router.post("/scan-selected-batch-folders", response_model=SelectedBatchFolderValidationResponse) def scan_selected_batch_folders_job( payload: SelectedBatchFolderValidationRequest, @@ -371,6 +386,132 @@ def generate_all_folders_job(payload: BatchGenerationRequest) -> BatchGeneration ) +@router.post("/upscale-images", response_model=UpscaleResponse) +def upscale_images_job(payload: UpscaleRequest) -> UpscaleResponse: + try: + return _upscale_single_folder( + folder_path=payload.folder_path, + output_path=payload.output_path, + scale=payload.scale, + dpi=payload.dpi, + ) + except ValueError as exc: + raise HTTPException(status_code=400, detail=str(exc)) from exc + + +@router.post("/upscale-images-batch", response_model=BatchUpscaleResponse) +def upscale_images_batch_job(payload: BatchUpscaleRequest) -> BatchUpscaleResponse: + parent_folder = resolve_folder_path(payload.parent_folder_path) + created_root = Path(payload.created_output_root_path).expanduser() + + if not parent_folder.exists() or not parent_folder.is_dir(): + raise HTTPException(status_code=400, detail=f"Invalid parent folder: {parent_folder}") + + discovered_bundle_folders = _list_detected_bundle_folders( + parent_folder=parent_folder, + created_root_name=created_root.name, + ) + bundle_folders = _resolve_selected_bundle_folders( + parent_folder=parent_folder, + selected_subfolders=payload.selected_subfolders, + bundle_folder_lookup={str(path): path for path in discovered_bundle_folders}, + ) + + bundle_results = [ + _upscale_batch_result( + bundle_folder=bundle_folder, + created_root=created_root, + scale=payload.scale, + dpi=payload.dpi, + ) + for bundle_folder in bundle_folders + ] + generated_folders = sum(1 for result in bundle_results if result.total_images > 0) + + return BatchUpscaleResponse( + success=True, + scale=payload.scale, + processed_folders=len(bundle_folders), + generated_folders=generated_folders, + total_detected_folders=len(discovered_bundle_folders), + selected_folders=len(bundle_folders), + bundle_results=bundle_results, + created_output_root_path=str(created_root), + message=( + f"Detected {len(discovered_bundle_folders)} eligible folder(s) under the parent. " + f"Processed {len(bundle_folders)} selected folder(s). " + f"Upscaled images for {generated_folders} folder(s)." + ), + ) + + +def _upscale_single_folder(*, folder_path: str, output_path: str, scale: int, dpi: int) -> UpscaleResponse: + """Upscale one folder's images into ``output_path``/Upscaled. + + Raises ValueError, with a message for the user, when there is nothing to + upscale or the files can't be saved. + """ + try: + scan_result = scan_folder(folder_path) + if not scan_result["valid"]: + raise ValueError(str(scan_result["message"])) + image_paths = list_supported_image_paths(resolve_folder_path(folder_path)) + except OSError as exc: + raise ValueError(f"Unable to access folder: {exc}") from exc + + try: + summary = upscale_images( + image_paths, + Path(output_path).expanduser() / UPSCALED_FOLDER_NAME, + scale=scale, + dpi=dpi, + ) + except OSError as exc: + raise ValueError(f"Couldn't save the upscaled images: {exc}") from exc + + return UpscaleResponse( + success=bool(summary.upscaled), + scale=scale, + total_images=len(summary.upscaled), + output_path=str(summary.output_dir), + png_folder=str(summary.png_dir), + jpg_folder=str(summary.jpg_dir), + output_files=summary.output_files, + images_skipped=len(summary.unreadable), + too_large_files=summary.too_large, + same_name_files=summary.same_name, + message=summary.user_message, + ) + + +def _upscale_batch_result(*, bundle_folder: Path, created_root: Path, scale: int, dpi: int) -> BatchUpscaleResult: + bundle_output = _build_batch_bundle_output_path(created_root=created_root, bundle_folder=bundle_folder) + try: + response = _upscale_single_folder( + folder_path=str(bundle_folder), + output_path=str(bundle_output), + scale=scale, + dpi=dpi, + ) + except ValueError as exc: + return BatchUpscaleResult( + folder_name=bundle_folder.name, + success=False, + total_images=0, + output_path=str(bundle_output / UPSCALED_FOLDER_NAME), + message=str(exc), + ) + + return BatchUpscaleResult( + folder_name=bundle_folder.name, + success=response.success, + total_images=response.total_images, + output_path=response.output_path, + images_skipped=response.images_skipped, + too_large_files=response.too_large_files, + same_name_files=response.same_name_files, + message=response.message, + ) def _generate_cover_collage_batch_result( @@ -672,7 +813,7 @@ def _list_detected_bundle_folders(*, parent_folder: Path, created_root_name: str for item in parent_folder.iterdir() if item.is_dir() and item.name != created_root_name - and not is_generated_output_folder(item) + and not is_ignored_folder(item) ], key=lambda path: (path.name.lower(), path.name), ) diff --git a/src/pasteup/services/folder_browser.py b/src/pasteup/services/folder_browser.py new file mode 100644 index 0000000..f39dece --- /dev/null +++ b/src/pasteup/services/folder_browser.py @@ -0,0 +1,152 @@ +"""Listing folders for the folder browser in the UI. + +A page cannot learn the path of a folder picked with the browser's own file +chooser, and Pasteup needs paths: it reads the images where they are and writes +its results next to them. So the page browses the disk through these listings. +""" + +from __future__ import annotations + +import os +import stat +import string +import sys +from collections.abc import Iterator +from pathlib import Path + +from pasteup.services.folder_scanner import is_ignored_folder, is_image_file_name + +# Offered as shortcuts when the home folder has them. +STANDARD_FOLDER_NAMES = ("Desktop", "Documents", "Downloads", "Pictures") + +_ON_WINDOWS = os.name == "nt" +# Windows hides its own folders (AppData, $Recycle.Bin, System Volume +# Information) with these attributes rather than a leading dot. +_HIDDEN_ATTRIBUTES = stat.FILE_ATTRIBUTE_HIDDEN | stat.FILE_ATTRIBUTE_SYSTEM + + +def browse_folder(folder_path: str = "") -> dict[str, object]: + """List the folders inside ``folder_path`` and count its images. + + A blank path starts at the home folder. A folder that cannot be listed + gives way to the nearest one above it that can, with a message saying why, + so the browser always has somewhere to show. + """ + home = _home_folder() + requested = folder_path.strip() if isinstance(folder_path, str) else "" + target = _absolute_path(requested) if requested else home + + message = "" + shown, folder_names, image_count = target, [], 0 + for candidate in _folders_to_try(target, home): + try: + folder_names, image_count = _read_folder(candidate) + except (OSError, ValueError) as exc: + message = message or _describe_unreadable_folder(target, exc) + continue + shown = candidate + break + + parent = shown.parent + return { + "path": str(shown), + "parent_path": str(parent) if parent != shown else None, + "breadcrumbs": [_link(folder) for folder in (*reversed(shown.parents), shown)], + "folders": [{"name": name, "path": str(shown / name)} for name in folder_names], + "image_count": image_count, + "places": list_places(home), + "message": message, + } + + +def list_places(home: Path) -> list[dict[str, str]]: + """Shortcuts: home, its standard folders, and on Windows each drive.""" + places = [{"name": "Home", "path": str(home)}] + places.extend(_link(home / name) for name in STANDARD_FOLDER_NAMES if os.path.isdir(home / name)) + places.extend(_link(Path(root)) for root in _drive_roots()) + return places + + +def _link(folder: Path) -> dict[str, str]: + # A root has no name of its own: show it as "/" or "C:\". + return {"name": folder.name or str(folder), "path": str(folder)} + + +def _read_folder(folder: Path) -> tuple[list[str], int]: + """The folders inside ``folder`` worth showing, and how many images it holds. + + Both follow the scan's rules: no folder it ignores is shown, and images are + counted as it counts them, so the number shown matches what it then finds. + """ + folder_names: list[str] = [] + image_count = 0 + with os.scandir(folder) as entries: + for entry in entries: + try: + if entry.is_dir(): + if not is_ignored_folder(Path(entry.name)) and not _has_hidden_attribute(entry): + folder_names.append(entry.name) + elif entry.is_file() and is_image_file_name(entry.name): + image_count += 1 + except OSError: + continue + return sorted(folder_names, key=lambda name: (name.lower(), name)), image_count + + +def _has_hidden_attribute(entry: os.DirEntry) -> bool: + if not _ON_WINDOWS: + return False + # Not following links, so a hidden junction such as "Documents and + # Settings" is judged by itself rather than by the folder it points to. + return bool(entry.stat(follow_symlinks=False).st_file_attributes & _HIDDEN_ATTRIBUTES) + + +def _folders_to_try(target: Path, home: Path) -> Iterator[Path]: + """``target``, then each folder above it, then home.""" + yield target + yield from target.parents + if home != target and home not in target.parents: + yield home + + +def _describe_unreadable_folder(folder: Path, exc: Exception) -> str: + if isinstance(exc, (FileNotFoundError, NotADirectoryError)): + return f"There's no folder at {folder}, so this is the nearest one that exists." + if isinstance(exc, PermissionError): + message = f"Pasteup isn't allowed to open {folder}." + if sys.platform == "darwin": + message += ( + " To let it, turn on access for your terminal app under System Settings ›" + " Privacy & Security › Files and Folders." + ) + return message + reason = getattr(exc, "strerror", None) or str(exc) + return f"Couldn't open {folder} ({reason})." + + +def _absolute_path(folder_path: str) -> Path: + # Not resolved: a link such as ~/Dropbox stays where the user found it + # rather than jumping to wherever it points. + return Path(os.path.abspath(os.path.expanduser(folder_path))) + + +def _home_folder() -> Path: + try: + return Path.home() + except (RuntimeError, KeyError): + return Path(Path.cwd().anchor) + + +def _drive_roots() -> list[str]: + """Each drive's root on Windows, since nothing above C:\\ leads to D:\\.""" + if not _ON_WINDOWS: + return [] + try: + if hasattr(os, "listdrives"): # Python 3.12+ + return sorted(os.listdrives()) + import ctypes + + present = ctypes.windll.kernel32.GetLogicalDrives() + except (OSError, AttributeError): + return [] + return [f"{letter}:\\" for index, letter in enumerate(string.ascii_uppercase) if present >> index & 1] diff --git a/src/pasteup/services/folder_scanner.py b/src/pasteup/services/folder_scanner.py index d340002..4009080 100644 --- a/src/pasteup/services/folder_scanner.py +++ b/src/pasteup/services/folder_scanner.py @@ -2,8 +2,9 @@ from __future__ import annotations +import os from pathlib import Path -from typing import Iterable +from typing import Iterable, Iterator SUPPORTED_EXTENSIONS = {".jpg", ".jpeg", ".png", ".webp"} DEFAULT_SAMPLE_SIZE = 5 @@ -12,6 +13,10 @@ # offers real source bundles. Keep this list here alongside the discovery helpers to # avoid reintroducing output directories such as `---Created---` into batch selection. EXCLUDED_GENERATED_FOLDER_NAMES = frozenset({"---Created---"}) +# Never read for images nor offered as bundles, besides hidden folders: Pasteup's +# own output, and __MACOSX, which a zip made on a Mac fills with "._" metadata +# files and which appears when the zip is unpacked anywhere else. +IGNORED_FOLDER_NAMES = EXCLUDED_GENERATED_FOLDER_NAMES | {"__MACOSX"} def scan_folder(folder_path: str, sample_size: int = DEFAULT_SAMPLE_SIZE) -> dict[str, object]: @@ -132,7 +137,7 @@ def resolve_folder_path(folder_path: str) -> Path: def list_supported_image_paths(folder: Path, *, recursive: bool = False) -> list[Path]: iterator: Iterable[Path] if recursive: - iterator = (item for item in folder.rglob("*") if item.is_file()) + iterator = _walk_files(folder) else: iterator = folder.iterdir() @@ -140,16 +145,44 @@ def list_supported_image_paths(folder: Path, *, recursive: bool = False) -> list [ item for item in iterator - if item.is_file() and item.suffix.lower() in SUPPORTED_EXTENSIONS + if is_image_file_name(item.name) and item.is_file() ], key=lambda path: (path.name.lower(), path.name, str(path)), ) +def is_image_file_name(name: str) -> bool: + """Whether a file's name makes it one of the images a scan counts. + + Hidden files are left out, as Finder leaves them out. Most of them are the + "._name" files macOS writes beside files it copies to a drive that can't + hold their metadata (exFAT, FAT32, some network shares): they carry the + image's extension but hold no image. + """ + return not name.startswith(".") and Path(name).suffix.lower() in SUPPORTED_EXTENSIONS + + def is_generated_output_folder(folder: Path) -> bool: return folder.name in EXCLUDED_GENERATED_FOLDER_NAMES +def is_ignored_folder(folder: Path) -> bool: + """Whether a folder is never read for images nor offered as a bundle. + + Hidden folders are another program's: on drives a Mac has written to, + macOS keeps .Spotlight-V100, .Trashes and .fseventsd. + """ + return folder.name.startswith(".") or folder.name in IGNORED_FOLDER_NAMES + + +def _walk_files(folder: Path) -> Iterator[Path]: + """Every file under ``folder``, outside ignored folders.""" + for root, folder_names, file_names in os.walk(folder): + # Pruned in place, which keeps os.walk out of them. + folder_names[:] = [name for name in folder_names if not is_ignored_folder(Path(name))] + yield from (Path(root) / name for name in file_names) + + def list_child_folder_paths(folder: Path) -> list[dict[str, str]]: return [ {"display_name": path.name, "full_path": str(path)} @@ -157,7 +190,7 @@ def list_child_folder_paths(folder: Path) -> list[dict[str, str]]: ( item.resolve(strict=False) for item in folder.iterdir() - if item.is_dir() and not is_generated_output_folder(item) + if item.is_dir() and not is_ignored_folder(item) ), key=lambda path: (path.name.lower(), path.name, str(path)), ) diff --git a/src/pasteup/services/upscaler.py b/src/pasteup/services/upscaler.py new file mode 100644 index 0000000..15794e8 --- /dev/null +++ b/src/pasteup/services/upscaler.py @@ -0,0 +1,152 @@ +"""Upscaling: every image of a bundle made bigger, saved as PNG and as JPG. + +Lanczos resampling makes an image bigger without blocks or jagged edges, but +it can't add detail the image doesn't have, so at 4x clip art comes out soft. +Pillow premultiplies alpha as it resamples, so a cutout's edge doesn't pick +up a dark fringe from the transparent pixels around it. +""" + +from __future__ import annotations + +import logging +from dataclasses import dataclass, field +from pathlib import Path +from typing import Sequence + +from PIL import Image + +from mullion import flatten_onto, open_path, validate_dpi + +logger = logging.getLogger(__name__) + +UPSCALE_FACTORS = (2, 3, 4) +# Made inside the bundle's output folder, with a folder per format inside it. +UPSCALED_FOLDER_NAME = "Upscaled" +# As for pages (MAX_PAGE_MEGAPIXELS in routes/jobs.py): an image is held in +# memory several times over while it is resampled and saved, and a 3000 px +# image at 4x is already 144 MP. +MAX_UPSCALED_MEGAPIXELS = 100 +# Files sharing a name, as art.png and art.jpg often do, would be saved over +# each other, so one gives both formats: lossless and maybe transparent first. +_SOURCE_PREFERENCE = {".png": 0, ".webp": 1, ".jpg": 2, ".jpeg": 2} + + +class _TooLarge(Exception): + pass + + +@dataclass +class UpscaleSummary: + scale: int + output_dir: Path + upscaled: list[str] = field(default_factory=list) + output_files: list[str] = field(default_factory=list) + unreadable: list[str] = field(default_factory=list) + too_large: list[str] = field(default_factory=list) + same_name: list[str] = field(default_factory=list) + + @property + def png_dir(self) -> Path: + return self.output_dir / "PNG" + + @property + def jpg_dir(self) -> Path: + return self.output_dir / "JPG" + + @property + def user_message(self) -> str: + parts = [f"Upscaled {len(self.upscaled)} image(s) {self.scale}x, each saved as PNG and JPG."] + if self.unreadable: + parts.append(f"Skipped {len(self.unreadable)} unreadable file(s).") + if self.too_large: + parts.append( + f"Left out {len(self.too_large)} image(s) that would pass {MAX_UPSCALED_MEGAPIXELS} MP " + f"at {self.scale}x: {', '.join(self.too_large)}." + ) + if self.same_name: + parts.append( + f"Left out {len(self.same_name)} file(s) named like an image in another format, " + f"which gave both formats: {', '.join(self.same_name)}." + ) + return " ".join(parts) + + +def upscale_images(image_paths: Sequence[Path], output_dir: Path, *, scale: int, dpi: int) -> UpscaleSummary: + """Upscale each image ``scale`` times into ``output_dir``/PNG and /JPG. + + A file keeps its name, so upscaling again replaces what the last run made. + The PNG keeps any transparency; the JPG is flattened onto white. + """ + if scale not in UPSCALE_FACTORS: + raise ValueError(f"Scale must be one of {', '.join(map(str, UPSCALE_FACTORS))}.") + resolved_dpi = validate_dpi(dpi) + + summary = UpscaleSummary(scale=scale, output_dir=output_dir) + sources, summary.same_name = _one_file_per_name(image_paths) + for path in sources: + try: + upscaled = _upscale(path, scale) + except _TooLarge: + summary.too_large.append(path.name) + continue + except (OSError, ValueError) as exc: + summary.unreadable.append(path.name) + logger.warning("Skipping unreadable image '%s': %s", path, exc) + continue + + summary.output_files.extend(_save_both_formats(upscaled, summary, path.stem, resolved_dpi)) + summary.upscaled.append(path.name) + return summary + + +def _one_file_per_name(image_paths: Sequence[Path]) -> tuple[list[Path], list[str]]: + """The files to upscale, and the names of those left out for sharing a name. + + Names are compared ignoring case, as Mac and Windows file systems do. + """ + chosen: dict[str, Path] = {} + for path in image_paths: + current = chosen.get(path.stem.casefold()) + if current is None or _preference(path) < _preference(current): + chosen[path.stem.casefold()] = path + + kept = set(chosen.values()) + return [path for path in image_paths if path in kept], [path.name for path in image_paths if path not in kept] + + +def _preference(path: Path) -> int: + return _SOURCE_PREFERENCE.get(path.suffix.lower(), len(_SOURCE_PREFERENCE)) + + +def _upscale(path: Path, scale: int) -> Image.Image: + # The size is read from the header first, so an image too big to upscale + # is never decoded at all. + try: + with Image.open(path) as probe: + width, height = probe.size + except Image.DecompressionBombError as exc: + raise _TooLarge from exc + if width * height * scale * scale > MAX_UPSCALED_MEGAPIXELS * 1_000_000: + raise _TooLarge + + # Upright, with any transparency kept as RGBA. + image = open_path(path, keep_alpha=True) + if image.getchannel("A").getextrema() == (255, 255): + # Opaque after all: without an alpha channel the PNG is smaller. + image = image.convert("RGB") + return image.resize((image.width * scale, image.height * scale), Image.Resampling.LANCZOS) + + +def _save_both_formats(image: Image.Image, summary: UpscaleSummary, stem: str, dpi: int) -> list[str]: + summary.png_dir.mkdir(parents=True, exist_ok=True) + summary.jpg_dir.mkdir(parents=True, exist_ok=True) + png_path = summary.png_dir / f"{stem}.png" + jpg_path = summary.jpg_dir / f"{stem}.jpg" + + # Not optimize=True, as the exporter's pages are: on a 4x upscale it + # tripled the time for a file no smaller. + image.save(png_path, format="PNG", compress_level=6, dpi=(dpi, dpi)) + # JPEG has no transparency; the same settings as exported pages. + jpg_image = image if image.mode == "RGB" else flatten_onto(image) + jpg_image.save(jpg_path, format="JPEG", quality=95, optimize=True, subsampling=0, dpi=(dpi, dpi)) + return [str(png_path), str(jpg_path)] diff --git a/src/pasteup/static/css/styles.css b/src/pasteup/static/css/styles.css index c4c2e78..df77076 100644 --- a/src/pasteup/static/css/styles.css +++ b/src/pasteup/static/css/styles.css @@ -58,6 +58,7 @@ --wobble-soft: 14px 5px 12px 6px / 6px 12px 5px 14px; --blob: 52% 48% 55% 45% / 48% 54% 46% 52%; --pencil: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24' fill='none' stroke='%232b2a33' stroke-width='2' stroke-linecap='round' stroke-linejoin='round'%3E%3Cpath d='M4.2 19.8l1.1-4.5L16.4 4.2c.8-.8 2-.8 2.8 0l.6.6c.8.8.8 2 0 2.8L8.7 18.7z'/%3E%3Cpath d='M14.4 6.3l3.3 3.3'/%3E%3C/svg%3E"); + --folder: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24' fill='none' stroke='%232b2a33' stroke-width='1.6' stroke-linecap='round' stroke-linejoin='round'%3E%3Cpath fill='%23f7dc8c' d='M2.8 6.6c0-1 .7-1.7 1.7-1.7h4.2c.7 0 1.1.3 1.5.9l.9 1.3h8.3c1 0 1.7.8 1.7 1.8l-.2 9.2c0 1-.8 1.7-1.7 1.7H4.6c-1 0-1.7-.8-1.7-1.7z'/%3E%3Cpath d='M3 10.2c6-.4 12.1-.4 18.1-.1'/%3E%3C/svg%3E"); } /* ---------- Base ---------- */ @@ -153,6 +154,13 @@ a { fill: var(--marker); } +/* Inside a button, beside its label. */ +.doodle--small { + width: 1.5rem; + height: 1.5rem; + stroke-width: 3.4; +} + /* ---------- Masthead ---------- */ .masthead { @@ -256,7 +264,8 @@ a { /* The outline and its shadow are drawn by a pseudo-element so the roughness filter bends the ink line without blurring the text inside. */ -.card::after { +.card::after, +.folder-browser::after { content: ""; position: absolute; inset: 0; @@ -268,7 +277,8 @@ a { } /* A strip of masking tape holding the card to the page. */ -.card::before { +.card::before, +.folder-browser::before { content: ""; position: absolute; top: -0.8rem; @@ -675,6 +685,23 @@ select:disabled { overflow-wrap: anywhere; } +/* The folder's path stays editable, with Browse beside it. */ +.folder-field { + display: flex; + flex-wrap: wrap; + gap: 0.6rem; +} + +.folder-field input { + flex: 1 1 15rem; + width: auto; +} + +.folder-field .secondary-button { + flex: none; + padding-inline: 1rem 1.2rem; +} + .subfolder-picker-row { display: flex; align-items: flex-end; @@ -903,7 +930,8 @@ select:disabled { overflow-wrap: anywhere; } -.copy-button { +.copy-button, +.folder-browser__place { flex: none; min-height: 2rem; padding: 0.1rem 0.75rem; @@ -918,11 +946,13 @@ select:disabled { transition: transform 120ms ease-out, box-shadow 120ms ease-out; } -.copy-button:hover { +.copy-button:hover, +.folder-browser__place:hover { transform: translate(-1px, -1px) rotate(-1.5deg); } -.copy-button:active { +.copy-button:active, +.folder-browser__place:active { transform: translate(1px, 1px); box-shadow: none; } @@ -932,6 +962,207 @@ select:disabled { color: var(--ink-soft); } +/* ---------- Folder browser ---------- */ + +/* A card lifted off the page. The list scrolls and keeps one height, so the + window doesn't jump about from folder to folder. */ +.folder-browser { + width: min(44rem, calc(100vw - 2rem)); + max-width: none; + max-height: calc(100vh - 3rem); + max-height: calc(100dvh - 3rem); + padding: 0; + overflow: visible; + color: var(--ink); + background: var(--paper-raised); + border: 0; +} + +.folder-browser[open] { + display: flex; + flex-direction: column; +} + +.folder-browser::backdrop { + background: rgba(43, 42, 51, 0.45); +} + +/* Scrolls as a whole only when a short or zoomed window can't fit even the + list's least height, so "Use this folder" is never out of reach. */ +.folder-browser__body { + display: flex; + flex-direction: column; + gap: 0.9rem; + min-height: 0; + padding: 1.9rem 1.6rem 1.5rem; + overflow-y: auto; +} + +.folder-browser__places, +.folder-browser__crumbs, +.folder-browser__list { + margin: 0; + padding: 0; + list-style: none; +} + +.folder-browser__places { + display: flex; + flex-wrap: wrap; + gap: 0.55rem; +} + +.folder-browser__place[aria-current] { + background: var(--marker); + color: var(--marker-ink); +} + +.folder-browser__crumbs { + display: flex; + flex-wrap: wrap; + align-items: baseline; + gap: 0.1rem 0.4rem; + font-family: var(--font-label); + font-size: 1.2rem; + line-height: 1.3; +} + +.folder-browser__crumbs li { + min-width: 0; + overflow-wrap: anywhere; +} + +.folder-browser__crumbs li + li::before { + content: "\203A"; + margin-right: 0.4rem; + color: var(--ink-soft); +} + +.folder-browser__crumb { + padding: 0; + font: inherit; + color: var(--accent); + text-decoration: underline; + text-underline-offset: 0.2em; + background: none; + border: 0; + cursor: pointer; +} + +.folder-browser__here { + padding-bottom: 0.15em; + background: var(--squiggle) no-repeat left bottom / 100% 0.3em; +} + +.folder-browser__notice { + padding: 0.5rem 0.8rem; + color: var(--warning-ink); + background: var(--warning-bg); + border: 2px solid currentColor; + border-radius: var(--wobble-soft); + font-size: 0.95rem; + overflow-wrap: anywhere; +} + +.folder-browser__list { + flex: 1 1 20rem; + min-height: 7rem; + padding: 0.35rem; + overflow-y: auto; + overscroll-behavior: contain; + background: var(--paper); + border: 2px solid var(--line); + border-radius: var(--wobble-soft); +} + +/* Shown only when a listing is slow, so a quick one doesn't flicker. */ +.folder-browser__list[aria-busy="true"] { + cursor: progress; + animation: dim 0s linear 0.2s forwards; +} + +@keyframes dim { + to { + opacity: 0.55; + } +} + +.folder-browser__folder { + display: flex; + align-items: center; + gap: 0.65rem; + width: 100%; + padding: 0.4rem 0.6rem; + font: inherit; + color: var(--ink); + text-align: left; + background: transparent; + border: 0; + border-radius: var(--wobble-soft); + cursor: pointer; + overflow-wrap: anywhere; +} + +.folder-browser__folder::before, +.folder-browser__folder::after { + content: ""; + flex: none; + background: no-repeat center / contain; +} + +.folder-browser__folder::before { + width: 1.6rem; + height: 1.6rem; + background-image: var(--folder); +} + +/* The dropdown chevron, turned to say "opens". */ +.folder-browser__folder::after { + width: 1rem; + height: 1rem; + margin-left: auto; + background-image: var(--chevron); + transform: rotate(-90deg); + opacity: 0.55; +} + +.folder-browser__folder:hover, +.folder-browser__folder:focus-visible { + background-color: var(--note); +} + +/* Inside the scrolling list, where an outer ring would be cut off. */ +.folder-browser__folder:focus-visible { + outline-offset: -3px; +} + +.folder-browser__empty { + padding: 0.5rem 0.6rem; + color: var(--ink-soft); + font-family: var(--font-label); + font-size: 1.15rem; +} + +.folder-browser__footer { + display: flex; + flex-wrap: wrap; + align-items: center; + justify-content: space-between; + gap: 0.75rem 1.25rem; +} + +.folder-browser__status { + font-family: var(--font-label); + font-size: 1.15rem; +} + +.folder-browser__actions { + display: flex; + flex-wrap: wrap; + gap: 0.75rem; + margin-left: auto; +} + /* ---------- Footer ---------- */ .site-footer { @@ -962,6 +1193,19 @@ select:disabled { width: 100%; align-self: stretch; } + + .folder-browser__body { + padding: 1.7rem 1.1rem 1.2rem; + } + + .folder-browser__actions { + width: 100%; + } + + .folder-browser__actions > button { + flex: 1 1 auto; + padding-inline: 0.9rem; + } } /* ---------- Motion ---------- */ diff --git a/src/pasteup/static/js/app.js b/src/pasteup/static/js/app.js index 0fffd3a..aab90c1 100644 --- a/src/pasteup/static/js/app.js +++ b/src/pasteup/static/js/app.js @@ -16,8 +16,8 @@ const GENERATION_MODES = { helpText: "Point Pasteup at the folder that holds one bundle's images.", modeStepCopy: "Counts your images and checks they can be read.", folderStepCopy: "Results go in a ---Created--- folder next to it.", - outputStepCopy: "Preview sheets of every image, or one cover for the listing.", - outputHintReady: "Ready! Make preview sheets, a cover, or both.", + outputStepCopy: "Preview sheets of every image, a cover for the listing, or upscaled copies of the images.", + outputHintReady: "Ready! Make preview sheets, a cover or upscaled images.", }, batch: { inputLabel: "Folder of bundles", @@ -29,7 +29,7 @@ const GENERATION_MODES = { helpText: "Point Pasteup at a folder with one folder inside per bundle.", modeStepCopy: "Lists the bundle folders inside and checks each one.", folderStepCopy: "Each bundle gets its own folder inside a ---Created--- folder next to it.", - outputStepCopy: "Preview sheets or a cover for every bundle you checked.", + outputStepCopy: "Preview sheets, a cover or upscaled images for every bundle you checked.", outputHintReady: "Ready! Make images for the checked bundles.", }, }; @@ -61,9 +61,12 @@ const backgroundToleranceEl = document.getElementById("background-tolerance"); const backgroundFeatherEl = document.getElementById("background-feather"); const includePartialPageEl = document.getElementById("include-partial-page"); +const upscaleScaleEl = document.getElementById("upscale-scale"); + const scanButton = document.getElementById("scan-button"); const generateButton = document.getElementById("generate-button"); const coverCollageButton = document.getElementById("cover-collage-button"); +const upscaleButton = document.getElementById("upscale-button"); const outputActionsSectionEl = document.getElementById("output-actions-section"); const outputActionsHintEl = document.getElementById("output-actions-hint"); const scanResultEl = document.getElementById("scan-result"); @@ -81,6 +84,16 @@ const batchSelectionSummaryEl = document.getElementById("batch-selection-summary const batchSelectionStatusEl = document.getElementById("batch-selection-status"); const videoSettingsEl = document.getElementById("video-settings"); const backgroundCleanSettingsEl = document.getElementById("background-clean-settings"); +const browseFolderButton = document.getElementById("browse-folder-button"); +const folderBrowserEl = document.getElementById("folder-browser"); +const folderBrowserTitleEl = document.getElementById("folder-browser-title"); +const folderBrowserPlacesEl = document.getElementById("folder-browser-places"); +const folderBrowserCrumbsEl = document.getElementById("folder-browser-crumbs"); +const folderBrowserNoticeEl = document.getElementById("folder-browser-notice"); +const folderBrowserListEl = document.getElementById("folder-browser-list"); +const folderBrowserStatusEl = document.getElementById("folder-browser-status"); +const folderBrowserCancelButton = document.getElementById("folder-browser-cancel"); +const folderBrowserChooseButton = document.getElementById("folder-browser-choose"); const generationState = { mode: document.querySelector('input[name="generation-mode"]:checked')?.value || "single", @@ -249,14 +262,22 @@ function validateVideoGenerationSettings() { return true; } -function validateOutputQualitySettings() { - const quality = readOutputQualitySettings(); - - if (!Number.isFinite(quality.dpi) || quality.dpi <= 0 || quality.dpi > 1200) { +function validateDpiSetting() { + const { dpi } = readOutputQualitySettings(); + if (!Number.isFinite(dpi) || dpi <= 0 || dpi > 1200) { reportSettingsProblem("DPI must be between 1 and 1200.", dpiEl); return false; } + return true; +} + +function validateOutputQualitySettings() { + if (!validateDpiSetting()) { + return false; + } + + const quality = readOutputQualitySettings(); if (quality.background_clean_mode === "off") { return true; } @@ -458,6 +479,30 @@ function buildBatchCoverCollagePayload() { }; } +function readUpscaleScale() { + return Number(upscaleScaleEl?.value) || 2; +} + +// Upscaling uses only the DPI from step 2: the images keep their own shape. +function readUpscalePayload() { + return { + folder_path: readWorkingFolderPath(), + output_path: outputFolderEl.value.trim(), + scale: readUpscaleScale(), + dpi: readOutputQualitySettings().dpi, + }; +} + +function buildBatchUpscalePayload() { + return { + parent_folder_path: inputFolderEl.value.trim(), + created_output_root_path: outputFolderEl.value.trim(), + selected_subfolders: getSelectedBatchFolderPaths(), + scale: readUpscaleScale(), + dpi: readOutputQualitySettings().dpi, + }; +} + function splitFolderPath(folderPath) { if (typeof folderPath !== "string") { @@ -694,6 +739,13 @@ function getCoverCollageButtonLabel() { return selectedCount === 1 ? "Make cover for 1 bundle" : `Make covers for ${selectedCount} bundles`; } +function getUpscaleButtonLabel() { + const { selectedCount } = getBatchGenerationSelectionDetails(); + return generationState.mode === "batch" && selectedCount > 0 + ? `Upscale ${pluralize(selectedCount, "bundle")}` + : "Upscale images"; +} + function getGenerationStatusMessage() { if (generationState.mode !== "batch") { return getSelectedGenerationModeConfig().statusMessage; @@ -752,11 +804,16 @@ function updateOutputActionState() { ? generationReady || coverCollageReady : generationState.singleFolderReady; + // Ready on the same terms as a cover: it too takes every image as it is. + const upscaleReady = coverCollageReady; + generateButton.textContent = getGenerationButtonLabel(); coverCollageButton.textContent = getCoverCollageButtonLabel(); + upscaleButton.textContent = getUpscaleButtonLabel(); generateButton.disabled = !generationReady || generationState.busy; coverCollageButton.disabled = !coverCollageReady || generationState.busy; - [scanButton, batchScanButton].forEach((button) => { + upscaleButton.disabled = !upscaleReady || generationState.busy; + [scanButton, batchScanButton, browseFolderButton].forEach((button) => { if (button) { button.disabled = generationState.busy; } @@ -765,6 +822,8 @@ function updateOutputActionState() { generateButton.classList.toggle("secondary-button", !generationReady); coverCollageButton.classList.toggle("primary-button", coverCollageReady); coverCollageButton.classList.toggle("secondary-button", !coverCollageReady); + upscaleButton.classList.toggle("primary-button", upscaleReady); + upscaleButton.classList.toggle("secondary-button", !upscaleReady); outputActionsSectionEl.classList.toggle("workflow-step--disabled", !outputActionsEnabled); outputActionsHintEl.textContent = getOutputActionsHint(); updateBatchSelectionStatus(); @@ -1179,12 +1238,12 @@ function describeVideoProblem(message) { return start >= 0 ? message.slice(start) : "The preview video couldn't be made."; } -function renderBatchFolderOutcomeSection(results, kind) { +function renderBatchFolderOutcomeSection(results, kind, scale) { if (!Array.isArray(results) || results.length === 0) { return ""; } - const requestedPreviewVideo = kind !== "cover" && Boolean(createPreviewVideoEl?.checked); + const requestedPreviewVideo = kind === "sheets" && Boolean(createPreviewVideoEl?.checked); const lines = results.map((result) => { const folderName = result.folder_name || result.displayName || fileName(result.output_path || "") || "Bundle"; if (!result.success) { @@ -1196,6 +1255,17 @@ function renderBatchFolderOutcomeSection(results, kind) { return `${folderName}: ${outputFiles.length > 0 ? outputFiles.map(fileName).join(", ") : "cover made"}`; } + if (kind === "upscale") { + // An image missing from the upscaled set is worth a mention; one left + // out for its name is not, as its twin gave both formats. + const tooLargeCount = Array.isArray(result.too_large_files) ? result.too_large_files.length : 0; + return [ + `${folderName}: ${pluralize(result.total_images ?? 0, "image")} upscaled`, + tooLargeCount > 0 ? `${tooLargeCount} too big to upscale ${scale}x` : "", + result.images_skipped > 0 ? `${pluralize(result.images_skipped, "file")} unreadable` : "", + ].filter(Boolean).join(", "); + } + if ((result.pages_generated ?? 0) === 0) { return `${folderName}: no sheets made. ${result.message || ""}`.trim(); } @@ -1214,14 +1284,37 @@ function renderBatchFolderOutcomeSection(results, kind) { function renderBatchSummary(result, kind) { const bundleResults = Array.isArray(result.bundle_results) ? result.bundle_results : []; const isCover = kind === "cover"; + const isUpscale = kind === "upscale"; // A bundle counts as done when something was made for it. - const doneResults = bundleResults.filter((bundleResult) => bundleResult?.success && (isCover || (bundleResult.pages_generated ?? 0) > 0)); + const doneResults = bundleResults.filter((bundleResult) => (isUpscale + ? bundleResult?.success && (bundleResult.total_images ?? 0) > 0 + : bundleResult?.success && (isCover || (bundleResult.pages_generated ?? 0) > 0))); const doneCount = doneResults.length; const totalCount = result.processed_folders ?? bundleResults.length; - const outputName = isCover ? "covers" : "preview sheets"; - const requestedPreviewVideo = !isCover && Boolean(createPreviewVideoEl?.checked); + const outputName = { cover: "covers", upscale: "upscaled images" }[kind] || "preview sheets"; + const requestedPreviewVideo = kind === "sheets" && Boolean(createPreviewVideoEl?.checked); const sum = (field) => doneResults.reduce((total, bundleResult) => total + (bundleResult[field] ?? 0), 0); + let summaryRows; + if (isCover) { + summaryRows = [ + { label: "Bundles done", value: `${doneCount} of ${totalCount}` }, + { label: "Cover files", value: pluralize(doneResults.flatMap((bundleResult) => bundleResult.output_files ?? []).length, "file") }, + ]; + } else if (isUpscale) { + summaryRows = [ + { label: "Bundles done", value: `${doneCount} of ${totalCount}` }, + { label: "Images upscaled", value: pluralize(sum("total_images"), "image") }, + ]; + } else { + summaryRows = [ + { label: "Bundles done", value: `${doneCount} of ${totalCount}` }, + { label: "Sheets", value: pluralize(sum("pages_generated"), "sheet") }, + { label: "Images used", value: pluralize(sum("total_images"), "image") }, + { label: "Videos", value: requestedPreviewVideo ? pluralize(bundleResults.filter((bundleResult) => bundleResult.video_created).length, "video") : null }, + ]; + } + let badgeLabel = "Done"; let message = `Made ${outputName} for ${pluralize(doneCount, "bundle")}.`; if (doneCount === 0) { @@ -1236,30 +1329,74 @@ function renderBatchSummary(result, kind) { tone: doneCount > 0 && doneCount === totalCount ? "success" : "warning", badgeLabel, message, - summaryRows: isCover - ? [ - { label: "Bundles done", value: `${doneCount} of ${totalCount}` }, - { label: "Cover files", value: pluralize(doneResults.flatMap((bundleResult) => bundleResult.output_files ?? []).length, "file") }, - ] - : [ - { label: "Bundles done", value: `${doneCount} of ${totalCount}` }, - { label: "Sheets", value: pluralize(sum("pages_generated"), "sheet") }, - { label: "Images used", value: pluralize(sum("total_images"), "image") }, - { label: "Videos", value: requestedPreviewVideo ? pluralize(bundleResults.filter((bundleResult) => bundleResult.video_created).length, "video") : null }, - ], + summaryRows, sections: [ - renderBatchFolderOutcomeSection(bundleResults, kind), + renderBatchFolderOutcomeSection(bundleResults, kind, result.scale), renderPathSection("Saved in", result.created_output_root_path || "", "Copy the folder path"), ], }); } +// Matches MAX_UPSCALED_MEGAPIXELS in services/upscaler.py. +const MAX_UPSCALED_MEGAPIXELS = 100; + +function describeTooLargeToUpscale(names, scale) { + if (names.length === 0) { + return ""; + } + + const advice = scale > 2 ? "Try a smaller size." : "They're big enough already."; + return names.length === 1 + ? `${names[0]} would pass ${MAX_UPSCALED_MEGAPIXELS} megapixels at ${scale}x, so it was left out. ${advice}` + : `${names.length} images would pass ${MAX_UPSCALED_MEGAPIXELS} megapixels at ${scale}x, so they were left out: ${names.join(", ")}. ${advice}`; +} + +function describeSameNameFiles(names) { + if (names.length === 0) { + return ""; + } + + return names.length === 1 + ? `${names[0]} was left out: an image with the same name in another format gave both the PNG and the JPG.` + : `${names.length} files were left out because an image with the same name in another format gave both the PNG and the JPG: ${names.join(", ")}.`; +} + +function renderUpscaleSummary(result) { + const imageCount = result.total_images ?? 0; + const tooLarge = Array.isArray(result.too_large_files) ? result.too_large_files : []; + const sameName = Array.isArray(result.same_name_files) ? result.same_name_files : []; + + renderResultCard(generationResultEl, { + tone: imageCount > 0 ? "success" : "warning", + badgeLabel: imageCount > 0 ? "Done" : "Nothing made", + message: imageCount > 0 + ? `Upscaled ${pluralize(imageCount, "image")} ${result.scale}x, each as a PNG and a JPG.` + : "No images were upscaled.", + notes: [ + result.images_skipped > 0 ? `Skipped ${pluralize(result.images_skipped, "file")} that couldn't be read.` : "", + describeTooLargeToUpscale(tooLarge, result.scale), + describeSameNameFiles(sameName), + ], + sections: imageCount > 0 + ? [ + renderPathSection("PNG files", result.png_folder, "Copy the PNG folder path"), + renderPathSection("JPG files", result.jpg_folder, "Copy the JPG folder path"), + ] + : [], + }); +} + function renderGenerationSummary(result, kind = "sheets") { if (Array.isArray(result.bundle_results)) { renderBatchSummary(result, kind); return; } + if (kind === "upscale") { + renderUpscaleSummary(result); + return; + } + const outputFiles = Array.isArray(result.output_files) ? result.output_files : []; const pagesGenerated = result.pages_generated ?? 0; const totalImages = result.total_images ?? 0; @@ -1426,15 +1563,17 @@ async function scanCurrentFolder({ preserveSubfolderPicker = false } = {}) { return result; } +async function scanFolderAndCheckBundles() { + await scanCurrentFolder({ preserveSubfolderPicker: true }); + // Check the picked bundles straight away, so making images for all the + // bundles in a folder takes one click. + if (generationState.mode === "batch" && hasSelectedBatchFolders()) { + await validateSelectedBatchFolders(); + } +} + scanButton?.addEventListener("click", async () => { - await runExclusive(async () => { - await scanCurrentFolder({ preserveSubfolderPicker: true }); - // Check the picked bundles straight away, so making images for all the - // bundles in a folder takes one click. - if (generationState.mode === "batch" && hasSelectedBatchFolders()) { - await validateSelectedBatchFolders(); - } - }); + await runExclusive(scanFolderAndCheckBundles); }); // The check card describes the picked bundles, so it follows the picks: @@ -1566,6 +1705,213 @@ useSubfolderButton?.addEventListener("click", async () => { await runExclusive(() => scanCurrentFolder()); }); +// The folder browser. A page can't learn the path of a folder picked with the +// browser's own file chooser, so this one lists folders through Pasteup: +// clicking a folder opens it, and "Use this folder" takes the one shown. +const folderBrowserState = { + path: "", + parentPath: null, + breadcrumbs: [], + // Where browsing got to last time, to start from when no path is typed. + lastPath: "", + // Only the latest listing is drawn, so a slow one can't replace a newer one. + requestId: 0, +}; + +function openFolderBrowser() { + if (!folderBrowserEl || folderBrowserEl.open || generationState.busy) { + return; + } + + folderBrowserTitleEl.textContent = `Choose the ${getSelectedGenerationModeConfig().inputLabel.toLowerCase()}`; + folderBrowserState.path = ""; + folderBrowserState.parentPath = null; + folderBrowserState.breadcrumbs = []; + folderBrowserPlacesEl.replaceChildren(); + folderBrowserCrumbsEl.replaceChildren(); + folderBrowserListEl.replaceChildren(); + showFolderBrowserNotice(""); + folderBrowserStatusEl.textContent = "Opening…"; + folderBrowserChooseButton.disabled = true; + folderBrowserEl.showModal(); + // A typed path is where browsing starts, so it works as a shortcut too. + showFolderInBrowser(readMainInputFolderPath() || folderBrowserState.lastPath); +} + +async function showFolderInBrowser(folderPath, { focusPath = "" } = {}) { + const requestId = ++folderBrowserState.requestId; + folderBrowserListEl.setAttribute("aria-busy", "true"); + const { ok, data, errorMessage } = await postJson("/api/jobs/browse-folder", { folder_path: folderPath }); + if (requestId !== folderBrowserState.requestId) { + return; + } + + folderBrowserListEl.setAttribute("aria-busy", "false"); + if (!ok) { + showFolderBrowserNotice(errorMessage); + if (!folderBrowserState.path) { + folderBrowserStatusEl.textContent = ""; + } + return; + } + + renderFolderBrowser(data, focusPath); +} + +function renderFolderBrowser(listing, focusPath = "") { + const folders = Array.isArray(listing.folders) ? listing.folders : []; + const breadcrumbs = Array.isArray(listing.breadcrumbs) ? listing.breadcrumbs : []; + const places = Array.isArray(listing.places) ? listing.places : []; + folderBrowserState.path = listing.path; + folderBrowserState.lastPath = listing.path; + folderBrowserState.parentPath = listing.parent_path || null; + folderBrowserState.breadcrumbs = breadcrumbs; + + folderBrowserPlacesEl.replaceChildren(...places.map((place) => { + const button = createFolderButton(place, "folder-browser__place"); + if (place.path === listing.path) { + button.setAttribute("aria-current", "true"); + } + return wrapInListItem(button); + })); + + folderBrowserCrumbsEl.replaceChildren(...breadcrumbs.map((crumb, index) => { + if (index < breadcrumbs.length - 1) { + return wrapInListItem(createFolderButton(crumb, "folder-browser__crumb")); + } + + const here = document.createElement("span"); + here.className = "folder-browser__here"; + here.setAttribute("aria-current", "location"); + here.textContent = crumb.name; + return wrapInListItem(here); + })); + + folderBrowserListEl.setAttribute("aria-label", `Folders inside ${breadcrumbs.at(-1)?.name || listing.path}`); + if (folders.length > 0) { + folderBrowserListEl.replaceChildren(...folders.map((folder) => wrapInListItem(createFolderButton(folder, "folder-browser__folder")))); + } else { + const emptyItem = document.createElement("li"); + emptyItem.className = "folder-browser__empty"; + emptyItem.textContent = "No folders inside."; + folderBrowserListEl.replaceChildren(emptyItem); + } + folderBrowserListEl.scrollTop = 0; + + showFolderBrowserNotice(listing.message || ""); + folderBrowserStatusEl.textContent = describeFolderContents(listing.image_count ?? 0, folders.length); + folderBrowserChooseButton.disabled = false; + + // Keyboard focus stays in the list: on the folder just come up out of, or + // else the first. Only that one is in the tab order; arrow keys move on. + const folderButtons = [...folderBrowserListEl.querySelectorAll("button")]; + const focusTarget = folderButtons.find((button) => button.dataset.path === focusPath) || folderButtons[0]; + folderButtons.forEach((button) => { + button.tabIndex = button === focusTarget ? 0 : -1; + }); + (focusTarget || folderBrowserChooseButton).focus(); +} + +function createFolderButton({ name, path }, className) { + const button = document.createElement("button"); + button.type = "button"; + button.className = className; + button.dataset.path = path; + button.textContent = name; + return button; +} + +function wrapInListItem(child) { + const item = document.createElement("li"); + item.append(child); + return item; +} + +function showFolderBrowserNotice(message) { + folderBrowserNoticeEl.textContent = message; + folderBrowserNoticeEl.hidden = !message; +} + +function describeFolderContents(imageCount, folderCount) { + const images = imageCount > 0 ? pluralize(imageCount, "image") : ""; + const folders = folderCount > 0 ? pluralize(folderCount, "folder") : ""; + if (images && folders) { + return `${images} and ${folders} here.`; + } + if (images) { + return `${images} here.`; + } + return folders ? `${folders} here, no images.` : "No images or folders here."; +} + +// Going up to a folder on the current path lands on the folder just left. +function childOnCurrentPath(folderPath) { + const { breadcrumbs } = folderBrowserState; + const index = breadcrumbs.findIndex((crumb) => crumb.path === folderPath); + return index >= 0 && index < breadcrumbs.length - 1 ? breadcrumbs[index + 1].path : ""; +} + +async function useBrowsedFolder() { + const chosenPath = folderBrowserState.path; + if (!chosenPath || generationState.busy) { + return; + } + + folderBrowserEl.close(); + // As if the path had been typed in, then scanned. + inputFolderEl.value = chosenPath; + // A long path shows its end, the folder's own name, rather than its start. + inputFolderEl.scrollLeft = inputFolderEl.scrollWidth; + resetSingleFolderSelectionState(); + syncDerivedOutputPath(); + resetScanReadyState(); + resetCoverCollageIncludeList(); + await runExclusive(scanFolderAndCheckBundles); + // The scan disables Browse, which drops keyboard focus back to the page. + if (document.activeElement === document.body) { + browseFolderButton?.focus(); + } +} + +browseFolderButton?.addEventListener("click", openFolderBrowser); +folderBrowserCancelButton?.addEventListener("click", () => folderBrowserEl.close()); +folderBrowserChooseButton?.addEventListener("click", useBrowsedFolder); + +// Places, the path and the folders are all buttons carrying the folder they open. +folderBrowserEl?.addEventListener("click", (event) => { + const button = event.target.closest?.("button[data-path]"); + if (button) { + showFolderInBrowser(button.dataset.path, { focusPath: childOnCurrentPath(button.dataset.path) }); + } +}); + +folderBrowserEl?.addEventListener("keydown", (event) => { + // Backspace goes up a level, as in a file manager. + if (event.key === "Backspace" && folderBrowserState.parentPath) { + event.preventDefault(); + showFolderInBrowser(folderBrowserState.parentPath, { focusPath: folderBrowserState.path }); + return; + } + + const folderButtons = [...folderBrowserListEl.querySelectorAll("button")]; + const index = folderButtons.indexOf(document.activeElement); + const nextIndex = { ArrowDown: index + 1, ArrowUp: index - 1, Home: 0, End: folderButtons.length - 1 }[event.key]; + if (index < 0 || nextIndex === undefined) { + return; + } + + event.preventDefault(); + const next = folderButtons[Math.min(Math.max(nextIndex, 0), folderButtons.length - 1)]; + folderButtons[index].tabIndex = -1; + next.tabIndex = 0; + next.focus(); +}); + +// A listing still on its way when the browser closes is dropped. +folderBrowserEl?.addEventListener("close", () => { + folderBrowserState.requestId += 1; +}); + async function runCoverCollage() { const selectedTemplateLabel = readSelectedCoverCollageTemplateLabel(); @@ -1609,6 +1955,49 @@ coverCollageButton?.addEventListener("click", async () => { await runExclusive(runCoverCollage); }); +async function runUpscale() { + const scale = readUpscaleScale(); + + if (generationState.mode !== "batch") { + renderPendingResult(generationResultEl, "Working", `Upscaling every image ${scale}x…`); + revealResult(generationResultEl); + + const { ok, data: result, errorMessage } = await postJson("/api/jobs/upscale-images", readUpscalePayload()); + if (!ok) { + renderFailedResult(generationResultEl, errorMessage); + return; + } + + renderGenerationSummary(result, "upscale"); + return; + } + + const payload = buildBatchUpscalePayload(); + if (payload.selected_subfolders.length === 0 || !hasValidatedBatchSelection()) { + renderPendingResult(generationResultEl, "Check bundles", "Check the picked bundles before upscaling."); + return; + } + + renderPendingResult(generationResultEl, "Working", `Upscaling images ${scale}x for ${pluralize(payload.selected_subfolders.length, "bundle")}…`); + revealResult(generationResultEl); + + const { ok, data: batchResult, errorMessage } = await postJson("/api/jobs/upscale-images-batch", payload); + if (!ok) { + renderFailedResult(generationResultEl, errorMessage); + return; + } + + renderGenerationSummary(batchResult, "upscale"); +} + +upscaleButton?.addEventListener("click", async () => { + if (upscaleButton.disabled || !validateDpiSetting()) { + return; + } + + await runExclusive(runUpscale); +}); + generateButton?.addEventListener("click", async () => { if (generateButton.disabled) { return; diff --git a/src/pasteup/templates/index.html b/src/pasteup/templates/index.html index 12b8b9a..64cc44e 100644 --- a/src/pasteup/templates/index.html +++ b/src/pasteup/templates/index.html @@ -62,7 +62,16 @@

- +
+ + +
@@ -188,7 +197,7 @@

Make your images

-

Preview sheets of every image, or one cover for the listing.

+

Preview sheets of every image, a cover for the listing, or upscaled copies of the images.

Scan a folder first. These unlock once it has images.

@@ -279,6 +288,33 @@

Cover collage

+ +
+
+ +
+

Upscaled images

+

Every image made bigger, as PNG and JPG.

+
+
+ +
+ + +

Saved in an Upscaled folder, one folder per format, at the DPI from step 2. PNGs keep their transparency; JPGs go on white.

+
+ + +
@@ -304,6 +340,35 @@

Pasteup runs on your computer; your images never leave it. API docs

+ +
+
+

Choose the bundle folder

+

Click a folder to open it. Once you're in the right one, click “Use this folder”.

+
+ + + + + + + +
    + + +
    +
    + diff --git a/tests/test_browse_folder_api.py b/tests/test_browse_folder_api.py new file mode 100644 index 0000000..4db58cf --- /dev/null +++ b/tests/test_browse_folder_api.py @@ -0,0 +1,165 @@ +import os +import stat +from pathlib import Path +from types import SimpleNamespace + +from fastapi.testclient import TestClient + +from pasteup.main import app +from pasteup.services import folder_browser +from pasteup.services.folder_scanner import scan_folder + +client = TestClient(app) + + +def _browse(folder_path: str) -> dict: + response = client.post("/api/jobs/browse-folder", json={"folder_path": folder_path}) + assert response.status_code == 200 + return response.json() + + +def _names(links: list[dict]) -> list[str]: + return [link["name"] for link in links] + + +def test_lists_the_folders_inside_and_counts_the_images(tmp_path: Path) -> None: + for name in ("bundle-b", "Bundle A", "bundle-c", ".git", "---Created---"): + (tmp_path / name).mkdir() + for name in ("one.png", "two.JPG", "three.webp", "notes.txt"): + (tmp_path / name).write_bytes(b"fake") + + listing = _browse(str(tmp_path)) + + assert listing["path"] == str(tmp_path) + # Hidden folders and Pasteup's own output folder are left out. + assert listing["folders"] == [ + {"name": name, "path": str(tmp_path / name)} for name in ("Bundle A", "bundle-b", "bundle-c") + ] + assert listing["image_count"] == 3 + assert listing["image_count"] == scan_folder(str(tmp_path))["image_count"] + assert listing["message"] == "" + + +def test_shows_the_way_up_from_the_top_of_the_drive(tmp_path: Path) -> None: + folder = tmp_path / "clip-art" / "bundle-01" + folder.mkdir(parents=True) + + listing = _browse(str(folder)) + + assert listing["parent_path"] == str(folder.parent) + assert listing["breadcrumbs"][0] == {"name": folder.anchor, "path": folder.anchor} + assert listing["breadcrumbs"][-2:] == [ + {"name": "clip-art", "path": str(folder.parent)}, + {"name": "bundle-01", "path": str(folder)}, + ] + assert len(listing["breadcrumbs"]) == len(folder.parts) + + +def test_the_top_of_the_drive_has_no_parent() -> None: + root = Path.cwd().anchor + + listing = _browse(root) + + assert listing["path"] == root + assert listing["parent_path"] is None + assert listing["breadcrumbs"] == [{"name": root, "path": root}] + + +def test_a_blank_path_starts_at_home(tmp_path: Path, monkeypatch) -> None: + monkeypatch.setenv("HOME", str(tmp_path)) + (tmp_path / "Pictures").mkdir() + + listing = _browse(" ") + + assert listing["path"] == str(tmp_path) + assert _names(listing["folders"]) == ["Pictures"] + + +def test_a_home_relative_path_is_expanded(tmp_path: Path, monkeypatch) -> None: + monkeypatch.setenv("HOME", str(tmp_path)) + (tmp_path / "Pictures" / "clip-art").mkdir(parents=True) + + assert _browse("~/Pictures")["path"] == str(tmp_path / "Pictures") + + +def test_a_missing_folder_shows_the_nearest_one_that_exists(tmp_path: Path) -> None: + """A typo in the folder field opens the browser as close to it as it can.""" + (tmp_path / "clip-art").mkdir() + typo = tmp_path / "clip-art" / "bundel-01" / "deeper" + + listing = _browse(str(typo)) + + assert listing["path"] == str(tmp_path / "clip-art") + assert listing["message"] == f"There's no folder at {typo}, so this is the nearest one that exists." + + +def test_a_file_shows_the_folder_it_is_in(tmp_path: Path) -> None: + image = tmp_path / "one.png" + image.write_bytes(b"fake") + + listing = _browse(str(image)) + + assert listing["path"] == str(tmp_path) + assert listing["image_count"] == 1 + assert "no folder at" in listing["message"] + + +def test_a_folder_it_may_not_open_gives_way_to_the_one_above(tmp_path: Path, monkeypatch) -> None: + locked = tmp_path / "Desktop" + locked.mkdir() + real_scandir = os.scandir + + def scandir(path): + if Path(path) == locked: + raise PermissionError(1, "Operation not permitted", str(path)) + return real_scandir(path) + + monkeypatch.setattr(folder_browser.os, "scandir", scandir) + + listing = _browse(str(locked)) + + assert listing["path"] == str(tmp_path) + assert listing["message"].startswith(f"Pasteup isn't allowed to open {locked}.") + + +def test_a_linked_folder_keeps_the_path_it_was_opened_by(tmp_path: Path) -> None: + """~/Dropbox stays ~/Dropbox, rather than jumping to where the link points.""" + target = tmp_path / "Library" / "CloudStorage" / "Dropbox" + target.mkdir(parents=True) + link = tmp_path / "Dropbox" + link.symlink_to(target, target_is_directory=True) + + listing = _browse(str(link)) + + assert listing["path"] == str(link) + assert listing["breadcrumbs"][-1] == {"name": "Dropbox", "path": str(link)} + + +def test_places_are_home_and_the_standard_folders_it_has(tmp_path: Path, monkeypatch) -> None: + for name in ("Pictures", "Desktop", "Music"): + (tmp_path / name).mkdir() + monkeypatch.setattr(folder_browser, "_drive_roots", lambda: ["C:\\", "D:\\"]) + + places = folder_browser.list_places(tmp_path) + + assert places == [ + {"name": "Home", "path": str(tmp_path)}, + {"name": "Desktop", "path": str(tmp_path / "Desktop")}, + {"name": "Pictures", "path": str(tmp_path / "Pictures")}, + {"name": "C:\\", "path": "C:\\"}, + {"name": "D:\\", "path": "D:\\"}, + ] + + +def test_windows_hidden_and_system_folders_are_left_out(monkeypatch) -> None: + monkeypatch.setattr(folder_browser, "_ON_WINDOWS", True) + + def entry(name: str, attributes: int = 0) -> SimpleNamespace: + result = SimpleNamespace(st_file_attributes=attributes) + return SimpleNamespace(name=name, stat=lambda follow_symlinks=True: result) + + assert folder_browser._has_hidden_attribute(entry("AppData", stat.FILE_ATTRIBUTE_HIDDEN)) + assert folder_browser._has_hidden_attribute( + entry("$Recycle.Bin", stat.FILE_ATTRIBUTE_HIDDEN | stat.FILE_ATTRIBUTE_SYSTEM) + ) + assert not folder_browser._has_hidden_attribute(entry("Pictures", stat.FILE_ATTRIBUTE_DIRECTORY)) diff --git a/tests/test_macos_metadata_files.py b/tests/test_macos_metadata_files.py new file mode 100644 index 0000000..b1fc53c --- /dev/null +++ b/tests/test_macos_metadata_files.py @@ -0,0 +1,127 @@ +"""A bundle on a drive a Mac has written to. + +A drive that can't hold macOS's file metadata itself (exFAT, FAT32, some network +shares) gets it as a hidden "._name" file beside each file copied there, and +macOS adds hidden folders of its own (.Spotlight-V100, .Trashes, .fseventsd). A zip +made on a Mac carries the same "._" files in a __MACOSX folder, which appears +when the zip is unpacked anywhere else. None of it is clip art. +""" + +from pathlib import Path + +from fastapi.testclient import TestClient +from PIL import Image + +from pasteup.main import app +from pasteup.services.folder_scanner import list_supported_image_paths + +client = TestClient(app) + +MACOS_FOLDERS = (".Spotlight-V100", ".Trashes", ".fseventsd", "__MACOSX") + + +def _image(path: Path) -> None: + Image.new("RGB", (40, 30), (200, 80, 60)).save(path) + + +def _apple_double(path: Path) -> None: + # What macOS writes: the AppleDouble magic number and version, then filler. + path.write_bytes(b"\x00\x05\x16\x07\x00\x02\x00\x00Mac OS X ".ljust(4096, b"\x00")) + + +def _bundle_copied_by_a_mac(folder: Path, image_count: int) -> Path: + folder.mkdir(parents=True) + for index in range(image_count): + _image(folder / f"art_{index:02}.png") + _apple_double(folder / f"._art_{index:02}.png") + return folder + + +def test_the_scan_counts_only_the_images(tmp_path: Path) -> None: + bundle = _bundle_copied_by_a_mac(tmp_path / "bundle", 12) + + data = client.post("/api/jobs/scan-folder", json={"folder_path": str(bundle)}).json() + + assert data["image_count"] == 12 + assert data["estimated_pages"] == 1 + assert data["sample_files"] == [f"art_{index:02}.png" for index in range(5)] + + +def test_sheets_do_not_report_the_metadata_as_unreadable_images(tmp_path: Path) -> None: + bundle = _bundle_copied_by_a_mac(tmp_path / "bundle", 12) + + data = client.post( + "/api/jobs/generate", + json={ + "folder_path": str(bundle), + "output_path": str(tmp_path / "out"), + "page_width": 400, + "page_height": 300, + "gap": 4, + "margin": 4, + "format": "PNG", + }, + ).json() + + assert data["pages_generated"] == 1 + assert data["total_images"] == 12 + assert data["images_skipped"] == 0 + + +def test_a_cover_draws_on_no_hidden_file_or_folder(tmp_path: Path) -> None: + """Covers look through every folder inside, where hidden ones hold copies.""" + bundle = _bundle_copied_by_a_mac(tmp_path / "bundle", 2) + _bundle_copied_by_a_mac(bundle / "extras", 1) + for name in (".thumbnails", "__MACOSX", "---Created---"): + (bundle / name).mkdir() + _image(bundle / name / "art_00.png") + + found = list_supported_image_paths(bundle, recursive=True) + + assert sorted(found) == [bundle / "art_00.png", bundle / "art_01.png", bundle / "extras" / "art_00.png"] + + +def test_macos_folders_are_not_offered_as_bundles(tmp_path: Path) -> None: + parent = tmp_path / "drive" + bundle = _bundle_copied_by_a_mac(parent / "bundle-01", 12) + for name in MACOS_FOLDERS: + (parent / name).mkdir() + _apple_double(parent / "__MACOSX" / "._bundle-01") + + scan = client.post("/api/jobs/scan-folder", json={"folder_path": str(parent)}).json() + assert scan["available_subfolders"] == [{"display_name": "bundle-01", "full_path": str(bundle)}] + + created_root = parent / "---Created---" + sheets = client.post( + "/api/jobs/generate-all-folders", + json={ + "parent_folder_path": str(parent), + "created_output_root_path": str(created_root), + "page_width": 400, + "page_height": 300, + "gap": 4, + "margin": 4, + "format": "PNG", + }, + ).json() + assert sheets["total_detected_folders"] == 1 + assert [result["folder_name"] for result in sheets["bundle_results"]] == ["bundle-01"] + + picked_anyway = client.post( + "/api/jobs/scan-selected-batch-folders", + json={"parent_folder_path": str(parent), "selected_subfolders": [str(parent / "__MACOSX")]}, + ) + assert picked_anyway.status_code == 400 + + +def test_the_folder_browser_counts_and_lists_as_the_scan_does(tmp_path: Path) -> None: + parent = tmp_path / "drive" + bundle = _bundle_copied_by_a_mac(parent / "bundle-01", 12) + for name in MACOS_FOLDERS: + (parent / name).mkdir() + + def browse(folder: Path) -> dict: + return client.post("/api/jobs/browse-folder", json={"folder_path": str(folder)}).json() + + assert browse(bundle)["image_count"] == 12 + assert [folder["name"] for folder in browse(parent)["folders"]] == ["bundle-01"] diff --git a/tests/test_request_safety.py b/tests/test_request_safety.py index 0c4065a..9fea5c9 100644 --- a/tests/test_request_safety.py +++ b/tests/test_request_safety.py @@ -123,3 +123,4 @@ def test_requests_for_another_host_are_refused() -> None: assert rebinding_client.get("/").status_code == 400 assert rebinding_client.post("/api/jobs/scan-folder", json={"folder_path": "/"}).status_code == 400 + assert rebinding_client.post("/api/jobs/browse-folder", json={"folder_path": "/"}).status_code == 400 diff --git a/tests/test_ui_generation_modes.py b/tests/test_ui_generation_modes.py index 2e1e4a4..7445e53 100644 --- a/tests/test_ui_generation_modes.py +++ b/tests/test_ui_generation_modes.py @@ -62,7 +62,10 @@ def test_app_js_checks_every_listed_bundle_after_a_batch_scan() -> None: script = APP_JS.read_text() assert 'generationState.selectedBatchFolders = generationState.mode === "batch" && selectedBatchFolders.length === 0' in script - assert 'if (generationState.mode === "batch" && hasSelectedBatchFolders()) {\n await validateSelectedBatchFolders();' in script + assert 'if (generationState.mode === "batch" && hasSelectedBatchFolders()) {\n await validateSelectedBatchFolders();' in script + # Both ways of picking a folder, scanning a typed path and choosing one in + # the folder browser, go on to check the bundles. + assert script.count("await runExclusive(scanFolderAndCheckBundles);") == 2 def test_app_js_scans_the_folder_picked_inside_a_single_bundle_folder() -> None: @@ -73,10 +76,53 @@ def test_app_js_scans_the_folder_picked_inside_a_single_bundle_folder() -> None: assert 'await runExclusive(() => scanCurrentFolder());' in script +def test_index_offers_a_folder_browser_beside_the_folder_field() -> None: + html = INDEX_TEMPLATE.read_text() + normalized = _squash_whitespace(html) + + # Named after the field's label, so it reads "Browse… Bundle folder". + assert 'aria-labelledby="browse-folder-button input-folder-label"' in html + assert 'Use this folder' in normalized + + +def test_app_js_browses_through_the_api_and_scans_the_folder_chosen() -> None: + script = APP_JS.read_text() + + assert 'postJson("/api/jobs/browse-folder", { folder_path: folderPath })' in script + assert "folderBrowserEl.showModal();" in script + # Only the latest listing is drawn, so a slow one can't replace a newer one. + assert "if (requestId !== folderBrowserState.requestId) {\n return;\n }" in script + assert "inputFolderEl.value = chosenPath;" in script + # Disabled with the scan buttons while a job runs. + assert "[scanButton, batchScanButton, browseFolderButton].forEach((button) => {" in script + + +def test_index_offers_upscaling_as_an_output_of_its_own() -> None: + html = INDEX_TEMPLATE.read_text() + normalized = _squash_whitespace(html) + select = html[html.index('Upscale images' in normalized + + +def test_app_js_upscales_one_bundle_or_many() -> None: + script = APP_JS.read_text() + + assert 'postJson("/api/jobs/upscale-images", readUpscalePayload())' in script + assert 'postJson("/api/jobs/upscale-images-batch", payload)' in script + assert script.count('renderGenerationSummary(result, "upscale");') == 1 + assert script.count('renderGenerationSummary(batchResult, "upscale");') == 1 + # Unlocked on the same terms as a cover, and locked while a job runs. + assert "const upscaleReady = coverCollageReady;" in script + assert "upscaleButton.disabled = !upscaleReady || generationState.busy;" in script + + def test_app_js_renders_batch_cover_collage_results_per_folder() -> None: script = APP_JS.read_text() - assert 'function renderBatchFolderOutcomeSection(results, kind) {' in script + assert 'function renderBatchFolderOutcomeSection(results, kind, scale) {' in script assert 'function renderBatchSummary(result, kind) {' in script assert 'if (Array.isArray(result.bundle_results)) {' in script assert '{ label: "Bundles done", value: `${doneCount} of ${totalCount}` },' in script @@ -108,9 +154,11 @@ def test_app_js_routes_every_request_through_the_error_aware_helper() -> None: assert script.count("fetch(") == 1 assert "if (response.ok && data) {" in script assert "typeof data?.detail === \"string\"" in script - assert script.count("renderFailedResult(generationResultEl, errorMessage);") == 3 + # Sheets, one cover, covers for many bundles, and upscaling one or many. + assert script.count("renderFailedResult(generationResultEl, errorMessage);") == 5 assert "await runExclusive(runPageGeneration);" in script assert "await runExclusive(runCoverCollage);" in script + assert "await runExclusive(runUpscale);" in script def test_app_js_reads_and_resets_manual_cover_collage_include_list() -> None: diff --git a/tests/test_upscale_images.py b/tests/test_upscale_images.py new file mode 100644 index 0000000..08c1575 --- /dev/null +++ b/tests/test_upscale_images.py @@ -0,0 +1,249 @@ +from pathlib import Path + +import pytest +from fastapi.testclient import TestClient +from PIL import Image + +from pasteup.main import app +from pasteup.services import upscaler + +client = TestClient(app) + +RED = (220, 40, 40) + + +def _photo(path: Path, size: tuple[int, int] = (40, 30)) -> None: + Image.new("RGB", size, (90, 160, 220)).save(path) + + +def _cutout(path: Path) -> None: + """A red square on a transparent page: clip art as it is usually sold.""" + image = Image.new("RGBA", (50, 40), (0, 0, 0, 0)) + image.paste(RED + (255,), (10, 10, 40, 30)) + image.save(path) + + +def _upscale(folder: Path, output: Path, **overrides) -> dict: + response = client.post( + "/api/jobs/upscale-images", + json={"folder_path": str(folder), "output_path": str(output), "scale": 2, "dpi": 300, **overrides}, + ) + assert response.status_code == 200, response.text + return response.json() + + +def test_each_image_is_upscaled_into_a_png_and_a_jpg_folder(tmp_path: Path) -> None: + source = tmp_path / "bundle" + source.mkdir() + _photo(source / "photo.jpg") + _cutout(source / "cutout.png") + output = tmp_path / "---Created---" / "bundle" + + result = _upscale(source, output, scale=3) + + upscaled = output / "Upscaled" + assert result["success"] is True + assert result["total_images"] == 2 + assert result["output_path"] == str(upscaled) + assert result["png_folder"] == str(upscaled / "PNG") + assert result["jpg_folder"] == str(upscaled / "JPG") + assert sorted(result["output_files"]) == sorted( + str(upscaled / folder / f"{name}.{folder.lower()}") + for name in ("cutout", "photo") + for folder in ("PNG", "JPG") + ) + for path in map(Path, result["output_files"]): + with Image.open(path) as image: + assert image.size == ((150, 120) if path.stem == "cutout" else (120, 90)) + assert round(image.info["dpi"][0]) == 300 + + +def test_the_png_keeps_transparency_without_a_dark_fringe_and_the_jpg_goes_on_white(tmp_path: Path) -> None: + source = tmp_path / "bundle" + source.mkdir() + _cutout(source / "cutout.png") + + _upscale(source, tmp_path / "out", scale=4) + + with Image.open(tmp_path / "out" / "Upscaled" / "PNG" / "cutout.png") as png: + assert png.mode == "RGBA" + assert png.getpixel((0, 0))[3] == 0 + # Resampled without premultiplying, the square's edge would blend + # towards the black of the transparent pixels around it (red ~130). + reds, alphas = png.getchannel("R").tobytes(), png.getchannel("A").tobytes() + edge_reds = [red for red, alpha in zip(reds, alphas) if 128 <= alpha < 255] + assert edge_reds and min(edge_reds) > 200 + with Image.open(tmp_path / "out" / "Upscaled" / "JPG" / "cutout.jpg") as jpg: + assert jpg.mode == "RGB" + assert jpg.getpixel((0, 0)) == (255, 255, 255) + + +def test_an_opaque_image_makes_a_png_without_an_alpha_channel(tmp_path: Path) -> None: + source = tmp_path / "bundle" + source.mkdir() + _photo(source / "photo.png") + + _upscale(source, tmp_path / "out") + + with Image.open(tmp_path / "out" / "Upscaled" / "PNG" / "photo.png") as png: + assert png.mode == "RGB" + + +def test_a_phone_photo_is_upscaled_upright(tmp_path: Path) -> None: + source = tmp_path / "bundle" + source.mkdir() + exif = Image.Exif() + exif[0x0112] = 6 # Orientation: rotate 90 degrees to display. + Image.new("RGB", (40, 30), "white").save(source / "phone.jpg", exif=exif) + + _upscale(source, tmp_path / "out") + + with Image.open(tmp_path / "out" / "Upscaled" / "JPG" / "phone.jpg") as jpg: + assert jpg.size == (60, 80) + + +def test_one_image_of_a_name_gives_both_formats(tmp_path: Path) -> None: + """art.png and art.jpg would be saved over each other; the PNG is used.""" + source = tmp_path / "bundle" + source.mkdir() + _cutout(source / "art.png") + _photo(source / "Art.jpg") + + result = _upscale(source, tmp_path / "out") + + assert result["total_images"] == 1 + assert result["same_name_files"] == ["Art.jpg"] + with Image.open(tmp_path / "out" / "Upscaled" / "PNG" / "art.png") as png: + assert png.mode == "RGBA" + assert "Art.jpg" in result["message"] + + +def test_an_image_too_big_to_upscale_is_left_out(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setattr(upscaler, "MAX_UPSCALED_MEGAPIXELS", 0.01) # 10,000 pixels + source = tmp_path / "bundle" + source.mkdir() + _photo(source / "small.png", (40, 30)) # 80 x 60 at 2x: 4,800 pixels + _photo(source / "big.png", (60, 60)) # 120 x 120 at 2x: 14,400 pixels + + result = _upscale(source, tmp_path / "out") + + assert result["total_images"] == 1 + assert result["too_large_files"] == ["big.png"] + assert not (tmp_path / "out" / "Upscaled" / "PNG" / "big.png").exists() + + +def test_an_image_too_big_even_to_open_is_left_out_before_it_is_decoded( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # Pillow refuses to open an image over twice this many pixels. + monkeypatch.setattr(Image, "MAX_IMAGE_PIXELS", 1_000) + source = tmp_path / "bundle" + source.mkdir() + _photo(source / "huge.png", (60, 60)) + + result = _upscale(source, tmp_path / "out") + + assert result["success"] is False + assert result["too_large_files"] == ["huge.png"] + assert result["output_files"] == [] + + +def test_an_unreadable_file_is_skipped(tmp_path: Path) -> None: + source = tmp_path / "bundle" + source.mkdir() + _photo(source / "good.png") + (source / "broken.png").write_bytes(b"not an image") + + result = _upscale(source, tmp_path / "out") + + assert result["total_images"] == 1 + assert result["images_skipped"] == 1 + + +def test_upscaling_again_replaces_what_the_last_run_made(tmp_path: Path) -> None: + source = tmp_path / "bundle" + source.mkdir() + _photo(source / "photo.png") + + _upscale(source, tmp_path / "out", scale=2) + _upscale(source, tmp_path / "out", scale=4) + + png_folder = tmp_path / "out" / "Upscaled" / "PNG" + assert [path.name for path in png_folder.iterdir()] == ["photo.png"] + with Image.open(png_folder / "photo.png") as png: + assert png.size == (160, 120) + + +@pytest.mark.parametrize("scale", [1, 5, 2.5]) +def test_the_scale_must_be_2_3_or_4(tmp_path: Path, scale: float) -> None: + response = client.post( + "/api/jobs/upscale-images", + json={"folder_path": str(tmp_path), "output_path": str(tmp_path / "out"), "scale": scale}, + ) + + assert response.status_code == 422 + assert not (tmp_path / "out").exists() + + +def test_a_folder_without_images_is_refused(tmp_path: Path) -> None: + response = client.post( + "/api/jobs/upscale-images", + json={"folder_path": str(tmp_path), "output_path": str(tmp_path / "out")}, + ) + + assert response.status_code == 400 + assert "No supported image files" in response.json()["detail"] + + +def test_many_bundles_are_each_upscaled_into_their_own_folder(tmp_path: Path) -> None: + parent = tmp_path / "clip-art" + for bundle, count in (("bundle-a", 2), ("bundle-b", 1), ("empty", 0)): + (parent / bundle).mkdir(parents=True) + for index in range(count): + _photo(parent / bundle / f"art_{index}.png") + (parent / "bundle-b" / "broken.png").write_bytes(b"not an image") + created_root = parent / "---Created---" + + response = client.post( + "/api/jobs/upscale-images-batch", + json={ + "parent_folder_path": str(parent), + "created_output_root_path": str(created_root), + "selected_subfolders": [str(parent / name) for name in ("bundle-a", "bundle-b", "empty")], + "scale": 3, + }, + ) + + assert response.status_code == 200 + result = response.json() + assert result["scale"] == 3 + assert result["generated_folders"] == 2 + assert [(r["folder_name"], r["success"], r["total_images"]) for r in result["bundle_results"]] == [ + ("bundle-a", True, 2), + ("bundle-b", True, 1), + ("empty", False, 0), + ] + # What a bundle couldn't use is reported per bundle, as for one bundle. + assert result["bundle_results"][1]["images_skipped"] == 1 + assert (created_root / "bundle-a" / "Upscaled" / "JPG" / "art_1.jpg").exists() + assert (created_root / "bundle-b" / "Upscaled" / "PNG" / "art_0.png").exists() + + +def test_many_bundles_refuses_a_folder_outside_the_parent(tmp_path: Path) -> None: + parent = tmp_path / "clip-art" + (parent / "bundle").mkdir(parents=True) + outside = tmp_path / "elsewhere" + outside.mkdir() + _photo(outside / "art.png") + + response = client.post( + "/api/jobs/upscale-images-batch", + json={ + "parent_folder_path": str(parent), + "created_output_root_path": str(parent / "---Created---"), + "selected_subfolders": [str(outside)], + }, + ) + + assert response.status_code == 400 + assert not (parent / "---Created---").exists()