Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,20 @@

All notable public changes to `sprite-gen` are recorded here. Versions track the `version:` field in `SKILL.md` and `pyproject.toml`.

## Unreleased (v2.5.0)
## v2.5.0 - API image generation and smoother loop cuts

- New `openai` image provider: `sprite-gen gen --provider openai` calls the OpenAI Images REST API with nothing but `OPENAI_API_KEY` — the credential a headless container (a Modal worker, CI, a SaaS backend) can have, where the `codex` route's interactive ChatGPT login cannot exist. New images go to `/v1/images/generations`, `--ref` switches to `/v1/images/edits` as multipart with the references as repeated `image[]` parts in order (up to 16), and gpt-image's inline base64 is decoded and published as a verified PNG without resizing. Default model `gpt-image-2.5-flare`. `--transparent` asks for `background: transparent` with `output_format: png` — the same measured `native` strategy as codex, and a live run came back 84 % alpha-0 with the subject at alpha 251–254. `--aspect-ratio` maps to one of the gpt-image `size` values that satisfy the API's constraints (both sides divisible by 16, ratio within 1:3..3:1, 655,360–8,294,400 pixels); a ratio with no exact size is refused rather than rounded to a nearby one you would be billed for. A missing or empty key, a rejected key and a failed request are all terminal: this provider never falls back to codex, to another credential, or to a retry.
- sprite-gen stays subscription-first. `codex` and `grok` run on a subscription you already pay for; `openai` bills per call, so it runs **only** when `--provider openai` names it. `SPRITE_GEN_DEFAULT_PROVIDER=openai` is refused rather than honoured, the guided `workflow` flow neither offers it nor saves it as a preference, no availability fallback targets it (a codex outage still reaches grok, never metered credit), and having `OPENAI_API_KEY` in the environment changes no route by itself. Every call that does spend API credit — image or video — prints one stderr line naming the charge before the request leaves.
- `gen --quality low|medium|high|xhigh|max|auto` and `gen --resolution 1k|1.5k|2k`: the two knobs an image is billed on, as one shared vocabulary with a per-provider subset. grok Imagine carries `auto|low|medium` and all three resolution tiers and prices an image on the pair; openai carries the whole quality range and takes its size from `--aspect-ratio`; codex `image_gen` exposes neither dial. A knob a provider cannot honour fails loudly instead of being dropped from a request body you are about to pay for. Omitting both keeps every existing call byte-identical. The resolution names are tiers, not pixel counts — `1.5k` rendered 1408x1408 at 1:1 (2026-09-20 measurement) — and the grok subsets were read off the server rather than the prose: `quality` deserializes the wider shared enum and then refuses per model (`high` → HTTP 400 "This model only supports the following quality value(s): low, medium, auto."), `resolution: 1.5k` renders although the capability guide lists only 1k and 2k, and an unknown field is not refused at all, which is why these names are checked locally. Grok image reports carry `extra.quality` and `extra.resolution` when the request sent them.
- `video`, `video-extend` and `video-edit` announce the per-call charge on stderr when they run on `XAI_API_KEY` instead of the Grok subscription login — one line per submitted job, naming the duration and resolution it is priced on. The subscription route stays silent and no other video behaviour changed.
- Registering a provider now fails loudly when the workflow catalog has no label for it. The guided flow built its provider choices by zipping labels onto the provider tuple positionally, so a third provider was dropped without a word; labels are a checked mapping (a missing one fails at import) and the flow offers the subscription routes only. `workflow` access probing knows `openai` — ready when the key is set, with `billing: api-credit` and the billing confirmation that goes with it.

- `video-loop` automatic cuts now score the transition actually played from the last displayed frame back to the first. The chooser prefers a step comparable to the cycle's ordinary motion rather than the smallest distance, which could repeat a pose and pause at the wrap. Reports retain `next_frame_distance` as a separate diagnostic.
- `video-frames --spill full` now lowers the tint threshold as well as lifting the cluster-size limit, so faint key-colour reflections are corrected. `--spill auto` examines the reference at the same threshold before selecting full correction, preserving the character's own key-coloured material. The conservative `small` mode and the default 3-second video-set duration are unchanged.

- Gait auto-detection now considers an ambiguous double-period candidate even when the short candidate exceeds the duration floor. It preserves both phase occurrences at source speed when the longer local minimum is similarly credible and passes the periodicity gate; near-exact short repeats, worse doubles, non-gait states and caller window bounds are preserved. Reports identify this conservative choice with `half_period_guard.reason = "ambiguous-harmonic"` and `cycle.review_recommended`, rather than claiming anatomical stride certainty.
- Added a [loop review guide](docs/loop-review.md) defining what numeric gates establish, when sequential visual review is needed, and how to record explicit cut/alignment choices without presenting them as automatic decisions.

## v2.4.1 - A visible hop is no longer refused as "never leaves its rest pose"

- `video-loop` one-shot detection no longer refuses a visible hop because the rest pose is not one pose. The excursion is admitted by either the peak's height in MADs of the rest noise (as before) or the fraction of the subject's pixel mass the peak moves (new, ≥ 0.4); the report records `excursion_moved` and `excursion_rule`. A body that walked a few steps, hopped 42 px and froze scored 1.8 MADs because the walking preamble and the frozen tail inflate the "rest noise"; by moved mass it scores 1.07. A jittering stand still fails both rules, and clips the MAD rule already accepted are cut exactly as before.
Expand Down
2 changes: 1 addition & 1 deletion SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: sprite-gen
version: 2.4.1
version: 2.5.0
description: "Generates images and game sprites through GPT or Grok with guided provider choices, separate saved defaults, automatic cleanup and optional curation. Handles sprite requests, ordinary image generation/editing, standalone image-to-video clips (i2v, animate this still, 그록 영상, 이매진 비디오, 스틸 움직여줘, first/last frame, reference-to-video, 영상 이어붙이기, 영상 편집, extend/edit a clip), chroma removal, animation atlases, video loops, 큐레이션뷰, image candidates, 팔레트 스왑, palette swap, recolor, rig layers, engine exports, repeating backgrounds, projected shadows, motion/contact inspection and optional scene composition from existing assets."
license: Apache-2.0
depends_on:
Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ grouping is derived from `sprite_gen/_modules.py`, the one taxonomy table.
| [gen.md](gen.md) | `sprite-gen gen` / `gen-set`: providers, default resolution, transparency strategy per provider, row usage |
| [video.md](video.md) | `sprite-gen video` / `video-extend` / `video-edit`: stills and clips to mp4 through Grok Imagine (image-to-video, last-frame pin, references, extension, editing) with the user's own credential |
| [video-pipeline.md](video-pipeline.md) | Pipeline B engine contract: state canvas, keyed frames, true-period and one-shot cycles, strip/GIF/WebP, the batch |
| [loop-review.md](loop-review.md) | Automatic loop decisions, ambiguous gait review, visual evidence and explicit cut/alignment overrides |
| [frame-interpolation.md](frame-interpolation.md) | Generative in-betweens for sprite frames, recorded as a take |
| [seamless-video-loop.md](seamless-video-loop.md) | Making a non-looping ambient clip loop forever (RIFE seam bridge) — a different job from pipeline B |

Expand Down
86 changes: 86 additions & 0 deletions docs/loop-review.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Loop automation and visual review

Use the normal pipeline first. No model needs to choose every cut:

```bash
sprite-gen video-loop --frames-dir keyed/ --out-dir loop/ --state run --fps 24
```

The caller must pass the **measured source fps**, not always 24. `video-set` obtains it
from `video-frames`. The default anchor is `none` in both commands. The default clip
request remains 3 seconds; it does not determine the playback speed.

## What the script decides

- It measures repeat candidates from the keyed frames and the state's time window.
- It scores the last displayed frame -> first frame against ordinary adjacent motion,
reducing repeated-pose stalls at a loop boundary.
- Gaits below the duration floor may use a double period, as before. Above that floor,
similarly credible non-exact candidates at P and approximately 2P now retain 2P once
when it fits the window and passes the periodicity gate. Keeping two real repetitions
is preferable to discarding a potentially distinct phase. This may produce a two-cycle
loop of a genuine short gait; frames keep their original timing.
- Near-exact short repeats, implausible longer repeats, and non-gait states are not
extended by that ambiguity policy. There is no character-specific frame count.
- Alpha, frame counts and seam metrics are checked before the output is accepted.

A numeric pass establishes those facts. It does **not** establish correct foot contact,
anatomical left/right alternation, a believable gait, or that the source was drawn well.
`cycle.review_recommended = true` marks an ambiguous-harmonic choice; `false` means
that this particular heuristic did not flag ambiguity, not that visual quality is certified.
The direct CLI summary preserves that flag and the guard evidence. `video-set` also
propagates them in its item report and marks the table row `OK (review gait)`.

## Material for a vision-capable reviewer

Provide the original clip, the generated loop played at source speed for at least three
repeats, the JSON report, and an ordered contact sheet with original frame indices. For
an ambiguous harmonic include both P and 2P candidates using the same size and fps.
A single still cannot establish a gait cycle. If the model cannot consume video, provide
ordered frames spanning at least two candidate cycles, including several frames on both
sides of the wrap. Do not ask it to judge motion from an unordered image collage.

A human or a vision-capable model should check:

1. Do the limbs alternate, or does one phase restart before its counterpart appears?
2. At the wrap, does motion stall, snap, or reverse compared with nearby frames?
3. Does the torso translate across the canvas, or are only the limbs/clothing moving?
4. Are deformation or contact errors already present in the source clip?

Return the evidence frame indices, chosen mode/start/length, anchor, reason and any
uncertainty. Keep that decision separate from the unmodified automatic report. A model
without the required visual input must not claim it reviewed the gait. Ambiguous or
contradictory visual evidence should remain unresolved rather than inventing a precise cut.

## Explicit adjustments

If the visible full cycle is clear but the automatic candidate is wrong, make the choice
explicit and keep the report's `kind = fixed`:

```bash
sprite-gen video-loop --frames-dir keyed/ --out-dir reviewed/ --state run \
--fps 24 --cycle fixed --start <zero-based-index> --length <frame-count>
```

Do not double a frame count unconditionally or duplicate existing frames. Choose a
continuous interval in the original sequence. `--min-len/--max-len` can instead narrow
an evidenced search interval while leaving the exact cut to the detector.

Use `--anchor feet` only when the source shows slow in-canvas translation. It removes a
linear horizontal trend; it is not a universal stabilizer, does not pin each planted foot,
and can alter an already stationary subject. Start with the default `none`. Compare with
`feet` on identical frames before selecting it. Normal body bounce, clothing motion and
a model's changing anatomy are not position drift.

Do not raise `--seam-max` merely to make a failed result pass. A source lacking a complete
usable cycle may require a longer/new clip. Regeneration is a new generation request,
not a hidden fallback inside loop extraction.

## Reproducibility

Store the source identity, engine version, exact CLI arguments, selected report and
whether a person/model overrode the automatic choice. Keep the same fps for comparisons.
Render comparison videos from PNG strips or lossless frame outputs, not palette GIFs.
A reviewed example is not evidence that all future inputs will automatically attain its
quality. Consumers can route `review_recommended` to review while retaining the automatic
result as an explicitly labelled candidate.
24 changes: 19 additions & 5 deletions docs/video-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,16 +140,17 @@ The still the clip was made from settles it. `video-frames --spill` takes:
| Mode | What is corrected |
|---|---|
| `small` (default) | only small key-tinted clusters — the still pipeline's rule, byte-identical output |
| `full` | every key-tinted cluster, whatever its size (colour only: alpha is unchanged) |
| `auto` | keys `--reference` (the still) with the same matte and counts its strongly key-tinted pixels; a share ≤ 0.5 % means the still has no key-coloured material of its own → `full`, otherwise `small` |
| `full` | key-tinted clusters of any size, including faint tints (colour only: alpha is unchanged) |
| `auto` | keys `--reference` (the still) with the same matte and counts its key-tinted pixels at the same threshold used by `full`; a share ≤ 0.5 % means the still has no key-coloured material of its own → `full`, otherwise `small` |

`video-set` passes `--spill auto` with each item's `canvas.png` as the reference (override
with `--spill small|full`), so a green-free character loses the reflections while a
character that *is* green keeps its colour. The decision and its numbers are recorded in
the frames report under `spill`. The correction is the engine's own `despill_color` blend
model (observed = (1−k)·subject + k·key, solved for the subject), so colours without key
tint are untouched. Faint tints whose cluster never crosses the strong-tint bar are left
as they are.
tint are untouched. `small` keeps the conservative tint threshold of 40; `full` lowers it to 8.
The `auto` reference check also uses 8, so faint key-coloured material in the
original character keeps the conservative correction.

## 3b. Canvas shape for raised limbs and wide costumes

Expand All @@ -173,7 +174,10 @@ was 17). `video-loop` therefore:
*smallest* local minimum that is within 15 % of the deepest one — exact repeats dip
again at 2× and 3× the period, the half-period look-alike dips noticeably less;
3. only then picks the **start** with the best seam for that period (± 1 frame):
`seam = D[i][i+L]` over the mean adjacent distance inside the cycle.
`seam = D[i+L-1][i]` over the mean adjacent distance inside the cycle.
Choose the ratio closest to 1 in log space, so a repeated pose at the wrap does
not win just because its distance is small. `next_frame_distance` retains the
distance to the frame after the cycle for diagnostics.

Windows come from the state profile (`STATE_PROFILES`). **Gait states take theirs in
seconds**, because a stride is a fact about the body, not about the clip length: walk
Expand All @@ -192,6 +196,16 @@ periodicity gate measures the period's dip against the profile mean over the who
window, and a ceiling that grows with the clip inflates that mean until a single hop
in a jittering stand passes as a walk.

For gait states, a duration above the floor is not proof that both phases are present.
If a local minimum near twice the chosen period is within the existing 25 % repeat-error
tolerance and still passes the periodicity gate, the detector retains the longer candidate
once, inside the requested window. Near-exact repeats (repeat error at most 10 % of an
ordinary adjacent step) stay short. This is a conservative response to ambiguous harmonics:
a genuine short gait may be shown twice, at the same source speed. It does not identify
anatomical left/right contacts. The report records `half_period_guard.reason =
"ambiguous-harmonic"` and `cycle.review_recommended = true`. See [loop review](loop-review.md)
for the visual review contract and manual overrides.

Gates, all fail-loud: no period (profile flat, `periodicity < 0.15`), loop seam ratio
above `--seam-max` (2.0), GIF/WebP re-opened and checked (frame count, `loop=0`,
transparent corners, no RGB under alpha 0 in the WebP).
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ build-backend = "setuptools.build_meta"
name = "sprite-gen"
# Release discipline: keep this package metadata version synchronized with
# SKILL.md's `version:` field in the same release commit.
version = "2.4.1"
version = "2.5.0"
description = "Component-row pipeline for clean 2D game sprites and animation atlases"
readme = "README.md"
license = "Apache-2.0"
Expand Down
16 changes: 12 additions & 4 deletions sprite_gen/frames/cutout.py
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,8 @@ def _matte_route(
}


def extract_route(image: Image.Image, kind: str, *, spill_max_fraction: float | None = None) -> tuple[Image.Image, dict[str, Any]]:
def extract_route(image: Image.Image, kind: str, *, spill_max_fraction: float | None = None,
spill_min_tint: float | None = None) -> tuple[Image.Image, dict[str, Any]]:
"""Magenta/green key background → reuse the verified `extract` chroma engine (no drift).

The engine keys from the background colour it detects on the borders
Expand All @@ -285,7 +286,11 @@ def extract_route(image: Image.Image, kind: str, *, spill_max_fraction: float |

target = KEY_TARGETS[kind]
painted = detect_background_key_rgb(image, target)
extra = {} if spill_max_fraction is None else {"spill_max_fraction": spill_max_fraction}
extra: dict[str, float] = {}
if spill_max_fraction is not None:
extra["spill_max_fraction"] = spill_max_fraction
if spill_min_tint is not None:
extra["spill_min_tint"] = spill_min_tint
result = remove_chroma_background(
image, target, _EXTRACT_KEY_THRESHOLD, _EXTRACT_FRINGE_THRESHOLD, _EXTRACT_FRINGE_DELTA, **extra
)
Expand All @@ -307,12 +312,14 @@ def cutout(
tolerance: int = CHROMA_TOLERANCE,
white_check_dir: Path | None = None,
spill_max_fraction: float | None = None,
spill_min_tint: float | None = None,
) -> dict[str, Any]:
"""Cut a uniform-background imported image to a clean transparent RGBA PNG.

`key`: "auto" (detect from corners) | "white" (matte) | "magenta" | "green"
(reuse the extract chroma engine). `spill_max_fraction` overrides the chroma
engine's trapped-spill cluster cap (None = the engine default). Returns a stats dict. Raises SystemExit if
engine's trapped-spill cluster cap and `spill_min_tint` its tint bar (None = the
engine default for either). Returns a stats dict. Raises SystemExit if
the key is unknown, the background cannot be located, or any transparent pixel
keeps non-zero RGB (No Silent Fallback).
"""
Expand All @@ -324,7 +331,8 @@ def cutout(

route = _detect_key_kind(_corner_average(image)) if key == "auto" else key
if route in ("magenta", "green"):
result, route_stats = extract_route(image, route, spill_max_fraction=spill_max_fraction)
result, route_stats = extract_route(image, route, spill_max_fraction=spill_max_fraction,
spill_min_tint=spill_min_tint)
else:
result, route_stats = _matte_route(image, input_path, strength, band, erode, tolerance)

Expand Down
Loading
Loading