Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

All notable public changes to `sprite-gen` are recorded here. Versions track the `version:` field in `SKILL.md` and `pyproject.toml`.

## v2.5.4 - Room behind the attack canvas

- Attack canvases keep 20 % of their width empty behind the subject (`--trail`, a new wide-canvas margin next to `--lead`). A long weapon drawn back before the strike reached the back edge of the old layout and the frames gate refused the clip; the samurai and katana samples that needed hand-padded stills in v2.5.3 now pass from the raw still. Headroom, lead and every other profile are unchanged, and `--trail 0` restores the previous placement.
- `video-set --base` refuses a direction the prompt table does not know (`back`, `front`, `side`) instead of accepting `left=` and animating it with the default facing.
- The facing test suite names its orchestrator scrub sample neutrally.

## v2.5.3 - Reference facing controls and observed action returns

- Reference edits can opt into `gen --facing right|left` to request orientation in the generation prompt. The default `preserve` leaves existing generation callers unchanged.
Expand Down
2 changes: 1 addition & 1 deletion SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: sprite-gen
version: 2.5.3
version: 2.5.4
description: "Generates images and game sprites through GPT or Grok with guided provider choices, separate saved defaults, automatic cleanup and optional curation. Handles sprite requests, ordinary image generation/editing, standalone image-to-video clips (i2v, animate this still, 그록 영상, 이매진 비디오, 스틸 움직여줘, first/last frame, reference-to-video, 영상 이어붙이기, 영상 편집, extend/edit a clip), chroma removal, animation atlases, video loops, 큐레이션뷰, image candidates, 팔레트 스왑, palette swap, recolor, rig layers, engine exports, repeating backgrounds, projected shadows, motion/contact inspection and optional scene composition from existing assets."
license: Apache-2.0
depends_on:
Expand Down
7 changes: 4 additions & 3 deletions docs/video-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,13 @@ is a property of the motion state, owned by one table (`STATE_CANVAS`):
| State | Shape | Ratio | Room | Why |
|---|---|---|---|---|
| `jump` | tall | 3:4 | 34 % head-room above the still | airborne frames need height |
| `attack` | wide | 16:9 | 35 % above the still, at least 28 % in front (facing side) | weapon swings rise overhead and extend in front |
| `attack` | wide | 16:9 | 35 % above the still, at least 28 % in front (facing side), 20 % behind | weapon swings rise overhead and extend in front; a long weapon drawn back reaches behind |
| `projectile` | wide | 16:9 | 34 % in front | the projectile travels away |
| everything else | square | 1:1 | — | in-place motion fits the still |

`--shape tall|wide|square` overrides the row; `--headroom` / `--lead` tune the room;
`--facing left` mirrors the wide layout. Headroom is a fraction of the full canvas
`--shape tall|wide|square` overrides the row; `--headroom` / `--lead` / `--trail` tune the room
(`--trail` is the empty fraction of the width kept behind the subject, for a weapon drawn
back before the strike); `--facing left` mirrors the wide layout. Headroom is a fraction of the full canvas
height; wide canvases grow both dimensions to preserve their ratio without shrinking
the still. A still whose corners are not one flat colour
is refused — a non-flat background cannot be extended without guessing.
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ build-backend = "setuptools.build_meta"
name = "sprite-gen"
# Release discipline: keep this package metadata version synchronized with
# SKILL.md's `version:` field in the same release commit.
version = "2.5.3"
version = "2.5.4"
description = "Component-row pipeline for clean 2D game sprites and animation atlases"
readme = "README.md"
license = "Apache-2.0"
Expand Down
5 changes: 4 additions & 1 deletion sprite_gen/video/batch.py
Original file line number Diff line number Diff line change
Expand Up @@ -267,7 +267,10 @@ def _parse_bases(values: list[str]) -> dict[str, Path]:
if "=" not in v:
raise SystemExit(f"video-set: --base expects direction=path, got {v!r}")
d, p = v.split("=", 1)
bases[d.strip()] = Path(p).expanduser().resolve()
d = d.strip()
if d not in VIEW_TEXT:
raise SystemExit(f"video-set: --base direction must be one of {', '.join(sorted(VIEW_TEXT))}, got {d!r}")
bases[d] = Path(p).expanduser().resolve()
if not bases:
raise SystemExit("video-set: at least one --base direction=path is required")
return bases
Expand Down
35 changes: 21 additions & 14 deletions sprite_gen/video/canvas.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,21 +48,22 @@ class CanvasProfile:
ratio: float # width / height
headroom: float # fraction of the canvas height kept empty ABOVE the still (tall/wide)
lead: float # fraction of the canvas width kept empty IN FRONT of the subject (wide)
trail: float # fraction of the canvas width kept empty BEHIND the subject (wide)
why: str


# The one table. Keys are state names as the sprite-request uses them; unknown
# states fall through to `default`.
STATE_CANVAS: dict[str, CanvasProfile] = {
"jump": CanvasProfile(SHAPE_TALL, 3 / 4, 0.34, 0.0, "airborne frames need head-room; hair clipped at 1:1"),
"attack": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.35, 0.28, "weapon swings rise overhead and extend in front"),
"projectile": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.0, 0.34, "projectile travels away from the body"),
"jump": CanvasProfile(SHAPE_TALL, 3 / 4, 0.34, 0.0, 0.0, "airborne frames need head-room; hair clipped at 1:1"),
"attack": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.35, 0.28, 0.2, "weapon swings rise overhead and extend in front; a long weapon drawn back reaches behind"),
"projectile": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.0, 0.34, 0.0, "projectile travels away from the body"),
# Raised-limb celebrations leave a square frame at the top corners; wide with a
# symmetric margin keeps them inside (lead applies in front, the rest pads the back).
"cheer": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.0, 0.30, "arms raised and spread leave a 1:1 frame"),
"wave": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.0, 0.30, "a raised waving arm leaves a 1:1 frame"),
"celebrate": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.0, 0.30, "same envelope as cheer"),
"default": CanvasProfile(SHAPE_SQUARE, 1.0, 0.0, 0.0, "in-place motion fits the still's own frame"),
"cheer": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.0, 0.30, 0.0, "arms raised and spread leave a 1:1 frame"),
"wave": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.0, 0.30, 0.0, "a raised waving arm leaves a 1:1 frame"),
"celebrate": CanvasProfile(SHAPE_WIDE, 16 / 9, 0.0, 0.30, 0.0, "same envelope as cheer"),
"default": CanvasProfile(SHAPE_SQUARE, 1.0, 0.0, 0.0, 0.0, "in-place motion fits the still's own frame"),
}
SHAPE_DEFAULTS: dict[str, CanvasProfile] = {
SHAPE_SQUARE: STATE_CANVAS["default"],
Expand Down Expand Up @@ -161,6 +162,7 @@ def pad_canvas(
facing: str = "right",
headroom: float | None = None,
lead: float | None = None,
trail: float | None = None,
key: str = "auto",
) -> tuple[Image.Image, dict[str, Any]]:
"""Return (padded RGB image, placement report). Never downsizes the still.
Expand All @@ -178,8 +180,9 @@ def pad_canvas(
w, h = src.size
head = profile.headroom if headroom is None else headroom
front = profile.lead if lead is None else lead
if not 0 <= head < 0.9 or not 0 <= front < 0.9:
raise SystemExit("video-canvas: --headroom/--lead must be in [0, 0.9)")
back = profile.trail if trail is None else trail
if not 0 <= head < 0.9 or not 0 <= front < 0.9 or not 0 <= back < 0.9 or not front + back < 0.9:
raise SystemExit("video-canvas: --headroom/--lead/--trail must be in [0, 0.9) and lead + trail below 0.9")
if profile.shape == SHAPE_SQUARE:
side = max(w, h)
canvas_w, canvas_h = side, side
Expand All @@ -190,12 +193,13 @@ def pad_canvas(
canvas_h = max(h, round(h / (1 - head)), round(w / profile.ratio))
canvas_w = max(w, round(canvas_h * profile.ratio))
x, y = (canvas_w - w) // 2, canvas_h - h
else: # wide: extra width goes in front of the facing direction; at least the profile ratio
else: # wide: `trail` of the width stays empty behind the subject, the rest of the extra width goes in front; at least the profile ratio
required_h = max(h, round(h / (1 - head)))
canvas_w = max(w, round(w / (1 - front)), round(required_h * profile.ratio))
canvas_w = max(w, round(w / (1 - front - back)), round(required_h * profile.ratio))
canvas_h = max(h, round(canvas_w / profile.ratio))
y = canvas_h - h
x = 0 if facing == "right" else canvas_w - w
behind = min(round(canvas_w * back), canvas_w - w)
x = behind if facing == "right" else canvas_w - w - behind
canvas = Image.new("RGB", (canvas_w, canvas_h), fill)
canvas.paste(src, (x, y))
report = {
Expand All @@ -206,6 +210,7 @@ def pad_canvas(
"offset": [x, y],
"headroom": head,
"lead": front,
"trail": back,
"facing": facing,
"key_rgb": list(fill),
"corner_rgb": list(corner),
Expand All @@ -226,12 +231,13 @@ def run_canvas(
lead: float | None,
report_path: Path | None,
key: str = "auto",
trail: float | None = None,
) -> dict[str, Any]:
still = still.expanduser().resolve()
if not still.is_file():
raise SystemExit(f"video-canvas: still not found: {still}")
profile = profile_for(state, shape)
canvas, report = pad_canvas(Image.open(still), profile, facing=facing, headroom=headroom, lead=lead, key=key)
canvas, report = pad_canvas(Image.open(still), profile, facing=facing, headroom=headroom, lead=lead, trail=trail, key=key)
out = out.expanduser().resolve()
out.parent.mkdir(parents=True, exist_ok=True)
tmp = out.with_name(out.name + ".part")
Expand All @@ -251,6 +257,7 @@ def add_arguments(parser: argparse.ArgumentParser) -> None:
parser.add_argument("--facing", choices=("right", "left"), default="right", help="which way the subject faces (wide canvases add room in front)")
parser.add_argument("--headroom", type=float, help="tall/wide: empty fraction of canvas height above the still (default from the profile)")
parser.add_argument("--lead", type=float, help="wide: empty fraction in front of the subject (default from the profile)")
parser.add_argument("--trail", type=float, help="wide: empty fraction behind the subject, for a weapon drawn back (default from the profile)")
parser.add_argument("--key", choices=KEYS, default="auto", help="chroma key of the still (auto reads the corners; green/magenta are normalized to the exact key; white pads with the corner colour)")
parser.add_argument("--report", type=Path, help="write the canvas report JSON here")

Expand All @@ -259,7 +266,7 @@ def run(**kwargs: object) -> int:
payload = run_canvas(
Path(str(kwargs["still"])), Path(str(kwargs["out"])),
state=kwargs.get("state"), shape=kwargs.get("shape"), facing=str(kwargs.get("facing") or "right"), # type: ignore[arg-type]
headroom=kwargs.get("headroom"), lead=kwargs.get("lead"), report_path=kwargs.get("report"), # type: ignore[arg-type]
headroom=kwargs.get("headroom"), lead=kwargs.get("lead"), trail=kwargs.get("trail"), report_path=kwargs.get("report"), # type: ignore[arg-type]
key=str(kwargs.get("key") or "auto"),
)
print(json.dumps(payload, ensure_ascii=False, indent=2))
Expand Down
4 changes: 2 additions & 2 deletions tests/gen/test_facing.py
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ def http(*a, **kw):

def test_codex_vision_reads_new_answer_and_scrubs_identity(monkeypatch, tmp_path):
seen = []
monkeypatch.setenv("KUMA_MEMBER_ID", "synthetic")
monkeypatch.setenv("ORCHESTRATOR_MEMBER_ID", "synthetic")
def run(command, **kwargs):
seen.append((command, kwargs))
Path(command[command.index("-o") + 1]).write_text('right')
Expand All @@ -227,7 +227,7 @@ def run(command, **kwargs):
text, metadata = facing_vision.codex_inspect(FIXTURES / "left.png", tmp_path)
assert text == "right" and metadata["auth_source"] == "codex-login"
assert len(seen) == 1 and "--ephemeral" in seen[0][0] and "read-only" in seen[0][0]
assert "KUMA_MEMBER_ID" not in seen[0][1]["env"]
assert "ORCHESTRATOR_MEMBER_ID" not in seen[0][1]["env"]
assert not list(tmp_path.iterdir())


Expand Down
31 changes: 31 additions & 0 deletions tests/video/test_batch_bases.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
"""`video-set --base` accepts only the directions the prompt table knows.

An undocumented key such as `left=` used to be accepted silently and then
animated with the default facing, so the caller's intent was inverted without
a word. The parser now names the allowed directions and refuses the rest.
"""
from __future__ import annotations

import pytest

from sprite_gen.video import batch


def test_known_directions_parse(tmp_path):
still = tmp_path / "s.png"
still.write_bytes(b"")
bases = batch._parse_bases([f"side={still}", f"front={still}", f"back={still}"])
assert set(bases) == {"side", "front", "back"}


@pytest.mark.parametrize("key", ["left", "right", "Side", "profile"])
def test_unknown_direction_is_refused_by_name(key, tmp_path):
with pytest.raises(SystemExit) as exc:
batch._parse_bases([f"{key}={tmp_path / 's.png'}"])
message = str(exc.value)
assert repr(key) in message and "back, front, side" in message


def test_missing_equals_still_refused():
with pytest.raises(SystemExit):
batch._parse_bases(["side"])
7 changes: 5 additions & 2 deletions tests/video/test_batch_facing.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,14 @@ def video(image, prompt, out, report, **kw):
if not options:
assert (tmp_path / "side.facing.png").read_bytes() == source_bytes
assert result["items"][0]["facing"]["final_direction_source"] == "observation"
assert (canvas["offset"][0] > 0) == (facing == "left")
behind = round(canvas["canvas"][0] * canvas["trail"])
assert canvas["facing"] == facing
assert canvas["offset"][0] == (behind if facing == "right" else canvas["canvas"][0] - canvas["still"][0] - behind)
else:
assert "facing" not in result["items"][0]
assert prompts[0] == batch.build_prompt(direction, "attack", None, facing="right")
assert canvas["offset"][0] == 0
assert canvas["facing"] == "right"
assert canvas["offset"][0] == round(canvas["canvas"][0] * canvas["trail"])


def test_changed_facing_cannot_reuse_an_opposite_prompt_clip(tmp_path, offline, monkeypatch):
Expand Down
64 changes: 64 additions & 0 deletions tests/video/test_canvas_trail.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
"""A wide canvas keeps `trail` of its width empty behind the subject.

A long weapon drawn back before a strike reaches behind the body; with every
spare pixel placed in front, that swing met the back edge and the frames gate
refused the clip. The attack profile now reserves room on both sides.
"""
from __future__ import annotations

import pytest
from PIL import Image

from sprite_gen.video import canvas


def _still(w: int = 200, h: int = 200) -> Image.Image:
img = Image.new("RGB", (w, h), (0, 255, 0))
img.paste((120, 80, 40), (60, 40, 140, 200))
return img


def test_attack_profile_reserves_room_behind_and_reports_it():
padded, report = canvas.pad_canvas(_still(), canvas.STATE_CANVAS["attack"], facing="right")
canvas_w, canvas_h = padded.size
x, y = report["offset"]
assert report["trail"] == canvas.STATE_CANVAS["attack"].trail > 0
assert x == round(canvas_w * report["trail"])
assert x + 200 <= canvas_w and y == canvas_h - 200
# still pixels are preserved at the offset
assert padded.getpixel((x + 100, y + 100)) == (120, 80, 40)
assert padded.getpixel((x // 2, canvas_h - 1)) == (0, 255, 0)


def test_trail_mirrors_for_a_left_facing_subject():
profile = canvas.STATE_CANVAS["attack"]
_, right = canvas.pad_canvas(_still(), profile, facing="right")
padded, left = canvas.pad_canvas(_still(), profile, facing="left")
canvas_w = padded.size[0]
assert left["canvas"] == right["canvas"]
assert left["offset"][0] == canvas_w - 200 - right["offset"][0]


def test_trail_zero_keeps_the_old_placement():
_, report = canvas.pad_canvas(_still(), canvas.STATE_CANVAS["attack"], facing="right", trail=0.0)
assert report["offset"][0] == 0 and report["trail"] == 0.0


def test_other_wide_profiles_are_unchanged():
for state in ("projectile", "cheer", "wave", "celebrate"):
_, report = canvas.pad_canvas(_still(), canvas.STATE_CANVAS[state], facing="right")
assert report["trail"] == 0.0 and report["offset"][0] == 0


def test_lead_and_trail_are_bounded():
with pytest.raises(SystemExit):
canvas.pad_canvas(_still(), canvas.STATE_CANVAS["attack"], trail=0.95)
with pytest.raises(SystemExit):
canvas.pad_canvas(_still(), canvas.STATE_CANVAS["attack"], lead=0.5, trail=0.45)


def test_cli_trail_reaches_the_report(tmp_path):
still = tmp_path / "still.png"
_still().save(still)
payload = canvas.run_canvas(still, tmp_path / "out.png", state="attack", shape=None, facing="right", headroom=None, lead=None, report_path=tmp_path / "r.json", trail=0.1)
assert payload["trail"] == 0.1 and payload["offset"][0] == round(payload["canvas"][0] * 0.1)
10 changes: 6 additions & 4 deletions tests/video/test_video_pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,9 @@ def test_wide_canvas_puts_room_in_front_of_the_facing(tmp_path: Path) -> None:
still = Image.open(_still(tmp_path))
right, rep_r = canvas_mod.pad_canvas(still, canvas_mod.profile_for("attack"), facing="right")
left, rep_l = canvas_mod.pad_canvas(still, canvas_mod.profile_for("attack"), facing="left")
assert rep_r["offset"][0] == 0 and rep_l["offset"][0] == left.width - 120
behind = round(right.width * rep_r["trail"])
assert behind > 0 and rep_r["offset"][0] == behind and rep_l["offset"][0] == left.width - 120 - behind
assert right.width - 120 - behind > behind # more room in front than behind
assert abs(right.width / right.height - 16 / 9) < 0.02


Expand All @@ -80,9 +82,9 @@ def test_attack_canvas_reserves_overhead_room_and_reports_placement(tmp_path: Pa
assert image.crop((x, y, x + 120, y + 160)).tobytes() == Image.open(still).tobytes()
assert image.crop((0, 0, image.width, y)).getextrema() == ((0, 0), (255, 255), (0, 0))
assert json.loads(report.read_text()) == rep
assert rep["why"] == "weapon swings rise overhead and extend in front"
# An explicit zero still gives the pre-headroom wide layout.
zero, zero_rep = canvas_mod.pad_canvas(Image.open(still), canvas_mod.profile_for("attack"), headroom=0)
assert rep["why"] == "weapon swings rise overhead and extend in front; a long weapon drawn back reaches behind"
# Explicit zeros still give the pre-headroom, pre-trail wide layout.
zero, zero_rep = canvas_mod.pad_canvas(Image.open(still), canvas_mod.profile_for("attack"), headroom=0, trail=0)
assert zero.size == (284, 160) and zero_rep["offset"] == [0, 0]


Expand Down
Loading