Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions PROVIDERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ them.
| PixelLab | Production; paid generation and account workflows live-tested | `PIXELLAB_API_KEY` | generations |
| Retro Diffusion | Experimental; authenticated paid still generation, download, provenance, and recovery live-tested; advanced workflows pending | `RD_API_KEY` | USD |
| ComfyUI | Experimental; local generation, review, recovery, and image-to-image revision smoke live-tested on Apple MPS; inpaint/outpaint remain mock-tested | none; optional `COMFYUI_BASE_URL` | free |
| Scenario | Experimental; BFL Flux 2 Dev authentication, CU preflight, paid single/two-output generation, review, download, and durable recovery live-tested | `SCENARIO_SDK_API_KEY` and `SCENARIO_SDK_API_SECRET` | compute-units |
| Scenario | Experimental; BFL Flux 2 Dev authentication, CU preflight, paid single/two-output generation, review, download, and durable recovery live-tested; reference-image edits, tool models, and multi-model chains run once each | `SCENARIO_SDK_API_KEY` and `SCENARIO_SDK_API_SECRET` | compute-units |
| FakeProvider | Test-only deterministic lifecycle | none | free |

Live tests now cover single-candidate RD Fast and RD Plus stills from cost quote
Expand Down Expand Up @@ -267,9 +267,11 @@ to ComfyUI. Provider-keyed budgets, independent orchestration, lock-authoritativ
recovery, and explicit account-provider selection ship with it.

The Scenario still-image adapter from
[GitHub issue #52](https://github.com/gfargo/pixelkiln/issues/52) now has mocked
edge coverage and a paid BFL Flux 2 Dev lifecycle smoke. It remains
experimental until more model schemas and representative art briefs pass.
[GitHub issue #52](https://github.com/gfargo/pixelkiln/issues/52) has mocked edge
coverage, a paid BFL Flux 2 Dev lifecycle smoke, and a broader evaluation (ten
more models, six tool models, reference-image edits, multi-model chains, and a real
game's art) recorded in [Set up Scenario](./docs/SCENARIO.md#where-the-evidence-is).
It remains experimental until more model schemas and representative art briefs pass.

| Candidate | What it adds | Fit with PixelKiln | Main cost or risk | Priority |
|---|---|---|---|---:|
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -366,7 +366,7 @@ verification. See [Library API](./docs/LIBRARY.md).
| [Set up PixelLab](./docs/PIXELLAB.md) | Production-provider credentials, manifest, generators, and account workflows. |
| [Set up Retro Diffusion](./docs/RETRO_DIFFUSION.md) | Experimental-provider credentials, styles, formats, cost checks, and limits. |
| [Set up ComfyUI](./docs/COMFYUI.md) | Self-hosted stills, revisions, ordered frame sets, per-asset inputs, and quality limits. |
| [Set up Scenario](./docs/SCENARIO.md) | Experimental hosted models, two-part credentials, CU preflight, review, and durable downloads. |
| [Set up Scenario](./docs/SCENARIO.md) | Experimental hosted models, two-part credentials, CU preflight, reference-image edits, multi-model chains, measured findings, and durable downloads. |
| [Versioned recipes](./docs/RECIPES.md) | Pinned workflow packs, model hashes, manifest templates, and quality contracts. |
| [Controlled revisions](./docs/REVISIONS.md) | Image-to-image, inpaint, palette cleanup, animation, and interpolation from a parent; masks, fail-closed readiness, provenance, and ComfyUI bindings. |
| [CLI reference](./docs/CLI.md) | Every command, flag, JSON mode, and exit contract. |
Expand Down
2 changes: 1 addition & 1 deletion docs/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ replacement:
| PixelLab adapter | The current production and live-tested generation backend: stills, tiles, UI panels and elements, `objectPro` objects, controlled revisions, and the character family (four engines, states, loops, mirrors, portraits, outfits, adoption). |
| Retro Diffusion adapter | Experimental backend; authenticated paid single-still lifecycle plus mocked advanced-workflow tests. |
| ComfyUI adapter | Experimental self-hosted still, controlled-revision, and atomic frame-set backend; PixelKiln automates lineage and mechanical quality checks while a person retains visual approval. |
| Scenario adapter | Experimental hosted still-image backend; BFL Flux 2 Dev quote, paid single/two-output generation, human review, and durable recovery live-tested. |
| Scenario adapter | Experimental hosted still-image backend with reference-image edits and cross-style chains; BFL Flux 2 Dev quote, paid generation, human review, and durable recovery live-tested, plus ten more models, six tool models, and a real game's art run once each. |

PixelKiln's core is provider-neutral, but PixelLab remains the only production
adapter. Retro Diffusion generation support is experimental. Paid RD Fast and
Expand Down
2 changes: 1 addition & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@ coverage. Live Apple MPS runs cover a small Stable
Diffusion 1.5 plumbing smoke and
the SDXL Base plus Pixel Art XL composition benchmark, including four-candidate
review, cache recovery, and native-grid refinement. `ScenarioProvider` is an experimental hosted still-image
adapter. It keeps offline planning conservative, records the provider's free
adapter that also uploads references for `styleImages` and `image-to-image` revisions. It keeps offline planning conservative, records the provider's free
submit-time Compute Unit quote, and stores durable `scenario://` job and asset
references that refresh signed URLs. Authentication and CU preflight have
passed live with BFL Flux 2 Dev, along with paid single/two-output jobs, human
Expand Down
2 changes: 1 addition & 1 deletion docs/CLI.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ work accordingly. The experimental `retrodiffusion` adapter supports
still-image `map`/`pixflux`, `tiles` sheets, and `animation` GIF/spritesheet
work. The experimental `comfyui` adapter runs committed API-format `map` and
ordered still-frame workflows on a self-hosted server. The experimental `scenario` adapter runs
hosted still models with Compute Unit preflight and durable asset recovery.
hosted still models, with reference-image edits, Compute Unit preflight, and durable asset recovery.

Which command:

Expand Down
2 changes: 1 addition & 1 deletion docs/GETTING_STARTED.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ remains the production adapter. See
provider options and current limits, or
[provider comparison](../PROVIDERS.md) for selection guidance.

For experimental Scenario still generation, use both credentials:
For experimental Scenario still generation and reference-image edits, use both credentials:

```dotenv
SCENARIO_SDK_API_KEY=...
Expand Down
95 changes: 81 additions & 14 deletions docs/SCENARIO.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,29 @@
# Set up Scenario

[Scenario](https://www.scenario.com/) is an experimental hosted provider for
project-specific models and third-party image models. PixelKiln currently
supports still PNG generation through Scenario's universal model endpoint,
free Compute Unit preflight, asynchronous jobs, multi-output review, and
refreshable asset downloads.

Every code path in the adapter has mocked coverage. BFL Flux 2 Dev has also passed
live authentication, cost preflight, paid single- and two-output generation,
human review, PNG download, and provider-backed recovery. Start every untested
model with one disposable asset and a small ceiling. This integration remains
experimental because Scenario model schemas vary.
project-specific models and third-party image models. PixelKiln supports still
PNG generation through Scenario's universal model endpoint, free Compute Unit
preflight, asynchronous jobs, multi-output review, and refreshable asset
downloads. It also uploads your images as references, so a style's
`styleImages` and `image-to-image` revisions work, and a manifest can chain
several Scenario models across styles with
[`revision.fromStyle`](./REVISIONS.md#across-styles). The adapter handles
still images only; audio models are reachable through the same account but not
through PixelKiln.

Every code path in the adapter has mocked coverage. BFL Flux 2 Dev passed live
authentication, cost preflight, paid single- and two-output generation, human
review, PNG download, and provider-backed recovery. A later evaluation ran ten
more text-to-image models, six tool models, reference-image edits, multi-model
chains, and a real game's art; its findings are on this page and its benchmark
projects are listed in [Where the evidence is](#where-the-evidence-is). Start
every untested model with one disposable asset and a small ceiling. This
integration remains experimental because Scenario model schemas vary.

The short version of what that evaluation found: price does not predict fit;
large or illustrative art (landmarks, covers, flat backdrops) comes out well
through render, Pixelate, cutout, and `refine`; small native-size sprites, tiles,
and characters do not; and a reference image keeps an existing design.

## Requirements

Expand Down Expand Up @@ -371,6 +384,57 @@ and post steps beside the file.
Music models (ACE-Step, MusicGen, ElevenLabs Music, Lyria 3.5) quoted 10 to 30
CU; Sonilo and Lyria 3 Pro need the `cu-pro-q3-25` plan. Music was not generated.

### Calling the API directly: the dry run and the legacy endpoint

Two facts for anyone calling Scenario outside PixelKiln, because both cost compute
units to find:

- **The free dry run is only the query parameter** `?dryRun=true`. A `dryRun: true`
in the JSON body is ignored, and the request submits and bills a real job. A probe of
the legacy `/generate/txt2img` endpoint made that mistake and billed 10 CU for a
FLUX.1 LoRA image, and jobs there cannot be cancelled ("Cannot cancel this type of
job"). The adapter always sends the query parameter.
- **LoRA models do not use `/generate/custom/{modelId}`.** A public pixel-art LoRA
(Flux Retro Aesthetics) answered "Custom models only are supported for this
endpoint" there. The older `/generate/txt2img` endpoint accepted it at 10 CU an
image, returned a JPEG (which the adapter refuses), and is absent from the current
API reference. The adapter does not call it, and the same-quality Klein render costs
1 CU.

### Budget and billing notes

- **The adapter cannot read your balance.** `gen` enforces the budget you pass
(`scenario does not expose an account balance; enforcing its run budget`) and
the per-style `maxComputeUnits`, but the remaining balance is only on Scenario's
dashboard. Whether the allowance renews depends on the plan and billing
setup; a dashboard showing "No monthly reset scheduled" means it may not.
- **Check the dashboard against your own tally.** In our evaluation a hand total
of quoted and billed amounts came to about 715 CU while the dashboard's balance
had fallen by 753 CU, about 5% more. The cause was not identified, so budget with a
margin.
- **Quotes were exact where the adapter made them.** Every quote the adapter took
matched the billed amount in the lockfile, including failed assets that the
adapter refused after billing (a JPEG or WebP result is billed even though no
file lands).
- **Flat prices.** GPT Image 2 `quality: medium` was 11 CU at any canvas size (`high`
44), ElevenLabs sound effects 30 CU at any length, and tool models 2 to 10 CU.

### Where the evidence is

Each is a committed project with its manifest, lockfile, outputs, and README:

| Project | What it ran | CU |
|---|---|---:|
| `benchmarks/provider-scenario-smoke` | The first live run: FLUX.2 Dev, one and two outputs, recovery | 48 |
| `benchmarks/provider-scenario-bakeoff` | Ten models on one brief | 33 |
| `benchmarks/provider-scenario-edit` | One reference-image edit | 1 |
| `benchmarks/provider-scenario-tools` | Birefnet, Pixel Snapper, Pixelate, three upscalers, and the refine gate | 34 |
| `benchmarks/provider-scenario-pipeline` | Generate, pixelate, cut out, refine a character, with a sweep of Pixelate grid sizes | 41 |
| `benchmarks/provider-scenario-chain` | The same chain declared with `revision.fromStyle` and run by one `gen` | 8 |
| `benchmarks/provider-scenario-game-pilot` | A real game's landmarks, covers, backdrops, structures, and sound effects (settings and measurements only; the game's art is not included) | about 630 |

Scratch projects for the game pilot are not committed, so its README is the record.

## Plan before spending

Scenario prices depend on model, size, steps, and output count, so PixelKiln
Expand Down Expand Up @@ -523,10 +587,13 @@ quote, final billing, selected asset ID, output index, and output hash without
retaining credentials or signed query strings. Once cached, `restore` prefers
the validated local content cache.

Scenario account listing, adoption, salvage, tagging, deletion, balance, image
uploads, training, background removal, and upscaling are not part of this first
adapter. PixelKiln reports unavailable account commands instead of pretending
they succeeded.
Scenario account listing, adoption, salvage, tagging, deletion, balance, and
training are not part of the adapter. Image uploads exist only to send
references (see [Reference images and edits](#reference-images-and-edits)), and
background removal, pixel snapping, and upscaling run as ordinary models through
`image-to-image` revisions ([Tool models](#tool-models-as-image-to-image)).
PixelKiln reports unavailable account commands instead of pretending they
succeeded.

## Mixed-provider projects

Expand Down
11 changes: 10 additions & 1 deletion skills/pixelkiln/references/scenario.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ schemas remain unverified. Require a one-asset smoke before widening a batch.
`referenceArray` for a model that names it differently. Other revision modes
and `strength` are refused.

Scenario model parameters vary. The MVP supplies prompt, 16–4096px dimensions
Scenario model parameters vary. The adapter supplies prompt, 16–4096px dimensions
(each model has tighter bounds, reported by the free dry run), one to four outputs, and optional seed. Put only documented
model inputs in `parameters`; PixelKiln rejects attempts to override its prompt,
dimensions, seed, output count, project routing, or CU ceiling.
Expand All @@ -45,5 +45,14 @@ See `docs/SCENARIO.md` for settings and measurements.
PixelKiln validates the job, bytes, hashes, and recovery path. It does not prove
the output is good pixel art. Inspect one result at 1× before approving a batch.

When calling Scenario outside PixelKiln (audio, say), the free dry run is only the
query parameter `?dryRun=true`: a `dryRun: true` in the JSON body is ignored and
submits and bills a real job. LoRA models need the legacy `/generate/txt2img`
endpoint (10 CU, JPEG output, not callable by the adapter). The adapter cannot read
the CU balance; check the dashboard and budget with a margin (a hand tally ran about
5% under the dashboard). A reference image keeps an existing design (silhouette
overlap 0.93 to 1.00 against 0.39 to 0.66 from text alone), though some models shift
its colours; use it before a text-only prompt when the look must be kept.

See `docs/SCENARIO.md` for setup, manifest examples, unsupported account
operations, and the live validation checklist.
19 changes: 18 additions & 1 deletion website/app/globals.css
Original file line number Diff line number Diff line change
Expand Up @@ -768,6 +768,23 @@ a { color: inherit; text-decoration: none; }
line-height: 1.4;
}
.scenario-strip figcaption b { display: block; color: var(--ember-light); font-weight: 600; }
.scenario-findings { margin-top: 18px; }
.scenario-findings ul {
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 0 34px;
margin: 0;
padding: 0;
list-style: none;
}
.scenario-findings li {
padding: 15px 0;
border-top: 1px solid var(--line);
color: #aaa293;
font-size: 13px;
line-height: 1.65;
}
.scenario-findings li b { color: #e6dccb; font-weight: 600; }
.scenario-bottom {
display: grid;
grid-template-columns: minmax(0, 1.1fr) minmax(0, 1fr);
Expand Down Expand Up @@ -1083,7 +1100,7 @@ a { color: inherit; text-decoration: none; }
.review-grid { grid-template-columns: 1fr; }
.review-copy { max-width: 720px; }
.provider-showcase-grid { grid-template-columns: 1fr; }
.scenario-grid, .scenario-bottom { grid-template-columns: minmax(0, 1fr); }
.scenario-grid, .scenario-bottom, .scenario-findings ul { grid-template-columns: minmax(0, 1fr); }
.lock-visual { width: min(680px, calc(100% - 35px)); }
.docs-layout { grid-template-columns: 170px minmax(0, 1fr); }
.docs-toc { display: none; }
Expand Down
Loading
Loading