Add admin collection-style LoRA training and 64-card batch workflow - #823
Conversation
Co-authored-by: driver727-pixel <269849721+driver727-pixel@users.noreply.github.com>
Co-authored-by: driver727-pixel <269849721+driver727-pixel@users.noreply.github.com>
Co-authored-by: driver727-pixel <269849721+driver727-pixel@users.noreply.github.com>
Co-authored-by: driver727-pixel <269849721+driver727-pixel@users.noreply.github.com>
Co-authored-by: driver727-pixel <269849721+driver727-pixel@users.noreply.github.com>
Co-authored-by: driver727-pixel <269849721+driver727-pixel@users.noreply.github.com>
Co-authored-by: driver727-pixel <269849721+driver727-pixel@users.noreply.github.com>
Co-authored-by: driver727-pixel <269849721+driver727-pixel@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
A critical dataset exposure and multiple moderate resource and workflow issues remain unresolved.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds an admin-only collection-style LoRA workflow for deterministic 64-card generation with approval gating and recovery support.
Changes:
- Added authenticated training, batching, generation, and recovery APIs.
- Added admin UI, shared lore data, access rules, tests, and documentation.
- Added configuration and collection-style data-model support.
File summaries
| File | Summary / final review notes |
|---|---|
src/services/collectionStyle.ts |
Client API wrappers and workflow types. |
src/pages/AssetGenerator.tsx |
Collection Style tab integration. |
src/lib/lore.ts |
Shared lore-name pool import. |
src/index.css |
Collection Style panel styling. |
src/components/AdminCollectionStylePanel.tsx |
Admin workflow interface. Moderate (2 votes): newly created batches can be hidden by a stale active-batch closure. |
shared/loreCharacterNames.json |
Shared 75-name lore pool. |
server/test/collectionStyle.test.js |
Workflow and recovery tests. |
server/routes/collectionStyle.js |
Training and batch API routes. Critical (1 vote): dataset archives remain publicly retrievable via bearer download tokens. Moderate (3 votes): unbounded dataset buffering, unrecoverable preparing state, and incorrect pending: 0 creation responses. |
server/lib/collectionStyle.js |
Deterministic plans, prompts, validation, and ZIP utilities. |
server/index.js |
Route registration and configuration. |
README.md |
Environment and workflow documentation. |
firestore.rules |
Admin-only workflow access rules. |
docs/DATA_MODEL.md |
Collection-style data-model documentation. |
.env.example |
New workflow configuration defaults. |
Review details
Suppressed comments (8)
server/index.js:220
buildRateLimiterdefaultspassOnStoreErrortotrue(seeserver/lib/rateLimit.js:33-36), so when Redis is configured but unavailable this limiter fails open. Because these routes can start paid LoRA jobs and image inference, this should fail closed likeimageRateLimit; otherwise a Redis outage removes the intended cost-protection limit.
windowMs: 60 * 1000,
max: 12,
message: { error: 'Too many collection-style requests — please wait a moment and try again.' },
store: sharedRateLimitStore,
});
server/lib/collectionStyle.js:477
- Variety is counted on raw Firestore values, so three distinct objects or arrays in
archetype,district, orstylesatisfy theSetchecks even thoughbuildCollectionStyleCaptionignores non-string values and falls back. This lets malformed sources bypass the promised variety enforcement; count only trimmed non-empty strings (or reject malformed prompt fields).
export function assertCollectionStyleSourceVariety(cards) {
const distinct = (field) => new Set(cards.map((card) => card?.prompts?.[field]).filter(Boolean)).size;
if (distinct('archetype') < 3 || distinct('district') < 3 || distinct('style') < 3) {
throw badRequest('Training sources need at least three archetypes, districts, and styles to reduce overfitting.');
server/routes/collectionStyle.js:887
- This write leaves
item.cardas the original plan, which has nocharacterImageUrl; only the immediate response includescard: completedCard. The panel refetches the item after every generation and renders previews fromitem.card.characterImageUrl, so completed cards appear without thumbnails after that refetch. Persist the completed card in the batch item or hydrate it fromadminBossAssets.
transaction.set(itemRef, {
status: 'completed',
resultCardId: card.id,
completedAt,
updatedAt: completedAt,
leaseExpiresAt: null,
lastError: null,
}, { merge: true });
server/routes/collectionStyle.js:839
- The recovery branch marks the item complete but still leaves its stored
cardplan without the recovered character URL. If a crash occurs after the Boss Asset write, the subsequent batch fetch therefore still has no image for the preview. Update the persisted item card withexistingAsset.data().characterImageUrl(and return that hydrated card).
itemRef.set({
status: 'completed',
resultCardId: card.id,
recoveredAt,
completedAt: recoveredAt,
updatedAt: recoveredAt,
leaseExpiresAt: null,
}, { merge: true }),
server/routes/collectionStyle.js:435
- The 64-document limit is applied before
toSourceAssetSummaryfilters eligibility. Once the Boss Asset library grows beyond 64 records, recent ineligible entries can hide older eligible character layers, leaving the UI unable to reach the required 12 sources despite enough valid assets. Page or filter eligible assets rather than truncating the raw collection.
const snap = await adminDb
.collection(BOSS_ASSETS_COLLECTION)
.orderBy('createdAt', 'desc')
.limit(MAX_COLLECTION_STYLE_SOURCES)
.get();
server/routes/collectionStyle.js:548
fal.queue.submitis a paid external side effect, but its request ID is persisted only in the later Firestore update. If Fal accepts the job and that update fails, the catch path marks this job failed without retainingfalRequestId; retrying then submits a second paid training job while the first is orphaned. Persist/reconcile the request ID with an idempotency key before allowing a retry.
const submitted = await fal.queue.submit(trainingModel, {
input: {
images_data_url: datasetUrl,
trigger_word: triggerToken,
is_style: true,
server/routes/collectionStyle.js:614
- When Fal reports
COMPLETEDbutextractFalLoraUrlrejects the result, this throws into the generic catch without marking either record failed. The job remainstrainingand the profile remains active, so every status poll repeats the same 502 and new training is blocked. Record this terminal result as failed before responding.
const result = await fal.queue.result(job.trainingModel || trainingModel, {
requestId: job.falRequestId,
});
const loraUrl = extractFalLoraUrl(result);
if (!loraUrl) throw badRequest('Fal training completed without a compatible LoRA URL.', 502);
src/components/AdminCollectionStylePanel.tsx:377
- The approval results render only the transparent
characterImageUrl; they never compose the district background, rarity frame, approved board, weapon, or stored placement presets. An explicit “Approve visuals” action therefore cannot inspect the final card composition that production will use. Show a composed card preview for the approval items (for example, via the existing card preview component).
{completedItems.map((item) => (
<article key={item.id} className="collection-style-result">
{item.card?.characterImageUrl && <img src={item.card.characterImageUrl} alt={item.name} />}
<strong>{item.name}</strong>
<small>{item.card?.prompts?.district ?? "Collection"} · persisted to Boss Assets</small>
- Files reviewed: 14/14 changed files
- Comments generated: 5
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| const archiveEntries = []; | ||
| for (let index = 0; index < sourceCards.length; index += 1) { | ||
| const { id, card } = sourceCards[index]; | ||
| const image = await downloadTrainingImage(fetchImpl, card.characterImageUrl); | ||
| const stem = `${String(index + 1).padStart(3, '0')}-${id.replace(/[^a-zA-Z0-9_-]/g, '-')}`; |
| }); | ||
| } | ||
| await writeBatch.commit(); | ||
| res.status(201).json({ batch: summarizeBatch(batch) }); |
| if ( | ||
| profile?.training?.status === 'preparing' | ||
| || profile?.training?.status === 'training' | ||
| ) { | ||
| throw badRequest('A collection-style LoRA training job is already active.', 409); |
| cacheControl: FIREBASE_STORAGE_CACHE_CONTROL, | ||
| metadata: { firebaseStorageDownloadTokens: token }, | ||
| }, | ||
| }); | ||
| return `${FIREBASE_STORAGE_BASE_URL}/v0/b/${encodeURIComponent(storageBucket)}/o/${encodeURIComponent(storagePath)}?alt=media&token=${token}`; |
| await loadBatch(result.batch.id); | ||
| await loadWorkspace(); |
Adds an admin-only style-LoRA workflow for producing a consistent card collection without regenerating backgrounds, frames, boards, or weapons. Training incurs a one-time cost; each card still performs character-layer inference.
Training and profile management
adminBossAssetscharacter layers only; reject full-card, board, frame, and background sources.Approval-gated batch generation
Admin experience and access control