Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
**/node_modules
**/.pnpm-store
**/dist
92 changes: 92 additions & 0 deletions MASTER_PLAN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# MASTER PLAN — Crypto Animal Studio × Jellyfish

Governance document for integrating **Crypto Animal Studio (CAS / Creative OS)** into
**Jellyfish** as the production platform. This file is the single high-level source of
truth for scope, approach, and the approved architectural decisions. Detailed current
architecture lives in `docs/architecture-analysis.md`; the contract lives in
`docs/crypto-animal-studio/episode-package-v1.md`.

---

## 1. Goal

Jellyfish is the **main project** (an end-to-end AI short-drama production workspace:
FastAPI + LangChain/LangGraph + SQLAlchemy, async task center, asset consistency,
image/video generation). CAS is the **upstream creative brain** that turns a news or
original premise into a fully-formed episode (script + storyboard + dialogue + character
and asset definitions). CAS delivers that episode to Jellyfish as a single validated
**EpisodePackage**, which Jellyfish then produces and exports.

```
CAS (creative brain) Jellyfish (production platform)
premise → episode + storyboard ──▶ EpisodePackage (contract)
+ dialogue + characters + assets ──▶ Chapter + Shots → assets → generation → export
```

## 2. Approach

Integrate CAS as a **bounded module** inside the Jellyfish backend
(`backend/app/crypto_animal_studio/`), reusing Jellyfish's existing Project / Chapter /
Shot / Asset / Media / Prompt / Task / Provider systems. Do **not** fork a parallel
platform. Build additively and keep every change inside the module plus a thin router
registration.

## 3. Approved final decisions (authoritative)

These are approved and binding. The ADR table with reasons/consequences is in
`docs/architecture-analysis.md` → "Final Architecture Decisions".

1. **A CAS Episode maps to one Jellyfish Chapter.**
2. **A Jellyfish Project represents a series / production / season** and may contain
multiple CAS Chapters/Episodes.
3. **EpisodePackage directly creates Shots** (Chapter / Shot / ShotDetail /
ShotDialogLine + character & asset links) from the completed storyboard.
4. **Completed CAS storyboards must not be passed through `ScriptDividerAgent`** —
re-dividing would destroy comedy beats, timing, dialogue alignment, and shot structure.
5. **`Chapter.raw_text` stores the complete generated script for traceability**, but
Shots are created from the EpisodePackage, not by re-dividing `raw_text`.
6. **No duplicate systems**: do not create parallel CAS Project / Episode / Shot / Asset
/ Media / Prompt / Task / Provider systems; reuse Jellyfish's.
7. **Provider configuration ultimately converges on Jellyfish** (`Provider` / `Model` /
`ModelSettings`). A temporary adapter is allowed during transition; a second
independent provider configuration system is not.
8. **The initial importer is synchronous** (no Celery for the first milestone).
9. **No new `ProjectStyle` or `ProjectVisualStyle` enum values** in the initial phase.

## 4. Sprint roadmap

| Sprint | Scope | Status |
|---|---|---|
| 2 — CAS Foundation | Bounded module + EpisodePackage v1 schema + validation + sample + docs + health endpoint + tests | Done |
| 2.1 — Foundation Hardening | Governance docs (this file + architecture-analysis) + structured `CameraSpec` (shot_type/angle/movement, CAS-local enums) + doc/sample/test/log updates | Done |
| 3 — Synchronous importer | EpisodePackage → validation → **synchronous** import service → create Chapter + Shots (+ ShotDetail/dialogue/links); no Celery/LLM/frontend | Planned |
| 4 — Consistency & generation hand-off | Seed characters/assets as Jellyfish entities; wire shots into existing generation workspace | Planned |
| 5 — Provider convergence & UX | Retire temporary provider adapter onto Jellyfish Provider/Model/ModelSettings; frontend entry; OpenAPI regen | Planned |

## 5. Scope guardrails (until explicitly lifted)

Do not, in the foundation phases: create DB tables, modify ORM models, create SQL
migrations, create Chapter/Shot records (before the importer sprint), add Celery tasks,
add Redis/S3 logic, invoke an LLM, migrate Creative OS agents, modify generated frontend
clients, build frontend UI, add a second provider system, or add
`ProjectStyle`/`ProjectVisualStyle` enum values.

## 6. Module boundary

```
backend/app/crypto_animal_studio/
├── api/ # thin routes → ApiResponse (health now; import later)
├── application/ # use cases (future synchronous import service)
├── domain/ # constants/enums/helpers (SCHEMA_VERSION, SourceType, camera enums)
├── schemas/ # the ONLY Pydantic models for EpisodePackage
├── agents/ # (future) CAS-specific agents
└── integrations/ # (future) bridge to Jellyfish Provider/Model/ModelSettings
```

## 7. References

- `docs/architecture-analysis.md` — full current architecture + Final Architecture Decisions (ADR).
- `docs/crypto-animal-studio/episode-package-v1.md` — EpisodePackage v1 contract.
- `docs/crypto-animal-studio/samples/sample-episode-package-v1.json` — valid sample.
- `docs/implementation-log.md` — per-sprint implementation log.
- `AGENTS.md` / `.cursor/rules/*` — repository conventions and definition of done.
20 changes: 20 additions & 0 deletions backend/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,23 @@ API_V1_PREFIX=/api/v1
# S3_BASE_PATH=jellyfish/dev
# 若有 CDN/自定义域名,可配置成 https://cdn.example.com
# S3_PUBLIC_BASE_URL=

# ---------------------------------------------------------------------------
# Crypto Animal Studio — single-shot render (Step 7)
# ---------------------------------------------------------------------------
# Render provider: comfyui | volcengine | openai
CAS_RENDER_PROVIDER=comfyui
# Self-hosted ComfyUI instance. No default: rendering fails clearly if unset.
# From inside Docker use host.docker.internal to reach a host-side ComfyUI.
CAS_COMFYUI_BASE_URL=http://host.docker.internal:8188
# Path to the workflow input/output mapping JSON.
# The workflow itself must be exported in ComfyUI API format.
# The mapping MUST map "width" and "height", otherwise a render fails fast.
CAS_COMFYUI_WORKFLOW_MAPPING=/app/workflows/cas_txt2video.mapping.json
# Poll interval and per-render timeout (seconds).
CAS_RENDER_POLL_INTERVAL_S=3.0
CAS_RENDER_TIMEOUT_S=1800.0
# Preview profile resolution. 432x768 is exact 9:16 and both are multiples of 8;
# roughly 16% of the 1080x1920 final profile, suitable for Intel Iris Xe.
CAS_RENDER_PREVIEW_WIDTH=432
CAS_RENDER_PREVIEW_HEIGHT=768
7 changes: 7 additions & 0 deletions backend/app/api/v1/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from fastapi import APIRouter

from app.api.v1.routes import film, health, llm, studio, script_processing
from app.crypto_animal_studio.api import router as crypto_animal_studio_router

router = APIRouter()

Expand All @@ -11,3 +12,9 @@
router.include_router(llm.router, prefix="/llm", tags=["llm"])
router.include_router(studio.router, prefix="/studio")
router.include_router(script_processing.router)
# Crypto Animal Studio(CAS)受限边界模块:通过既有聚合机制挂载,不新建独立 FastAPI app。
router.include_router(
crypto_animal_studio_router,
prefix="/crypto-animal-studio",
tags=["crypto-animal-studio"],
)
16 changes: 14 additions & 2 deletions backend/app/api/v1/routes/studio/files.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,22 @@ async def list_files_api(
project_id: str | None = Query(None, description="按 file_usages 限定项目;提供后仅返回该项目下有关联记录的文件"),
chapter_title: str | None = Query(None, description="章节标题(精确匹配,与 project_id 联用)"),
shot_title: str | None = Query(None, description="镜头标题(精确匹配,与 project_id 联用)"),
chapter_id: str | None = Query(
None, description="按 file_usages.chapter_id 精确过滤(与 project_id 联用;比标题稳定)"
),
usage_kind: str | None = Query(
None, description="按 file_usages.usage_kind 精确过滤,如 subtitle(与 project_id 联用)"
),
) -> ApiResponse[PaginatedData[FileRead]]:
if chapter_title is not None or shot_title is not None:
scoped = (chapter_title, shot_title, chapter_id, usage_kind)
if any(value is not None for value in scoped):
if not project_id:
raise HTTPException(
status_code=400,
detail="project_id is required when chapter_title or shot_title is set",
detail=(
"project_id is required when chapter_title, shot_title, "
"chapter_id or usage_kind is set"
),
)

if project_id is not None:
Expand All @@ -50,6 +60,8 @@ async def list_files_api(
project_id=project_id,
chapter_title=chapter_title,
shot_title=shot_title,
chapter_id=chapter_id,
usage_kind=usage_kind,
q=q,
order=order,
is_desc=is_desc,
Expand Down
20 changes: 20 additions & 0 deletions backend/app/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,26 @@ def cors_origins_list(self) -> list[str]:
# 可选:对外访问基址(CDN 或自定义域名),为空则使用 S3 自带 URL 或预签名 URL
s3_public_base_url: str | None = None

# CAS 单镜头渲染(Step 7)。全部通过环境变量提供,仓库内不存放任何地址或凭据。
#: 渲染使用的供应商标识:comfyui | volcengine | openai。
cas_render_provider: str = "comfyui"
#: ComfyUI 实例地址,例如 http://127.0.0.1:8188(无缺省值:未配置即明确失败)。
cas_comfyui_base_url: str | None = None
#: 工作流「输入/输出映射」JSON 的路径,见 app.core.integrations.comfyui.workflow。
cas_comfyui_workflow_mapping: str | None = None
#: 轮询间隔(秒)。
cas_render_poll_interval_s: float = 3.0
#: 单次渲染超时(秒)。视频生成通常远慢于图像,缺省给足余量。
cas_render_timeout_s: float = 1800.0

# 分辨率档位:预览档用于低成本试跑(如 Intel Iris Xe 等核显),
# 成片档保持 EP001 的 1080×1920 输出规格。两者都必须是 8 的倍数。
#: 预览档宽度。
cas_render_preview_width: int = 432
#: 预览档高度。432×768 = 精确 9:16(0.5625),两者都是 8 的倍数,
#: 像素量约为成片档 1080×1920 的 16%,适合 Intel Iris Xe 等核显试跑。
cas_render_preview_height: int = 768

def model_post_init(self, __context: object) -> None:
if not self.celery_broker_url or not str(self.celery_broker_url).strip():
password_part = f":{self.redis_password}@" if self.redis_password else ""
Expand Down
5 changes: 4 additions & 1 deletion backend/app/core/contracts/provider.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@
from dataclasses import dataclass
from typing import Literal

ProviderKey = Literal["openai", "volcengine"]
#: 受支持的供应商标识。
#: ``comfyui`` 为自托管推理服务:无 API key,凭 base_url 直连,见
#: ``app.core.integrations.comfyui``。
ProviderKey = Literal["openai", "volcengine", "comfyui"]


@dataclass(frozen=True, slots=True)
Expand Down
20 changes: 20 additions & 0 deletions backend/app/core/contracts/video_generation.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,26 @@ class VideoGenerationInput(BaseModel):
)
watermark: Optional[bool] = Field(None, description="是否包含水印,供应商/模型可能有差异")

#: 可选的显式渲染分辨率。省略时由 ratio 推导(既有行为完全不变)。
#: 用途:预览档(低分辨率试跑)与成片档(全分辨率)共用同一条渲染路径。
width: Optional[int] = Field(None, gt=0, description="渲染宽度(像素);须与 height 同时提供")
height: Optional[int] = Field(None, gt=0, description="渲染高度(像素);须与 width 同时提供")

@model_validator(mode="after")
def check_explicit_dimensions(self) -> "VideoGenerationInput":
"""width/height 必须成对出现,且为 8 的倍数。

成对要求:只给一个无法推导另一个,静默补齐会产生非预期的画面比例。
8 的倍数:扩散模型的潜空间按 8 下采样,非 8 倍数会在供应商侧报错或被
悄悄取整;在此处提前拒绝,错误信息比 ComfyUI 的节点报错清晰得多。
"""
if (self.width is None) != (self.height is None):
raise ValueError("width and height must be provided together or both omitted")
for name, value in (("width", self.width), ("height", self.height)):
if value is not None and value % 8 != 0:
raise ValueError(f"{name} must be a multiple of 8, got {value}")
return self

@model_validator(mode="after")
def require_prompt_or_any_reference(self) -> "VideoGenerationInput":
has_prompt = bool((self.prompt or "").strip())
Expand Down
4 changes: 4 additions & 0 deletions backend/app/core/db.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,10 @@ async def init_db() -> None:
import app.models.studio # noqa: F401
import app.models.task # noqa: F401
import app.models.task_links # noqa: F401
# CAS 边界模块的导入台账表(幂等记账);导入以注册到 Base.metadata。
import app.crypto_animal_studio.domain.import_ledger # noqa: F401
# CAS 生产流水线表(任务/镜头/产物)。
import app.crypto_animal_studio.production.models # noqa: F401

async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
Expand Down
31 changes: 31 additions & 0 deletions backend/app/core/integrations/comfyui/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
"""ComfyUI 集成(自托管推理服务)。"""

from app.core.integrations.comfyui.video import (
ComfyUIError,
ComfyUIVideoApiAdapter,
extract_video_output,
is_video_filename,
read_execution_status,
)
from app.core.integrations.comfyui.workflow import (
REQUIRED_RENDER_INPUTS,
WorkflowConfigError,
WorkflowMapping,
apply_inputs,
load_mapping,
require_render_inputs,
)

__all__ = [
"REQUIRED_RENDER_INPUTS",
"ComfyUIError",
"ComfyUIVideoApiAdapter",
"WorkflowConfigError",
"require_render_inputs",
"WorkflowMapping",
"apply_inputs",
"extract_video_output",
"is_video_filename",
"load_mapping",
"read_execution_status",
]
Loading