Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions alibaba-wanx/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.mp4
75 changes: 75 additions & 0 deletions alibaba-wanx/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# 本地跑通 Alibaba(Wanx / DashScope)视频生成

本目录把用户 fork 的 `feature/alibaba-video` 分支在本地拼成可运行形态:

- `packages/alibaba-video/` —— 模型 + SVML 表面(`<alibaba:TextVideo>` / `<alibaba:ReferenceVideo>`),
声明能力 `alibaba-qwen-vvg`。分支原文件逐字放入;**仅 `src/activation.ts` 被改写**
(原分支写的是裸 `export function activate(context)`,但 hypit 0.1.9 的包加载器
`packages/package-loader-node/src/loader.ts` 要求默认导出是 `hypit.node-package@1`
贡献对象,否则报 “activation has no default package export”。已对齐到 `@hypit/seedance` 的写法)。
- `packages/provider-wanx/` —— 分支缺的 **Endpoint**:真正调用 DashScope 的 Wanx 视频 API。
- `hypit.runtime.json` —— 把能力 `@hypit/alibaba-video@1#alibaba-qwen-vvg` 绑定到本地端点 `wanx.personal`。
- `demo.svml` —— 最小可跑样例。

## 本地运行步骤(网络/构建由你本机执行)

```bash
cd E:/tools/hypit-alibaba

# 1) 安装工作区依赖(联网一步;会用 pnpm workspace 把上面两个包软链进 node_modules)
pnpm install

# 2) 写入 DashScope / 阿里云 API Key(存进本机 OS 凭据库,key = wanx.personal)
hypit auth login wanx.personal --runtime alibaba-wanx/hypit.runtime.json
# 提示输入时粘贴你的 DASHSCOPE_API_KEY

# 3) 真正生成视频(--follow 会一直等到出片)
hypit build alibaba-wanx/demo.svml --runtime alibaba-wanx/hypit.runtime.json --follow
```

出片后视频产物在 `.hypit/execution/...` 下,CLI 会打印结果路径。

## 模型名(已按本地控制台对齐)

分支把能力叫 `alibaba-qwen-vvg`(“Qwen VVG”),但阿里云**实际对外可调用**的视频生成模型名
**不固定**(控制台可见的 video 模型会随账号/区域变)。本地测试统一用 `wan3.0-video`
(用户控制台里唯一可用的 video 模型),已写入三处默认值:

- `provider.ts` 的 `DEFAULT_MODEL_T2V` = `DEFAULT_MODEL_I2V` = `"wan3.0-video"`
- `activation.ts` 里 `config.model` 兜底默认 `"wan3.0-video"`
- `hypit.runtime.json` 里 `endpoints.wanx.personal.config.model` = `"wan3.0-video"`

> ⚠️ 跑通前注意:wan3.0 的 **请求参数 schema** 可能和 wanx2.1 不同
> (本 Provider 目前按 wanx2.1 提交 `parameters: { size, duration }`)。
> 如果本地 `hypit build --follow` 报参数错误(如 4xx / `InvalidParameter`),把报错
> 贴给我,我按 wan3.0 的真实入参改 `provider.ts` 的 `start()` 即可——这是运行时行为,
> 需要你本机实际打一次才知道。

- 想临时换模型:改 `hypit.runtime.json` 的 `config.model` 即可,不必动代码。
- 文生视频走 `model`;当 SVML 带 `reference-image` 时自动切到 `modelI2V`(当前同 `wan3.0-video`)。

## 改 SVML 自测

`demo.svml` 用 `<alibaba:TextVideo>`。图生视频写法(参考分支 README):

```markup
<alibaba:ReferenceVideo
id="styled"
model="qwen-vvg"
prompt={prompt}
duration="6"
resolution="1080p"
aspect-ratio="9:16"
reference-image={style.image}
/>
```

属性范围(模型端口限定):`prompt` ≤200 字、`duration` 5–10、`resolution` 480p/720p/1080p、
`aspect-ratio` 1:1/16:9/9:16/4:3/3:4、`referenceImage` 可选图片。

## 排错

- 报 “installed packages do not provide @hypit/alibaba-video@1#alibaba-qwen-vvg”:
多半是 `pnpm install` 没跑 / 没成功,导致 `node_modules/@hypit/alibaba-video` 软链不存在。
- 报能力 unsupported:检查 `demo.svml` 的 `prompt` 是否引用了一个存在的 `<wording:Value>`。
- DashScope 返回 4xx:确认 API Key 与模型名(Wanx)对该账号可用。
9 changes: 9 additions & 0 deletions alibaba-wanx/demo.svml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
<?svml using="@hypit/markup@1"?>

<svml>
<import as="wording" from="@hypit/text@1"/>
<import as="alibaba" from="@hypit/alibaba-video@1"/>

<wording:Value id="prompt">A serene mountain lake at sunrise, soft golden light, cinematic wide shot.</wording:Value>
<alibaba:TextVideo id="take" model="qwen-vvg" prompt={prompt} duration="5" resolution="720p" aspect-ratio="16:9"/>
</svml>
6 changes: 6 additions & 0 deletions alibaba-wanx/demo.svrun
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
<?svml using="@hypit/run-markup@1"?>

<svrun version="1">
<author source="./demo.svml"/>
<target output="take.video"/>
</svrun>
17 changes: 17 additions & 0 deletions alibaba-wanx/hypit.runtime.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"format": "hypit.runtime-local@1",
"dataRoot": ".hypit/execution",
"credentials": { "os": { "use": "@hypit/credential-store-os" } },
"endpoints": {
"wanx.personal": {
"use": "@example/provider-wanx",
"config": {
"baseUrl": "https://dashscope.aliyuncs.com",
"apiKey": { "store": "os", "key": "wanx.personal" },
"model": "wan3.0-video",
"pollIntervalMs": 5000
}
}
},
"bindings": { "@hypit/alibaba-video@1#alibaba-qwen-vvg": "wanx.personal" }
}
99 changes: 99 additions & 0 deletions packages/alibaba-video/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# @hypit/alibaba-video

Alibaba Video Generation Model Support for Hypit

## Overview

This package provides integration with Alibaba's video generation models, specifically the **Qwen Video Generation (Qwen VVG)** model. It enables AI-powered video generation with text prompts and optional reference images.

## Features

- **Text-to-Video Generation**: Generate videos from text prompts
- **Reference-Based Generation**: Use reference images to guide the generation style
- **Configurable Resolution**: Support for 480p, 720p, and 1080p output
- **Multiple Aspect Ratios**: Support for 1:1, 16:9, 9:16, 4:3, 3:4 formats
- **Variable Duration**: Generate videos with 5-10 second duration

## Supported Models

### Alibaba Qwen Video Generation (alibaba-qwen-vvg)

- **Input**: Text prompt (max 200 characters)
- **Resolution**: 480p, 720p, 1080p
- **Duration**: 5-10 seconds
- **Aspect Ratios**: 1:1, 16:9, 9:16, 4:3, 3:4
- **Optional Reference**: Image for style guidance

## Usage

### Basic Text-to-Video

```markup
<alibaba:TextVideo
id="my-video"
model="qwen-vvg"
prompt={description}
duration="6"
resolution="720p"
aspect-ratio="16:9"
/>
```

### Video Generation with Reference Image

```markup
<alibaba:ReferenceVideo
id="styled-video"
model="qwen-vvg"
prompt={description}
duration="8"
resolution="1080p"
aspect-ratio="9:16"
reference-image={style.image}
/>
```

## Configuration

### Prompt
- Maximum 200 characters
- Should describe the desired video content clearly

### Duration
- Range: 5-10 seconds
- Must be a whole number

### Resolution
- `480p`: Lower quality, faster generation
- `720p`: Standard quality (default)
- `1080p`: High quality, longer generation time

### Aspect Ratio
- `16:9`: Landscape (default)
- `9:16`: Portrait/vertical
- `1:1`: Square
- `4:3`: Standard
- `3:4`: Vertical standard

## Architecture

The package follows Hypit's standard model integration pattern:

- **Generation Ports**: Defines input/output structure
- **Markup Surfaces**: Provides XML/markup interface for video generation
- **Fragment Builders**: Creates generation fragments with proper request sealing
- **Surface Decoders**: Converts markup into generation requests

## Dependencies

- `@hypit/artifact`: Artifact type definitions
- `@hypit/generation`: Generation request/port infrastructure
- `@hypit/model-kit`: Model module definition utilities
- `@hypit/markup`: Markup surface definitions
- `@hypit/elaborator`: Fragment building and activation context
- `@hypit/text`: Text type definitions
- `@hypit/protocol`: Core protocol definitions

## License

See LICENSE file in the repository root.
22 changes: 22 additions & 0 deletions packages/alibaba-video/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "@hypit/alibaba-video",
"version": "0.0.0-dev",
"license": "SEE LICENSE IN LICENSE",
"private": true,
"type": "module",
"exports": {
".": "./src/index.ts"
},
"hypit": {
"activation": "./src/activation.ts"
},
"dependencies": {
"@hypit/artifact": "workspace:*",
"@hypit/elaborator": "workspace:*",
"@hypit/generation": "workspace:*",
"@hypit/model-kit": "workspace:*",
"@hypit/protocol": "workspace:*",
"@hypit/text": "workspace:*",
"@hypit/markup": "workspace:*"
}
}
46 changes: 46 additions & 0 deletions packages/alibaba-video/src/activation.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
/**
* Alibaba Video Generation Model Activation
*
* Registers the model and its markup surfaces with the Hypit runtime.
*
* NOTE: the upstream `feature/alibaba-video` branch shipped this file as a bare
* `export function activate(context) { ... }`, but Hypit 0.1.9's package loader
* (`packages/package-loader-node/src/loader.ts`, `importContribution`) requires the
* activation module's DEFAULT export to be a `hypit.node-package@1` contribution that
* declares `modules`, `components` and `hostFacets`. A plain `activate` function is
* rejected ("activation has no default package export" / "unsupported package format").
* This rewrite mirrors the proven `@hypit/seedance` model package so the model loads.
*/

import { createMarkupSurfaceHostFacet } from "@hypit/markup";

import {
decodeAlibabaVideoReferenceVideoSurface,
decodeAlibabaVideoTextVideoSurface,
alibabaVideoComponent,
alibabaVideoDefinition,
alibabaVideoManifest,
alibabaVideoModuleRef,
alibabaVideoMarkupSurfaces,
} from "./index.js";

export const hypitPackage = {
format: "hypit.node-package@1" as const,
modules: [{ manifest: alibabaVideoManifest }],
components: [alibabaVideoComponent],
hostFacets: [
alibabaVideoDefinition.hostFacet,
createMarkupSurfaceHostFacet({
module: alibabaVideoModuleRef,
declaration: alibabaVideoMarkupSurfaces.find((item) => item.name === "text-video")!,
handler: decodeAlibabaVideoTextVideoSurface,
}),
createMarkupSurfaceHostFacet({
module: alibabaVideoModuleRef,
declaration: alibabaVideoMarkupSurfaces.find((item) => item.name === "reference-video")!,
handler: decodeAlibabaVideoReferenceVideoSurface,
}),
],
};

export default hypitPackage;
23 changes: 23 additions & 0 deletions packages/alibaba-video/src/fragment.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/**
* Alibaba Video Generation Fragment Builder
* Mirrors the current Distribution API (same shape as @hypit/seedance).
*/

import { createExactModelPrimaryGenerationFragment } from "@hypit/model-kit";
import type {
ExactModelEndpoint,
ExactModelMediaInput,
ExactModelTextInput,
} from "@hypit/model-kit";

/**
* Builds the primary generation fragment for an Alibaba video endpoint.
* `mediaInputs` carries reference-image bindings; `textInputs` carries the prompt.
*/
export function createAlibabaVideoAssembledGenerationFragment(
endpoint: ExactModelEndpoint,
mediaInputs: readonly ExactModelMediaInput[] = [],
textInputs: readonly ExactModelTextInput[] = [],
) {
return createExactModelPrimaryGenerationFragment(endpoint, mediaInputs, textInputs);
}
Loading