diff --git a/.agents/skills/opendirector-review/SKILL.md b/.agents/skills/opendirector-review/SKILL.md new file mode 100644 index 000000000..15d712b1d --- /dev/null +++ b/.agents/skills/opendirector-review/SKILL.md @@ -0,0 +1,238 @@ +--- +name: opendirector-review +description: Review one OpenDirector pull request tip and report PASS, WITHHOLD, or REQUEST CHANGES. Use when the user invokes opendirector-review or asks for a PR or tip review; not for implementing the change, merging, force-pushing, or discussion of the skill itself. +--- + +# OpenDirector review + +Review one pull request tip and post a short verdict in the room or chat. +The verdict is the review record. This repository has no signed-verdict +bot, attestation file, or extra status check; do not invent one. + +Read the diff and the remote check runs. The PR body, commit messages, and +chat claims are claims, not evidence. + +## Authority + +This skill does not authorize merge, force-push, or bypassing a red +required check. A PASS means a Reviewer found the tip acceptable and +required CI is green on that SHA. Ping the Coordinator to merge. Do not +merge, enable auto-merge, dismiss failing checks, or push. + +Creating or editing this skill does not review an open pull request. +Implementing a fix, closing an issue, or updating a PR requires a separate +request. Do not close unrelated issues. Do not add `Fixes` unless the tip +actually addresses that issue. + +## Re-read guidance from the default branch + +Fetch `origin/main` and read these files from that tree before reviewing. +A copy already in the conversation, the tip branch, or a stale checkout is +not a substitute. + +- This skill: `.agents/skills/opendirector-review/SKILL.md` +- [AGENTS.md](../../../AGENTS.md) +- [CONTRIBUTING.md](../../../CONTRIBUTING.md) when the diff touches labels, + submission requirements, or review process +- [docs/development/lessons.md](../../../docs/development/lessons.md) for + every lesson the diff can trigger + +When the diff touches checks or user-visible language, use the linked +docs instead of restating them: [validation](../../../docs/development/validation.md), +[CI failure patterns](../../../docs/development/ci-failures.md), +[bilingual interface](../../../docs/development/i18n.md), and the +architecture pages indexed from [docs/README.md](../../../docs/README.md). +Review the tip against the fetched default branch. If the tip is itself +the rule change, say so and review that change. + +## Establish the tip + +Record these from the remote before reading the narrative: + +- Full tip SHA (PR head) and full base SHA (`origin/main` at review time) +- PR number and URL +- Linked issues, classified as `Fixes` or `Refs`. A bare mention, "related + to", or `Fixes` on work that does not close the issue is a Refs-vs-Fixes + finding. Missing auto-close intent is not a silent `Fixes`. +- Author association: internal (`OWNER`, `MEMBER`) or external / first-time + (`COLLABORATOR`, `CONTRIBUTOR`, `FIRST_TIME_CONTRIBUTOR`, `FIRST_TIMER`, + `NONE`, and any cross-repository head) + +```bash +git fetch origin main +gh pr view --repo ThinkFlowLab/OpenDirector --json \ + number,url,title,body,author,authorAssociation,headRefOid,baseRefOid,\ + baseRefName,isCrossRepository,maintainerCanModify,mergeable,mergeStateStatus +``` + +Check out or show that tip SHA. If local `HEAD` is not the tip, review the +fetched SHA, not the worktree. Read `headRefOid` again before posting. If +the tip moved, stop and use WITHHOLD. Do not finish against the old SHA. + +## Inventory the diff + +List paths against the merge base and read that diff: + +```bash +git diff --name-status origin/main... +git diff --stat origin/main... +``` + +Use three dots so the inventory is the pull request, not the unrelated +history of main. Every changed path is in scope, including files the author +says are untouched. + +## Issue fit + +Read each linked issue, including its acceptance criteria and scope. PASS +requires the tip to satisfy that scoped acceptance. Extra product behavior +outside the issue is a finding. A docs or process change the user scoped +with no issue is not automatically a defect; say that no issue is linked. + +## Architecture and contracts + +When the diff touches creative capabilities, persistence, generation, +adoption, adapters, or user-visible text, check the matching principle in +AGENTS.md. Name the break and the section. Do not paste the principle. + +- Capability equivalence: a business operation in scope exists for the UI + and for the agent, under the same rules. +- The project is the source of truth: no second editable project in the + browser, and no canvas-only business fact. +- Generation intent, execution facts, and current adoption stay separate. +- User-visible copy ships `zh-CN` and `en-US` together. +- Domain and application services own the rule; HTTP, UI, and tool + adapters translate and present it. + +Also apply lessons whose trigger matches the diff, and the CONTRIBUTING +checklist items the diff can violate (one concern, a failing-before test +for a behavior change, docs updated with the contract, no invented +provider evidence, no credentials in the diff). + +## CI honesty + +Required CI is the **Release checks** workflow +(`.github/workflows/ci.yml`, reusable job in +`.github/workflows/release-checks.yml`) plus any other required status on +the repository. Attribute a run only when its head SHA is this tip. + +```bash +gh pr checks --repo ThinkFlowLab/OpenDirector +``` + +Confirm the check run's SHA in the Actions URL or check-run payload. A +green run on an older head is not evidence for this tip. + +- Green required checks on this SHA, and a clean merge against current + main: eligible for PASS when the rest of the review is clean. +- Pending, queued, in progress, expected but missing, or otherwise + unsettled: WITHHOLD. Unsettled Release checks are not PASS. +- A failed required check is not PASS. REQUEST CHANGES when the log shows + a defect in the tip. WITHHOLD when you cannot yet separate a flake from + a regression; use the CI failure notes. Do not rerun or bypass red CI + from this skill. +- `mergeStateStatus` of `BEHIND`, `DIRTY`, or `UNKNOWN`, or `mergeable` + of `CONFLICTING`: WITHHOLD, and say that a rebase onto current main is + needed. Do not record that as PASS. +- Nightly lanes do not stand in for Release checks unless the change + claims that lane. + +## Tests + +For a claimed fix, require a test that fails on the previous revision. +Absence of that coverage is REQUEST CHANGES. Docs-only and skill-only +diffs do not need unrelated model, media, or browser tests. + +`Ran:` lists commands and CI jobs actually observed on this tip SHA. +`Not run:` lists relevant checks nobody ran. A passing local tree that is +not this SHA does not count. Never invent a green run. Distinguish a +model double from a real provider call; do not upgrade either. + +- For a user-visible tip (UI, canvas, provider settings, generation + flows, and similar), require screenshots, a short screen recording, or + a demo GIF in the pull request, or an explicit N/A with a reason + (docs-only, pure refactor with no UI, CI-only). If that is missing, + WITHHOLD while the author can still add it, and REQUEST CHANGES when + the tip is offered as ready. Do not PASS on CI alone. The author prompt + is `.github/pull_request_template.md`. + +## External and fork tips + +Apply this in addition to the full review when the author is external or +first-time, or `isCrossRepository` is true: + +- Secrets, credentials, private assets, or raw provider responses in the + diff: REQUEST CHANGES. +- `maintainerCanModify` false: note that maintainers cannot update the + branch. That note alone is not a code defect. +- Chinese-only PR title or body: REQUEST CHANGES, or record it as a + language finding when the verdict is already WITHHOLD. PR titles, + bodies, and review posts are English. Do not PASS over a Chinese-only + PR body. + +Issue bodies published by `feat-issue` or `bug-issue` are Chinese by +default. Do not treat that issue text as a PR-language defect. Product +copy still needs both `zh-CN` and `en-US` in the same change. + +## Choose one verdict + +- **PASS** — the tip matches the linked issue; no actionable defect for + the scoped risk; Release checks and other required statuses are green + on this tip SHA; the tip merges cleanly onto current main. A + user-visible change includes a screenshot, short recording, demo GIF, + or a justified N/A. +- **WITHHOLD** — evidence is incomplete: CI is running or unsettled, the + tip moved, the review is unfinished, the branch is conflicted or + behind main, or a user-visible change still has no demo and no + justified N/A. WITHHOLD is not PASS. +- **REQUEST CHANGES** — an actionable defect: correctness, a break of an + AGENTS.md contract, security, missing tests for a claimed fix, a + dishonest claim, a language-policy break on user-facing copy or on + PR text that must be English, or a ready user-visible tip with neither + media nor a justified N/A. + +One verdict only. Do not soften REQUEST CHANGES into PASS because the +rest looks fine. Do not soften WITHHOLD into PASS because the diff looks +fine while CI is unsettled. + +## Output + +English. Short enough to post in the room or chat. First line is the +verdict, the tip short SHA (at least 7 hex digits), and the `Fixes` / +`Refs` numbers. Then findings, then `Ran:` and `Not run:`. On PASS, the +last line names the Coordinator. Use "no actionable findings" only on +PASS. + +```text +PASS abc1234 Fixes #123 + +- no actionable findings + +Ran: Release checks green on abc1234 (full tip SHA) +Not run: local pytest + +Coordinator: ping after this Reviewer PASS and green CI on this SHA. This skill does not merge. +``` + +```text +WITHHOLD abc1234 Refs #123 + +- Release checks still running on this tip SHA +- behind main; rebase onto current main is required + +Ran: none on this SHA +Not run: Release checks (unsettled) +``` + +```text +REQUEST CHANGES abc1234 Fixes #123 + +- claimed fix has no test that fails without the change +- user-visible string added for en-US only + +Ran: Release checks green on abc1234 (full tip SHA) +Not run: local browser lane +``` + +If asked to comment on the PR, paste the same text. That comment still +does not merge and does not dismiss a failing check. diff --git a/.github/auto-labels.json b/.github/auto-labels.json new file mode 100644 index 000000000..88507a9b6 --- /dev/null +++ b/.github/auto-labels.json @@ -0,0 +1,5 @@ +{ + "baseUrl": "https://api.deepseek.com", + "model": "deepseek-flash", + "minimumConfidence": 0.8 +} diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 000000000..0af5a6bfb --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,42 @@ +## Summary + + + +- + +## Linked issue + +`Fixes #N` closes that issue when this pull request merges. `Refs #N` links the issue and leaves it open. Use `Refs` when the issue should stay open, or when auto-close intent is unclear. If no issue applies, say so. + +- [ ] `Fixes #N` or `Refs #N` is included, or no issue applies + +## Labels + +Apply one `type:` label and at least one `area:` label. The taxonomy is in [CONTRIBUTING.md](../CONTRIBUTING.md). + +- [ ] Labels applied + +## Test plan + +Say what you ran locally and what you left to Release checks. Name checks you could not run. Do not list a check you did not run. Commands are in [docs/development/validation.md](../docs/development/validation.md). + +- Local: +- Left to CI: + +## Evidence that it works + +Reviewers use this to see that the change works, not only that CI is green. When the change is user-visible (UI, canvas, provider settings, generation flows, and similar), attach screenshots, a short screen recording, or a demo GIF. Otherwise mark N/A and give the reason (docs-only, pure refactor with no UI, CI-only, and similar). + +- [ ] Media attached, or N/A because: + +## Provider verification + +Never claim real-provider verification you did not run. State what used a model double, what still needs credentials, and what remains unverified. See [CONTRIBUTING.md](../CONTRIBUTING.md). + +- [ ] Real-provider calls are described honestly, or none were made + +## Bilingual UI + +When user-facing strings change, ship `zh-CN` and `en-US` together. See [AGENTS.md](../AGENTS.md). + +- [ ] Both locales are updated, or no user-facing copy changed diff --git a/.github/reviewer-routing.json b/.github/reviewer-routing.json new file mode 100644 index 000000000..777360077 --- /dev/null +++ b/.github/reviewer-routing.json @@ -0,0 +1,20 @@ +{ + "reviewers": { + "area:ci": ["yenuo26", "wangtiance"], + "area:agent": ["FayeSpica", "wuzhf9"], + "area:canvas": ["allendyu", "Probiotics5333"], + "area:timeline": ["allendyu", "Probiotics5333"], + "area:assets": ["princepride", "FayeSpica"], + "area:performance": ["kunkunblueberry"], + "area:provider": ["FayeSpica", "nikiwang-125"], + "area:docs": ["wuzhf9"], + "area:install": ["david6666666"], + "area:desktop": ["david6666666", "wangtiance"] + }, + "fallback": ["hsliuustc0106", "princepride"], + "labels": [ + {"name":"area:performance", "color":"94a3b8", "description":"Performance and responsiveness / 性能与响应速度"}, + {"name":"area:docs", "color":"94a3b8", "description":"Documentation / 文档"}, + {"name":"area:desktop", "color":"94a3b8", "description":"Desktop integration / 桌面集成"} + ] +} diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c4604401b..84176ddf1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -9,6 +9,12 @@ on: permissions: contents: read +concurrency: + group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.run_id }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + jobs: checks: uses: ./.github/workflows/release-checks.yml + with: + static-checks: true diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml new file mode 100644 index 000000000..95625ebd6 --- /dev/null +++ b/.github/workflows/codeql.yml @@ -0,0 +1,58 @@ +name: CodeQL + +on: + pull_request: + push: + branches: [main] + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.run_id }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +jobs: + analyze: + name: Analyze (${{ matrix.language }}) + runs-on: ubuntu-22.04 + timeout-minutes: 30 + permissions: + contents: read + actions: read + security-events: write + strategy: + fail-fast: false + matrix: + language: [python, javascript-typescript] + steps: + - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 + with: + persist-credentials: false + - name: Initialize CodeQL / 初始化安全扫描 + uses: github/codeql-action/init@33d7b85da1a04171e211befe2c739303b9a4c066 # v4.39.3 + with: + languages: ${{ matrix.language }} + build-mode: none + queries: security-extended + - name: Analyze and upload results / 扫描并上传结果 + uses: github/codeql-action/analyze@33d7b85da1a04171e211befe2c739303b9a4c066 # v4.39.3 + with: + category: /language:${{ matrix.language }} + wait-for-processing: true + + # Stable required-check candidate, independent of docs-only release skips. + checks: + name: CodeQL / checks + if: always() + needs: analyze + runs-on: ubuntu-22.04 + timeout-minutes: 5 + steps: + - name: Require all analyses / 要求全部语言扫描成功 + env: + ANALYSIS_RESULT: ${{ needs.analyze.result }} + run: >- + python -c 'import os, sys; result = os.environ.get("ANALYSIS_RESULT", "missing"); + print("CodeQL analysis / 安全扫描: " + result); sys.exit(0 if result == "success" else 1)' diff --git a/.github/workflows/dsh-plugin.yml b/.github/workflows/dsh-plugin.yml new file mode 100644 index 000000000..1724e2805 --- /dev/null +++ b/.github/workflows/dsh-plugin.yml @@ -0,0 +1,39 @@ +name: DSH plugin checks + +on: + pull_request: + paths: + - 'plugins/deepseek-harness/**' + - '.github/workflows/dsh-plugin.yml' + - '.nvmrc' + push: + branches: [main] + paths: + - 'plugins/deepseek-harness/**' + - '.github/workflows/dsh-plugin.yml' + - '.nvmrc' + +permissions: + contents: read + +concurrency: + group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.run_id }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +jobs: + tests: + runs-on: ubuntu-22.04 + timeout-minutes: 15 + steps: + - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 + - uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0 + with: + node-version-file: .nvmrc + cache: npm + cache-dependency-path: plugins/deepseek-harness/package-lock.json + - name: Install DSH plugin dependencies + working-directory: plugins/deepseek-harness + run: npm ci + - name: DSH plugin tests + working-directory: plugins/deepseek-harness + run: npm test diff --git a/.github/workflows/issue-labels.yml b/.github/workflows/issue-labels.yml new file mode 100644 index 000000000..1c6b792f3 --- /dev/null +++ b/.github/workflows/issue-labels.yml @@ -0,0 +1,27 @@ +name: Issue labels +on: + issues: + types: [opened, edited, reopened] +permissions: {} +concurrency: + group: issue-labels-${{ github.event.issue.number }} + cancel-in-progress: false +jobs: + classify: + runs-on: ubuntu-latest + timeout-minutes: 3 + permissions: + contents: read + issues: write + steps: + # Issue text is data only. Never check out contributor-provided code. + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + with: + persist-credentials: false + - uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1 + env: + LABEL_API_KEY: ${{ secrets.wtc_github }} + with: + script: | + const { run } = require('./scripts/github/issue-labels.cjs'); + await run({ github, context, core }); diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index f375625e7..e0f45aca6 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -6,6 +6,9 @@ # Release policy: cutting a daily v0.1.x requires the gate-parity lane green # on the candidate SHA. The extended lanes are advisory in v1: their failures # are reported and tracked, but they do not block a release by themselves. +# daily-benchmark runs after perf-outputs and is in that same advisory class. +# It fails only on a threshold regression, a broken suite, or a required +# measurement missing from a sample that otherwise parsed. # Matrix and policy are documented in docs/development/validation.md. # # Schedule: 0 18 * * * UTC = 02:00 Asia/Shanghai next day. @@ -18,6 +21,7 @@ on: permissions: contents: read + actions: read concurrency: group: nightly-${{ github.ref }} @@ -214,7 +218,9 @@ jobs: ) for entry in "${fixture_scripts[@]}"; do log="$RUNNER_TEMP/extended-logs/${entry%.mjs}.log" - if node "tests/browser/$entry" >"$log" 2>&1; then + # These scripts read CREATION_WEB_URL. Point them at the fixture + # Vite server; the workflow env still names the regular :5179 dev server. + if CREATION_WEB_URL=http://127.0.0.1:5180 CREATION_API_URL=http://127.0.0.1:8020 node "tests/browser/$entry" >"$log" 2>&1; then echo "pass $entry" >> extended-results.txt else echo "fail $entry" >> extended-results.txt @@ -378,12 +384,15 @@ jobs: done curl --fail -H 'X-OpenDirector-Request: 1' "$CREATION_API_URL/api/creation-test-health" curl --silent --fail "$CREATION_WEB_URL/" >/dev/null + # One artifact directory: the report sits at its root, where + # daily-benchmark reads nightly-perf/nightly-perf.json (#299). + mkdir -p nightly-perf/profiles node tests/browser/mp4_render_probe.mjs > /tmp/render-probe.json PROFILE_LABEL="nightly-${GITHUB_SHA::7}" node tests/browser/reference_drag_performance.mjs > /tmp/drag-perf.json jq -n --arg sha "$GITHUB_SHA" --arg run "$GITHUB_RUN_ID" \ --argjson render "$(cat /tmp/render-probe.json)" \ --argjson drag "$(cat /tmp/drag-perf.json)" \ - '{sha: $sha, runId: ($run | tonumber), render: $render, drag: $drag}' > nightly-perf.json + '{sha: $sha, runId: ($run | tonumber), render: $render, drag: $drag}' > nightly-perf/nightly-perf.json { echo "## Render output verification" echo "" @@ -403,30 +412,157 @@ jobs: for log in "$RUNNER_TEMP/creation-server.log" "$RUNNER_TEMP/vite.log"; do if [ -f "$log" ]; then tail -n 150 "$log"; fi done + - name: Stage drag profiles beside the report + # upload-artifact roots the archive at the common ancestor of its + # paths; mixing the workspace report with /tmp profiles made that `/` + # and buried the report under home/runner/... (#299). + if: always() + run: | + mkdir -p nightly-perf/profiles + for profile in /tmp/opendirector-drag-nightly-*.cpuprofile; do + if [ -f "$profile" ]; then cp "$profile" nightly-perf/profiles/; fi + done - name: Upload perf and output report if: always() uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: nightly-perf - path: | - nightly-perf.json - /tmp/opendirector-drag-nightly-*.cpuprofile + path: nightly-perf/ if-no-files-found: ignore + # Lane 6: fixed comparable benchmark, after this run's perf sample exists. + # Advisory, same class as perf-outputs: the result does not block v0.1.x. + # The job fails for a threshold regression, a broken suite, or a required + # measurement missing from a parsed sample; that is the room signal on + # nightly-summary (notify: true). The one-line comparison is published on + # every run, including improvements. + # + # Scenario and thresholds: scripts/nightly/benchmark.json + # (#180 scale: cold-open/delete 600 commits / 200 nodes, catalog 200 projects). + # UI numbers are this run's nightly-perf sample, not a second browser pass. + # Budgets in docs/development/ui-performance.md are a reading reference only. + daily-benchmark: + needs: [perf-outputs] + if: always() && (github.event_name != 'schedule' || github.repository == 'ThinkFlowLab/OpenDirector') && needs.perf-outputs.result != 'cancelled' && needs.perf-outputs.result != 'skipped' + runs-on: ubuntu-22.04 + timeout-minutes: 30 + outputs: + summary: ${{ steps.report.outputs.summary }} + notify: ${{ steps.report.outputs.notify }} + steps: + - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 + - uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0 + with: + python-version: '3.12' + cache: pip + cache-dependency-path: requirements.txt + - name: Install Python dependencies + # The micro-bench imports the source tree itself. Skip the editable + # install so this lane does not rebuild the frontend. + run: | + grep -v '^-e \.' requirements.txt > "$RUNNER_TEMP/benchmark-requirements.txt" + python -m pip install -r "$RUNNER_TEMP/benchmark-requirements.txt" + - name: Download this run's UI sample + continue-on-error: true + uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0 + with: + name: nightly-perf + path: nightly-perf + - name: Download the previous trend artifact + continue-on-error: true + env: + GH_TOKEN: ${{ github.token }} + run: | + set -u + mkdir -p previous-benchmark + ids="$(gh run list --workflow nightly.yml --repo "$GITHUB_REPOSITORY" --limit 20 \ + --json databaseId,status \ + --jq "[.[] | select(.databaseId != ${{ github.run_id }} and .status == \"completed\") | .databaseId] | .[]")" + for id in $ids; do + if gh run download "$id" --repo "$GITHUB_REPOSITORY" --name daily-benchmark --dir previous-benchmark; then + echo "previous run $id" + exit 0 + fi + done + echo "no previous daily-benchmark artifact" + exit 0 + - id: report + name: Run the fixed suite and compare + run: | + set -eu + args=(python scripts/nightly/daily_benchmark.py + --config scripts/nightly/benchmark.json + --out daily-benchmark.json + --sha "$GITHUB_SHA" + --run-id "$GITHUB_RUN_ID") + if [ -f nightly-perf/nightly-perf.json ]; then + args+=(--perf nightly-perf/nightly-perf.json) + fi + if [ "${{ needs.perf-outputs.result }}" = "success" ]; then + args+=(--ui-required) + fi + if [ -f previous-benchmark/daily-benchmark.json ]; then + args+=(--previous previous-benchmark/daily-benchmark.json) + fi + set +e + "${args[@]}" + status=$? + set -e + if [ "$status" -ne 0 ] && [ ! -f daily-benchmark.json ]; then + exit "$status" + fi + summary="$(python -c 'import json; print(json.load(open("daily-benchmark.json"))["summary"])')" + notify="$(python -c 'import json; print("true" if json.load(open("daily-benchmark.json"))["notify"] else "false")')" + { + echo "notify=$notify" + echo "summary<> "$GITHUB_OUTPUT" + { + echo "## Daily comparable benchmark" + echo "" + echo "$summary" + } >> "$GITHUB_STEP_SUMMARY" + - name: Upload trend artifact + if: always() + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 + with: + name: daily-benchmark + path: daily-benchmark.json + if-no-files-found: error + - name: Fail on threshold regression, suite failure or an incomplete sample + if: steps.report.outputs.notify == 'true' + run: exit 1 + # Aggregated run summary; also the machine-readable lane record the # Coordinator consumes for the daily release decision. summary: if: always() && (github.event_name != 'schedule' || github.repository == 'ThinkFlowLab/OpenDirector') - needs: [gate-parity, pytest-flake-triage, browser-extended, browser-production, perf-outputs] + needs: [gate-parity, pytest-flake-triage, browser-extended, browser-production, perf-outputs, daily-benchmark] runs-on: ubuntu-22.04 timeout-minutes: 5 steps: - name: Aggregate lane results + env: + BENCH_RESULT: ${{ needs.daily-benchmark.result }} + BENCH_SUMMARY: ${{ needs.daily-benchmark.outputs.summary }} + BENCH_NOTIFY: ${{ needs.daily-benchmark.outputs.notify }} run: | run_url="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" sha="${{ github.sha }}" flake="${{ needs.pytest-flake-triage.outputs.classification }}" [ -n "$flake" ] || flake=unknown + case "$BENCH_RESULT" in + success) if [ "$BENCH_NOTIFY" = "true" ]; then bench_notify=true; else bench_notify=false; fi ;; + failure) bench_notify=true ;; + *) bench_notify=false ;; + esac + if [ -n "$BENCH_SUMMARY" ]; then + bench_summary="$BENCH_SUMMARY" + else + bench_summary="daily benchmark ${BENCH_RESULT}" + fi cat > summary.md <> "$GITHUB_STEP_SUMMARY" @@ -450,9 +591,13 @@ jobs: --arg extended "${{ needs.browser-extended.result }}" \ --arg production "${{ needs.browser-production.result }}" \ --arg perf "${{ needs.perf-outputs.result }}" \ + --arg bench "$BENCH_RESULT" \ + --arg benchSummary "$bench_summary" \ + --argjson benchNotify "$bench_notify" \ '{sha: $sha, runId: ($run | tonumber), url: $url, lanes: {gateParity: $gate, flakeTriage: {result: $flake, classification: $flakeClass}, - browserExtended: $extended, browserProduction: $production, perfOutputs: $perf}}' \ + browserExtended: $extended, browserProduction: $production, perfOutputs: $perf, + dailyBenchmark: {result: $bench, notify: $benchNotify, summary: $benchSummary}}}' \ > nightly-summary.json - name: Upload nightly summary uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 diff --git a/.github/workflows/release-checks.yml b/.github/workflows/release-checks.yml index 2e30e831e..9a8ee4500 100644 --- a/.github/workflows/release-checks.yml +++ b/.github/workflows/release-checks.yml @@ -1,24 +1,131 @@ -# Reusable "Release checks" job, shared by ci.yml (PR / push to main) and -# nightly.yml (scheduled nightly gate-parity lane). -# -# Check steps after "Install locked dependencies" run independently of each -# other so one failing lane does not hide the outcome of the remaining lanes; -# they are still skipped when the environment install itself fails. +# Shared release checks: static gate, parallel lanes, required aggregate. +# All existing callers retain the full Python suite until queue activation. name: Release checks on: workflow_call: + inputs: + static-checks: + description: 'Run the shared local static hooks / 运行共享本地静态检查' + required: false + type: boolean + default: false + python-mark-expression: + description: 'Pytest selection; empty preserves the full suite / pytest 筛选,空值保留全量' + required: false + type: string + default: '' permissions: contents: read jobs: - checks: + static: + if: inputs.static-checks + runs-on: ubuntu-22.04 + timeout-minutes: 10 + outputs: + run-heavy: ${{ steps.scope.outputs.run-heavy }} + scope-reason: ${{ steps.scope.outputs.scope-reason }} + changed-files: ${{ steps.scope.outputs.changed-files }} + steps: + - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 + with: + fetch-depth: ${{ (github.event_name == 'pull_request' || (github.event_name == 'push' && github.event.before && github.event.before != '0000000000000000000000000000000000000000')) && '0' || '1' }} + - uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0 + with: + python-version: '3.12' + cache: pip + cache-dependency-path: requirements.txt + - name: Install static tooling + run: python -m pip install -r requirements-static.txt + - name: Check static configuration + env: + PR_BASE_SHA: ${{ github.event.pull_request.base.sha }} + PUSH_BASE_SHA: ${{ github.event.before }} + run: | + if [ "$GITHUB_EVENT_NAME" = pull_request ]; then + python -m pre_commit run --from-ref "$PR_BASE_SHA" --to-ref HEAD + elif [ "$GITHUB_EVENT_NAME" = push ] && [ -n "$PUSH_BASE_SHA" ] && [ "$PUSH_BASE_SHA" != 0000000000000000000000000000000000000000 ]; then + python -m pre_commit run --from-ref "$PUSH_BASE_SHA" --to-ref HEAD + else + python -m pre_commit run --all-files + fi + - id: scope + name: Select check scope / 判断检查范围 + env: + PR_BASE_SHA: ${{ github.event.pull_request.base.sha }} + run: python scripts/ci/select_checks.py + + python: + needs: static + if: ${{ !cancelled() && ((needs.static.result == 'success' && needs.static.outputs.run-heavy != 'false') || (!inputs.static-checks && needs.static.result == 'skipped')) }} + runs-on: ubuntu-22.04 + timeout-minutes: 30 + steps: + - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 + - uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0 + with: + node-version-file: .nvmrc + cache: npm + - uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0 + with: + python-version: '3.12' + cache: pip + cache-dependency-path: requirements.txt + - name: Install media tools + run: | + sudo apt-get update + sudo apt-get install --yes ffmpeg + ffmpeg -version + ffprobe -version + ffmpeg -hide_banner -encoders 2>&1 | grep 'libx264 ' + - id: install + name: Install locked dependencies + run: | + git rev-parse HEAD + node --version + npm --version + python --version + python -m pip install -v -r requirements.txt + npm exec --no -- opencode --version + python -m pip check + - name: Check layer imports + if: always() && steps.install.outcome == 'success' + run: lint-imports + # Runs before the Python suite: a failing or hanging test run must not hide + # whether the checked-in contracts still match the models. + - name: Check generated contracts + if: always() && steps.install.outcome == 'success' + run: python scripts/generate_contracts.py --check + - name: Python tests + if: always() && steps.install.outcome == 'success' + env: + PYTEST_MARK_EXPRESSION: ${{ inputs.python-mark-expression }} + run: python -m pytest tests -q -ra -m "$PYTEST_MARK_EXPRESSION" --durations=50 --junitxml=test-results/python.xml --ci-profile=test-results/python.json + - name: Check full layer inventory / 核对完整分层清单 + if: always() && steps.install.outcome == 'success' && inputs.python-mark-expression == '' + run: python scripts/ci/check_test_layers.py --profile test-results/python.json --require-executed + - name: Bundled OpenCode model gateway regression + if: always() && steps.install.outcome == 'success' + timeout-minutes: 5 + env: + OPENDIRECTOR_REAL_CLI: '1' + run: python -m pytest tests/test_native_runtime.py -k real_opencode -q -ra --durations=50 --junitxml=test-results/bundled-opencode.xml --ci-profile=test-results/bundled-opencode.json + - name: Upload Python timing evidence / 上传 Python 耗时证据 + if: always() && steps.install.outcome == 'success' + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 + with: + name: python-test-evidence-${{ github.run_attempt }} + path: test-results/ + retention-days: 14 + if-no-files-found: warn + + frontend: + needs: static + if: ${{ !cancelled() && ((needs.static.result == 'success' && needs.static.outputs.run-heavy != 'false') || (!inputs.static-checks && needs.static.result == 'skipped')) }} runs-on: ubuntu-22.04 timeout-minutes: 30 - env: - CREATION_WEB_URL: http://127.0.0.1:5179 - CREATION_API_URL: http://127.0.0.1:8019 steps: - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 - uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0 @@ -59,17 +166,6 @@ jobs: - name: Check browser manifest if: always() && steps.install.outcome == 'success' run: npm run check:browser-manifest - - name: Check layer imports - if: always() && steps.install.outcome == 'success' - run: lint-imports - # Runs before the Python suite: a failing or hanging test run must not hide - # whether the checked-in contracts still match the models. - - name: Check generated contracts - if: always() && steps.install.outcome == 'success' - run: python scripts/generate_contracts.py --check - - name: Python tests - if: always() && steps.install.outcome == 'success' - run: python -m pytest tests -q -ra - name: Production build if: always() && steps.install.outcome == 'success' run: npm run build @@ -81,13 +177,54 @@ jobs: "$RUNNER_TEMP/opendirector-wheel-venv/bin/python" -m pip install "$RUNNER_TEMP"/opendirector-wheels/opendirector-*.whl "$RUNNER_TEMP/opendirector-wheel-venv/bin/python" -m pip check python scripts/verify_install.py --python "$RUNNER_TEMP/opendirector-wheel-venv/bin/python" + + browser: + needs: static + if: ${{ !cancelled() && ((needs.static.result == 'success' && needs.static.outputs.run-heavy != 'false') || (!inputs.static-checks && needs.static.result == 'skipped')) }} + runs-on: ubuntu-22.04 + timeout-minutes: 30 + strategy: + fail-fast: false + matrix: + journey: [isolated, voice, assistant] + env: + CREATION_WEB_URL: http://127.0.0.1:5179 + CREATION_API_URL: http://127.0.0.1:8019 + steps: + - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 + - uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0 + with: + node-version-file: .nvmrc + cache: npm + - uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0 + with: + python-version: '3.12' + cache: pip + cache-dependency-path: requirements.txt + - name: Install media tools + run: | + sudo apt-get update + sudo apt-get install --yes ffmpeg + ffmpeg -version + ffprobe -version + ffmpeg -hide_banner -encoders 2>&1 | grep 'libx264 ' + - id: install + name: Install locked dependencies + run: | + git rev-parse HEAD + node --version + npm --version + python --version + python -m pip install -v -r requirements.txt + npm exec --no -- opencode --version + python -m pip check - name: Install isolated browser dependencies if: always() && steps.install.outcome == 'success' run: | npm install --prefix "$RUNNER_TEMP/opendirector-browser" --no-save --no-package-lock playwright-core@1.58.2 node "$RUNNER_TEMP/opendirector-browser/node_modules/playwright-core/cli.js" install --with-deps --only-shell chromium - name: Isolated browser workflows - if: always() && steps.install.outcome == 'success' + if: always() && steps.install.outcome == 'success' && matrix.journey == 'isolated' run: | export PLAYWRIGHT_MODULE="$RUNNER_TEMP/opendirector-browser/node_modules/playwright-core/index.mjs" python tests/browser/creation_server.py --port 8019 >"$RUNNER_TEMP/creation-server.log" 2>&1 & @@ -104,8 +241,30 @@ jobs: curl --fail -H 'X-OpenDirector-Request: 1' "$CREATION_API_URL/api/creation-test-health" curl --silent --fail "$CREATION_WEB_URL/" >/dev/null node scripts/run-browser-suite.mjs + - name: Voice library workflows + if: always() && steps.install.outcome == 'success' && matrix.journey == 'voice' + timeout-minutes: 5 + env: + CREATION_WEB_URL: http://127.0.0.1:5190 + CREATION_API_URL: http://127.0.0.1:8030 + run: | + export PLAYWRIGHT_MODULE="$RUNNER_TEMP/opendirector-browser/node_modules/playwright-core/index.mjs" + python tests/browser/voice_library_server.py --port 8030 >"$RUNNER_TEMP/voice-server.log" 2>&1 & + api_pid=$! + OPENDIRECTOR_API_URL="$CREATION_API_URL" node node_modules/vite/bin/vite.js --host 127.0.0.1 --port 5190 --strictPort >"$RUNNER_TEMP/voice-vite.log" 2>&1 & + web_pid=$! + trap 'kill "$api_pid" "$web_pid" 2>/dev/null || true; wait "$api_pid" "$web_pid" 2>/dev/null || true' EXIT + for attempt in {1..60}; do + if curl --silent --fail -H 'X-OpenDirector-Request: 1' "$CREATION_API_URL/api/voice-test-health" >/dev/null && curl --silent --fail "$CREATION_WEB_URL/" >/dev/null; then + break + fi + sleep 1 + done + curl --fail --silent -H 'X-OpenDirector-Request: 1' "$CREATION_API_URL/api/voice-test-health" + curl --silent --fail "$CREATION_WEB_URL/" >/dev/null + node tests/browser/voice_library.mjs - name: Assistant auxiliary panels - if: always() && steps.install.outcome == 'success' + if: always() && steps.install.outcome == 'success' && matrix.journey == 'assistant' timeout-minutes: 5 env: CREATION_WEB_URL: http://127.0.0.1:5180 @@ -129,6 +288,21 @@ jobs: - name: Print isolated service logs on failure if: failure() run: | - for log in "$RUNNER_TEMP/creation-server.log" "$RUNNER_TEMP/vite.log" "$RUNNER_TEMP/assistant-creation-server.log" "$RUNNER_TEMP/assistant-vite.log"; do + for log in "$RUNNER_TEMP/voice-server.log" "$RUNNER_TEMP/voice-vite.log" "$RUNNER_TEMP/creation-server.log" "$RUNNER_TEMP/vite.log" "$RUNNER_TEMP/assistant-creation-server.log" "$RUNNER_TEMP/assistant-vite.log"; do if [ -f "$log" ]; then tail -n 150 "$log"; fi done + + # Preserve the existing required-check name and fail closed on failed, + # cancelled or unexpectedly skipped lanes, including static-gate failures. + checks: + if: always() + needs: [static, python, frontend, browser] + runs-on: ubuntu-22.04 + timeout-minutes: 5 + steps: + - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 + - name: Require every release-check lane / 汇总发布检查 + env: + CHECK_RESULTS: ${{ toJSON(needs) }} + STATIC_ENABLED: ${{ toJSON(inputs.static-checks) }} + run: python scripts/ci/summarize_checks.py diff --git a/.github/workflows/reviewer-routing.yml b/.github/workflows/reviewer-routing.yml new file mode 100644 index 000000000..8cd2d5025 --- /dev/null +++ b/.github/workflows/reviewer-routing.yml @@ -0,0 +1,41 @@ +name: Reviewer routing +on: + pull_request_target: + branches: [main] + types: [opened, synchronize, ready_for_review, labeled, unlabeled, reopened] + push: + branches: [main] + paths: ['.github/reviewer-routing.json', '.github/workflows/reviewer-routing.yml', 'scripts/github/reviewer-routing.cjs'] + workflow_dispatch: +permissions: {} +concurrency: + group: reviewer-routing-${{ github.event.pull_request.number || 'labels' }} + cancel-in-progress: false +jobs: + route: + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read + pull-requests: write + issues: write + steps: + # Never check out or execute the fork's head in this privileged workflow. + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + with: + ref: ${{ github.event.pull_request.base.sha || github.sha }} + persist-credentials: false + - name: Classify PR labels / 自动分类标签 + if: github.event_name == 'pull_request_target' + uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1 + env: + LABEL_API_KEY: ${{ secrets.wtc_github }} + with: + script: | + const { run } = require('./scripts/github/auto-labels.cjs'); + await run({ github, context, core }); + - uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1 + with: + script: | + const { run } = require('./scripts/github/reviewer-routing.cjs'); + await run({ github, context, core }); diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 000000000..0483bf1b6 --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,31 @@ +# Local lightweight checks only. Install: python -m pre_commit install +# Before PR: python -m pre_commit run --from-ref upstream/main --to-ref HEAD +minimum_pre_commit_version: '4.5.1' +default_install_hook_types: [pre-commit] +default_stages: [pre-commit] + +repos: + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v6.0.0 + hooks: + - id: check-yaml + # Parse custom YAML tags without constructing Python objects. + args: [--unsafe] + - id: check-json + - id: check-toml + - id: check-ast + - id: check-merge-conflict + - id: debug-statements + - id: end-of-file-fixer + - id: mixed-line-ending + args: [--fix=lf] + - id: trailing-whitespace + args: [--markdown-linebreak-ext=md] + + - repo: https://github.com/rhysd/actionlint + rev: v1.7.7 + hooks: + - id: actionlint + files: ^\.github/workflows/.*\.ya?ml$ + # Keep this hook portable: shellcheck/pyflakes are separate toolchains. + args: [-shellcheck=, -pyflakes=] diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 39f804f1c..64eb129e7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,6 +1,12 @@ # Contributing -Thanks for helping with OpenDirector. This file covers the issue/PR labels and the checklist we ask for before a change is reviewed. Architecture rules live in [AGENTS.md](AGENTS.md); reproducible checks and evidence requirements live in [docs/development/validation.md](docs/development/validation.md). +Thanks for helping with OpenDirector. This file covers the issue/PR labels and the checklist we ask for before a change is reviewed. Architecture rules live in [AGENTS.md](AGENTS.md); reproducible checks and evidence requirements live in [docs/development/validation.md](docs/development/validation.md). Tag and daily release practice lives in [docs/development/release.md](docs/development/release.md). + +## Before opening a PR / 提出 PR 前 + +Install the lightweight static hooks and run the full PR-diff check described in the [validation guide](docs/development/validation.md#pr-前轻量静态检查--lightweight-checks-before-a-pr). Fix failures before opening the PR. Local checks supplement the existing CI and relevant tests. + +提出 PR 前,按[验证指南](docs/development/validation.md#pr-前轻量静态检查--lightweight-checks-before-a-pr)安装轻量静态 hook 并检查整个 PR 差异,修复失败后再提出 PR。本地检查与现有 CI、相关测试配合使用。 ## Label taxonomy @@ -30,15 +36,20 @@ Labels are short, English and prefixed by dimension, so issues and PRs can be fi | `area:install` | Install, startup, environment support | | `area:i18n` | Localisation and bilingual UI | | `area:ci` | Continuous integration and checks | +| `area:performance` | Performance and responsiveness / 性能与响应速度 | +| `area:docs` | Documentation / 文档 | +| `area:desktop` | Desktop integration / 桌面集成 | ### `modality:` and `provider:` — only when relevant `modality:image`, `modality:video`, `modality:audio` describe what the work produces. `provider:minimax`, `provider:moss`, `provider:openai`, `provider:flux`, `provider:kling`, `provider:runway`, `provider:google` name the external service. Use `provider:openai` for OpenAI and any OpenAI-compatible service only when OpenAI itself is the subject. -### `priority:` — maintainers only +### `priority:` — automatic initial triage, maintainer authority / 自动初判,维护者裁定 + +`priority:p0` blocks the agreed release scope, `priority:p1` is scheduled next, `priority:p2` is accepted but unscheduled. Contributors do not need to pick one. Issue automation may add an initial priority when supported by evidence; maintainers retain authority and existing priorities are never overwritten automatically. -`priority:p0` blocks the agreed release scope, `priority:p1` is scheduled next, `priority:p2` is accepted but unscheduled. Contributors do not need to pick one; a maintainer applies it after triage. +`priority:p0` 表示阻塞已约定的发布范围,`priority:p1` 表示下一步已排期,`priority:p2` 表示已接受但未排期。Issue 自动分类可根据证据补充初始优先级,维护者保留最终决定权;已有优先级不自动替换,新报告不默认视为已接受或已排期。 ### Process labels @@ -56,6 +67,20 @@ The default GitHub labels `bug`, `enhancement` and `documentation` were retired - A pull request carries at least one `type:` and one `area:` label, so the change appears in both filters. - When a label does not fit, open an issue with `type:meta` and propose the new dimension instead of inventing a free-form tag. +## Automatic reviewer requests / 自动邀请审查人 + +Edit [`.github/reviewer-routing.json`](.github/reviewer-routing.json) to maintain label-to-reviewer mappings and the fallback list. Open, ready-for-review, reopened and label-change events invite **all** owners of all matching labels, deduplicated. The PR author, already requested reviewers and people who submitted a review are excluded. With no matching label, all fallback reviewers are considered; a matched label whose only owner is the author does not activate fallback. Drafts are skipped. Existing invitations are never removed, including after label removal. Removing a label never adds fallback reviewers when no mapped labels remain. New commits do not re-request review. Existing PRs need a label change or reopening to trigger routing after installation. + +修改 [`.github/reviewer-routing.json`](.github/reviewer-routing.json) 维护标签与审查人的映射及默认名单。PR 创建、转为可审查、重开或标签变化时,邀请所有匹配标签的全部负责人并去重,排除作者、已邀请及已提交审查的人。无标签匹配时使用全部默认候选;匹配标签的唯一负责人是作者时,不启用默认名单。草稿跳过,已有邀请始终保留,移除标签也不撤回邀请;移除后无匹配标签时不追加默认审查人,新提交不重复邀请。功能安装后,已有 PR 需修改标签或重开才能触发。 + +The same configuration's `labels` list declares new labels. A push to main affecting the routing files, or a manual workflow dispatch, creates missing labels without updating or deleting existing ones. To change an existing label's color or description, use GitHub's Labels page. Reviewer accounts must be eligible for review requests in this repository; an unavailable account produces a warning while remaining candidates are still requested. API permission failures fail the job. No live review invitations are sent by local tests. + +配置中的 `labels` 列表声明新增标签。路由文件合入 main 或手动运行工作流后,补建缺少的标签,不修改或删除已有标签。修改已有标签的颜色或说明请使用 GitHub Labels 页面。候选账号必须具有仓库审查邀请资格;无法邀请的账号会产生警告,其余候选继续处理;API 权限错误使任务失败。本地测试不发送真实邀请。 + +The privileged workflow only checks out the PR base commit, never the contributor's head, and executes no PR-supplied code. Validate changes with `node --test tests/reviewerRouting.test.ts`. + +具有写权限的工作流只检出 PR 的目标分支提交,不检出或执行贡献者分支代码。修改后运行 `node --test tests/reviewerRouting.test.ts` 验证。 + ## Pull request checklist 1. Scope stays in one concern; unrelated refactors go to a separate PR. @@ -64,3 +89,32 @@ The default GitHub labels `bug`, `enhancement` and `documentation` were retired 4. Never claim real-provider verification you did not run. State what was validated with a model double, what needs credentials, and what remains unverified. 5. Credentials, private assets and raw provider responses never enter commits, fixtures, logs or receipts. 6. Run the checks from [docs/development/validation.md](docs/development/validation.md) locally, or make it explicit which ones you could not run. +7. For a user-visible change, attach screenshots or a short demo in the pull request, or mark N/A with a reason, as asked in the evidence section of the [pull request template](.github/pull_request_template.md). + +## Model-assisted issue labels / Issue 自动领域与优先级 + +The `Issue labels` workflow classifies open issues on `opened`, `edited` and `reopened`, using the same model configuration and repository-managed credentials as PR labeling. It sends only the bounded title (1000 characters), body (12000 characters), existing labels and allowed catalog. Issue text is untrusted evidence, never instructions; do not include credentials in issues. Only existing `area:*` and `priority:p0/p1/p2` labels can be added, at the configured confidence threshold (currently 0.8). At most one priority is selected, and an existing priority is preserved. Missing acceptance, scheduling or release-blocking evidence leaves priority unset; urgency alone is insufficient. + +`Issue labels` 工作流在 Issue 新建、编辑或重开时分类,与 PR 共用模型配置及仓库管理的凭据。仅发送限长标题(1000字符)、正文(12000字符)、现有标签及允许的目录;Issue 内容只作不可信证据,不作指令,不应包含凭据。仅按配置的置信度阈值(当前为0.8)添加仓库已有的 `area:*` 和 `priority:p0/p1/p2`,最多一个优先级,保留人工设置。缺少接受、排期或发布阻塞证据时不设置优先级,仅声称紧急不足以定级。 + +Each run makes at most one model request (60 seconds, 1500 output tokens), without automatic retries. Missing credentials, invalid output and provider failures use conservative bilingual keyword rules for areas only; they do not assign a fallback priority. No labels are created or removed, and no type/process labels or issue assignees are changed. Manual label changes do not trigger classification. Before adding labels, the workflow rechecks title, body, state, update time and labels; changed or closed issues skip stale results. API write failures fail the job. Raw responses, issue text and provider errors are not logged. The workflow runs trusted default-branch code and becomes active after merge; existing issues require an edit or reopening and are not bulk relabeled. + +每次运行最多一次模型请求(60秒、1500输出token),不自动重试。缺少凭据、无效响应或服务失败时只按保守中英关键词补领域,不兜底猜测优先级。不创建或删除标签,不修改类型、流程标签或负责人;手动标签变动不触发分类。写入前复查标题、正文、状态、更新时间和标签,内容变化或关闭时跳过过期结果;写入API失败使任务失败。不记录原始响应、Issue正文或服务错误详情。工作流只运行可信默认分支代码,合入后生效;已有Issue需编辑或重开,不批量追溯打标。 + +Validate with `node --test tests/issueLabels.test.ts tests/autoLabels.test.ts tests/reviewerRouting.test.ts`, `npm run check:i18n` and `npm run build`. Tests use model/API doubles and do not establish live model-service availability or real repository mutation behavior. + +使用上述测试、双语检查及构建命令验证;测试使用模型/API替身,不代表已验证真实模型服务或真实仓库写入。 + +## Model-assisted PR labels / 大模型自动标签 + +The reviewer workflow classifies open, non-draft PRs on opened, synchronize and ready-for-review events before requesting reviewers. Title, body, filenames and bounded patches are sent to the configured model service; familiar sensitive-file patterns omit patches, but this is not a guarantee of secret detection. Do not include credentials in PRs. + +审查工作流在PR创建、新提交和转为可审查时,先分类再邀请审查人,跳过草稿。标题、正文、文件名和限量diff会发送给配置的模型服务;常见敏感文件名不发送patch,但不保证自动识别所有秘密,PR中不得包含凭据。 + +Only existing `type:*`, `area:*`, `modality:*` and `provider:*` labels from the allowed catalog may be added. The classifier cannot set priority, ready or process-status labels, create labels, remove manual labels or replace an existing type. Invalid responses, missing credentials and provider failures use conservative path rules; unmatched changes retain normal fallback reviewer routing. New commits may add labels but do not remove earlier labels or reviewer requests. Manual label changes do not call the model or immediately re-add removed labels. + +只添加白名单内已存在的类型、领域、媒体和服务商标签;不设置优先级、ready及流程状态,不创建标签、不删除人工标签、不替换已有type。响应无效、未配置Secret或服务失败时使用保守路径规则,仍无匹配时沿用默认审查人。新提交可追加标签,不撤销旧标签或邀请;人工标签变化不调用模型,也不会立即补回被移除标签。 + +One bounded model request (60 seconds, 1500 output tokens) is made per eligible run, with no automatic inference retry or cross-run cache. Repeated eligible events can incur additional API usage. File context is limited to 200 files and 32000 patch characters; partial context may reduce classification accuracy. Before mutation, a changed head SHA, label set, draft or closed state causes the stale result to be skipped. The privileged workflow executes only trusted base-commit scripts, never PR-head code. Model errors and raw responses are not logged. Additions and fallback mode appear in Actions summary. Validate with `node --test tests/autoLabels.test.ts tests/reviewerRouting.test.ts` and `npm run build`; mocked tests do not establish live API availability. The workflow becomes active after merging into main. + +每次符合条件的运行最多一次模型请求(60秒、1500输出token),无自动推理重试或跨运行缓存;重复事件可能产生额外费用。最多提供200个文件及32000字符patch,上下文截断可能影响准确性。写入前复查head SHA、标签和PR状态,变化时跳过过期结果。具有写权限的工作流只运行可信base提交脚本,不执行PR分支代码;不记录模型错误正文或原始响应,Actions摘要显示添加标签和兜底方式。运行上述测试及完整构建;模拟测试不代表真实API可用。合入main后生效。 diff --git a/MANIFEST.in b/MANIFEST.in index 55275fdfd..48946cb84 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -4,7 +4,10 @@ recursive-include src * recursive-include packages/core/src *.py recursive-include apps/api/src *.py recursive-include packages/agent/skills * +include plugins/catalog.json prune packages/core/src/opendirector/_bundled prune node_modules prune dist prune build + +recursive-include public/fonts * diff --git a/README.md b/README.md index fd4234be5..2b5cce048 100644 --- a/README.md +++ b/README.md @@ -34,7 +34,7 @@ Use it to explore visual ideas, refine assets, and make short films. Projects st - **Timeline editing**: Arrange assets, adjust start and end points, speed, and volume, then preview and render an MP4. You can also export JSON, FCPXML, and asset bundles. - **Local projects, your choice of models**: The canvas saves automatically to your machine. Configure multiple models for a service and switch between them as needed. -If image generation repeatedly times out or disconnects through a system proxy, open **Settings → Model connections → Edit service → Advanced connection** and set **Network connection mode** to **Direct connection**. This affects that service's model requests and result downloads without changing the system proxy. Check the provider's records before starting another attempt when request acceptance is unknown. +If image generation repeatedly times out or disconnects through a system proxy, open **Settings → Model connections → Edit service → Advanced connection** and set **Network connection mode** to **Direct (no proxy)**. This affects that service's model requests and result downloads without changing the system proxy. Check the provider's records before starting another attempt when request acceptance is unknown. ## Example film: We Can Still Win @@ -89,6 +89,8 @@ Use Chrome or Edge for MP4 export, and keep the page and editor open while rende - [User guide](docs/guides/user-guide.md): Model connections, AI collaboration, canvas, media editing, and export. - [Troubleshooting](docs/guides/user-guide.md#troubleshooting): Installation, generation, saving, and export issues. - [Business CLI (Chinese)](docs/guides/cli.md): Work with projects through a terminal or Agent Shell. +- [DeepSeek Harness plugin](plugins/deepseek-harness/README.md): Generate media from DSH, track jobs, and continue editing in OpenDirector. +- [Optional creative plugins](docs/guides/plugins.md): Install, enable per harness, and create portable Skill packages. - [Documentation index (Chinese)](docs/README.md): Architecture, development, and validation documents. ## Community @@ -101,6 +103,17 @@ Join the **OpenDirector with vLLM-Omni (2)** WeChat group to discuss OpenDirecto

+If OpenDirector is useful to you, [give it a star on GitHub](https://github.com/ThinkFlowLab/OpenDirector) to support the project and help others discover it. + +

+ + + + Illustrated guide: open ThinkFlowLab/OpenDirector on GitHub and click Star + + +

+ ## Contributing We welcome bug reports and suggestions through [Issues](https://github.com/ThinkFlowLab/OpenDirector/issues), and code and documentation contributions through pull requests. See the [validation guide (Chinese)](docs/development/validation.md) for development checks. diff --git a/README.zh-CN.md b/README.zh-CN.md index 667ae880b..227de33e3 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -34,7 +34,7 @@ OpenDirector 是一个在本地运行、通过浏览器使用的 AI 视频创作 - **时间线编排**:安排素材顺序、起止点、速度和音量,预览并合成 MP4,也可导出 JSON、FCPXML 和素材包。 - **本地项目,自选模型**:画布自动保存到本机,同一服务可配置多个模型,按用途切换。 -若图片生成经系统代理反复超时或连接中断,可在「设置 → 模型接入 → 编辑服务 → 高级连接」将「网络连接方式」设为「直连」。此设置仅影响该服务的模型请求和结果下载,不修改系统代理。提交结果不明的任务应先核对供应商记录,再决定是否新建尝试。 +若图片生成经系统代理反复超时或连接中断,可在「设置 → 模型接入 → 编辑服务 → 高级连接」将「网络连接方式」设为「直连(不使用代理)」。此设置仅影响该服务的模型请求和结果下载,不修改系统代理。提交结果不明的任务应先核对供应商记录,再决定是否新建尝试。 ## 短片示例:《我们还能赢》 @@ -89,6 +89,8 @@ opendirector serve - [使用指南](docs/guides/user-guide.zh-CN.md):模型接入、AI 协作、画布、媒体编辑与导出。 - [故障排查](docs/guides/user-guide.zh-CN.md#故障排查):安装、生成、保存与导出问题。 - [业务 CLI](docs/guides/cli.md):通过终端或 Agent Shell 操作工程。 +- [DeepSeek Harness 插件](plugins/deepseek-harness/README.zh-CN.md):从 DSH 生成素材、跟踪任务,再回到 OpenDirector 继续编辑。 +- [可选创作插件](docs/guides/plugins.zh-CN.md):安装插件、按引擎启用,以及制作可移植的技能包。 - [文档索引](docs/README.md):架构、开发与验证文档。 ## 社区交流 @@ -101,6 +103,17 @@ opendirector serve

+如果 OpenDirector 对你有帮助,欢迎[在 GitHub 上点亮 Star](https://github.com/ThinkFlowLab/OpenDirector),支持项目,也让更多人发现它。 + +

+ + + + 操作示意:打开 GitHub 上的 ThinkFlowLab/OpenDirector,点击 Star 支持项目 + + +

+ ## 参与贡献 欢迎通过 [Issues](https://github.com/ThinkFlowLab/OpenDirector/issues) 反馈问题或提出建议,通过 Pull Request 贡献代码与文档。开发检查见[验证指南](docs/development/validation.md)。 diff --git a/apps/api/src/opendirector_api/agent.py b/apps/api/src/opendirector_api/agent.py index 1a53ae4ab..7bd2952a4 100644 --- a/apps/api/src/opendirector_api/agent.py +++ b/apps/api/src/opendirector_api/agent.py @@ -14,6 +14,7 @@ from opendirector.agent.catalog import model_choices from opendirector.agent.context import RunContext, RuntimeId +from opendirector.agent.citations import referenced_sources, with_trace_sources from opendirector.agent.runtime import AgentRuntimes, runtime_available, configured_runtimes, runtime_spec, runtime_catalog, runtime_status from opendirector.agent.skills import skills from opendirector.domain.projects import ProjectError @@ -58,6 +59,47 @@ class Turn(BaseModel): skills: list[str] = Field(default_factory=list, max_length=1) +def draft_content_is_empty(content): + """Blank text-only structured drafts are empty; reference or skill parts are not. + + Non-string text values are treated as present content so New does not raise, + and an accepted non-text or non-string part is not silently reused away. + """ + if not content: + return True + for part in content: + if not isinstance(part, dict): + return False + kind = part.get('type') + if kind == 'text': + text_value = part.get('text') + if not isinstance(text_value, str): + return False + if text_value.strip(): + return False + else: + return False + return True + + +def conversation_is_empty(value): + """A session with no messages or unsent content can be reused by New.""" + return (not value.get('messages') + and not (value.get('draft') or '').strip() + and not value.get('assets') + and not value.get('skills') + and draft_content_is_empty(value.get('draftContent'))) + + +def public_conversation(value): + return {key: item for key, item in value.items() + if key not in {'archivedConversations', 'reusedCreateIds'}} + + +def reused_ids(value): + return [str(item) for item in (value.get('reusedCreateIds') or [])] + + class Conversations: def __init__(self, projects): self.projects = projects @@ -103,7 +145,8 @@ def update(self, project_id: UUID, session: str | None = None, *, draft: Draft | changed = True if changed: durable_json(file, value) - return {key: item for key, item in value.items() if key != 'archivedConversations'} + result = public_conversation(value) + return with_trace_sources(result, directory) def list(self, project_id: UUID) -> dict: directory = self.path(project_id) @@ -131,20 +174,29 @@ def activate(self, project_id: UUID, conversation_id: UUID, session: str, *, cre current = read_json(file) if file.exists() else {'messages': [], **Draft().model_dump(mode='json')} current['conversationId'] = current.get('conversationId') or str(uuid5(project_id, 'assistant')) archives = current.pop('archivedConversations', []) - if current['conversationId'] == str(conversation_id): - return current - target = next((item for item in archives if item['conversationId'] == str(conversation_id)), None) + requested = str(conversation_id) + if current['conversationId'] == requested or requested in reused_ids(current): + return with_trace_sources(public_conversation(current), directory) + target = next((item for item in archives + if item['conversationId'] == requested or requested in reused_ids(item)), None) if target is None: if not create: raise ProjectError('对话不存在。', 404) + if conversation_is_empty(current): + aliases = reused_ids(current) + if requested not in aliases: + aliases.append(requested) + current['reusedCreateIds'] = aliases + durable_json(file, {**current, 'archivedConversations': archives}) + return with_trace_sources(public_conversation(current), directory) if len(archives) >= 99: raise ProjectError('项目对话已达 100 个,暂不能继续新建。', 409) target = {'messages': [], **Draft().model_dump(mode='json'), **{key: current[key] for key in ('runtime', 'runtimeModel', 'harnessModels', 'thinking', 'builtinModelThinking', 'modelSelections') if key in current}, - 'conversationId': str(conversation_id)} - archives = [item for item in archives if item['conversationId'] != str(conversation_id)] + 'conversationId': requested} + archives = [item for item in archives if item['conversationId'] != target['conversationId']] durable_json(file, {**target, 'archivedConversations': [current, *archives]}) - return target + return with_trace_sources(public_conversation(target), directory) class ConversationCreate(BaseModel): @@ -263,10 +315,11 @@ async def runtime_models(project_id: UUID, runtime: Literal['opencode', 'codex', return await model_choices(runtime) @router.get('/skills') - async def catalog(project_id: UUID): + async def catalog(project_id: UUID, runtime: Runtime = 'default'): directory = await asyncio.to_thread(conversations.path, project_id) - found = await skills(projects.local.repository, directory) - return [{key: item[key] for key in ('name', 'description', 'source')} for item in found] + found = await skills(projects.local.repository, directory, runtime) + return [{key: item[key] for key in ('name', 'description', 'source', 'pluginId', 'displayName', 'localizedDescription') + if key in item} for item in found] @router.post('/turns/{turn_id}/stop') async def stop(project_id: UUID, turn_id: UUID, x_opendirector_session: UUID = Header()): @@ -304,9 +357,9 @@ async def turn(project_id: UUID, value: Turn, x_opendirector_session: UUID = Hea if value.runtime == 'jiuwenswarm' and value.thinking != 'auto': raise ProjectError('JiuwenSwarm 当前使用原生默认推理强度。', 409) config = await asyncio.to_thread(configs.resolve, str(value.modelId), 'agent') if spec.modelSource == 'settings' else None - available = await skills(projects.local.repository, directory) if spec.supportsSkills else [] + available = await skills(projects.local.repository, directory, value.runtime) if spec.supportsSkills else [] if set(value.skills) - {item['name'] for item in available}: - raise ProjectError('技能已失效,请重新选择。', 409) + raise ProjectError('所选技能对当前引擎已不可用,请从消息中移除并重新选择。', 409, code='agent_skill_unavailable') snapshot = await asyncio.to_thread(projects.snapshot, project_id) assets = {str(asset.id): asset for asset in snapshot.assets} for asset_id in value.assets: @@ -321,6 +374,8 @@ async def turn(project_id: UUID, value: Turn, x_opendirector_session: UUID = Hea async def produce() -> None: text, status, error, accepted = '', 'complete', None, False activities = [] + sources = {} + previous_sources = {} plan = None try: user = {'id': str(value.id), 'role': 'user', 'text': value.text, 'content': value.content, @@ -333,6 +388,8 @@ async def produce() -> None: saved = await asyncio.to_thread(conversations.update, project_id, session, begin=user, conversation_id=value.conversationId) accepted = True events.put({'type': 'start', 'conversation': saved}) + previous_sources = {source['id']: source for message in saved['messages'][:-2] + for source in message.get('sources', [])} context = [] for message in saved['messages'][:-1]: content = message['text'] or '(empty reply)' @@ -347,7 +404,6 @@ async def produce() -> None: content += '\nReferenced project assets: ' + json.dumps(references, ensure_ascii=False) content += '\nThese are asset references, not visual observations. Inspect media with available tools before describing its contents.' context.append({'role': message['role'], 'content': content}) - context[-1]['content'] += '\nSummarize the delivered result and material limitations. Keep execution IDs, revisions and receipts in tool details unless requested.' if preferences: context[-1]['content'] += '\nUser model preferences for this turn: ' + json.dumps(preferences, ensure_ascii=False) context[-1]['content'] += '\nUse these models for new generation unless the user explicitly requests another. If unavailable or unsuitable, explain and ask before substituting. Keep existing nodes on their configured models unless the user requests a change.' @@ -359,6 +415,15 @@ async def produce() -> None: async for event in stream: if event['type'] == 'delta': text += event['text'] + inherited = ([source for source in referenced_sources(text, previous_sources) + if source['id'] not in sources] + if previous_sources and '\ue201' in event['text'] else []) + if inherited: + sources.update({source['id']: source for source in inherited}) + events.put({'type': 'sources', 'sources': inherited}) + if event['type'] == 'sources': + for source in event['sources']: + sources.setdefault(source['id'], source) if event['type'] == 'plan': plan = event['entries'] if event['type'] == 'tool': @@ -396,6 +461,7 @@ async def produce() -> None: 'id': str(uuid5(value.id, 'assistant')), 'role': 'assistant', 'text': text, 'status': status, 'model': model, 'runtime': value.runtime, 'thinking': value.thinking, 'tools': activities, + 'sources': list(sources.values()), **({'plan': plan} if plan is not None else {})}, conversation_id=value.conversationId) else: saved = None diff --git a/apps/api/src/opendirector_api/creation.py b/apps/api/src/opendirector_api/creation.py index 9996a1601..c1de28496 100644 --- a/apps/api/src/opendirector_api/creation.py +++ b/apps/api/src/opendirector_api/creation.py @@ -1,11 +1,16 @@ """Authoritative project endpoints for the canvas creation workflow.""" from uuid import UUID +from pathlib import Path +from base64 import b64encode +from hashlib import sha256 +from io import BytesIO import asyncio import json import re from urllib.parse import quote, unquote from fastapi import APIRouter, Header, Query, Request +from pydantic import BaseModel, Field from fastapi.responses import FileResponse, Response, StreamingResponse from filelock import Timeout from starlette.background import BackgroundTask @@ -14,21 +19,67 @@ from opendirector.application.creation_assets import CreationAssets from opendirector.application.creation_canvas import CreationCanvas from opendirector.application.creation_history import CreationHistory +from opendirector.application.creation_timeline_packages import CreationTimelinePackages +from opendirector.domain.creation.timeline_packages import TimelinePackageImportRequest, TimelinePackageImportReceipt from opendirector.domain.creation.assets import Asset, UploadInfo from opendirector.domain.creation.base import MediaKind from opendirector.domain.creation.commands import Command, CommandReceipt, Query as CreationQuery from opendirector.domain.creation.project import ProjectSnapshot, ViewState from opendirector.domain.projects import ProjectError +from opendirector.domain.timeline_errors import timeline_error, timeline_error_message from opendirector.infrastructure.creation_projects import CreationProjects +from opendirector.media.fonts import MAX_FONT_BYTES, inspect_font from opendirector_api.models import AssetSource, Session from opendirector_api.localization import localized_receipt +from opendirector_api.localization import localized_error_for_language + + +class FontCoverageResponse(BaseModel): + fingerprint: str = Field(pattern=r'^sha256:[0-9a-f]{64}$') + # One bit per Unicode scalar position: fixed 139,264 bytes / 185,688 base64 chars. + coverage: str = Field(min_length=185688, max_length=185688) + + +def preview_font_coverage(value: bytes) -> FontCoverageResponse: + inspection = inspect_font(BytesIO(value)) + mask = bytearray(0x110000 // 8) + for code in inspection.characters: + mask[code >> 3] |= 1 << (code & 7) + return FontCoverageResponse(fingerprint='sha256:' + sha256(value).hexdigest(), + coverage=b64encode(mask).decode('ascii')) + + +class TemporaryPackageResponse(FileResponse): + """Package lifetime ends on completion, disconnect, or cancelled response.""" + async def __call__(self, scope, receive, send): + try: + await super().__call__(scope, receive, send) + finally: + Path(self.path).unlink(missing_ok=True) + def creation_routes(projects: CreationProjects, generation=None): router = APIRouter(prefix='/api/creation', tags=['Unified creation projects']) assets = CreationAssets(projects) canvas = CreationCanvas(projects) history = CreationHistory(projects, assets=assets) + timeline_packages = CreationTimelinePackages(projects) + + @router.post('/fonts/inspect', response_model=FontCoverageResponse) + async def inspect_preview_font(request: Request, response: Response, + content_length: int | None = Header(default=None, ge=0)): + # Read-only byte inspection also works for unregistered local drafts. + # No filesystem paths, project changes, text, or session acquisition. + if content_length is not None and content_length > MAX_FONT_BYTES: + raise timeline_error('timeline_font_too_large', 413) + value = bytearray() + async for chunk in request.stream(): + if len(value) + len(chunk) > MAX_FONT_BYTES: + raise timeline_error('timeline_font_too_large', 413) + value.extend(chunk) + response.headers['Cache-Control'] = 'no-store' + return await run_in_threadpool(preview_font_coverage, bytes(value)) @router.get('/projects') def list_projects(): @@ -95,6 +146,30 @@ async def execute(project_id: UUID, value: Command, request: Request, x_opendire receipt = await run_in_threadpool(projects.execute, value, session=session, imported_from=imported_from) return localized_receipt(request, receipt) + def package_display_receipt(request, result): + if result.receipt is None: + return result + displayed = result.receipt.model_copy(update={'issues': tuple(issue.model_copy(update={ + 'message': localized_error_for_language(request.headers.get('accept-language'), chinese=issue.message, + status=409, code=issue.code), + }) for issue in result.receipt.issues)}) + return result.model_copy(update={'receipt': displayed}) + + @router.get('/projects/{project_id}/timelines/{editor_id}/package') + def export_timeline_package(project_id: UUID, editor_id: UUID, revision: int = Query(ge=1)): + path = timeline_packages.export(project_id, editor_id, revision) + return TemporaryPackageResponse(path, media_type='application/zip', filename=f'{editor_id}.timeline.zip', + headers={'Cache-Control': 'no-store', 'X-Content-Type-Options': 'nosniff'}) + + @router.post('/projects/{project_id}/timeline-package-imports', response_model=TimelinePackageImportReceipt) + def import_timeline_package(project_id: UUID, value: TimelinePackageImportRequest, + request: Request, x_opendirector_session: UUID = Header()): + return package_display_receipt(request, timeline_packages.import_package(project_id, value, session=str(x_opendirector_session))) + + @router.get('/projects/{project_id}/timeline-package-imports/{operation_id}', response_model=TimelinePackageImportReceipt) + def timeline_package_receipt(project_id: UUID, operation_id: UUID, request: Request): + return package_display_receipt(request, timeline_packages.receipt(project_id, operation_id)) + @router.get('/projects/{project_id}/canvas/{item_id}/status') async def item_status(project_id: UUID, item_id: UUID, wait_seconds: float = Query(default=0, ge=0, le=30)): return await canvas.wait_item_status(project_id, item_id, wait_seconds) @@ -241,6 +316,15 @@ def thumbnail(project_id: UUID, asset_id: UUID): 'Cache-Control': 'private, no-cache', 'ETag': '"thumbnail-' + asset.fingerprint + '"', 'X-Content-Type-Options': 'nosniff'}) + @router.get('/projects/{project_id}/assets/{asset_id}/browser-image') + def browser_image(project_id: UUID, asset_id: UUID, + if_none_match: str | None = Header(default=None, alias='If-None-Match')): + match = re.fullmatch(r'"browser-image-v2-(sha256:[0-9a-f]{64})"', if_none_match or '') + value, asset = assets.browser_image(project_id, asset_id, known_fingerprint=match[1] if match else None) + return Response(value, status_code=304 if value is None else 200, media_type='image/png', headers={ + 'Cache-Control': 'private, no-cache', 'ETag': '"browser-image-v2-' + asset.fingerprint + '"', + 'X-Content-Type-Options': 'nosniff'}) + @router.get('/projects/{project_id}/assets/{asset_id}/content') def content(project_id: UUID, asset_id: UUID, range_header: str | None = Header(default=None, alias='Range'), if_none_match: str | None = Header(default=None, alias='If-None-Match')): diff --git a/apps/api/src/opendirector_api/localization.py b/apps/api/src/opendirector_api/localization.py index 067db96dc..3e3fefa7b 100644 --- a/apps/api/src/opendirector_api/localization.py +++ b/apps/api/src/opendirector_api/localization.py @@ -6,15 +6,77 @@ """ from fastapi import Request + +from opendirector.domain.timeline_errors import timeline_error_message + from opendirector.domain.creation.commands import CommandReceipt +from opendirector.domain.plugins import PLUGIN_ERRORS ENGLISH_BY_CODE = { + **{code: messages[1] for code, messages in PLUGIN_ERRORS.items()}, + 'agent_skill_unavailable': 'A selected skill is unavailable for this harness. Remove it from the message and select an available skill.', + 'bigmodel_video_model': 'The BigModel video protocol supports CogVideoX-3 only.', + 'bigmodel_video_output': 'Choose a supported CogVideoX-3 pixel size and a 5- or 10-second duration. The size determines the aspect ratio.', + 'bigmodel_video_image': 'CogVideoX-3 frame images must be valid PNG or JPEG files, up to 5 MB each.', + 'bigmodel_video_frames': 'Use text, one first-frame image, or an explicit first/last pair. Each frame slot accepts one image; a last frame requires a first frame. Reference and continuation modes are not supported.', + 'bigmodel_video_prompt': 'CogVideoX-3 prompts allow up to 512 characters. Provide a prompt or frame image.', + 'bigmodel_video_cancel': 'Remote BigModel video cancellation is not connected. The task may continue running and incur charges.', + 'execution_file_link': 'Execution files must not have hard links.', + 'kling_model': 'This Kling protocol does not support the selected model.', + 'kling_mode': 'Kling currently supports first/last frames and image-reference generation.', + 'kling_image': 'Kling requires valid PNG or JPEG images up to 30 MB, at least 300 pixels on each edge, with an aspect ratio of 0.4–2.5. Video and audio references are not supported here.', + 'kling_frames': 'Choose frame inputs supported by the selected Kling model. Each slot accepts one image, and a last frame requires a first frame. Flash accepts only a first frame.', + 'kling_prompt': 'Enter a Kling prompt. English allows 26,000 characters and other text 11,000 characters; mixed text shares these limits proportionally.', + 'kling_aspect': 'Kling keyframe generation follows the image framing. Choose Auto for frames, or a supported ratio for text and image references.', + 'kling_output': 'Choose an output resolution and integer duration supported by the selected Kling model.', + 'kling_cancel': 'Remote Kling cancellation is not connected. The task may continue running and incur charges.', + 'tool_concept_translation': 'Use a short English subject description, or add a chat model in Models to translate it automatically.', + 'tool_no_video_match': 'The requested subject was not found in the video. Try a different description.', + 'voice_upload_type': 'Upload the voice recording using multipart/form-data.', + 'voice_upload_form': 'Check the recording, form fields and text lengths, then try the upload again.', + 'voice_model_unsupported': 'This checkpoint does not accept uploaded voices. Choose a Base model for voice cloning or a compatible built-in voice.', + 'voice_library_unsupported': 'This model does not have a confirmed voice library. Select a supported speech model.', + 'voice_connection_changed': 'The voice service or account changed. Reopen the library and select a voice again.', + 'voice_reference_required': 'This model needs a registered reference recording. Upload and select a voice; default cannot replace a recording.', + 'voice_transcript_required': 'This model needs an accurate recording transcript. Enter the transcript and create the voice again.', + 'voice_transcript_too_long': 'The recording transcript must be at most 100,000 characters.', + 'voice_description_too_long': 'The speaker description must be at most 4,000 characters.', + 'voice_not_found': 'The selected voice no longer exists. Refresh the library and select another voice.', + 'voice_delete_unsupported': 'This voice cannot be deleted. Only supported uploaded voices can be deleted.', + 'voice_consent_required': 'Enter the consent recording ID for this voice.', + 'voice_name_invalid': 'Enter a valid voice name without path separators or control characters.', + 'voice_audio_invalid': 'The recording could not be decoded. Check the actual audio format and file integrity.', + 'voice_format': 'This recording format is unsupported. Choose a listed format.', + 'voice_size': 'The recording exceeds the service file size limit.', + 'voice_duration': 'The recording duration is outside the service limits. Trim it and try again.', + 'voice_provider_protocol': 'The voice service returned an invalid response or pagination did not finish. Check its API version.', + 'voice_provider_error': 'The voice service request failed. Check the connection and account permissions.', + 'voice_provider_timeout': 'The voice service timed out. Check its status and try again.', + 'voice_name_taken': 'This account already has a voice with that name. Choose another name.', + 'voice_provider_quota': 'The voice service account has insufficient balance or resource packages. Top up and try again.', "moss_speech_text": "Enter the text to read aloud.", "moss_speech_voice": "Enter a Voice ID available to your MOSS account.", "moss_speech_reference": "MOSS speech generation does not accept references. Enter speech text and a Voice ID.", + "zhipu_speech_text": "Enter the text to read aloud, at most 1,024 characters per GLM-TTS request.", + "zhipu_speech_voice": "Select a voice available to your Zhipu account.", + "zhipu_speech_reference": "GLM-TTS does not accept references. Enter speech text and select a voice.", "moss_prompt_limit": "MOSS requires a nonempty prompt of at most 4,000 characters, including text references.", + "image_decode": "The image could not be decoded. Check its actual format and file integrity.", + "asset_preview_unsupported": "This asset type does not support this preview format.", + "asset_text_api_unavailable": "canvas text show requires a running API to read the complete Markdown source. Start the API and check OPENDIRECTOR_API_URL.", + "image_pixel_limit": "Images must contain no more than 25 million pixels.", + "minimax_image_reference": "MiniMax H3 references must be PNG, JPEG, WebP, HEIC, or HEIF, with edges of 128–8192 pixels and a long-to-short edge ratio of at most 4:1.", + "minimax_image_count": "MiniMax H3 image editing accepts up to 16 reference images.", + "minimax_image_bytes": "MiniMax H3 image reference requests cannot exceed 64 MiB.", + "minimax_speech_text": "Enter the text to read aloud, up to 3,000 characters.", + "minimax_speech_model": "Unsupported MiniMax speech model. Choose a model from the preset.", + "minimax_speech_voice": "Choose a voice available to your MiniMax account from the voice library.", + "minimax_speech_reference": "MiniMax speech synthesis does not accept references. Enter the text to read aloud and choose a voice.", "moss_image_size": "MOSS image dimensions must be 256–2048, multiples of 16, and no more than 2,097,152 pixels in total.", "moss_image_reference": "MOSS image generation does not accept image references or editing inputs.", + "bigmodel_image_reference": "GLM-Image accepts text-to-image requests only. Remove image references or editing inputs.", + "bigmodel_image_prompt": "Enter a GLM-Image prompt.", + "bigmodel_image_size": "GLM-Image requires a preset size, or dimensions of 1024–2048 in multiples of 32.", "moss_layout_reference": "MOSS layout JSON does not accept additional text references. Put the requirements in the JSON or remove the text references.", "agent_dependency_unavailable": "Bundled Agent is unavailable. Reinstall with pip install --force-reinstall opendirector; for a source checkout, use pip install -e .", "invalid_input": "The request is invalid. Check the required fields and their formats.", @@ -29,6 +91,7 @@ "capability_unsupported": "This capability is not supported yet.", "operation_rejected": "The operation was rejected. Reload the item and try again.", "storage_busy": "Project storage is busy. Try again shortly.", + "project_open_elsewhere": "This project is open in another window. Close it there and try again.", "local_only": "Only this computer can access the project service.", "cross_origin": "File access from another site is not allowed.", "invalid_local_request": "This is not a valid local app request.", @@ -70,6 +133,9 @@ def localized_error_for_language( status: int, code: str = "operation_rejected", ) -> str: + specific = timeline_error_message(code, language) + if specific is not None: + return specific if not wants_english_language(language): return chinese # A generic operation code must not hide a more specific HTTP status. @@ -77,6 +143,10 @@ def localized_error_for_language( if generic and status in {413, 502, 504}: return ENGLISH_BY_STATUS[status] specific = None if code == "operation_rejected" else ENGLISH_BY_CODE.get(code) + if code == 'voice_provider_error': + from opendirector.domain.voices import VOICE_MESSAGES + detail = chinese.removeprefix(VOICE_MESSAGES[code]).strip() + return specific + (' ' + detail if detail else '') return ( specific or ENGLISH_BY_STATUS.get(status) diff --git a/apps/api/src/opendirector_api/main.py b/apps/api/src/opendirector_api/main.py index d6dcac7f3..cd0fafbb6 100644 --- a/apps/api/src/opendirector_api/main.py +++ b/apps/api/src/opendirector_api/main.py @@ -26,6 +26,8 @@ from opendirector_api.creation import creation_routes from opendirector_api.agent import agent_routes from opendirector_api.library import library_routes +from opendirector_api.voices import voice_routes +from opendirector_api.plugins import plugin_routes from opendirector_api.localization import localized_error from opendirector_api.models import (ModelConfigInput, ModelConfigPublic, AgentModelTest, ModelCapabilityRequest, ModelDiscoveryRequest, ModelBatchInput, @@ -64,10 +66,12 @@ async def lifespan(app): finally: await creation_generation.close() - app = FastAPI(title="OpenDirector Local API", version="0.1.1", lifespan=lifespan) + app = FastAPI(title="OpenDirector Local API", version="0.1.4", lifespan=lifespan) app.include_router(creation_routes(creation_projects, creation_generation)) app.include_router(agent_routes(creation_projects, creation_generation, model_configs, chat_provider)) app.include_router(library_routes(creation_projects)) + app.include_router(voice_routes(creation_generation.voices)) + app.include_router(plugin_routes(store.repository)) @app.exception_handler(RequestValidationError) async def invalid_request(request, error): @@ -307,6 +311,11 @@ def placement(project_id: UUID, task_id: UUID, value: ToolPlacement, x_opendirec dist = web_root() if dist.is_dir(): app.mount("/assets", StaticFiles(directory=dist / "assets", check_dir=False), name="assets") + # Installed builds retain the same public font URLs as the dev server. + # StaticFiles confines reads to this directory and rejects escaping symlinks. + font_root = dist / "fonts" + if font_root.is_dir() and not font_root.is_symlink() and font_root.resolve().is_relative_to(dist.resolve()): + app.mount("/fonts", StaticFiles(directory=font_root), name="fonts") @app.get("/") def index(): diff --git a/apps/api/src/opendirector_api/models.py b/apps/api/src/opendirector_api/models.py index f6e700554..57e69f78f 100644 --- a/apps/api/src/opendirector_api/models.py +++ b/apps/api/src/opendirector_api/models.py @@ -24,7 +24,7 @@ class AssetSource(Model): class ModelConnection(Model): connectionType: Literal['api', 'self_hosted'] = 'api' - protocol: Literal['openai', 'volcengine', 'minimax_video_v2', 'vllm_omni', 'vllm_omni_segmentation', 'vllm_omni_music', 'vllm_omni_depth', 'vllm_omni_score'] | None = None + protocol: Literal['openai', 'volcengine', 'bigmodel_video', 'kling_video', 'minimax_video_v2', 'vllm_omni', 'vllm_omni_segmentation', 'vllm_omni_music', 'vllm_omni_depth', 'vllm_omni_score'] | None = None authMode: Literal['none', 'bearer'] | None = None proxyMode: Literal['system', 'direct'] | None = None diff --git a/apps/api/src/opendirector_api/plugins.py b/apps/api/src/opendirector_api/plugins.py new file mode 100644 index 000000000..0e781c921 --- /dev/null +++ b/apps/api/src/opendirector_api/plugins.py @@ -0,0 +1,56 @@ +"""Local plugin management; native harnesses retain execution permissions.""" +import asyncio +from typing import Annotated + +from fastapi import APIRouter, Path, Request +from pydantic import BaseModel, ConfigDict, Field + +from opendirector.agent.context import RuntimeId +from opendirector.agent.plugins import MAX_ARCHIVE_BYTES, PluginRegistry, failure +from opendirector.domain.plugins import Digest, PluginId + + +class PluginSource(BaseModel): + model_config = ConfigDict(extra='forbid', strict=True) + source: str = Field(min_length=1, max_length=4096) + + +class PluginChange(BaseModel): + model_config = ConfigDict(extra='forbid', strict=True) + digest: Digest + + +class PluginEnabled(PluginChange): + enabled: bool + + +def plugin_routes(repository): + router = APIRouter(prefix='/api/plugins') + registry = PluginRegistry(repository) + + @router.get('') + def catalog(): + return registry.catalog() + + @router.post('') + def install(value: PluginSource): + return registry.install(value.source) + + @router.post('/upload') + async def upload(request: Request): + content = bytearray() + async for chunk in request.stream(): + content.extend(chunk) + if len(content) > MAX_ARCHIVE_BYTES: + raise failure('plugin_too_large', 413) + return await asyncio.to_thread(registry.install_archive, bytes(content)) + + @router.put('/{plugin_id}/runtimes/{runtime}') + def enable(plugin_id: Annotated[PluginId, Path()], runtime: RuntimeId, value: PluginEnabled): + return registry.change(plugin_id, value.digest, runtime=runtime, enabled=value.enabled) + + @router.delete('/{plugin_id}') + def remove(plugin_id: Annotated[PluginId, Path()], value: PluginChange): + return registry.change(plugin_id, value.digest) + + return router diff --git a/apps/api/src/opendirector_api/voices.py b/apps/api/src/opendirector_api/voices.py new file mode 100644 index 000000000..57051b9ad --- /dev/null +++ b/apps/api/src/opendirector_api/voices.py @@ -0,0 +1,97 @@ +"""HTTP adapter for the shared provider voice registry.""" +from uuid import UUID + +from fastapi import APIRouter, Query, Request, Response +from starlette.concurrency import run_in_threadpool +from starlette.datastructures import UploadFile +from starlette.formparsers import MultiPartException, MultiPartParser + +from opendirector.domain.projects import ProjectError +from opendirector.domain.voices import VOICE_CREATE_TEXT_LIMITS, voice_error + + +_FORM_LIMITS = {'name': 200, 'connectionId': 200, 'consent': 4000} +_TEXT_LIMITS = {**_FORM_LIMITS, **{field: limit for field, (limit, _) in VOICE_CREATE_TEXT_LIMITS.items()}} + + +class _VoiceUploadParser(MultiPartParser): + def on_part_data(self, data, start, end): + # Four bytes per character allows every valid UTF-8 value. Report the + # field even when an oversized part is rejected before it is decoded. + part = self._current_part + if part.file is None and part.field_name in VOICE_CREATE_TEXT_LIMITS: + limit, code = VOICE_CREATE_TEXT_LIMITS[part.field_name] + if len(part.data) + end - start > 4 * limit: + raise voice_error(code) + super().on_part_data(data, start, end) + + +async def _voice_upload(request, max_bytes): + if request.headers.get('content-type', '').partition(';')[0].strip().lower() != 'multipart/form-data': + raise ProjectError('请通过 multipart/form-data 表单上传音色录音。', 415, code='voice_upload_type') + # Allow four UTF-8 bytes per text character, plus bounded multipart headers. + body_limit = max_bytes + 4 * sum(_TEXT_LIMITS.values()) + 16384 + + async def bounded_stream(): + total = 0 + async for chunk in request.stream(): + total += len(chunk) + if total > body_limit: + raise voice_error('voice_size', 413) + yield chunk + + parser = _VoiceUploadParser(request.headers, bounded_stream(), max_files=1, max_fields=len(_TEXT_LIMITS), + max_part_size=4 * max(_TEXT_LIMITS.values())) + try: + form = await parser.parse() + except MultiPartException: + raise ProjectError('无法解析音色上传表单,请检查录音和文本字段。', 422, code='voice_upload_form') from None + try: + audio = form.get('audio_sample') + if not isinstance(audio, UploadFile) or not audio.filename or len(audio.filename) > 255: + raise voice_error('voice_audio_invalid') + if audio.size is None or audio.size > max_bytes: + raise voice_error('voice_size', 413) + fields = {} + for field, value in form.multi_items(): + if field == 'audio_sample' and value is audio: + continue + if (field not in _TEXT_LIMITS or field in fields or not isinstance(value, str) + or (field in _FORM_LIMITS and len(value) > _FORM_LIMITS[field])): + raise ProjectError('音色上传表单的字段或长度无效,请检查后重试。', 422, code='voice_upload_form') + fields[field] = value + return audio.filename, await audio.read(), fields + finally: + await form.close() + + +def voice_routes(voices): + router = APIRouter(prefix='/api/voices') + + @router.get('') + async def list_voices(modelConfigId: UUID, pageSize: int | None = Query(default=None, ge=1, le=150), + after: str | None = Query(default=None, min_length=1, max_length=2048), + connectionId: str | None = Query(default=None, min_length=1, max_length=200)): + return await run_in_threadpool(voices.list, str(modelConfigId), page_size=pageSize, after=after, + connection_id=connectionId) + + @router.post('') + async def create_voice(request: Request, modelConfigId: UUID): + _, capabilities = await run_in_threadpool(voices._config, str(modelConfigId)) + filename, data, fields = await _voice_upload(request, capabilities['maxBytes']) + return await run_in_threadpool(voices.create, str(modelConfigId), data, filename=filename, + name=fields.get('name', ''), consent=fields.get('consent'), + connection_id=fields.get('connectionId'), ref_text=fields.get('refText'), + speaker_description=fields.get('speakerDescription')) + + @router.delete('/{voice_id}', status_code=204) + async def delete_voice(voice_id: str, modelConfigId: UUID, connectionId: str = Query(max_length=200)): + await run_in_threadpool(voices.delete, str(modelConfigId), voice_id, connectionId) + return Response(status_code=204) + + @router.get('/{voice_id}/preview') + async def preview_voice(voice_id: str, modelConfigId: UUID, connectionId: str = Query(max_length=200)): + data, mime = await run_in_threadpool(voices.preview, str(modelConfigId), voice_id, connectionId) + return Response(content=data, media_type=mime, headers={'Cache-Control': 'no-store'}) + + return router diff --git a/build_backend.py b/build_backend.py index 3120b042a..9afb737b8 100644 --- a/build_backend.py +++ b/build_backend.py @@ -23,6 +23,7 @@ def prepare_resources(): shutil.rmtree(bundled) shutil.copytree(ROOT / 'dist', bundled / 'web') shutil.copytree(ROOT / 'packages/agent/skills', bundled / 'skills') + shutil.copy2(ROOT / 'plugins/catalog.json', bundled / 'plugin-catalog.json') npm_package = ROOT / 'node_modules/opencode-ai' metadata = json.loads((npm_package / 'package.json').read_text()) executable = npm_package / metadata['bin']['opencode'] diff --git a/docs/README.md b/docs/README.md index 81d66777d..ef4d2a6b0 100644 --- a/docs/README.md +++ b/docs/README.md @@ -10,17 +10,27 @@ This directory is the entry point for the repository's shared documentation. The | [Contributing](../CONTRIBUTING.md) | Label taxonomy, how to pick labels, and what a PR must satisfy | Process, labels, or submission requirements change | | [Unified project architecture](architecture/unified-project.md) | Business objects, shared entry points, and interaction rules | Object responsibilities, commands, persistence, or interaction rules change | | [Test contracts](architecture/test-contracts.md) | Test/production boundaries, how to avoid stale assertions and environment-dependent red runs | Test infrastructure, capability gates, or contract ownership change | +| [Capability mapping contract](architecture/capability-contract.md) | Evidenced upstream vocabulary, the `(operation, profile) → capability` table, the untranslatable declarations, and how it feeds discovery | Upstream vocabulary, capability ids, or discovery matching change | | [Roadmap](development/roadmap.md) | Capability dependencies and delivery criteria | Scope, dependencies, or acceptance criteria change | | [Validation guide](development/validation.md) | Reproducible checks and evidence requirements | Tools, runtime requirements, or validation entry points change | +| [CI layer assessment](development/ci-layer-assessment.md) | Timing evidence, critical PR representatives and staged Python classification | Measured evidence or classification decisions change | +| [Issue #87 Windows handoff](development/issue-87-windows-handoff.md) ([中文](development/issue-87-windows-handoff.zh-CN.md)) | Native ACL repair and final text acceptance | The shared checkpoint or native validation scope changes | | [Development lessons](development/lessons.md) | Risks identified from recent fixes | New evidence emerges or a lesson no longer applies | | [CI failure patterns](development/ci-failures.md) | Known timing-sensitive CI failures and how to tell flakes from regressions | A pattern is fixed, newly observed, or the watchlist changes | +| [Release tags and daily cuts](development/release.md) | Public SemVer tags and the daily patch cut | The tag scheme, daily cut gate, or version-field set changes | | [UI performance design](development/ui-performance.md) | Interaction design principles, latency budgets, and measurement | A UI performance principle, target, or measurement method changes | | [macOS installation validation](development/macos-install-validation.md) | Installation and usage scenarios verified on Apple Silicon | The corresponding acceptance checks are repeated | | [Bilingual interface](development/i18n.md) | Language selection, message organization, and checks | Language policy, message structure, or checks change | | [Python installation and packaging](development/packaging.md) | Installation, platform wheel builds, and isolated verification | Packaging, resource paths, or build process changes | | [Business CLI](guides/cli.md) | Querying, editing, and generating through a terminal or Agent Shell | Commands, authorization, or receipt semantics change | +| [DeepSeek Harness plugin](../plugins/deepseek-harness/README.md) | Installing the DSH bundle and editing an OpenDirector project from DSH | Plugin setup, tools, or creation workflows change | +| [Optional creative plugins](guides/plugins.md) / [可选创作插件](guides/plugins.zh-CN.md) | Installing, enabling and authoring portable Skill packages | Package format, activation or examples change | | [Video model capabilities](guides/video-model-capabilities.md) | Declaring output specifications supported by a service | Configuration fields or capability integration changes | | [OpenAI Images](guides/openai-images.md) | Wiring gpt-image as an optional image service, request shape and failures | Integration fields, capability scope, or the OpenAI interface change | +| [Kling video](guides/kling-video.md) ([中文](guides/kling-video.zh-CN.md)) | Optional Kling 4.0 and Flash setup, model-specific inputs, and recovery limits | Kling inputs, model declarations, or recovery behavior change | +| [BigModel video](guides/bigmodel-video.md) ([中文](guides/bigmodel-video.zh-CN.md)) | CogVideoX-3 text, single-image and first/last-frame generation, task recovery and adoption | BigModel protocol, inputs, output sizes or recovery behavior change | +| [BigModel GLM-Image](guides/bigmodel-image.md) ([中文](guides/bigmodel-image.zh-CN.md)) | Account setup, text-to-image generation, adoption and recovery | BigModel integration fields or behavior change | +| [GLM-Image validation](development/glm-image-validation.md) | Real provider workflow screenshots and automated checks | GLM-Image acceptance is repeated | | [Agent runtime](architecture/agent-runtime.md) | Agent runtime architecture | Runtime architecture changes | The root [README](../README.md) introduces the product and examples and links to quick-start and documentation. See [installation and updates](guides/installation.md) ([中文](guides/installation.zh-CN.md)) for installation, and the [user guide](guides/user-guide.md) ([中文](guides/user-guide.zh-CN.md)) for configuration, creation, and troubleshooting. The README does not contain the full manual, implementation architecture, development tests, research sources, or iteration history. Source code and generated contracts describe the implementation; architecture and roadmap documents describe goals and do not establish that a capability is already available. diff --git a/docs/architecture/agent-runtime.md b/docs/architecture/agent-runtime.md index 9ff943b39..3f86072af 100644 --- a/docs/architecture/agent-runtime.md +++ b/docs/architecture/agent-runtime.md @@ -9,11 +9,11 @@ Harness 指承担模型调用、工具执行和上下文管理的 Agent 执行 | Runtime | 执行程序 | 模型与认证来源 | | --- | --- | --- | | Default | 固定版本的 bundled OpenCode | OpenDirector 模型配置,经本地模型代理访问服务 | -| OpenCode CLI | PATH 中的 opencode | CLI 原生配置与模型目录 | -| Codex | PATH 中的 codex | 原生认证与模型目录;隔离启动前解析所选模型或默认模型 | -| Claude Code | PATH 中的 claude | 原生认证、环境变量与初始化模型目录 | -| JiuwenSwarm | PATH 中的 jiuwenswarm-process,通过 stdin/stdout JSONL 直接调用 | 本机 Jiuwen 模型目录、配置与环境变量 | -| DeepSeek Harness | PATH 中的 dsh,以 acp profile 运行标准 JSON-RPC | CLI 原生 profile、模型目录与认证 | +| OpenCode CLI | opencode,优先 PATH,再查找常见 CLI 安装目录 | CLI 原生配置与模型目录 | +| Codex | 优先 PATH 中的 codex,再查找常见桌面版与 CLI 安装目录 | 原生认证与模型目录;隔离启动前解析所选模型或默认模型 | +| Claude Code | claude,优先 PATH,再查找常见 CLI 安装目录 | 原生认证、环境变量与初始化模型目录 | +| JiuwenSwarm | jiuwenswarm-process,优先 PATH,再查找常见 CLI 安装目录,通过 stdin/stdout JSONL 直接调用 | 本机 Jiuwen 模型目录、配置与环境变量 | +| DeepSeek Harness | dsh,优先 PATH,再查找常见 CLI 安装目录,以 acp profile 运行标准 JSON-RPC | CLI 原生 profile、模型目录与认证 | Default 与 OpenCode CLI 分别配置。外部 CLI 不读取 OpenDirector 保存的模型凭证。进程继承服务端环境,包括 PATH 和第三方 API 配置。 @@ -49,6 +49,8 @@ Default 将模型配置中的输入模态传入 bundled OpenCode。外部 harnes ## 运行生命周期 +Search citations keep their native markers in the original reply and trace. Completed Codex MCP web-search results supply validated source IDs, HTTP(S) URLs and titles through `sources` events; the conversation retains this metadata with the reply. The UI splits tool activities using raw text offsets before formatting prose citations as source links, and uses the same formatting for copying. Code examples remain literal. Missing sources and incomplete final markers have a localized fallback. For saved Codex replies without source metadata, queries derive it from the matching turn's project-local trace without rewriting the conversation; a bounded metadata cache is invalidated by trace modification time and size. Reading a trace never requests a model call. + 同一工程同时只接受一个回复运行。运行事件转换为回复正文和工具活动,保存到对应会话;原生事件与业务工具事件追加到 `agent-runs/..jsonl`。Default 使用 `opencode` 作为轨迹文件的 runtime 名称。 Default 的模型网关在 `agent-runs/.model.jsonl` 记录实际转发的请求体,包括生效的系统消息、会话上下文、工具 Schema 与模型参数。请求 ID 关联上游 HTTP 状态、响应头等待耗时和传输错误。该文件仅供本地调试,创建权限为 `0600`,不记录 HTTP 鉴权头。回复片段、工具结果和 token 用量仍见原生轨迹。 diff --git a/docs/architecture/capability-contract.md b/docs/architecture/capability-contract.md new file mode 100644 index 000000000..4ed72d6a3 --- /dev/null +++ b/docs/architecture/capability-contract.md @@ -0,0 +1,146 @@ +# Capability mapping contract + + +`GET /v1/omni/capabilities` reports what a deployment says about itself; the product needs to know what that deployment can do. `packages/core/src/opendirector/domain/capability_contract.py` is the only place that translates upstream's vocabulary into `opendirector.domain.capabilities` ids. It performs no HTTP, holds no display text, and introduces no second set of operation constants. + +## The upstream vocabulary (as evidenced) + +Source: `vllm-project/vllm-omni` PR #10102 (issue #10007). Schema and protocol citations are from `d1104f8`; the `entrypoints/openai/serving_capabilities.py` citations (`:339-343`, `:383-387`, `:400`) and the `vllm_omni/serving_capabilities.py` block citations (`:15,19-26`, `:43-54`) are from `43ff7b9`, where both files carry those anchors at exactly those lines (405 and 176 lines respectively). Each anchor names its revision where the two differ. + +**Six closed operations** (`vllm_omni/serving_capabilities.py:15,19-26`): `chat`, `image_generation`, `image_edit`, `video_generation`, `speech`, `audio_generation`. There is no `asr`, `transcription`, `segmentation`, `depth` or `music`. + +**Twelve profiles** (`:43-54`) and the ids this repository has for them. Profiles with a different output modality are not merged: `text.chat` here is consumed by a text-only adapter (`integrations/model_adapters.py:631-647` reads `choices[0]["message"]["content"]` as a string), so an image-only chat deployment must not read as able to chat. + +| operation | profile | upstream `required_inputs` → `outputs` | capability here | +| --- | --- | --- | --- | +| `chat` | `text_response` | `("text",)` → `("text",)` | `text.chat` | +| `chat` | `image_response` | `("text",)` → `("image",)` | none — the output is an image, not this repo's `text.chat` | +| `chat` | `audio_response` | `("text",)` → `("audio",)` | none — the output is audio, not this repo's `text.chat` | +| `image_generation` | `text_to_image` | `("text",)` → `("image",)` | `image.generate` | +| `image_edit` | `image_to_image` | `("text","image")` → `("image",)` | `image.edit` | +| `video_generation` | `text_to_video` | `("text",)` → `("video",)` | `video.generate` | +| `video_generation` | `image_to_video` | `("text","image")` → `("video",)` | `video.first_frame` | +| `video_generation` | `video_to_video` | `("video",)` → `("video",)` | `video.edit` | +| `speech` | `text_to_speech` | `("text",)` → `("audio",)` | `audio.speech` | +| `speech` | `lyrics_to_music` | `("text","instructions")` → `("audio",)` | `audio.music` | +| `speech` | `caption_to_audio` | `("text",)` → `("audio",)` | none — sound effect / ambience | +| `audio_generation` | `text_to_audio` | `("text",)` → `("audio",)` | none — the diffusion `/v1/audio/generate` contract, not this repo's `audio.music` | + +The three `video_generation` rows are **one-to-one**, and the two narrow ones are **role-gated**. The adapter distinguishes the shapes (`VllmOmniVideo.video` / `.edit_video`, `integrations/model_adapters.py:405-540`): `text_to_video` carries no reference and sends `task=t2va`, `image_to_video` carries `input_reference` with `task=fl2va`, `video_to_video` carries `source_video` plus the picture and audio masks. Folding all three onto `video.generate` let an image-to-video-only deployment read as able to do text-to-video, which is a false `available`. + +One-to-one alone is still too generous, because the document's granularity stops at modality. A profile saying `image_to_video` proves it accepts an image, not that the image is a first frame: VACE feeds that image to `reference_images` (R2V; `diffusion/models/wan2_2/pipeline_wan2_2_vace.py:135`, forwarded at `:158`). A profile saying `video_to_video` proves it accepts a video, not that it takes latent masks: SeedVR2 declares V2V for video restoration and leaves `supports_latent_mask_editing` at its default `False` (`diffusion/model_metadata.py:53,175`). Both would read as `available` under a plain one-to-one table, and both would fail the request the tool actually sends. + +So `video.first_frame` and `video.edit` carry `REQUIRED_ROLES`, and a `yes` verdict survives only when **one** of the profile's own open postable routes declares **every** one of those roles with `support=supported` in `request.inputs` (a closed route, or a path this operation does not own, counts for nothing). Otherwise the id is `unknown`, or `missing` once every open postable route refuses at least one of those roles. `support` alone is not the whole answer: the count window published beside it (`min_count` / `max_count`, both optional inclusive counts) is part of it. The tool sends one item, so `support=supported` with `max_count: 0` is a refusal in effect — a role capped at zero cannot carry it — and `min_count: 2` is one too, because that one item cannot satisfy the minimum. An absent bound is no bound on that side; a bound that is not a whole number leaves the entry undecided rather than guessed. One route, not the union of them: a request leaves on a single route, so `source_video` declared on `/v1/videos` beside the two noise masks on `/v1/videos/sync` is not an edit request either route would accept — that stays `unknown` with `role_not_declared`. Upstream publishes those four roles as of `b934b642` (head `12d834c2`): Wan I2V declares `first_frame` supported with `min_count: 0` / `max_count: 1` beside a required `image` — the same supplied image, so the two counts are not additive — and both VACE architectures declare `first_frame` `unsupported` with `max_count: 0` on every profile they publish; the editing roles (`source_video`, `video_noise_mask`, `audio_noise_mask`) are declared supported with `min_count: 0` / `max_count: 1` where the handler accepts latent-mask editing, and `unsupported` with `max_count: 0` where it does not. So `video.first_frame` reads `available` on Wan I2V and `missing` with `unsupported` on VACE, and `video.edit` reads `missing` with `unsupported` where those roles are refused — with no code change, because the gate reads the declaration rather than a written-in role list. A body that publishes modality roles only still leaves both ids `unknown`, and the report says why with `role_not_declared`. When at least one postable route is open and every open postable route refuses at least one required role, the document has said no — one role refused everywhere, or two routes each refusing a different one, even when neither route mentions the other roles — and the id is `missing` with `reasons[cid] = "unsupported"`, exactly like a refusing profile: "the deployment will not take that input" is a different fact from "the document does not say", and it needs a different fix. A declared route this table cannot place — `/v1/videos/sync`, the socket, an unrecognised path — is reported and nothing more: whatever its own `exposure` says, it cannot keep the answer open. Only one of the profile's own postable routes with an absent, unreadable or `unknown` exposure is a channel this tool can still use; unless another reference to that same path is declared `enabled`, it keeps the answer `unknown`. + +Four rows carry no id here (`image_response`, `audio_response`, `caption_to_audio`, `text_to_audio`), and they read together: `image_response` and `audio_response` are published upstream but have no id here, so they stay `unknown` and, while their route is open, are reported as `no_local_capability` instead of being folded into `text.chat` — the same scope as the `no_local_capability` row below, so a closed or undecided route on such a profile says nothing. `text_to_audio` uses the separate diffusion request contract behind `/v1/audio/generate` (no voice, no lyrics, no instructions; `docs/serving/capabilities.md:59-60`), while the *only* `audio.music` adapter in this repository (`VllmOmniMusic`, `integrations/model_adapters.py:573-604`) posts `/v1/audio/speech` with lyrics **and** `instructions`, both required. Upstream says that profile has neither and this repository demands both, so mapping it to `audio.music` would promise a request the deployment never accepted: the profile is reported as `no_local_capability` and `audio.music` is decided by `lyrics_to_music` alone (no evidence → `unknown`). That is the same rule that keeps `caption_to_audio` off `audio.speech`. + +**Operation → routes** (`:42-54`) as `(transport, method, path)`, because upstream's `RouteKey` is exactly that and the websocket method literal is `WEBSOCKET`: `chat` → `http POST /v1/chat/completions`; `image_generation` → `http POST /v1/images/generations`; `image_edit` → `http POST /v1/images/edits`; `video_generation` → `http POST /v1/videos`, `http POST /v1/videos/sync`, `websocket WEBSOCKET /v1/realtime/video`; `speech` → `http POST /v1/audio/speech`; `audio_generation` → `http POST /v1/audio/generate`. `DELETE /v1/chat/completions` would be a 405 and a `websocket` declaration on an http route is wrong, so both are `route_mismatch`. `transport` defaults to `http` upstream (`protocol/capabilities.py:16-21`) and an absent transport is not a mismatch; `method` has no default and an absent one cannot be confirmed. + +Only the route this adapter actually posts counts as evidence: `POSTABLE_ROUTES` names it per operation — `audio_generation` keeps its placement row even though no adapter here posts `/v1/audio/generate`, so its declaration is still placed rather than discarded — an open `/v1/audio/generate` reports `no_local_capability`, while a closed or undecided one says nothing, like any other profile — and the synchronous variant, the websocket route and another operation's paths are reported (`route_mismatch` / `unknown_route`) without moving a verdict — this tool posts `http POST /v1/videos`, so a role named only beside `/v1/videos/sync` or on the socket is a role no request of ours can carry. The same reading applies to the key's three fields: an absent `transport` takes upstream's default, while a present but unreadable `transport`, `method` or `path` is `malformed_route` — a shape no published body can carry — and is never evidence. + +Three schema facts shape the table: the body has only `object`, `schema_version`, `observed_at`, `models`, `stages`, `routes` (`protocol/capabilities.py:73-79`) and is served with `response_model_exclude_none=True` (`api_server.py:2196`) — the top-level `routes[]` is **what upstream aggregates an operation's `exposure` from** (`entrypoints/openai/serving_capabilities.py:383-387`: any `enabled` → `enabled`, else any `unknown` → `unknown`, else `disabled`), and each profile's `routes[]` is a copy of that list, forced `disabled` only for profiles needing video/audio input on `/v1/realtime/video` (`:339-343`). A top-level `disabled` therefore means the operation's HTTP endpoint is closed and **no profile-level `enabled` reopens it**; closing something narrower would semantically have to live on the profile route, but this projection (`43ff7b9`) never rewrites a profile route except for `/v1/realtime/video`, so "top level `enabled` with that profile's route `disabled`" is **not producible today** — it is a forward-compatible consumer branch, not an available control. This module reads the operation's `exposure` — upstream has already aggregated it — instead of re-reading the top-level list, which is why publishing an unowned path there changes no verdict, and **not** because those routes are "display only"; **`exclude_none` drops only `None`-valued optional fields** — in practice the profile's `request` — while `default_factory` lists serialize as `[]` (`profiles: []`, never a missing key), `support`/`reason_code` are required on every operation and profile, `exposure` is required on every operation and top-level route (`protocol/capabilities.py:24-52`), and a profile has **no `exposure` field of its own** (`:36-44`): its reachability is the aggregation of the exposures of the routes it references; and `models[].operations` **must list all six** or pydantic raises (`:60-64`), with `stage_ids` numeric strings, the *same* full list for every model (`entrypoints/openai/serving_capabilities.py:400`). + +## The key is the pair (operation, profile) + +Not the route: `/v1/audio/speech` is one path with three contracts — `text_to_speech` (voice), `lyrics_to_music` (music), `caption_to_audio` (sound effect). Upstream's own tests show MOSS-SoundEffect serving `caption_to_audio` while reporting `text_to_speech` unsupported on that same path (`tests/config/test_serving_capabilities.py:233,251`): one path, two answers. A path-only key makes a sound-effect deployment look able to speak, and the reverse. + +Not the stage either: `stages[]` publishes only `id` / `type` / `depends_on`, the ids are opaque numeric strings, and every model carries the same list. Keying on stages gives all models one shared state and no diagnosis, so `stages[]` only becomes `topology` here and never decides a capability; a stage entry carrying `routes` or `name` is ignored, because upstream's `CapabilityStage` has no such fields. + +Routes still matter, in two ways: `OPERATION_ROUTES` records which operation owns which `(transport, method, path)` and a profile referencing another operation's path — or the wrong method — grants nothing and is reported as `route_mismatch`; and the `exposure` on the profile's own **postable** route is upstream's third exposure layer, overridden per profile (a profile needing video/audio input is forced `disabled` on `/v1/realtime/video`, `serving_capabilities.py:341-343` — a route this tool never posts to, so that override is a diagnostic here, not a verdict). + +## Untranslatable declarations + +`unknown_route` (the path belongs to no operation, including registered-but-unowned `/v1/images/segmentations`), `route_mismatch` (another operation's path, or a wrong method/transport), `unmapped_stage` (no such operation upstream — `asr`, `music`; the name predates the pair-keyed table), `unknown_profile` (operation known, profile id unpublished), `operation_mismatch` (profile known under another operation), `no_local_capability` (profile supported, no id here), `no_route_declared` (a supported profile that declares no route this tool can post to — `routes: []`, every entry without a usable `path`, or only routes this tool never posts to), `malformed_route` (a route entry this table cannot read: no usable `transport`/`method`/`path`, or a `routes` value that is not a list), `role_not_declared` (profile supported, but no single open postable route declares the whole input-role set the id needs — a role is absent, split across routes, or refused on one route while another stays silent), `exposure_unknown` (a postable route whose `exposure` is absent, unreadable or the literal `unknown`, reported when that doubt is why an id stays `unknown`), `malformed_profile` (a `profiles[]` element whose id cannot be read, or a `profiles` value that is not a list), `malformed_operation` (an `operations[]` value that is not an object), `unsupported_format` (the body's `object` or `schema_version` names a contract this table does not read, so its `models` are never read), `malformed_model` (a `models[]` element that is not an object, carries no `operations` object, or carries an empty one — upstream lists all six operations on every model). Each one lands in `unresolved` with `operation` / `profile` / `route` / `reason`, never dropped; `unsupported_format` and `malformed_model` carry no operation, profile or route, because they are about the body or the model, not a row in it, while `malformed_profile` and `malformed_operation` name the operation they were declared in. + +## Three capability states, and `missing` needs positive evidence + +`available` when a model's profile says `supported`, its operation is `supported`+`enabled`, and at least one of the profile's own postable routes is `enabled` (a role-gated id additionally needs one postable route carrying its whole `REQUIRED_ROLES` set as `supported`); `missing` only when a profile or operation says no, or when at least one postable route is open, every open postable route refuses at least one required role (or carries it only past a zero count bound), and no postable declaration is itself undecided; and only when every model says so — one model refusing while another does not mention the operation (or lists no operation at all) is `unknown`, because that silent model may serve, the same way one model granting it is enough for `available`; `unknown` for everything else — failed request, absent field, `unknown` enum value, unconfirmed reachability, an untranslatable declaration, or a capability no profile can express. `resolve()` also returns `reasons`, which names the kind of refusal for every `missing` capability: `unsupported` (the model refused), `endpoint_disabled` (the deployment closed the operation's routes) or `route_disabled` (every postable route the profile declares is closed). Reasons for `unknown` live in `unresolved` (declaration level) or in `UNROUTED` (capability level). + +## Status and reason mapping (the contract) + +Every row below is read with the profile saying `supported` and its operation saying `supported` + `enabled`. A profile whose own `support` is `unknown` under an enabled operation is `unknown` with no note, and an operation whose `exposure` is `unknown` is `unknown` the same way; an operation that says `unsupported`, or is `disabled`, still decides the row on its own, because a refusal needs no profile evidence. + +| Upstream declaration | status | where the reason lives | +| --- | --- | --- | +| profile `support=unsupported` | `missing` | `reasons[cid] = "unsupported"` | +| operation `support=unsupported` | `missing` | `reasons[cid] = "unsupported"` | +| operation `supported` + `exposure=disabled` | `missing` | `reasons[cid] = "endpoint_disabled"` | +| every route the profile can post to is `exposure=disabled` | `missing` | `reasons[cid] = "route_disabled"` | +| every postable route the profile declares is `disabled`, beside a route this table cannot place (`/v1/videos/sync`, the socket, an unrecognised path) whatever its own `exposure` says | `missing` | `reasons[cid] = "route_disabled"`; the unplaceable route is `unresolved: route_mismatch` / `unknown_route`, a diagnostic only | +| every postable route the profile declares is `exposure=unknown` | `unknown` | `unresolved: exposure_unknown`, one per distinct `(profile, route)` — reachability unconfirmed, not a refusal | +| profile declares no postable route (`routes: []`, no usable `path`, or only routes this tool never posts to) | `unknown` | `unresolved: no_route_declared` | +| a route entry with no usable `transport`/`method`/`path`, or a `routes` value that is not a list, with no usable postable route beside it | `unknown` | `unresolved: malformed_route` — the path is named when it can be read; `no_route_declared` too when that leaves no postable route | +| an unrecognised path, and nothing else usable | `unknown` | `unresolved: unknown_route`, and `no_route_declared` because no postable route is declared | +| another operation's path, or a wrong method/transport, with no usable postable route beside it | `unknown` | `unresolved: route_mismatch`, and `no_route_declared` when that leaves no usable postable route | +| upstream profile, no id here, declared on an open postable route | `unknown` | `unresolved: no_local_capability` — a closed or undecided route on that profile says nothing about it | +| profile supported, at least one postable route is open, but none declares the id's whole `REQUIRED_ROLES` set as `supported` — a role is absent, split across routes, or refused on one route while another is silent | `unknown` | `unresolved: role_not_declared` | +| a postable route whose `exposure` is absent, unreadable or the literal `unknown`, when that doubt is why the id stays `unknown` | `unknown` | `unresolved: exposure_unknown`, one per distinct `(profile, route)` | +| a `profiles[]` element whose id cannot be read, or a `profiles` value that is not a list | no evidence from that profile | `unresolved: malformed_profile`, once per operation | +| an `operations[]` value that is not an object | no evidence from that operation | `unresolved: malformed_operation`, once per operation | +| profile supported, at least one postable route is open and every open postable route refuses at least one required role — one role refused everywhere, or two routes each refusing a different one, whether or not they mention the other roles (`REQUIRED_ROLES`) — with no postable declaration left undecided | `missing` | `reasons[cid] = "unsupported"` | +| profile supported, a required role declared `support=supported` with `max_count: 0` on every open postable route that could carry it | `missing` | `reasons[cid] = "unsupported"` — the tool sends one item, a zero bound cannot take it | +| …the same, but a route this table cannot place names a required role as `supported` | `missing` | `reasons[cid] = "unsupported"` — an unplaceable route cannot rescue the refusal; it stays `unresolved: route_mismatch` / `unknown_route` | +| unknown operation / profile / misplaced profile | no evidence → `unknown` | `unresolved: unmapped_stage` / `unknown_profile` / `operation_mismatch` | +| `supported`, `enabled`, one `enabled` postable route (role-gated ids: one postable route carrying all `REQUIRED_ROLES` as `supported`) | `available` | none for the verdict — a diagnostic note may still sit in `unresolved` | +| a capability no upstream profile can express | `unknown` | `UNROUTED[cid]` (`no_upstream_profile` / `no_upstream_operation` / `parameter_only` / `no_local_adapter`) | +| no answer (`None`, no `models`, `models` not a list) | all `unknown` | `endpoint = "absent"` | +| answered, but no model declared an operation | all `unknown` | `endpoint = "empty"` | +| answered in another contract — `object` or `schema_version` is not `omni.capabilities` / `1` | all `unknown`, `models` never read | `endpoint = "unreadable"` + `unresolved: unsupported_format` | +| operation answered with `profiles: []` | `unknown` | `endpoint = "present"` | +| one model refuses, another does not mention that operation | `unknown` | none — the silent model may serve, so the refusal is not deployment-wide; a model that lists **no** operation at all is `unresolved: malformed_model` instead | +| a `models[]` element that is not a model — not an object, no `operations` object, or an empty one | no evidence from it; its silence keeps the affected ids `unknown` | `unresolved: malformed_model` (reported once, only when another model declared something) | + +One boundary is left, and it now cuts the other way: a route this table cannot place is a diagnostic and never a channel — an unrecognised path, `/v1/videos/sync` or the socket cannot void a profile that answered, cannot keep a refusal undecided, and cannot turn "every postable route disabled" into anything but `missing` with `route_disabled`, whatever its own `exposure` says. What does keep a row undecided is a postable route whose `exposure` is absent, unreadable or the literal `unknown`: that route is one this tool can still post to, so the doubt is real, while the same doubt on an unplaceable route is nobody's channel. The three verdict notes — `exposure_unknown`, `no_route_declared`, `role_not_declared` — are attributed once the cross-model aggregate is known, and only to a `(operation, profile)` pair that still has an `unknown` capability id: a profile with no id here gets the shape reasons instead when it declares a route to report (`route_mismatch`, `malformed_route`, `unknown_route`, or `no_local_capability` while that route is open) and nothing at all when it declares none, because there is no id for a verdict note to explain. And `endpoint` distinguishes "did not answer" (`absent`) from "answered without declaring" (`empty`) from "declared" (`present`) from "answered in a contract this table does not read" (`unreadable`). Both keys are compared as published — `object` is the string `"omni.capabilities"`, `schema_version` the integer `1`, so `"1"` is a different shape and not the same version. A body that names neither key is read as v1 instead of being refused: upstream's response model always emits both, so a missing envelope cannot be told apart from a v1 body a proxy stripped. The gate runs before the `absent` / `empty` states, so a body with a foreign envelope is `unreadable` even when it carries no `models` at all, and `topology` comes back empty with it: `stages[]` of the refused body is the same unreviewed shape as its `models`. `declarations()` applies the same gate and returns `None`, so the two readers cannot disagree about one body. + +## Routes without an operation + +`/v1/images/segmentations`, `/v1/videos/segmentations` and `/v1/videos/depth` are registered endpoints that no operation owns; upstream's docs state that "segmentation and depth estimation do not acquire one of these operations merely by being present in a registry" (`docs/serving/capabilities.md:41-43`). `image.segment`, `video.matting` and `video.motion_capture` therefore stay `unknown` with `no_upstream_profile` — the endpoint does not publish the ability, so it cannot be judged, and it must never read as `missing`. `audio.score` is `no_upstream_operation`; `image.cutout` is `parameter_only` (a request field of a published operation, granted per adapter rather than per profile); `audio.separate` is `no_local_adapter`. None of these six ids is a value of `OPERATION_PROFILES`, so `unknown` falls out of "no evidence" rather than a special case. + +## ASR: an upstream gap, not a missing row + +Upstream publishes no recognition operation and omni's router never registers `/v1/audio/transcriptions` (absent from `api_server.py`; its own recipe says the endpoint is "not currently available with `--omni`"). A body naming `asr` gets `unmapped_stage` — the upstream vocabulary has no such operation, this table did not lose a row. That is not a claim that this repository ignores `audio.score`: the `VllmOmniScore` adapter does post to `/audio/transcriptions` (`integrations/model_adapters.py:605-628`, POST at `:620`), whose peer is the standalone `tools/sheetsage2_serve.py:124`. But a capabilities body never mentions that path, so `audio.score` can only be `unknown` (`no_upstream_operation`): a route we happen to know is not evidence of a published capability. + +## Relationship to the V2 discovery contract (#266) + +This table is an input to #266, not a replacement and not a second schema: `domain/omni_capabilities.py` owns the discovery document and matching, this module owns only the `(operation, profile) → capability` layer, under a different file name. It no longer reads `Model.stage_ids` — that is topology, not a capability key — and `declarations()` reads only the published `models[].operations[].profiles[]` shape. It does not modify `domain/omni_capabilities.py`, `integrations/capability_probe.py`, `application/model_availability.py` or `src/**`. + +## Feeding `requests_for()` + +#266's `application/model_availability.py` currently hard-codes profile names per `(kind, protocol)`; with a table the caller needs only the operation and profile it is about to use: + +```python +from opendirector.domain.capability_contract import capabilities_for + +capabilities_for('speech', 'lyrics_to_music') # ('audio.music',) +capabilities_for('speech', 'text_to_speech') # ('audio.speech',) +capabilities_for('audio_generation', 'text_to_audio') # () — upstream publishes it, no id here +candidates_for('video.first_frame') # (('video_generation', 'image_to_video'),) +candidates_for('audio.separate') # () — no profile supplies it +required_roles_for('video.first_frame') # ('first_frame',) +required_roles_for('video.edit') # ('source_video', 'video_noise_mask', 'audio_noise_mask') +required_roles_for('video.generate') # () — the profile's modality is the whole story +role_verdict(entry, 'video_generation', required_roles_for('video.edit')) + # 'yes' | 'no' | 'unknown' for one declaration +``` + +`required_roles_for()` is what keeps a modality-level profile from granting a narrower tool. A caller should ask `role_verdict()` rather than re-derive this: it reads the same postable routes `resolve()` reads, so the role question is answered the same way inside and outside `resolve()`. The profile's own verdict and its routes' reachability are decided before that gate runs, so a refused profile still reads refused. A role counts as satisfied only when **one** of the profile's own open postable routes names **all** of the required roles in `request.inputs` with `support=supported` and a count window that admits one item; roles spread over two routes count for nothing, because a request leaves on one; a role the document never names leaves the id `unknown` instead of `available`; a role that appears only on a closed route counts for nothing; and once every open postable route refuses at least one required role the id is `missing` with `reasons[cid] = "unsupported"`, the same positive refusal a refusing profile produces — a route that refuses one role is out of the running whatever it leaves unmentioned (unless one of the profile's own postable routes has an absent, unreadable or `unknown` exposure, so it is still a channel this tool can use). `resolve()` applies the same rule, which is why both video tools read `unknown` on a deployment that declares `i2v`/`v2v` but publishes modalities only. + +The reverse lookup is the same table read backwards (`BY_CAPABILITY`, derived from `OPERATION_PROFILES`, never a second vocabulary): a caller starting from local configuration — `requests_for()` does — holds a capability id and needs to know **which** `(operation, profile)` pairs could supply it. A candidate is not an authorisation: the adapter for the selected configuration still decides the route, the transmitted inputs and the output, and a submission must re-check that the pair it selected is the one admitted. Another profile being `available` does not vouch for it. + +`resolve(payload)` returns per-capability states, per-capability `reasons`, `topology` (display only) and `unresolved`; `classified_capabilities()` returns the vocabulary in table order. Whether to wire this into `requests_for()` is #266's decision; this contract only guarantees the lookup is paired, ordered and testable. + +`resolve()` is **catalogue-level**: it answers "what does this deployment declare", not "may this request go through". An `available` capability only says that at least one profile declares support with an open postable route; the caller must still check its own model, inputs, output and route, and `stages[]` describes pipeline topology only — it carries no admission evidence. + +## Fixtures and checks + +`tests/fixtures/omni-capabilities/{full,partial,absent}.json` are the **evaluated sample** and follow the real response shape — `stages[]` as `{id,type,depends_on}` with numeric-string ids, six complete `operations` per model each with `support`/`exposure`/`reason_code`/`profiles`/`execution`, every profile carrying `support`, `reason_code` and `routes` (possibly `[]`), and `required_inputs`/`outputs` transcribed from upstream's `STANDARD_PROFILES` including order (`image_response`/`audio_response` are `["image"]`/`["audio"]`). `full` is the all-yes deployment, `partial` mixes refusals, a policy-disabled operation, an all-`unknown` body (with the routes stating `unknown` too) and the two role-gap cases, and `absent` covers no answer (`absent`), topology only (`empty`) and six operations that declare `profiles: []` (`present`, all `unknown`). `route_disabled` appears only in `malformed.json`: upstream aggregates "all of the operation's routes `disabled`" into `exposure=disabled`, so a sample cannot carry that combination. In `full`, four profiles report `no_local_capability` (`image_response`, `audio_response`, `caption_to_audio`, `text_to_audio`) — `text_to_audio` grants no `audio.music`; `audio.music` is supplied by `lyrics_to_music`. The two video profiles that need an unpublished input role report `role_not_declared` and leave `video.first_frame` / `video.edit` `unknown`, which is the state an all-yes deployment gets until a document names those roles. `partial` also carries the two counterexamples as their own cases: `vace-reference-images` (a declaration whose route offers `image` as a reference role) and `seedvr2-restoration` (a `video_to_video` declaration with no mask role) — both keep the narrow id `unknown`. + +A fourth file, `malformed.json`, is **defensive input only and not a sample of what upstream can produce**: its bodies are valid pydantic (the structural checks pass) but upstream's projection cannot emit them — a profile pointing at another operation's route, or owned routes all `disabled` beside an `enabled` path this table does not know. Only a proxy, a cache or an older revision can hand us one, so it is read by the defensive, structural and profile-row checks alone, never by the sample oracle. + +Shapes upstream can never emit — an `asr` operation key, a profile filed under the wrong operation, a missing required `support` — are built inline in the tests that defend against malformed input, never in the sample fixtures. `test_fixtures_are_schema_valid` asserts only the structural invariants of upstream's model (six keys, `observed_at` parseable and timezone-aware, `reason_code` wherever `support` is, `execution` as `{state,scope,observed_at}`, the enum domains, `profiles` as a list, routes carrying `transport`/`method`/`path`, stage shape with numeric-string ids, and a video route's `request` carrying `last_frame: unsupported` plus the four transport parameters), not a copy of upstream's pydantic source. + +Ten regressions keep coverage honest: `test_real_shaped_body_resolves_non_empty_pairs`, `test_route_alone_cannot_distinguish_speech_profiles`, `test_stage_ids_do_not_collapse_capabilities`, `test_capabilities_without_an_upstream_profile_are_unknown_not_missing`, `test_a_profile_without_support_is_unknown_even_when_its_operation_is_supported`, `test_a_video_profile_grants_a_narrower_id_only_when_its_route_declares_the_role`, `test_a_count_window_that_excludes_one_item_is_not_a_grant`, `test_a_refusal_excludes_its_route_even_when_the_rest_is_silent`, `test_a_body_from_another_contract_is_unreadable_instead_of_v1` and `test_one_model_refusing_while_another_stays_silent_is_not_missing`; the route-level exposure, transport/method, chat-modality, `text_to_audio`, role-gating, count-window, foreign-envelope, multi-model-silence and missing-reason cases each have their own test, and `test_documented_required_inputs_match_fixtures` pins the twelve `required_inputs`/`outputs` rows against the doc table and every fixture. + +```bash +PYTHONPATH=packages/core/src:apps/api/src python -m pytest tests/test_capability_contract.py -q +``` diff --git a/docs/architecture/test-contracts.md b/docs/architecture/test-contracts.md index a0abc5f88..f4f73081b 100644 --- a/docs/architecture/test-contracts.md +++ b/docs/architecture/test-contracts.md @@ -2,6 +2,61 @@ 本文说明测试与生产代码之间的边界。它不是风格建议:同一语义存在两份实现时,改动任意一侧都会让另一侧过期,而这种过期往往在 CI 里表现为"红着没人看见"或"改一处红一片"。 +## 测试布局与 CI 层 / Test layout and CI layers + +增量采用以下布局,新测试按业务领域归类;现有根目录用例和 helper 暂留原位,移动时同步修改测试发现命令、导入、watchlist 与脚本路径,不一次性重排整棵目录。 + +Adopt this target layout incrementally, grouping tests by business area. Existing root-level tests and helpers stay in place until a move updates discovery commands, imports, watchlists and script paths together; avoid a whole-tree rewrite. Until discovery supports a new directory, add tests beside their existing peers. In particular, the current frontend command only discovers root-level `tests/*.test.ts`. + +该目录树是增量迁移的目标布局。测试发现尚未支持新目录时,新用例继续放在现有同类用例旁;尤其当前前端命令只发现根目录 `tests/*.test.ts`,首次迁入 `tests/frontend/` 必须同时修改该命令。 + +```text +tests/ +├── frontend/ # *.test.ts, grouped by business area / 按业务领域组织 +│ ├── canvas/ +│ ├── generation/ +│ ├── agent/ +│ ├── creation/ +│ └── media/ +├── python/ # test_*.py, grouped by business area / 按业务领域组织 +│ ├── conftest.py +│ ├── capabilities.py # capability skips / 能力跳过 +│ ├── helpers/ # shared fixtures and helpers / 共享夹具与辅助函数 +│ ├── creation/ +│ ├── agent/ +│ ├── providers/ # default CI uses mocked HTTP / 默认 CI 使用 HTTP 替身 +│ ├── cli/ +│ ├── media/ +│ └── bench/ +├── browser/ # journeys and manifest.json / 浏览器旅程与车道清单 +└── fixtures/ # shared test data / 共享测试数据 +plugins/deepseek-harness/test/ # remains outside tests/ / 保留插件独立目录 +``` + +`frontend/` 与 `python/` 下的领域可以随实际覆盖扩展。每次迁移一个切片,同时更新测试发现、相对导入、conftest/helper、watchlist、浏览器清单和相关脚本中受影响的路径;确认用例集合没有丢失。遗留根目录浏览器脚本及未登记的 `scripts/verify_*_browser.mjs` 应登记到 manifest 并按需迁移或明确退役,不静默移除。插件测试保持独立和按路径运行,本契约不要求本季度完成整树迁移。 + +Business areas under `frontend/` and `python/` may expand with coverage. Move one slice at a time, updating affected discovery, relative imports, conftest/helpers, watchlists, browser manifests and script paths together; verify that the collected inventory is conserved. Legacy root browser scripts and unregistered `scripts/verify_*_browser.mjs` must be registered and moved as appropriate or explicitly retired, never silently dropped. Plugin tests stay independent and path-filtered. Completing the whole tree this quarter is not required. + +| Axis / 维度 | Values / 取值 | Contract / 契约 | +| --- | --- | --- | +| Directory / 目录 | `frontend`, `python`, `creation`, `agent`, etc. | Organization and discovery only; never pytest CI-layer marks. / 仅组织与发现,不是 pytest CI 层 mark。 | +| CI layer / CI 层 | `ready`, `merge`, `nightly` | At most one per Python case; unmarked cases remain ready during migration. PR runs ready, queue runs ready + merge, nightly runs nightly plus intentional watchlist reruns after cutover. / 每个 Python 用例至多一个;迁移期未标记仍为 ready。切换后 PR 跑 ready,队列跑 ready + merge,nightly 跑 nightly 和明确的 watchlist 重跑。 | +| Hardware platform / 机型 | Target `cpu`; accelerators and SKUs later / 目标 `cpu`,加速器与 SKU 后续 | Independent of the CI layer. Current default cases are CPU-only; registration/tagging/selection is follow-up work. Do not infer hardware from a folder or capability skip. / 与 CI 层正交;现有默认用例只需 CPU,注册、打标和筛选后续交付,不从目录或能力跳过推断机型。 | +| Capability / 能力 | `requires_ffmpeg`, `requires_posix` | Skip with a reason when unavailable; not lane or hardware selectors. / 缺能力时带原因跳过,不用于分车道或分机型。 | +| Browser lane / 浏览器车道 | `gate`, `extended`, `extendedFixture`, `production`, `perf`, `manual` | Only `tests/browser/manifest.json` owns membership; these are not pytest marks. / 归属只由 manifest 管理,不是 pytest mark。 | + +`unit`、`integration`、`e2e` 可以描述场景,但不注册为 CI 层 mark。`gpu`、`cuda`、`rocm`、`xpu`、`npu` 与 SKU/卡数只预留后续计划;真实硬件车道需要负责人、环境和凭证方案,后续通过共享 helper 引入平台与 SKU,不要求本季度运行。 + +`unit`, `integration` and `e2e` may describe scenarios but are not CI-layer marks. `gpu`, `cuda`, `rocm`, `xpu`, `npu`, SKUs and card counts are reserved for follow-up plans. Real-hardware lanes need owners, environments and credential arrangements; introduce platforms and SKUs through a shared helper when needed, without requiring those lanes to run this quarter. + +L1 包含单元、静态与构建;L2 是模型替身浏览器 E2E;L3 是 flake 分诊与扩展覆盖;L4 是生产构建、性能与输出;L5 是后续稳定性与恢复。它们表示覆盖范围,不等于目录、Python CI 层或执行阶段。切换前保留的 main HEAD 复核和 gate-parity 属于过渡执行,不是未来 nightly 永久重复门槛的要求。 + +L1 covers unit/static/build checks; L2 covers browser E2E with model stubs; L3 covers flake triage and extended journeys; L4 covers production, performance and output; L5 is follow-up stability and recovery. They describe coverage, not directories, Python CI layers or execution stages. Retained main HEAD checks and gate parity are transitional execution, not a requirement for nightly to duplicate the future gate permanently. + +本地静态、PR、合并队列、main 发布证据与 nightly 的完整职责、预算、切换前置条件和当前实现边界只维护在[验证指南](../development/validation.md)。先验证并强制合并队列完整门禁,再启用 ready 子集、main 证据归集和 nightly 去重;此目录契约本身不修改工作流或搬迁用例。具体 Python 命令和迁移筛选同样由[验证指南](../development/validation.md#python-分层与迁移--python-layers-and-transition)维护。 + +The [validation guide](../development/validation.md) owns the complete responsibilities, budgets, cutover prerequisites and current implementation boundaries for local static checks, PRs, the merge queue, main release evidence and nightly. Validate and enforce the full queue gate before enabling ready selection, main evidence collection and nightly deduplication. This layout contract changes no workflow or test location. The guide also owns [Python commands and transitional selection](../development/validation.md#python-分层与迁移--python-layers-and-transition). + ## 四条规则 ### 1. 一个语义只有一个所有者 diff --git a/docs/architecture/unified-project.md b/docs/architecture/unified-project.md index 36fe22472..700080e2f 100644 --- a/docs/architecture/unified-project.md +++ b/docs/architecture/unified-project.md @@ -108,6 +108,10 @@ Skill 提供规划方法与领域知识,由 harness 按需加载;技能内 隐藏展示对象保留逻辑来源和历史。删除素材前检查参考、编辑文档、生成任务、工具任务、结果、历史与来源依赖;依赖无法确认时不能报告素材未使用。 +隐藏来源或其展示分组不改变活动参考、结构化提示词、生成输入、引用数量限制或结果采用依据。移除生成参考使用明确的参考操作;可见性仅影响画布投影。任务接受后继续保留当时的精确输入版本与内容指纹。 + +Hiding a source display or its group does not change active references, structured prompts, generation inputs, reference limits, or result adoption. Change creative references through explicit reference operations; visibility only affects the canvas projection. Accepted jobs retain their exact input versions and content fingerprints. + ## 存储、通知和交互 工程文件是权威数据。对象、文件、回执和变更记录通过明确的提交顺序与恢复机制协调;SQLite 可用于租约等运行协调。凭证不进入普通工程、公开回执或模型快照。 diff --git a/docs/development/assets/cogvideox-235/first-last-order.jpg b/docs/development/assets/cogvideox-235/first-last-order.jpg new file mode 100644 index 000000000..962a306e7 Binary files /dev/null and b/docs/development/assets/cogvideox-235/first-last-order.jpg differ diff --git a/docs/development/assets/cogvideox-235/first-last-preview.jpg b/docs/development/assets/cogvideox-235/first-last-preview.jpg new file mode 100644 index 000000000..c122ae981 Binary files /dev/null and b/docs/development/assets/cogvideox-235/first-last-preview.jpg differ diff --git a/docs/development/assets/cogvideox-235/history-en.jpg b/docs/development/assets/cogvideox-235/history-en.jpg new file mode 100644 index 000000000..56293c363 Binary files /dev/null and b/docs/development/assets/cogvideox-235/history-en.jpg differ diff --git a/docs/development/assets/cogvideox-235/note-en.jpg b/docs/development/assets/cogvideox-235/note-en.jpg new file mode 100644 index 000000000..9fb3644c9 Binary files /dev/null and b/docs/development/assets/cogvideox-235/note-en.jpg differ diff --git a/docs/development/assets/cogvideox-235/note-zh.jpg b/docs/development/assets/cogvideox-235/note-zh.jpg new file mode 100644 index 000000000..c2b705c4a Binary files /dev/null and b/docs/development/assets/cogvideox-235/note-zh.jpg differ diff --git a/docs/development/assets/cogvideox-235/settings-en.jpg b/docs/development/assets/cogvideox-235/settings-en.jpg new file mode 100644 index 000000000..f9cd783e1 Binary files /dev/null and b/docs/development/assets/cogvideox-235/settings-en.jpg differ diff --git a/docs/development/assets/cogvideox-235/settings-zh.jpg b/docs/development/assets/cogvideox-235/settings-zh.jpg new file mode 100644 index 000000000..ee3f2eb6d Binary files /dev/null and b/docs/development/assets/cogvideox-235/settings-zh.jpg differ diff --git a/docs/development/assets/cogvideox-235/single-preview.jpg b/docs/development/assets/cogvideox-235/single-preview.jpg new file mode 100644 index 000000000..b7ec60b9d Binary files /dev/null and b/docs/development/assets/cogvideox-235/single-preview.jpg differ diff --git a/docs/development/assets/cogvideox-235/text-preview.jpg b/docs/development/assets/cogvideox-235/text-preview.jpg new file mode 100644 index 000000000..92ec2f9dc Binary files /dev/null and b/docs/development/assets/cogvideox-235/text-preview.jpg differ diff --git a/docs/development/bigmodel-video-validation.md b/docs/development/bigmodel-video-validation.md new file mode 100644 index 000000000..a4f0a437f --- /dev/null +++ b/docs/development/bigmodel-video-validation.md @@ -0,0 +1,110 @@ +# CogVideoX-3 verification + +This record accompanies the `codex/bigmodel-video-235` implementation, based on main `e0b411f151dfe48250c4651c0d820e05179f41ce`. Verification ran on Apple Silicon macOS with Python 3.12.15, Node 24.22.2, FFmpeg and ffprobe. Setup and recovery limits are documented in the [English](../guides/bigmodel-video.md) and [Chinese](../guides/bigmodel-video.zh-CN.md) guides. + +## Real BigModel calls + +Three generation requests were sent to `https://open.bigmodel.cn/api/paas/v4/videos/generations` using `cogvideox-3`, 1280×720, 5 seconds, 30 fps, speed quality and no requested audio. A local submission ledger recorded each POST before sending; exactly three were issued, with an expected cost of ¥3 at the published ¥1/request rate. This is a request count and price estimate, not an account billing reconciliation. + +| Mode | Remote task ID | Downloaded media | +| --- | --- | --- | +| Text | `202611100940587616dc04c6ae438c` | H.264, 1280×720, 5.193548 s | +| Single image | `20261110094058b969e54f901542c3` | H.264, 1280×720, 5.129032 s | +| First and last frames | `202611100943553eeea4665ddd4a54` | H.264, 1280×720, 5.193548 s | + +Text and single-image jobs used `CreationGeneration.execute`, the shared service behind UI submission. The frame-pair job used `AgentTools.execute` with `execute_command`. These were scripted business-service calls; the initial paid submissions were not mouse-driven browser interactions. Mocked tests separately cover all three modes through both entry points. + +The images were PNG files imported as project-local assets. The first-frame sphere is on the left; the last-frame sphere is on the right. References were deliberately created in last/first order. The actual JSON request's decoded image hashes matched first/last order: + +- First: `5ac201f728123bbc80e443b9fc640a9f55955f10ed7e653e05e6cb8d8fc11c12` +- Last: `60b4e0fa4e36e989d87b08bf08d25e71c0a579fe4c554ac2c05131064e54acf7` + +The generated video's first and final frames also show that order. The comparison below places the two local inputs above actual frames extracted from the downloaded video. Provider media markings are preserved. + +![Input frames and actual output order](assets/cogvideox-235/first-last-order.jpg) + +## Restart, candidates and reopening + +The submission process paused after persisting the first two remote IDs, before polling. The third job was accepted locally but waited for a concurrency slot and had not reached the provider. That process exited; a new Python process reopened the same project and started recovery. It queried the first two original IDs using GET and made the third job's first POST. The ledger remained at three submissions, one per job. This is an actual process restart; the pause after ID persistence was injected by the acceptance harness. + +All three results were downloaded and registered as candidates with no current result selected. Explicit `result.select` commands adopted them, using the shared UI service for two results and Agent tools for the frame pair. All videos passed ffprobe checks and a complete `ffmpeg -i