Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions CONTENT_REVIEW_ASYNC_BOUNDARY_TASK.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# 阶段 3:内容复盘异步边界与陈旧预览保护

## 背景

第二次工程复检确认两项仍会影响真实使用的问题:官方作品导出同步在 FastAPI
异步路由内执行阻塞式 Worker 请求;导入预览请求返回较慢时,旧账号或旧文件的
响应可能覆盖当前页面上下文。

## 目标

1. 将“账号解析、Worker 官方导出、行数校验、SQLite 提交”整体移出事件循环。
2. 为导入预览增加请求代际和取消保护,旧响应不得恢复旧批次或确认按钮。
3. 保持现有错误码、事务边界、真实发布和 Provider 行为不变。

## 允许修改范围

- `app/routers/content_review.py`
- `app/static/js/content-review.js`
- `app/templates/content_review.html`
- `tests/test_content_review.py`
- `tests/test_content_review_browser.py`
- `docs/UI_REFERENCE.md`
- `DEVELOPMENT_LOG.md`
- `NEXT_STEPS.md`
- 本任务文件

## 禁止修改范围

- 不改 Windows Worker、真实浏览器、发布调度器或平台投稿链路。
- 不改数据库结构、活动数据库、Provider 配置或依赖文件。
- 不把官方导出改为自动重试或可取消的后台任务。
- 不扩展为发布中心跨服务事务重构。

## 已确定实现要求

- 使用项目现有的 `starlette.concurrency.run_in_threadpool` 风格。
- 原有 `ContentReviewError`、`PublishError` 和 `PublishWorkerUnavailable` 映射保持不变。
- 预览请求捕获代际、账号和 `File` 对象;仅当前上下文可写入 DOM。
- 文件或账号切换时使旧预览失效,并安全忽略 `AbortError`。
- 更新静态脚本版本参数,避免浏览器继续使用旧缓存。

## 验收标准

- 阻塞导出链路在事件循环线程之外执行,并保持成功及错误响应契约。
- Worker 行数不一致时不提交任何导出数据。
- 预览期间切换文件或账号后,旧响应不能显示预览、恢复批次或启用确认按钮。
- 专项测试、全量测试、Ruff、JS 语法检查、PowerShell Parser、Compose 配置、
`compileall`、`pip check` 与 `git diff --check` 通过。
- 最终 diff 仅包含允许路径,无依赖变化、敏感信息、TODO/debug 或临时产物。

## 测试命令

```powershell
.venv\Scripts\python.exe -m pytest -q tests/test_content_review.py tests/test_content_review_browser.py tests/test_douyin_analytics_worker.py
.venv\Scripts\python.exe -m pytest -q
.venv\Scripts\python.exe -m ruff check app tests scripts
node --check app/static/js/content-review.js
```

## 返回格式

报告修改文件、专项/全量测试结果、静态检查结果、范围审计和未覆盖的外部边界;
不得宣称触发过真实导出、发布、Provider 调用或活动数据库迁移。
8 changes: 8 additions & 0 deletions DEVELOPMENT_LOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Development Log

## 2026-08-30 内容复盘异步边界与陈旧预览保护

- 官方作品导出路由将账号解析、同步 Worker 调用、行数校验和 SQLite 提交整体放入线程池;原有登录、验证、限流、页面变化、下载失败和 Worker 不可用错误码保持不变。
- 人工导入预览新增请求代际、账号、文件对象和 `AbortController` 四重上下文保护;切换账号或文件后,旧响应不能恢复旧批次、预览区或“确认导入”按钮。
- 独立复审补获并修正了保护逻辑自身的按钮状态回归:账号切换后当前文件仍可重新预览,旧确认请求返回后也不能重新启用无批次按钮或覆盖当前消息。
- Worker 返回行数与作品数组不一致时继续在数据库提交前以 502 停止;本轮不增加自动重试、后台队列或真实平台操作。
- 已新增线程边界、提交前校验和真实 Chrome 陈旧响应回归;专项 `39 passed`、全量 `867 passed`,Ruff、Compileall、9 个 JavaScript、20 个 PowerShell、三套 Compose、`pip check` 与 `git diff --check` 全部通过。验证仅使用 Mock Worker、隔离测试库和本地测试文件,不触发真实导出、AI 或投稿。

## 2026-08-29 内容复盘“数据 → 动作 → 验证”闭环

- 自动导出卡改为显示最近一次官方作品导出的北京时间、批次作品数及已匹配/待确认/未匹配统计;账号趋势表不再参与作品导出时间和周数口径,同一自然周多次导出只计一个官方导出周。
Expand Down
7 changes: 7 additions & 0 deletions NEXT_STEPS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Next Steps

## 2026-08-30 内容复盘异步边界验收

1. 合并并受控重启 Web 后,普通查看页面不需要点击任何导出、AI 或投稿按钮;先确认内容复盘、任务详情和系统状态可同时正常打开。
2. 若要人工检查预览保护,可选择测试表并点击“预览数据”,在返回前切换账号或重新选择文件;旧预览不应重新出现,“确认导入”应保持禁用,之后只对当前文件重新预览。
3. 官方导出仍是一次明确的人工操作,不会自动重试。关闭页面或客户端断开不会强制取消已经交给 Windows Worker 的只读导出,遇到登录、验证码、429 或页面变化仍按原规则安全停止。
4. 本轮不处理发送中心三段保存的跨服务全有或全无;实验归属接口自身仍为单事务且幂等。只有真实出现不可恢复的部分保存问题时,再单独评估跨服务事务,不为代码洁癖扩大改造。

## 2026-08-29 内容复盘闭环使用与上线检查

1. 本分支合并并受控重启 Web 前,先确认没有 `PUBLISHING` 任务并备份活动 SQLite;启动时会应用 `20260829_02_content_feedback_loop`。重启后检查 `/api/system/readiness?deep=1` 和发送中心,确认迁移账本、实验表及 Worker 状态均正常。
Expand Down
33 changes: 19 additions & 14 deletions app/routers/content_review.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from fastapi import APIRouter, File, Form, HTTPException, Query, UploadFile
from starlette.concurrency import run_in_threadpool

from app.models.content_review import (
ContentExperimentAssignmentRequest,
Expand All @@ -20,6 +21,23 @@ def _raise_content_review_http(exc: content_review_service.ContentReviewError):
raise HTTPException(status_code=exc.status_code, detail=str(exc)) from exc


def _sync_douyin_export(account_id_input: str) -> dict:
account_id = content_review_service._resolve_douyin_account_id(account_id_input)
worker_result = PublishWorkerClient().analytics_export_sync(account_id=account_id)
items = list(worker_result.get("items") or [])
if int(worker_result.get("row_count") or 0) != len(items):
raise content_review_service.ContentReviewError(
"Windows Worker 返回的作品行数校验失败",
status_code=502,
)
return content_review_service.commit_douyin_item_export(
account_id=account_id,
items=items,
captured_at=str(worker_result.get("captured_at") or content_review_service._now_iso()),
source_filename=str(worker_result.get("source_filename") or "作品列表导出.xlsx"),
)


@router.get("/accounts")
async def list_accounts() -> dict:
return {"accounts": content_review_service.list_douyin_accounts()}
Expand Down Expand Up @@ -59,20 +77,7 @@ async def commit_import(
@router.post("/douyin/export-sync")
async def export_sync_douyin_items(payload: DouyinAnalyticsExportSyncRequest) -> dict:
try:
account_id = content_review_service._resolve_douyin_account_id(payload.account_id)
worker_result = PublishWorkerClient().analytics_export_sync(account_id=account_id)
items = list(worker_result.get("items") or [])
if int(worker_result.get("row_count") or 0) != len(items):
raise content_review_service.ContentReviewError(
"Windows Worker 返回的作品行数校验失败",
status_code=502,
)
return content_review_service.commit_douyin_item_export(
account_id=account_id,
items=items,
captured_at=str(worker_result.get("captured_at") or content_review_service._now_iso()),
source_filename=str(worker_result.get("source_filename") or "作品列表导出.xlsx"),
)
return await run_in_threadpool(_sync_douyin_export, payload.account_id)
except content_review_service.ContentReviewError as exc:
_raise_content_review_http(exc)
except (PublishError, PublishWorkerUnavailable) as exc:
Expand Down
54 changes: 46 additions & 8 deletions app/static/js/content-review.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,25 @@ const contentReviewSync = document.querySelector("#content-review-sync");

let previewBatchId = "";
let contentReviewLoadSequence = 0;
let contentReviewPreviewSequence = 0;
let contentReviewPreviewController = null;

function currentAccountId() {
return contentReviewAccount?.value || "";
}

function invalidateContentReviewPreview() {
contentReviewPreviewSequence += 1;
contentReviewPreviewController?.abort();
contentReviewPreviewController = null;
previewBatchId = "";
if (contentReviewPreview) contentReviewPreview.hidden = true;
if (contentReviewCommit) contentReviewCommit.disabled = true;
if (contentReviewPreviewButton) {
contentReviewPreviewButton.disabled = !contentReviewFile?.files?.[0];
}
}

function showContentReviewMessage(message, tone = "info") {
if (!contentReviewMessage) return;
contentReviewMessage.hidden = !message;
Expand Down Expand Up @@ -605,12 +619,12 @@ async function loadContentReviewData() {

contentReviewFile?.addEventListener("change", () => {
const file = contentReviewFile.files?.[0];
previewBatchId = "";
if (contentReviewPreview) contentReviewPreview.hidden = true;
invalidateContentReviewPreview();
if (contentReviewFileName) contentReviewFileName.textContent = file?.name || "未选择文件";
if (contentReviewPreviewButton) contentReviewPreviewButton.disabled = !file;
const status = document.querySelector("#content-review-file-status");
if (status) status.textContent = file ? "已选择" : "尚未选择";
showContentReviewMessage("", "info");
});

contentReviewImportForm?.addEventListener("submit", async (event) => {
Expand All @@ -620,20 +634,32 @@ contentReviewImportForm?.addEventListener("submit", async (event) => {
showContentReviewMessage("请先选择 .xlsx 或 .csv 文件。", "error");
return;
}
contentReviewPreviewController?.abort();
const requestSequence = ++contentReviewPreviewSequence;
const accountId = currentAccountId();
const controller = new AbortController();
contentReviewPreviewController = controller;
const button = contentReviewImportForm.querySelector("button[type='submit']");
button.disabled = true;
showContentReviewMessage("正在只读解析并校验数据表…", "info");
const formData = new FormData();
formData.append("file", file);
formData.append("account_id", currentAccountId());
formData.append("account_id", accountId);
try {
const data = await contentReviewApi("/api/content-review/imports/preview", {
method: "POST",
body: formData,
signal: controller.signal,
});
if (
requestSequence !== contentReviewPreviewSequence
|| accountId !== currentAccountId()
|| file !== contentReviewFile?.files?.[0]
) return;
previewBatchId = data.batch_id || "";
document.querySelector("#content-review-file-status").textContent = data.already_imported ? "已导入" : "预览通过";
if (contentReviewPreview) contentReviewPreview.hidden = data.already_imported;
if (contentReviewCommit) contentReviewCommit.disabled = data.already_imported || !previewBatchId;
if (!data.already_imported) {
contentReviewPreview.querySelector("[data-preview-filename]").textContent = data.filename || file.name;
contentReviewPreview.querySelector("[data-preview-type]").textContent = data.report_type === "douyin_item_export" ? "官方作品列表" : "账号趋势表";
Expand All @@ -643,31 +669,40 @@ contentReviewImportForm?.addEventListener("submit", async (event) => {
}
showContentReviewMessage(data.message, data.already_imported ? "info" : "success");
} catch (error) {
if (error.name === "AbortError" || requestSequence !== contentReviewPreviewSequence) return;
previewBatchId = "";
if (contentReviewPreview) contentReviewPreview.hidden = true;
if (contentReviewCommit) contentReviewCommit.disabled = true;
showContentReviewMessage(`预览失败:${error.message}`, "error");
} finally {
button.disabled = !contentReviewFile?.files?.[0];
if (requestSequence === contentReviewPreviewSequence) {
if (contentReviewPreviewController === controller) contentReviewPreviewController = null;
button.disabled = !contentReviewFile?.files?.[0];
}
}
});

contentReviewCommit?.addEventListener("click", async () => {
if (!previewBatchId) return;
const batchId = previewBatchId;
const requestSequence = contentReviewPreviewSequence;
contentReviewCommit.disabled = true;
try {
const data = await contentReviewApi(`/api/content-review/imports/${encodeURIComponent(previewBatchId)}/commit`, {
const data = await contentReviewApi(`/api/content-review/imports/${encodeURIComponent(batchId)}/commit`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ confirm: true }),
});
if (requestSequence !== contentReviewPreviewSequence || batchId !== previewBatchId) return;
showContentReviewMessage(data.message, "success");
contentReviewPreview.hidden = true;
previewBatchId = "";
await loadContentReviewData();
} catch (error) {
if (requestSequence !== contentReviewPreviewSequence || batchId !== previewBatchId) return;
showContentReviewMessage(`导入失败:${error.message}`, "error");
} finally {
contentReviewCommit.disabled = false;
contentReviewCommit.disabled = !previewBatchId;
}
});

Expand All @@ -692,8 +727,11 @@ contentReviewSync?.addEventListener("click", async () => {
});

contentReviewAccount?.addEventListener("change", () => {
previewBatchId = "";
if (contentReviewPreview) contentReviewPreview.hidden = true;
invalidateContentReviewPreview();
const file = contentReviewFile?.files?.[0];
const status = document.querySelector("#content-review-file-status");
if (status) status.textContent = file ? "已选择" : "尚未选择";
showContentReviewMessage("", "info");
loadContentReviewData();
});

Expand Down
4 changes: 2 additions & 2 deletions app/templates/content_review.html
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ <h2>导入抖音官方数据表</h2>
<div><dt>行数</dt><dd data-preview-rows>—</dd></div>
<div><dt>归因</dt><dd data-preview-attribution>—</dd></div>
</dl>
<button class="primary-button" id="content-review-commit" type="button">确认导入</button>
<button class="primary-button" id="content-review-commit" type="button" disabled>确认导入</button>
</div>
</article>

Expand Down Expand Up @@ -170,5 +170,5 @@ <h2>作品级归因</h2>
{% endblock %}

{% block extra_scripts %}
{% if accounts %}<script src="{{ url_for('static', path='js/content-review.js') }}?v=20260829-feedback-loop-v1"></script>{% endif %}
{% if accounts %}<script src="{{ url_for('static', path='js/content-review.js') }}?v=20260830-async-boundary-v1"></script>{% endif %}
{% endblock %}
6 changes: 6 additions & 0 deletions docs/UI_REFERENCE.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# UI 参考说明

## 2026-08-30 更新:内容复盘异步与预览上下文保护

- “自动导出并同步全部作品”的视觉和操作语义不变;耗时的 Worker 导出及后续提交在服务端线程池执行,不再阻塞同一 Web 进程的异步请求。
- 人工预览期间若切换文件或账号,页面立即清空旧批次、隐藏预览并禁用“确认导入”;已经在途的旧响应不得覆盖当前文件名、账号或确认状态。
- 被账号/文件切换主动取消的旧预览不显示“预览失败”,用户可直接对当前上下文重新预览。

## 2026-08-29 更新:内容复盘可执行闭环

- 自动导出卡固定展示精确北京时间和本批作品/匹配统计;文件上传使用蓝色自定义按钮、独立文件名和禁用态预览按钮,不展示浏览器原生文件控件。
Expand Down
64 changes: 64 additions & 0 deletions tests/test_content_review.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
from __future__ import annotations

import asyncio
import csv
from datetime import datetime
import hashlib
import io
import json
import threading
from uuid import uuid4

from fastapi.testclient import TestClient
Expand All @@ -13,6 +15,8 @@

from app.db.database import get_connection, init_db
from app.main import app
from app.models.content_review import DouyinAnalyticsExportSyncRequest
from app.routers import content_review as content_review_router
from app.services import content_review_service
from app.services.publishers.base import PublishError
from app.services.publishers.worker_client import PublishWorkerClient
Expand Down Expand Up @@ -837,6 +841,66 @@ def worker_sync(_self, *, account_id):
}


def test_export_sync_offloads_worker_validation_and_commit(monkeypatch):
caller_thread_id = threading.get_ident()
observed_thread_ids = []

def resolve_account(account_id):
observed_thread_ids.append(threading.get_ident())
return account_id

def worker_sync(_self, *, account_id):
observed_thread_ids.append(threading.get_ident())
return {
"captured_at": "2026-08-28T12:00:00+08:00",
"source_filename": "作品列表导出.xlsx",
"row_count": 0,
"items": [],
}

def commit_export(**kwargs):
observed_thread_ids.append(threading.get_ident())
return {"row_count": len(kwargs["items"]), "message": "同步完成"}

monkeypatch.setattr(content_review_service, "_resolve_douyin_account_id", resolve_account)
monkeypatch.setattr(PublishWorkerClient, "analytics_export_sync", worker_sync)
monkeypatch.setattr(content_review_service, "commit_douyin_item_export", commit_export)

result = asyncio.run(
content_review_router.export_sync_douyin_items(
DouyinAnalyticsExportSyncRequest(account_id="test-account")
)
)

assert result["row_count"] == 0
assert len(set(observed_thread_ids)) == 1
assert observed_thread_ids[0] != caller_thread_id


def test_export_sync_rejects_row_count_mismatch_before_commit(monkeypatch):
account_id = _insert_account()
committed = False

def worker_sync(_self, *, account_id):
return {"row_count": 1, "items": []}

def commit_export(**_kwargs):
nonlocal committed
committed = True

monkeypatch.setattr(PublishWorkerClient, "analytics_export_sync", worker_sync)
monkeypatch.setattr(content_review_service, "commit_douyin_item_export", commit_export)

response = TestClient(app).post(
"/api/content-review/douyin/export-sync",
json={"account_id": account_id},
)

assert response.status_code == 502
assert response.json()["detail"] == "Windows Worker 返回的作品行数校验失败"
assert committed is False


def test_export_sync_api_preserves_fixed_worker_error_code(monkeypatch):
account_id = _insert_account()

Expand Down
Loading
Loading