Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Agent Note: 文稿检测 AI 痕迹维度合同

Status: implemented — #228 一期启发式统计;无持久化、无迁移

## Problem

用 AI 辅助写作的创作者想知道文稿「AI 味」重不重、具体重在哪。既有链路只有
风险把关(违禁词/极限词/限流词),没有写作风格信号。本轮新增 ai_traces 维度:
AI 高频词/黑话命中、套话连接词命中、句式均匀度,各自分档 + 综合建议。

## Decision

### 定位红线:启发式统计,不是模型检测

全链路(响应字段、advice、UI 文案、hover 提示)明示「启发式统计」,禁止
「AI 率 / AI 概率 / 检测模型」类口径——词库命中与句式统计都判不了文稿来源。
与风险把关语义分开:ai_traces 独立端点独立展示,**不并入 /sensitive 的
total,不影响「未发现违禁词,可以发布」结论**(风险把关与写作建议不同层)。
现链路无全局总分/评分体系,本轮不新造。

### 词库(backend/app/core/ai_traces/ai_lexicon.json)

`_meta` + 两组词条:`buzzword` 154 条(预算 100~200)、`connective` 39 条
(预算 30~50)。**人工整理**(大模型中文输出的词频观察 + 通行互联网黑话),
无单一权威来源,`_meta.source` 如实标注;若逐词要求外部背书会滑向 4/5 工作量,
首版放弃。词条校验同敏感词:不含空白/标点,非法词条显式 warning 跳过,
测试断言零跳过 + 无重复 + 预算区间。加词走普通 PR,更新改 `version_date`。

刻意排除:通用副词单用(显著/极大)、正常书面语常用词(但是/因此),只收
堆积才有信号的黑话与套话,控制口语化文稿的误报。

### 扫描与偏移(沿用敏感词既有合同,改动两边一起改)

分隔符 `[\s\W]` 1:1 掩码、每维度按词长降序 union 正则(同 start 保留最长)、
start/end 是原文 codepoint 偏移、不做 NFKC。两维度命中合并后按
(start 升序, 长度降序, buzzword 先于 connective) 贪心去重叠,区间两两不交
——前端 `_highlight.tsx` 按 codepoint 切分对齐(emoji 不漂移)。

### 分档阈值(service 显式常量,改阈值必须同步测试边界断言)

- 密度 = 命中次数 / 全文 codepoint 数 × 1000(次/千字,分母含标点空白)。
buzzword:≥3 起 medium、≥8 high——黑话在口语化写作几乎不出现,阈值从宽。
connective:≥2.5 起 medium、≥6 high——首先/其次在正常议论文有基础出现率,
阈值不能压太低误伤正常行文。
- 句式均匀度:按中文句末标点(。!?;…全半角)与换行分句,句长取 codepoint
数,用 **CV=pstdev/mean**(无量纲,长短文可比)而非原始方差;
CV<0.20 high、0.20≤CV<0.35 medium、≥0.35 low。自然中文写作句长长短交错
(口语样本实测 CV≈0.76),模板化行文句长趋同(密集样文实测 CV≈0.07)。
- **insufficient 边界**:句数 <3 时 CV 无统计意义,rhythm 返回 insufficient
且 cv=null,不编造分档。词证据与句证据独立:短文本黑话密集仍按词分档
(word hits 参与整体);只有「句数不足且词零命中」时整体才 insufficient——
短到没有任何可信信号时宁可不判,也不编造「痕迹少」。

### 入口与前端

- `POST /api/v1/content-check/ai-traces`:挂在 /content-check 路由器上共享
`get_current_user` 与限流;`max_length=20000` 与 sensitive 同口径(422)。
- 前端 `/content-check`:开始检测时两请求 `Promise.all` 并行,结果合并成
单一对象进 `_scan-state` 的快照合同(泛型化后两类结果一起 current/stale)。
右侧新增 AI 痕迹面板(三子指标 + 分档 Badge + advice);正文高亮合并渲染:
风险命中红/黄系不变,AI 痕迹命中**紫色系**(设计系统 --color-purple),
同区间重叠时风险命中优先。「未发现违禁词,可以发布」只看 sensitive.total。

### 测试口径

`tests/test_ai_traces.py`:词表零跳过/预算、偏移切片还原(含 emoji)、密度与
CV 分档边界等值行为、insufficient 边界、401/422、万字基准(均值 <500ms,
实测远低于此)。跑法同敏感词:一次性 PG 单跑或 `make test-backend`。
零回归验证:`tests/test_sensitive_word.py` 未改动、单跑通过;前端
`npm run test:coverage` 通过(_scan-state / _highlight 用例同步扩)。

## 后续(v2 再议)

全局总分/评分体系、逐句均匀度高亮(v1 只做词级定位)、改写建议生成、
词库入库与用户自定义、英文文稿。
31 changes: 27 additions & 4 deletions backend/app/api/v1/content_check.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,23 @@
"""文稿检测入口——发布前违禁词 / 广告法极限词扫描。
"""文稿检测入口——发布前违禁词 / 广告法极限词扫描,附 AI 痕迹启发式统计。

入口层只做三件事:路由、schema 校验、调 service(`sensitive_word_service.scan`
纯函数)。无 ORM、无 session。认证用 `Depends(get_current_user)` 防匿名刷大文本
(与 skill.py 同模式);限流由 `/api/v1` 前缀的全局中间件自动覆盖,无需单列规则。
与 `ai_trace_service.analyze` 纯函数)。无 ORM、无 session。认证用
`Depends(get_current_user)` 防匿名刷大文本(与 skill.py 同模式);限流由
`/api/v1` 前缀的全局中间件自动覆盖,无需单列规则。
"""

from __future__ import annotations

from fastapi import APIRouter, Depends

from app.api.v1.auth import get_current_user
from app.schemas.content_check import SensitiveCheckRequest, SensitiveCheckResponse
from app.schemas.content_check import (
AiTracesCheckRequest,
AiTracesReport,
SensitiveCheckRequest,
SensitiveCheckResponse,
)
from app.services.ai_trace_service import analyze as analyze_ai_traces
from app.services.sensitive_word_service import scan

router = APIRouter(prefix="/content-check", tags=["content-check"], dependencies=[Depends(get_current_user)])
Expand All @@ -32,3 +39,19 @@ async def check_sensitive(payload: SensitiveCheckRequest) -> SensitiveCheckRespo
total=len(hits),
hits=[{**hit} for hit in hits],
)


@router.post(
"/ai-traces",
response_model=AiTracesReport,
summary="文稿 AI 痕迹启发式统计",
description=(
"统计文稿的「AI 味」写作风格信号:AI 高频词/黑话命中(buzzword,含原文"
"字符偏移)、套话连接词命中(connective)、句式均匀度(句长变异系数分档)。"
"这是启发式统计,不是模型检测,不输出「AI 率/AI 概率」,也不影响违禁词"
"检测的把关结论。句数不足 3 句时句式维度返回 insufficient。"
"单次最多 20000 字符,超出返回 422。"
),
)
async def check_ai_traces(payload: AiTracesCheckRequest) -> AiTracesReport:
return AiTracesReport(**analyze_ai_traces(payload.text))
26 changes: 26 additions & 0 deletions backend/app/core/ai_traces/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
"""AI 痕迹维度的 dimension / level 常量(跨层共享的纯常量放 core)。

「AI 痕迹」是**启发式统计,不是模型检测**:词库命中与句式节奏只描述写作
风格信号,任何输出与文案不得暗示「AI 率 / AI 概率 / 检测模型」口径。
词表数据与来源记录在同目录 ai_lexicon.json 的 `_meta`;分档阈值与理由见
.agents/notes/implemented/feature/2026-11-04-content-check-ai-traces.md。
"""

from __future__ import annotations

# 词维度:buzzword=AI 高频词/黑话,connective=套话连接词(各有词表);
# rhythm=句式均匀度(无词表,按句长统计),单独成档不进 WORD_DIMENSIONS。
DIMENSION_BUZZWORD = "buzzword"
DIMENSION_CONNECTIVE = "connective"
DIMENSION_RHYTHM = "rhythm"

WORD_DIMENSIONS: tuple[str, ...] = (DIMENSION_BUZZWORD, DIMENSION_CONNECTIVE)

# 分档:high/medium/low 描述「AI 味」浓度(high=痕迹明显),insufficient=样本
# 不足宁可不判。与风险把关的 severity(high/medium/low)语义不同,不复用配色。
LEVEL_HIGH = "high"
LEVEL_MEDIUM = "medium"
LEVEL_LOW = "low"
LEVEL_INSUFFICIENT = "insufficient"

LEVELS: tuple[str, ...] = (LEVEL_HIGH, LEVEL_MEDIUM, LEVEL_LOW, LEVEL_INSUFFICIENT)
209 changes: 209 additions & 0 deletions backend/app/core/ai_traces/ai_lexicon.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,209 @@
{
"_meta": {
"description": "AI 痕迹维度的启发式词表:buzzword=AI 高频词/互联网黑话,connective=套话连接词。只统计写作风格信号,不代表文稿来源判定",
"source": [
"人工整理:中文互联网公开讨论中常见的大模型生成文风高频词与套话(对大模型中文输出的词频观察,社区共识性表述,无单一权威来源)",
"通行「互联网黑话」词表的公共词条(赋能/抓手/闭环类,社区广泛流传的事实性短语)"
],
"license": "人工整理词表,无第三方版权依赖",
"version_date": "2026-11-04",
"note": "启发式统计口径,不暗示「AI 率/AI 概率」。词条不得包含空白或标点字符(service 加载时校验,非法词条显式 warning 跳过,测试断言零跳过);buzzword 预算 100~200 条、connective 30~50 条,测试锁定。更新时改本 version_date 并复核 owning note 的阈值理由"
},
"buzzword": [
{ "word": "赋能" },
{ "word": "抓手" },
{ "word": "闭环" },
{ "word": "底层逻辑" },
{ "word": "顶层设计" },
{ "word": "方法论" },
{ "word": "组合拳" },
{ "word": "护城河" },
{ "word": "生态圈" },
{ "word": "生态链" },
{ "word": "私域" },
{ "word": "公域" },
{ "word": "流量池" },
{ "word": "风口" },
{ "word": "赛道" },
{ "word": "蓝海" },
{ "word": "红海" },
{ "word": "内卷" },
{ "word": "破圈" },
{ "word": "出圈" },
{ "word": "裂变" },
{ "word": "引爆" },
{ "word": "链路" },
{ "word": "全链路" },
{ "word": "颗粒度" },
{ "word": "对齐" },
{ "word": "拉通" },
{ "word": "拉齐" },
{ "word": "沉淀" },
{ "word": "复盘" },
{ "word": "复用" },
{ "word": "打通" },
{ "word": "联动" },
{ "word": "协同" },
{ "word": "耦合" },
{ "word": "解耦" },
{ "word": "中台" },
{ "word": "底座" },
{ "word": "矩阵" },
{ "word": "打法" },
{ "word": "深耕" },
{ "word": "布局" },
{ "word": "落地" },
{ "word": "聚焦" },
{ "word": "发力" },
{ "word": "锚定" },
{ "word": "卡位" },
{ "word": "助力" },
{ "word": "助推" },
{ "word": "驱动" },
{ "word": "激活" },
{ "word": "释放" },
{ "word": "解锁" },
{ "word": "契机" },
{ "word": "蓄势" },
{ "word": "破局" },
{ "word": "破题" },
{ "word": "迭代" },
{ "word": "升维" },
{ "word": "蜕变" },
{ "word": "蝶变" },
{ "word": "跃迁" },
{ "word": "飞跃" },
{ "word": "质变" },
{ "word": "彰显" },
{ "word": "凸显" },
{ "word": "迸发" },
{ "word": "激荡" },
{ "word": "澎湃" },
{ "word": "涌现" },
{ "word": "汇聚" },
{ "word": "凝聚" },
{ "word": "铸就" },
{ "word": "谱写" },
{ "word": "绘就" },
{ "word": "描绘" },
{ "word": "点亮" },
{ "word": "照亮" },
{ "word": "启航" },
{ "word": "扬帆" },
{ "word": "远航" },
{ "word": "起航" },
{ "word": "腾飞" },
{ "word": "跃升" },
{ "word": "领跑" },
{ "word": "领军" },
{ "word": "答卷" },
{ "word": "画卷" },
{ "word": "篇章" },
{ "word": "新篇章" },
{ "word": "华章" },
{ "word": "华丽转身" },
{ "word": "无缝衔接" },
{ "word": "一站式" },
{ "word": "一体化" },
{ "word": "全方位" },
{ "word": "多维度" },
{ "word": "立体化" },
{ "word": "场景化" },
{ "word": "智能化" },
{ "word": "数字化" },
{ "word": "精细化" },
{ "word": "体系化" },
{ "word": "系统化" },
{ "word": "结构化" },
{ "word": "标准化" },
{ "word": "流程化" },
{ "word": "规模化" },
{ "word": "常态化" },
{ "word": "差异化" },
{ "word": "多元化" },
{ "word": "多样化" },
{ "word": "深度融合" },
{ "word": "深度解析" },
{ "word": "深度剖析" },
{ "word": "深入探讨" },
{ "word": "深入剖析" },
{ "word": "全面提升" },
{ "word": "全面升级" },
{ "word": "显著提升" },
{ "word": "显著增强" },
{ "word": "显著改善" },
{ "word": "大幅提升" },
{ "word": "极大地" },
{ "word": "有效地" },
{ "word": "深刻洞察" },
{ "word": "独到见解" },
{ "word": "前所未有" },
{ "word": "日新月异" },
{ "word": "突飞猛进" },
{ "word": "如火如荼" },
{ "word": "方兴未艾" },
{ "word": "雨后春笋" },
{ "word": "蒸蒸日上" },
{ "word": "欣欣向荣" },
{ "word": "生机勃勃" },
{ "word": "蓬勃发展" },
{ "word": "波澜壮阔" },
{ "word": "气势磅礴" },
{ "word": "丰富多彩" },
{ "word": "五彩斑斓" },
{ "word": "大放异彩" },
{ "word": "相得益彰" },
{ "word": "相辅相成" },
{ "word": "扎实推进" },
{ "word": "稳步推进" },
{ "word": "持续推进" },
{ "word": "纵深推进" },
{ "word": "提质增效" },
{ "word": "降本增效" },
{ "word": "凝心聚力" },
{ "word": "走深走实" },
{ "word": "行稳致远" },
{ "word": "砥砺前行" }
],
"connective": [
{ "word": "首先" },
{ "word": "其次" },
{ "word": "再次" },
{ "word": "最后" },
{ "word": "此外" },
{ "word": "另外" },
{ "word": "与此同时" },
{ "word": "综上所述" },
{ "word": "综上" },
{ "word": "总而言之" },
{ "word": "总的来说" },
{ "word": "总之" },
{ "word": "由此可见" },
{ "word": "也就是说" },
{ "word": "换句话说" },
{ "word": "换言之" },
{ "word": "简而言之" },
{ "word": "简言之" },
{ "word": "具体而言" },
{ "word": "具体来说" },
{ "word": "总体而言" },
{ "word": "整体而言" },
{ "word": "综合来看" },
{ "word": "一方面" },
{ "word": "另一方面" },
{ "word": "不仅如此" },
{ "word": "除此之外" },
{ "word": "值得注意的是" },
{ "word": "需要注意的是" },
{ "word": "值得一提的是" },
{ "word": "更重要的是" },
{ "word": "毫无疑问" },
{ "word": "毋庸置疑" },
{ "word": "不可否认" },
{ "word": "众所周知" },
{ "word": "显然" },
{ "word": "无疑" },
{ "word": "事实上" },
{ "word": "进一步" }
]
}
Loading
Loading