Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Changelog / 更新日志

## 2026-08-06 — FFmpeg Agent permission boundary / FFmpeg Agent 权限边界

### 中文

#### 权限与产品边界

- Agent 可见的 FFmpeg 工具收紧为两个只读操作:`ffmpeg_probe` 和
`ffmpeg_loudness_analyze`。
- 移除 Agent 对 `ffmpeg_prepare_audio` 与 `ffmpeg_loudness_normalize` 的定义和执行路由,
防止对话模型未经用户确认直接生成音频文件。
- 写入能力没有从 Runtime 中删除:`prepare` 与 `loudness_normalize` 继续由
`pi_ffmpeg_job_start` C ABI 提供给 `pi-desktop`,由 Desktop 显示计划并获得用户确认。
- FFmpeg 组件安装、更新和卸载仍然只能由用户触发;Agent 不获得组件生命周期写权限。

#### 兼容性

- 保持现有六个 FFmpeg/组件 C ABI 导出及 JSON 请求、任务状态和错误结构不变。
- 保留四个有限白名单操作:探测、PCM WAV 准备、EBU R128 响度分析和双遍响度标准化。
- 未增加任意 FFmpeg 参数、远程 URL、系统 PATH 修改或开放式滤镜图入口。

#### 测试与文档

- 新增精确工具面测试,断言 Agent 只能看到两个只读工具。
- 新增拒绝测试,断言两个音频写入工具在执行前以未知工具失败。
- README 和 FFmpeg 组件契约增加 Desktop 确认写入的责任边界说明。
- 验证结果:Rust 工作区 57 项测试通过、2 项按设计忽略;Clippy `-D warnings` 通过;
本机真实 FFmpeg 四个白名单操作通过。

### English

#### Permission and product boundary

- Reduced the Agent-visible FFmpeg surface to two read-only operations:
`ffmpeg_probe` and `ffmpeg_loudness_analyze`.
- Removed the Agent definitions and execution routes for `ffmpeg_prepare_audio` and
`ffmpeg_loudness_normalize`, preventing a conversational model from creating audio files
without a user-reviewed Desktop action.
- Preserved the write-capable Runtime path: `prepare` and `loudness_normalize` remain available
through `pi_ffmpeg_job_start` for `pi-desktop`, which presents the plan and asks the user to
confirm it.
- Component install, update, and uninstall remain user-triggered lifecycle actions; the Agent
receives no component lifecycle mutation capability.

#### Compatibility

- Kept the existing six FFmpeg/component C ABI exports and their JSON request, job status, and
structured error contracts unchanged.
- Preserved all four finite allow-listed operations: probe, PCM WAV preparation, EBU R128
loudness analysis, and two-pass loudness normalization.
- Added no arbitrary FFmpeg arguments, remote URLs, global `PATH` changes, or open-ended filter
graph execution.

#### Tests and documentation

- Added an exact tool-surface test proving that the Agent sees only the two read-only tools.
- Added rejection coverage proving that both audio-writing tool names fail before execution.
- Updated the README and FFmpeg component contract with the Desktop confirmation boundary.
- Validation: 57 Rust workspace tests passed with 2 intentionally ignored; Clippy passed with
`-D warnings`; all four allow-listed operations passed against a real local FFmpeg build.
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ audio preparation and loudness operations, plus user-triggered install, update,
and uninstall lifecycle jobs. It does not change the global `PATH`, execute
caller-supplied FFmpeg arguments, or accept remote input URLs.

The Agent can only probe files and analyze loudness. Creating prepared or
normalized WAV files is reserved for `pi-desktop` through the C ABI, where the
user reviews the operation and confirms the write.

See [the FFmpeg component contract](docs/ffmpeg-component.md) for the C-ABI,
job JSON, release manifest, safety rules, licensing, and the hand-off contract
for `pi-desktop`.
Expand Down
51 changes: 16 additions & 35 deletions crates/pi-agent-ffi/src/component_ffi.rs
Original file line number Diff line number Diff line change
Expand Up @@ -652,7 +652,8 @@ pub unsafe extern "C" fn pi_job_destroy(job: *mut PiJob) {
}
}

/// Agent-visible processing tools. Lifecycle mutation is intentionally absent.
/// Agent-visible, read-only FFmpeg tools. Audio-file creation remains a
/// Desktop-owned action so the user can review and confirm it in the UI.
pub(crate) struct FfmpegTools {
config: FfmpegConfig,
}
Expand Down Expand Up @@ -682,21 +683,11 @@ impl ToolExecutor for FfmpegTools {
description: "Inspect one absolute local media file and return structured audio stream facts.".into(),
input_schema_json: r#"{"type":"object","additionalProperties":false,"properties":{"input":{"type":"string"}},"required":["input"]}"#.into(),
},
ToolDefinition {
name: "ffmpeg_prepare_audio".into(),
description: "Create a new contained WAV with optional trim, sample rate, mono/stereo, and s16/s24/f32 PCM conversion.".into(),
input_schema_json: r#"{"type":"object","additionalProperties":false,"properties":{"input":{"type":"string"},"output_name":{"type":"string"},"sample_rate":{"type":"integer","minimum":8000,"maximum":192000},"channels":{"type":"integer","minimum":1,"maximum":2},"sample_format":{"type":"string","enum":["s16","s24","f32"]},"start_seconds":{"type":"number","minimum":0},"duration_seconds":{"type":"number","exclusiveMinimum":0}},"required":["input","output_name"]}"#.into(),
},
ToolDefinition {
name: "ffmpeg_loudness_analyze".into(),
description: "Measure EBU R128 integrated loudness, true peak, loudness range, and threshold for a local file.".into(),
input_schema_json: r#"{"type":"object","additionalProperties":false,"properties":{"input":{"type":"string"}},"required":["input"]}"#.into(),
},
ToolDefinition {
name: "ffmpeg_loudness_normalize".into(),
description: "Create a two-pass EBU R128 normalized WAV; all loudness targets must be explicit.".into(),
input_schema_json: r#"{"type":"object","additionalProperties":false,"properties":{"input":{"type":"string"},"output_name":{"type":"string"},"target_lufs":{"type":"number","minimum":-70,"maximum":-5},"max_true_peak_db":{"type":"number","minimum":-9,"maximum":0},"target_lra":{"type":"number","minimum":1,"maximum":50}},"required":["input","output_name","target_lufs","max_true_peak_db","target_lra"]}"#.into(),
},
]
}

Expand All @@ -707,9 +698,7 @@ impl ToolExecutor for FfmpegTools {
};
let operation = match call.tool_name.as_str() {
"ffmpeg_probe" => "probe",
"ffmpeg_prepare_audio" => "prepare",
"ffmpeg_loudness_analyze" => "loudness_analyze",
"ffmpeg_loudness_normalize" => "loudness_normalize",
other => return Ok(tool_error(call, format!("unknown FFmpeg tool {other}"))),
};
let Some(object) = args.as_object_mut() else {
Expand Down Expand Up @@ -836,25 +825,25 @@ mod tests {
}

#[test]
fn agent_arguments_reject_wrong_optional_types_before_execution() {
fn agent_rejects_audio_creation_tools_before_execution() {
let tools = FfmpegTools {
config: FfmpegConfig::default(),
};
let result = tools
.execute(&ToolCall {
id: "call-1".into(),
tool_name: "ffmpeg_prepare_audio".into(),
arguments_json:
r#"{"input":"C:\\audio.wav","output_name":"out.wav","sample_rate":"44100"}"#
.into(),
})
.unwrap();
assert!(result.is_error);
assert!(result.result_json.contains("invalid arguments"));
for tool_name in ["ffmpeg_prepare_audio", "ffmpeg_loudness_normalize"] {
let result = tools
.execute(&ToolCall {
id: "call-1".into(),
tool_name: tool_name.into(),
arguments_json: r#"{"input":"C:\\audio.wav","output_name":"out.wav"}"#.into(),
})
.unwrap();
assert!(result.is_error);
assert!(result.result_json.contains("unknown FFmpeg tool"));
}
}

#[test]
fn agent_surface_contains_processing_only() {
fn agent_surface_contains_read_only_tools_only() {
let tools = FfmpegTools {
config: FfmpegConfig::default(),
};
Expand All @@ -863,14 +852,6 @@ mod tests {
.into_iter()
.map(|tool| tool.name)
.collect::<Vec<_>>();
assert_eq!(
names,
[
"ffmpeg_probe",
"ffmpeg_prepare_audio",
"ffmpeg_loudness_analyze",
"ffmpeg_loudness_normalize",
]
);
assert_eq!(names, ["ffmpeg_probe", "ffmpeg_loudness_analyze"]);
}
}
8 changes: 8 additions & 0 deletions docs/ffmpeg-component.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,14 @@ managed-then-PATH resolution in `auto` mode.
{"operation":"loudness_normalize","input":"C:\\audio\\take.wav","output_name":"take-normalized.wav","target_lufs":-16,"max_true_peak_db":-1.5,"target_lra":11}
```

## Agent permission boundary

The Agent-visible tool surface is deliberately read-only: it exposes only
`ffmpeg_probe` and `ffmpeg_loudness_analyze`. `prepare` and
`loudness_normalize` remain available through `pi_ffmpeg_job_start` for
`pi-desktop`, which must show the operation to the user and obtain their
confirmation before it starts a job that writes a WAV file.

`pi_job_status_json` returns a stable envelope. `progress` ranges from `0` to
`1`; it is `1` on success. `phase` reports the active lifecycle or processing
phase, and `error` is a structured object with `code`, `message`, and optional
Expand Down
Loading