diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 672186cc..fb73dcc1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -265,7 +265,9 @@ jobs: # 不加參數:dart_test.yaml 讓 live 預設跳過(ADR 0015 §決定 3)。 # 身分測試要 cmake,ubuntu runner 內建。插件執行環境的測試跑真的 # QuickJS:test/support/quickjs.dart 預先載入 flutter_js 內附的 .so, - # 不用先建置桌面版;載入失敗會讓測試失敗,不會跳過。 + # 不用先建置桌面版;載入失敗會讓測試失敗,不會跳過。插件契約執行器 + # (test/plugins/contract/contract_test.dart,ADR 0015 §決定 6)也在 + # 這一步:它以 fixture 重播跑 test/fixtures/plugins/ 的每個測試插件。 - name: Unit and widget tests run: flutter test diff --git a/.trellis/spec/app/plugins/index.md b/.trellis/spec/app/plugins/index.md index 300f64f9..4d7dbc12 100644 --- a/.trellis/spec/app/plugins/index.md +++ b/.trellis/spec/app/plugins/index.md @@ -27,6 +27,15 @@ lib/plugins/ plugin_installer.dart # PluginInstaller:解析 → 載入 → 寫入 → 註冊 dev_plugin_entry.dart # dev 的開發入口 types/fmp-plugin.d.ts # 給插件作者的 TypeScript 型別 + +test/plugins/contract/ # 契約執行器(只在測試裡,理由見 PR 9b 的 research/notes.md) + contract_runner.dart # PluginDirectory、runContract/runCheck/recordContract + checks.dart # checks.json、checkShapes、期望的比對 + fixture.dart # fixture 格式、fixtureShapes、遮蔽、重播的網址比對 + fixture_adapters.dart # ReplayAdapter、RecordingAdapter(dio 最底層的 adapter) + credential_scan.dart # fixture、log、串流 headers 的憑證檢查 + contract_test.dart # 重播的入口(FMP_PLUGIN_DIR) + record_test.dart # 錄製(live 測試是真實連線的入口) ``` ## 寫一個插件 @@ -69,7 +78,48 @@ export async function resolveStream({ sourceId, cid, formats }) { 對應表);`fmp.http.request` 自己丟的錯誤(網域不符、限流重試後仍失敗、傳輸錯誤)直接讓它往上拋。 - 要跨重啟的值(匿名 cookie 等)存 `fmp.storage`;`fmp.credentials.get()` 在 M1 一律是 `null`。 - 沒有 `setTimeout`、`fetch`、`require`,也不能 `import` 其他 module:一個檔案就是全部。 -- 用 `fmp-test` 當範本:`app/test/fixtures/plugins/test_plugin/test_plugin.js`。 +- 用 `fmp-test` 當範本:`app/test/fixtures/plugins/test_plugin/test_plugin.js`;會發請求的範本是 + `app/test/fixtures/plugins/http_test_plugin/`。 + +## 寫檢查案例與 fixture + +插件目錄是契約檢查的單位(格式在 `fmp-plugin.d.ts` 的 `FmpChecks`、`FmpFixture`): + +``` +<插件目錄>/ + <名稱>.js # 安裝檔,只能有一個 + checks.json # 每能力最多一條 + fixtures/<能力>/001.json … # 該案例依序的請求與回應 +``` + +```json +{ + "search": { + "input": { "keyword": "tone", "page": 1 }, + "expect": { "minItems": 2, "nonEmpty": ["sourceId", "title"] } + }, + "resolveStream": { + "input": { + "sourceId": "a1", + "purpose": "playback", + "formats": [{ "container": "mp4", "codec": "aac" }] + }, + "expect": { "error": "Unavailable", "reason": "copyright" } + } +} +``` + +- 成功的案例盡量用錄的:`FMP_PLUGIN_DIR=<絕對路徑> flutter test --run-skipped --tags live + test/plugins/contract/record_test.dart`(`app/` 內;真實連線,照 ADR 0027 §決定 2 回報)。 + 那個能力原本的 fixture 整組重寫,但只在結果符合 `expect` 時寫(不符就不動原本的檔案並回報); + 需要登入的案例錄不了(M1 沒有憑證,會以 `AuthRequired` 失敗)。 +- 錯誤案例(風控、下架)多半錄不到:手寫或把錄到的改掉,在 `meta.edited` 寫理由,錄製就不會蓋掉 + 那個案例。手寫的 fixture 也要是遮過的樣子(值寫 `***`),掃描不會放過。 +- 會變的 query 參數(時間戳、簽名)在 fixture 裡寫 `***` 就不比值;遮蔽名單上的參數錄的時候 + 已經是 `***`。 +- 跑:`FMP_PLUGIN_DIR=<絕對路徑> flutter test test/plugins/contract/contract_test.dart`;沒設 + `FMP_PLUGIN_DIR` 就是跑 `app/` 內的測試插件(裸 `flutter test` 已包含)。失敗訊息列出每一條 + 違反,例如 `search: request #1 (GET …) does not match fixtures/search/001.json (GET …)`。 ## 在 App 裡試插件 @@ -124,10 +174,15 @@ export async function resolveStream({ sourceId, cid, formats }) { 回訊息再拋錯),不在正式程式碼留開關。 - 走 Riverpod 的測試(安裝、清單)用 `ProviderContainer(retry: (_, _) => null, ...)`:App 關了重試, 不關的話失敗的 provider 會一直重試到測試逾時。 -- 錯誤 fixture 的契約測試、`checks.json` 在 PR 9b。 +- 契約執行器本身(`test/plugins/contract/`):改它的檢查時,在 `contract_runner_test.dart` 以 + `copyPlugin`/`edit`(`plugin_copy.dart`)在測試插件的副本上造一個會紅的變異,並留一個改無關處 + 不紅的案例。`edit` 找不到要換的字串會讓測試失敗(Windows checkout 可能是 CRLF,只換單行)。 ## Quality Check -- `test/plugins/` 全綠;新的宿主 API 有隔離案例;`type_definitions_test.dart` 綠。 +- `test/plugins/` 全綠(含契約執行器);新的宿主 API 有隔離案例;`type_definitions_test.dart` 綠。 - `lib/plugins/` 以外沒有 import `flutter_js`;沒有網址字面值(`fmp_url_literal`)。 - 改了 `fmp` 形狀或 DTO:`fmp-plugin.d.ts` 一起改,`app/AGENTS.md` § 插件若規則變了也改。 +- 加了 DTO 欄位:`test/plugins/contract/checks.dart` 的 `trackFields`/`candidateFields` 一起加 + (`checks_test.dart` 比對);`SourcePlugin` 加了能力的方法:`checkShapes['FmpChecks']`、 + `PluginCheck.run` 與 d.ts 的 `FmpChecks` 一起加。 diff --git a/.trellis/spec/app/testing/index.md b/.trellis/spec/app/testing/index.md index 099df946..abeb84f1 100644 --- a/.trellis/spec/app/testing/index.md +++ b/.trellis/spec/app/testing/index.md @@ -13,7 +13,8 @@ | 單元 | 純邏輯、平台層實作(注入路徑與 callback,在暫存目錄上跑) | `test/platform/app_data_directory_test.dart` | | widget | 畫面;依賴以建構子或 provider override 注入 | `test/app/fmp_app_test.dart` | | 建置設定 | 原生身分:能執行就執行(`cmake -P`),不能就解析設定檔並附變異案例 | `test/identity/` | -| 插件契約、整合、golden | M1 PR 9、PR 13 與設計系統元件加入時再寫 | — | +| 插件契約 | 插件目錄的 `checks.json` 以 fixture 重播(寫法見 `.trellis/spec/app/plugins/index.md` § 寫檢查案例與 fixture) | `test/plugins/contract/contract_test.dart` | +| 整合、golden | M1 PR 13 與設計系統元件加入時再寫 | — | - 碰資料庫的測試用 `test/support/memory_database.dart`,寫法見 `.trellis/spec/app/data/index.md` § 測試。 diff --git a/.trellis/tasks/09-28-m1-skeleton-tracer/design.md b/.trellis/tasks/09-28-m1-skeleton-tracer/design.md index b342686e..4de89692 100644 --- a/.trellis/tasks/09-28-m1-skeleton-tracer/design.md +++ b/.trellis/tasks/09-28-m1-skeleton-tracer/design.md @@ -84,10 +84,10 @@ app/ i18n/ # slang:zh-TW(base)、zh-CN、en packages/ fmp_lints/ # analysis_server_plugin;analyzer_testing 測試 - plugin_contract/ # 契約執行器(重播 fixture 跑 checks.json) test/ flutter_test_config.dart # HttpOverrides.global 擋真實 HttpClient fixtures/plugins/test_plugin/ # 合成資料、播放本機音檔 + plugins/contract/ # 契約執行器(重播 fixture 跑 checks.json;9b 改放這裡,ADR 0015 更正) integration_test/ ``` @@ -135,7 +135,7 @@ M1 的 drift 表: - `checks.json`。 - 插件以舊專案 `lib/data/sources/bilibili*` 為規格,用 JS 重寫(ADR 0008 檔頭補充)。M1 只需要 `search` 與 `resolveStream` 兩個能力。 - fixture 以真實連線錄製一次(ADR 0027 §決定 2),寫檔前經遮蔽函式;提交前人工確認沒有 cookie 或 token 原值。 -- fmp-plugins 的變更以該 repo 自己的 PR 合併,本機跑 `app/packages/plugin_contract` 的執行器驗證。 +- fmp-plugins 的變更以該 repo 自己的 PR 合併,本機以 `FMP_PLUGIN_DIR=<插件目錄> flutter test test/plugins/contract/contract_test.dart`(在 `app/` 內)驗證。 ## 5. CI(PR 2 起逐步擴充) diff --git a/.trellis/tasks/09-28-m1-skeleton-tracer/implement.md b/.trellis/tasks/09-28-m1-skeleton-tracer/implement.md index 6df3f093..a5f616c5 100644 --- a/.trellis/tasks/09-28-m1-skeleton-tracer/implement.md +++ b/.trellis/tasks/09-28-m1-skeleton-tracer/implement.md @@ -26,14 +26,16 @@ ## 進度與交接(2026-09-30 更新;compact 後從這裡接) -- **已合併進 `main`**:#173(設計文件)、#174(PR 1 指令檔分家)、#175(PR 2 骨架與 CI)、#177(PR 3 lint)、#178(PR 4 平台層)、#179(PR 5 drift)、#180(PR 6 log 與設定)、#181(PR 7 錯誤模型)、#182(PR 8 網路層)。#176 是 CI 路徑探測,已關閉。 +- **已合併進 `main`**:#173(設計文件)、#174(PR 1 指令檔分家)、#175(PR 2 骨架與 CI)、#177(PR 3 lint)、#178(PR 4 平台層)、#179(PR 5 drift)、#180(PR 6 log 與設定)、#181(PR 7 錯誤模型)、#182(PR 8 網路層)、#183(PR 9a JS 執行環境)。#176 是 CI 路徑探測,已關閉。 - **isar/sqlite3 共存探針**:已完成,兩平台共存、全部 16KB 對齊(`research/isar-sqlite3-coexistence.md`,ADR 0010 已補)。 -- **PR 9a 完成**(分支 `feat/js-runtime`,子任務已 archive 到 `.trellis/tasks/archive/2026-09/09-30-js-runtime/`):每插件一個背景 isolate 的 QuickJS、宿主 API v1、manifest、從檔案安裝與 dev 開發入口、測試插件 `fmp-test`;數字在該子任務 `research/notes.md` §4。 +- **PR 9a 完成**(#183,子任務已 archive 到 `.trellis/tasks/archive/2026-09/09-30-js-runtime/`):每插件一個背景 isolate 的 QuickJS、宿主 API v1、manifest、從檔案安裝與 dev 開發入口、測試插件 `fmp-test`;數字在該子任務 `research/notes.md` §4。 - 實機:Windows dev 開發入口裝上、重啟後從資料庫載入、prod 不理會旗標;Android 模擬器 dev 的前兩項。模擬器上的 `com.personal.fmp` 是舊版 1.11.0,prod 沒裝上去驗;prod 那一段由單元測試守(`devPluginPath` 對 prod 一律回 `null`,有變異驗證)。 -- **下一步**:9b(fixture 錄製重播、`checks.json`、契約執行器)→ 9c(建 `1morr/fmp-plugins`、B 站插件、錄一次 fixture)→ YouTube.js 探針(與 10–13 並行)→ 10 播放核心 → 11 verify-on-device → 12 UI → 13 五平台建置與發版 workflow → 里程碑驗收。 -- **擁有者決定**:1–7 都在父任務 `prd.md`「擁有者的決定」。9a 期間新增了兩項: +- **PR 9b 待合併**(分支 `feat/plugin-contract`,子任務已 archive 到 `.trellis/tasks/archive/2026-09/09-30-plugin-contract/`):fixture 錄製重播、`checks.json`、契約執行器;擁有者決定 8 讓執行器可在命令列做免登入的錄製(`live` tag),ADR 0015 §決定 7 已補更正。 +- **之後**:9c(建 `1morr/fmp-plugins`、B 站插件、錄一次 fixture)→ YouTube.js 探針(與 10–13 並行)→ 10 播放核心 → 11 verify-on-device → 12 UI → 13 五平台建置與發版 workflow → 里程碑驗收。 +- **擁有者決定**:1–8 都在父任務 `prd.md`「擁有者的決定」。9a、9b 期間新增了三項: - 決定 6:插件安裝檔是單一 `.js`,開頭帶 `==FMP Plugin==` manifest; - - 決定 7:插件在背景 isolate 執行;逾時先送存活探測,沒回應才停用到重啟。 + - 決定 7:插件在背景 isolate 執行;逾時先送存活探測,沒回應才停用到重啟; + - 決定 8(9b):M1 的 fixture 由契約執行器在命令列錄製,限免登入案例。 - **每個 PR 的固定流程**: 1. 從最新 `main` 開分支; 2. `task.py create … --parent .trellis/tasks/09-28-m1-skeleton-tracer --package app --no-start`; @@ -150,7 +152,7 @@ ### 9. JS 執行環境與插件(拆成 9a、9b、9c 三個 PR) - **9a** JS 執行環境、宿主 API v1、manifest 解析(安裝檔格式見 prd 擁有者決定 6)、`SourcePlugin` 轉接、從檔案安裝、測試插件;`flutter_js` 實測(含能否在 `flutter test` 內載入 QuickJS)。 -- **9b** fixture 錄製與重播 adapter、`checks.json`、`packages/plugin_contract` 契約執行器、CI 步驟。 +- **9b** fixture 錄製與重播 adapter、`checks.json`、契約執行器(放在 `app/test/plugins/contract/`)、CI 步驟。 - **9c** 在 `1morr/fmp-plugins` 建 repo 與 B 站插件(`search`、`resolveStream`),錄一次 fixture;FMP 端無程式碼,或只有文件。 原清單: @@ -161,7 +163,7 @@ - `apiVersion` 相容檢查; - 從檔案安裝。 - [x] TypeScript 型別定義(9a)。 -- [ ] `packages/plugin_contract/`:契約執行器、fixture 格式、`checks.json`。 +- [x] 契約執行器、fixture 格式、`checks.json`(9b,`app/test/plugins/contract/`)。 - [x] `test/fixtures/plugins/test_plugin/`:合成資料、本機音檔(9a)。 - [ ] 接真實 B 站時觀察:伺服器回不合法的 `Set-Cookie` 是否讓請求變成 `UnexpectedError`(`dio_cookie_manager` 的 `ignoreInvalidCookies` 預設 false;PR 8 檢查提出,沒有重現案例前不改)。 - [ ] 建立 `1morr/fmp-plugins`: @@ -181,6 +183,13 @@ - [ ] Android 的跨 isolate 成本:模擬器 debug 下每次宿主呼叫多約 7 ms,不經宿主的 `search` 也比純 Dart 對照慢,差額未查明。有實機時用 profile 模式重量(`integration_test/plugin_runtime_benchmark_test.dart`)。 - [ ] Android 上 prod 的開發入口實機驗證:需要一台沒裝舊版的模擬器或實機(PR 13 或 M9 前)。 +9b 留下的後續: + +- [ ] 錄製與重播的 adapter 現在在 `app/test/plugins/contract/`;M3 的 App 內開發工具要用時移進 `lib/core/network/`,格式不變。 +- [ ] 執行器看不到插件自己吞掉的「網域不在清單」錯誤(網路層照樣不送出),也分不出插件自拋的 `ParseError` 與 DTO 驗證失敗;要看到前者得讓網路層替被拒的請求寫紀錄。 +- [ ] 串流候選的網址本身(query 裡的 `access_key` 之類)不在憑證檢查內,現在只查 headers。9c 接 B 站時看要不要加。 +- [ ] 9b 審查時有一次 `flutter test` 兩個契約測試檔卡在 loading(編譯階段,20 分鐘無進度),之後 14 次沒重現。CI 若出現同樣的卡住,從 flutter_tools 層查。 + ### 探針:YouTube.js(擁有者決定 3) - [ ] 9 合併後開一個子任務,時限約 2–3 個 session。 diff --git a/.trellis/tasks/09-28-m1-skeleton-tracer/prd.md b/.trellis/tasks/09-28-m1-skeleton-tracer/prd.md index 289b9a3b..52bba638 100644 --- a/.trellis/tasks/09-28-m1-skeleton-tracer/prd.md +++ b/.trellis/tasks/09-28-m1-skeleton-tracer/prd.md @@ -61,6 +61,7 @@ 6. **插件安裝檔格式**(2026-09-30):單一 `.js` 檔,開頭以 `/* ==FMP Plugin== … ==/FMP Plugin== */` 包一段 JSON manifest,App 不執行腳本就能讀出 manifest 並先檢查網域與能力(沿用使用者腳本 metadata block 的慣例;MusicFree、LX Music 也是一檔一插件)。插件庫可把 manifest 另存、由腳本合併成安裝檔;圖示只能用網址或內嵌 base64。 7. **插件在背景 isolate 執行**(2026-09-30):`flutter_js` 0.8.7 的 QuickJS 沒有中斷機制,同步無窮迴圈會凍住 UI isolate(PR 9a 實測)。每個插件的 JS 執行環境放在自己的背景 isolate;呼叫逾時先送存活探測:背景 isolate 有回應就只讓這次呼叫以 `NetworkError` 失敗(只是在等網路),沒有回應或 isolate 已結束才把插件標成「沒有回應」並停用到 App 重啟(2026-09-30 擁有者確認);運算重的腳本(YouTube 解簽名)也不會讓畫面卡頓。不換引擎:可中斷的替代套件(`flutter_qjs_next`、`quickjs_engine`)都是單一作者、低採用的新套件。 +8. **M1 的 fixture 錄製**(2026-09-30):契約執行器加錄製模式,打 `live` tag、手動執行、只給免登入的案例,寫檔前經遮蔽;需要登入的錄製仍在 App 內(M3)。理由:App 內的插件開發工具在 M3,9c 要先錄 B 站 fixture,過時也要能重錄;ADR 0015 §決定 4 的冒煙測試本來就在命令列用真實連線。 ## 需求 diff --git a/.trellis/tasks/09-28-m1-skeleton-tracer/task.json b/.trellis/tasks/09-28-m1-skeleton-tracer/task.json index b7adb204..2464ebc7 100644 --- a/.trellis/tasks/09-28-m1-skeleton-tracer/task.json +++ b/.trellis/tasks/09-28-m1-skeleton-tracer/task.json @@ -27,7 +27,8 @@ "09-29-logging-settings", "09-29-error-model", "09-29-network-layer", - "09-30-js-runtime" + "09-30-js-runtime", + "09-30-plugin-contract" ], "parent": "09-26-fmp-rewrite", "relatedFiles": [], diff --git a/.trellis/tasks/archive/2026-09/09-30-plugin-contract/check.jsonl b/.trellis/tasks/archive/2026-09/09-30-plugin-contract/check.jsonl new file mode 100644 index 00000000..3b3d1b95 --- /dev/null +++ b/.trellis/tasks/archive/2026-09/09-30-plugin-contract/check.jsonl @@ -0,0 +1,8 @@ +{"file": "docs/adr/0015-testing-gates-and-dev-environment.md", "reason": "Fixture format, check cases, contract runner, recording correction"} +{"file": "docs/adr/0014-script-source-plugins.md", "reason": "Plugin model, manifest, host API v1, contract tests"} +{"file": "docs/adr/0011-settings-and-logging.md", "reason": "Redaction applied before fixtures are written"} +{"file": ".trellis/spec/app/plugins/index.md", "reason": "How plugins and the runtime are built (9a)"} +{"file": ".trellis/spec/app/network/index.md", "reason": "SourceHttpClient and createAdapter injection point"} +{"file": ".trellis/spec/app/testing/index.md", "reason": "Test conventions, live tag, zero-network guard"} +{"file": ".trellis/spec/app/lints/index.md", "reason": "Lint rules and layer imports"} +{"file": ".trellis/tasks/archive/2026-09/09-30-js-runtime/research/notes.md", "reason": "flutter_js facts, QuickJS in flutter test"} diff --git a/.trellis/tasks/archive/2026-09/09-30-plugin-contract/implement.jsonl b/.trellis/tasks/archive/2026-09/09-30-plugin-contract/implement.jsonl new file mode 100644 index 00000000..3b3d1b95 --- /dev/null +++ b/.trellis/tasks/archive/2026-09/09-30-plugin-contract/implement.jsonl @@ -0,0 +1,8 @@ +{"file": "docs/adr/0015-testing-gates-and-dev-environment.md", "reason": "Fixture format, check cases, contract runner, recording correction"} +{"file": "docs/adr/0014-script-source-plugins.md", "reason": "Plugin model, manifest, host API v1, contract tests"} +{"file": "docs/adr/0011-settings-and-logging.md", "reason": "Redaction applied before fixtures are written"} +{"file": ".trellis/spec/app/plugins/index.md", "reason": "How plugins and the runtime are built (9a)"} +{"file": ".trellis/spec/app/network/index.md", "reason": "SourceHttpClient and createAdapter injection point"} +{"file": ".trellis/spec/app/testing/index.md", "reason": "Test conventions, live tag, zero-network guard"} +{"file": ".trellis/spec/app/lints/index.md", "reason": "Lint rules and layer imports"} +{"file": ".trellis/tasks/archive/2026-09/09-30-js-runtime/research/notes.md", "reason": "flutter_js facts, QuickJS in flutter test"} diff --git a/.trellis/tasks/archive/2026-09/09-30-plugin-contract/prd.md b/.trellis/tasks/archive/2026-09/09-30-plugin-contract/prd.md new file mode 100644 index 00000000..fbfb3308 --- /dev/null +++ b/.trellis/tasks/archive/2026-09/09-30-plugin-contract/prd.md @@ -0,0 +1,66 @@ +# 插件契約執行器與 fixture 重播(M1 PR 9b) + +父任務:`../09-28-m1-skeleton-tracer`(implement「9.」的 9b)。前一個 PR:9a(#183,封存在 `../archive/2026-09/09-30-js-runtime/`)。 + +依據: + +- ADR 0015 §決定 4(檢查案例一份四用)、§5(fixture)、§6(契約執行器)、§7(App 內插件開發工具); +- ADR 0014 §如何確認(契約測試); +- ADR 0011(遮蔽)、0012(媒體請求不帶憑證、只連 manifest 網域)、0013(錯誤對到 `AppError` 類別); +- ADR 0027 §決定 2(真實連線的條件)。 + +## 做什麼 + +1. **fixture 的錄製與重播**:放在 `SourceHttpClient` 最底層的 dio `HttpClientAdapter`,接 PR 8 的 `createAdapter` 注入點。 + - 一次請求與回應存成一個 JSON 檔,內容是 `meta`、`request`、`response`。欄位形狀參考 WireMock 的 stub mapping。 + - 錄製:寫檔前一律經正式的遮蔽函式(ADR 0011)。 + - 重播: + - 比對 method、排序後的 URL、請求順序; + - 被遮蔽的欄位不參與比對; + - 比對不到就讓測試失敗,不發真實請求。 + - 錯誤案例可手改 fixture,並在 `meta` 標記。 +2. **`checks.json`**:每插件每能力最多一條檢查案例,內容是能力名、輸入、期望。 + - 期望分兩種:成功時的形狀條件,例如至少 N 筆、欄位非空;錯誤時的 `AppError` 類別。 + - 格式寫進 `fmp-plugin.d.ts` 的旁邊或同一份文件,給插件作者看。 +3. **契約執行器**:原定放在 `app/packages/plugin_contract/`,實作時改放 `app/test/plugins/contract/`(理由見 `research/notes.md`,ADR 0015 §決定 6 已補更正);用重播跑一個插件目錄裡的每個案例,並斷言: + - 能力與匯出一致; + - DTO 驗證通過; + - 案例的期望; + - 錯誤對到 `AppError` 類別; + - 媒體請求不帶憑證(M1 沒有媒體 client,就對 `resolveStream` 回傳的 headers 做檢查,寫明); + - 只連 manifest 宣告的網域; + - 插件的 log 經過遮蔽。 + + 執行方式: + - QuickJS 要在 `flutter test` 裡跑(9a 的結論),所以執行器以 `flutter test` 為入口,插件目錄用參數或環境變數傳入; + - `1morr/fmp-plugins` 的 CI 以固定的 FMP 版本執行; + - 指令寫進 `app/AGENTS.md` 與 `.trellis/spec/app/plugins/index.md`。 +4. **`app/` 內的案例**: + - 9a 的測試插件 `fmp-test` 補上 `checks.json`。 + - 另外加一個會發 HTTP 請求的測試插件,網域用 `*.test`,fixture 手寫,讓重播、網域、遮蔽的斷言真的被執行到。 + - 執行器自己的測試要有雙向變異驗證:造出違規讓它失敗,例如連到清單外的網域、log 沒遮蔽、期望不符、fixture 比對不到;再改一個無關處證明它不會失敗。 +5. **fixture 掃描**:`app/` 內所有 fixture 都不得有未遮蔽的憑證(ADR 0015 §如何確認)。 + - 用遮蔽函式重跑一次,結果要和原檔相同; + - 另外比對常見憑證欄位。 + - 要有雙向變異驗證。 +6. **CI**:`app` job 加上契約執行器的步驟,或確認它已包含在 `flutter test` 內,二擇一並寫明。 +7. **錄製模式**(擁有者 2026-09-30 決定,見父任務 prd 決定 8): + - 執行器加錄製模式:真實連線跑同一份 `checks.json`,經遮蔽後寫進插件目錄的 `fixtures/`。 + - 打上 `live` tag,只有明確帶 `--run-skipped --tags live` 才執行,CI 不跑。 + - 只給不需要登入的案例用;需要登入的錄製留在 App 內(M3)。 + - 本 PR 不對真實音源執行錄製,錄製模式的測試用本機假伺服器或假 adapter。實際錄 B 站在 9c。 + - ADR 0015 §決定 7 補一句更正:命令列可以做免登入的錄製。 + +## 驗收 + +- [ ] `app/` 驗證清單全過: + - format; + - build_runner 後沒有實質變動; + - `dart analyze --fatal-infos`、`flutter analyze`; + - `flutter test`; + - 哨兵。 +- [ ] 執行器對兩個測試插件都通過。每種違規都有會紅的測試,也有證明不會誤紅的測試。 +- [ ] fixture 掃描有雙向變異驗證。 +- [ ] CI 的 `app` job 綠,而且跑到契約執行器。 +- [ ] 錄製模式:對本機假伺服器錄出的 fixture 經過遮蔽,再用重播模式跑同一份案例會通過;不帶 `--tags live` 時不執行。 +- [ ] 用執行器跑外部插件目錄的指令寫進文件,並實際跑過一次:把測試插件複製到 `app/` 外的暫存目錄。 diff --git a/.trellis/tasks/archive/2026-09/09-30-plugin-contract/research/notes.md b/.trellis/tasks/archive/2026-09/09-30-plugin-contract/research/notes.md new file mode 100644 index 00000000..ce53dd6e --- /dev/null +++ b/.trellis/tasks/archive/2026-09/09-30-plugin-contract/research/notes.md @@ -0,0 +1,176 @@ +# 契約執行器與 fixture 的設計查證(M1 PR 9b) + +- 日期:2026-09-30 +- 環境:Flutter 3.47.5(Dart 3.13.4)、Windows 11。 +- 來源:dio 5.11.1、dio_cookie_manager 3.5.0、drift 2.35.0 的 pub cache 原始碼;成熟工具的官方文件 + (下列網址,2026-09-30 以 raw GitHub 取得)。 + +## 1. 執行器放在哪:`app/test/plugins/contract/`,不是 `app/packages/plugin_contract/` + +prd 寫的是 workspace 裡的獨立套件。實作時改成 `app/` 自己的測試目錄,理由是具體的: + +1. **要在 `flutter test` 裡跑 QuickJS,而那個準備是測試目錄層級的**:`test/flutter_test_config.dart` + 預先載入 flutter_js 的原生庫(`test/support/quickjs.dart`)並裝零聯網的 `HttpOverrides`。 + 獨立套件要在自己的 `test/` 各抄一份;而且 `quickjs.dart` 從工作目錄的 + `.dart_tool/package_config.json` 找 flutter_js,workspace 成員的工作目錄沒有那個檔(只在 + workspace 根 `app/`),要另外改。 +2. **依賴方向會成環**:執行器要 `package:fmp` 的載入器、網路層(`createAdapter` 注入點)、 + `Redactor`、`AppDatabase`。套件依賴 `fmp`;`app/` 的 CI 又要對 `test/fixtures/plugins/` 跑它 + (ADR 0015 §決定 6),`fmp` 就得 dev 依賴這個套件:`fmp` → `plugin_contract` → `fmp`。不成環 + 的做法是 CI 另在套件目錄跑一次 `flutter test`,等於為了目錄位置多一個步驟。 +3. **分層 lint**:dio 只准在 `lib/core/network/`、drift 只准在 `lib/data/`(`fmp_layer_imports`, + 以 package 根算路徑)。套件的 `lib/` 放錄製與重播的 adapter、記憶體資料庫就會違規或要加例外; + 放在測試目錄沒有這個問題(`test/support/fake_http_adapter.dart` 已是先例)。 +4. **插件庫的 CI 用法一樣簡單**:checkout FMP 的固定 ref,在 `app/` 內 + `FMP_PLUGIN_DIR=<插件目錄> flutter test test/plugins/contract/contract_test.dart`。 + 套件方案也是 checkout 整個 FMP 再進某個目錄跑,沒有比較省。 + +也沒有放進 `lib/`:錄製與重播的 adapter、checks 的模型現在只有測試在用。ADR 0015 §決定 7 的 App +內插件開發工具(M3)要在 App 裡切換真實、錄製、重播時,再把 `fixture.dart`、 +`fixture_adapters.dart` 移進 `lib/core/network/`(dio 的擁有者),格式不變。 + +ADR 0015 §決定 6 寫「一個通用套件」:實際是 `app/` 測試目錄裡的一組檔案。要不要在 ADR 加一行 +更正,由主 session 決定。 + +## 2. fixture 格式:WireMock stub mapping + +WireMock(`https://raw.githubusercontent.com/wiremock/wiremock.org/main/src/content/docs/docs/stubbing.mdx`) +一個 stub 是 `{"request": {"method", "url"}, "response": {"status", "headers", "body" | "jsonBody" | "base64Body"}}`, +`jsonBody` 讓 JSON 回應不必跳脫、可以直接手改。採用它的欄位名稱,外層加 ADR 0015 §決定 5 的 `meta`: + +```json +{ + "meta": { "recordedAt": "2026-09-30T00:00:00.000Z" }, + "request": { + "method": "GET", + "url": "https://api.fmp.test/search?page=1&keyword=tone&access_key=***", + "headers": { "referer": "https://www.fmp.test/" } + }, + "response": { + "status": 200, + "headers": { + "content-type": ["application/json"], + "set-cookie": ["demo_session=***; Path=/"] + }, + "jsonBody": { "demo_session": "***", "list": [], "more": false } + } +} +``` + +- 沒採用的 WireMock 欄位:`urlPath`、`queryParameters` 等比對器(比對規則固定,見 §3)、 + `base64Body`(見 §4)、`bodyFileName`(一個請求一個檔就夠)。 +- response headers 是「名稱(小寫)→ 字串陣列」,和宿主給插件的 `HttpResponse.headers` 相同; + request 的 `headers`、`body` 只供閱讀,不比對。 +- 一個請求一個檔(ADR 0015 §決定 5),放 `fixtures/<能力>/`,依檔名排序是請求順序,錄製寫 + `001.json`、`002.json`…。一個目錄相當於 VCR 的一卷 cassette。 +- `meta.recordedAt`:錄製時間;`meta.edited`:手寫或手改的理由(prd「錯誤案例可手改 fixture, + 並在 meta 標記」)。錄製遇到有 `edited` 的案例整個略過,不蓋掉手寫的錯誤案例。 + +## 3. 重播的比對:Polly.js 與 VCR + +- Polly.js(`https://raw.githubusercontent.com/Netflix/pollyjs/master/docs/configuration.md` + § matchRequestsBy)預設比 method、headers、body、order、url(query 是「Sorted query + string」)。VCR(`https://raw.githubusercontent.com/vcr/vcr/master/features/request_matching/README.md`) + 預設只比 method 與 URI,`:query` 比對不分順序;同樣的請求依序拿不同的回應。 +- 採用:**method+網址(query 不分順序)+順序**(ADR 0015 §決定 5 也這樣寫)。headers、body + 不比:headers 帶 cookie、UA 這類會變的值,照 Polly 的預設比 headers 只會讓錄好的 fixture 很快 + 過時。 +- 順序是嚴格的:第 n 個請求只對第 n 個 fixture。比 Polly 的 order(同一個識別碼的第幾次)更嚴, + 理由是插件的請求本來就是依序發的,順序變了通常就是插件的邏輯變了。 +- 用完與沒用到:VCR 的 `allow_unused_http_interactions: false` + (`features/cassettes/allow_unused_http_interactions.feature`)在 cassette 結束時若有沒用到的 + 互動就報錯。採用:沒被請求用到的 fixture 算違反;請求多於 fixture 也算。 +- 遮蔽過的欄位:VCR 的 `filter_sensitive_data` 寫佔位字串,重播時換回原值 + (`features/configuration/filter_sensitive_data.feature`)。這裡換不回(遮蔽不可逆,也不該留 + 原值),改成**比對前以同一個遮蔽函式遮過實際的網址**,被遮的欄位兩邊都是 `***`;另外 fixture + 裡值是 `***` 的 query 參數與路徑段不比值,給手寫時標出「這個會變」(時間戳、簽名)。 +- 對不上時 adapter 丟一個不是 `IOException` 的錯誤:網路層轉成 `UnexpectedError`、不重試 + (`interceptors.dart` 的 `_map`),執行器另外從 adapter 讀出哪一個請求對不上。 + +## 4. 錄製 + +- 接在同一個注入點:`RecordingAdapter` 包住真正的 adapter(live 用 dio 的 + `IOHttpClientAdapter`,測試用假的),回應原樣交給插件,遮蔽過的一份寫檔。 +- 遮蔽(ADR 0011 §決定 3,一律經 `Redactor`):網址與文字 body 用 `redact`;header 與 + `jsonBody` 用 `redactValue`(鍵在名單上的值整個換成 `***`,JSON 仍然合法。對整段 JSON 文字用 + `redact` 時,`"token": 123` 會變成 `"token": ***`,不是合法 JSON)。 +- `set-cookie`:`redactValue` 會把整個值換成 `***`,重播時 `dio_cookie_manager` 以 + `Cookie.fromSetCookieValue('***')` 解析會拋 `HttpException`、請求失敗(`cookie_mgr.dart` 的 + `_fromSetCookieValue`,`ignoreInvalidCookies` 預設 false)。所以只換 cookie 的值: + `名稱=***; 屬性`,名稱與屬性也經 `redact`。 +- 不留 `content-length`、`content-encoding`、`transfer-encoding`:body 已經解壓、遮蔽後重新編碼, + 長度與編碼都不對了(VCR 的 `update_content_length_header`、`decompress` 處理同一件事)。 +- 不是 UTF-8 的 body 不錄、報出來:fixture 沒有 `base64Body`,因為二進位內容無法經文字遮蔽函式; + M1 的案例(搜尋、解串流網址)都是 JSON。 +- 只錄得了免登入的案例(prd 擁有者決定 8):錄製的認證來源是 `NoCredentials`, + `auth: 'required'` 的請求以 `AuthRequired` 失敗,不需要另外的欄位標示。 +- 重試照真的時間等(重播時立刻重送):live 錄製遇到 429 不能連打。 +- live 入口在呼叫端 zone 以 `HttpOverrides.runWithHttpOverrides` 放行。插件的請求是背景 isolate + 送訊息、主 isolate 的 `ReceivePort` 監聽器發出的;監聽器在 `PluginRuntime.start` 時註冊,跑在 + 註冊當下的 zone,所以整個錄製包在放行的 zone 裡就行。`record_test.dart` 的 `sends real + requests from the zone it runs in` 以只計數、不連網的 `HttpOverrides` 證明這點。 + +## 5. checks.json + +- 一個物件,鍵是能力名稱:「每插件每能力最多一條」由格式保證。只收 `SourcePlugin` 已有方法的 + 能力(M1:`search`、`resolveStream`),其他鍵是不認得的欄位(封閉物件,比照 manifest 與 DTO)。 +- `input` 就是該能力的 DTO(`SearchQuery`、`StreamRequest`),以 `sourceDtoShapes` 解碼。 +- `expect` 兩種:成功(`minItems`、`nonEmpty`:清單每一筆的這些欄位都要有值,欄位名稱同 d.ts); + 失敗(`error`:`AppError` 類別名,`reason`:只有 `Unavailable`)。沒有 JSONPath 之類的通用斷言 + 語言:prd 要的只有「至少 N 筆、欄位非空、錯誤類別」。 +- 不強制每個宣告的能力都有案例(ADR 寫的是「最多一條」)。 + +## 6. 執行器檢查什麼、怎麼查 + +| 檢查 | 做法 | +|---|---| +| 能力與匯出一致 | `ScriptPluginLoader.load` 本來就拒絕(`Unsupported`),執行器先載入一次並報出原因 | +| DTO 驗證 | 經 `ScriptSourcePlugin` 呼叫,驗證失敗是 `ParseError`,期望成功時就不符 | +| 案例的期望 | §5 | +| 錯誤類別 | `AppError.typeName` 與 `Unavailable.reason`;原因以 `log.report` 取遮蔽過的字串(`AppError` 的 cause 是函式庫私有) | +| 媒體請求不帶憑證 | M1 沒有媒體 client:看 `resolveStream` 回傳的每個候選的 headers,名單上的 header 名稱或值會被遮蔽的都算 | +| 只連 manifest 網域 | 網路層本來就不送出(`Unsupported`,原因含 `not allowed`);執行器報出這種錯誤、檢查每個 fixture 的網址、adapter 再看一次每個送到底層的請求 | +| log 經遮蔽 | 每個案例的 log 歷史:再遮一次不變、名單上的欄位值是 `***` | +| fixture 沒有憑證 | 同上兩道,對每個 fixture 檔 | + +看不到的(已知限制,寫在 `app/AGENTS.md`):插件自己接住並吞掉網域錯誤;插件拋的 `ParseError` +與 DTO 驗證的 `ParseError` 分不出來。前者要網路層對被拒的請求也寫網路紀錄才看得到,屬於網路層的 +行為變更,這個 PR 不做。 + +每個案例各自一份記憶體資料庫、log、遮蔽函式與 HTTP client(VCR、Polly 每個測試一卷的做法), +案例之間不共用 storage 與 cookie;資料庫在案例結束時就關(drift 在 debug 下會對同時開著的多個 +`AppDatabase` 警告,`db_base.dart` 的 `_handleInstantiated`)。 + +## 7. CI + +選「已包含在 `flutter test` 內」:`contract_test.dart` 在裸 `flutter test` 裡跑 +`test/fixtures/plugins/` 的兩個測試插件;`ci.yml` 只補註解。`record_test.dart` 的 live 測試被 +`dart_test.yaml` 跳過(本機 `flutter test test/plugins/contract` 顯示 `~1`)。 + +## 8. 對 `app/` 以外的目錄實際跑一次 + +把兩個測試插件複製到 session scratchpad 的 `fmp-plugins/`(`http-test/`、`tone/`),在 `app/` 內: + +``` +$ FMP_PLUGIN_DIR='<暫存目錄>\fmp-plugins' \ + flutter test test/plugins/contract/contract_test.dart +00:00 +0: finds plugin directories +00:00 +1: fmp-test-http (http-test) install file, checks.json and fixtures +00:00 +2: fmp-test-http (http-test) search +00:00 +3: fmp-test-http (http-test) resolveStream +00:00 +4: fmp-test (tone) install file, checks.json and fixtures +00:00 +5: fmp-test (tone) search +00:00 +6: fmp-test (tone) resolveStream +00:00 +7: All tests passed! +``` + +指向單一插件目錄(`fmp-plugins/http-test`)也可以(4 個測試)。改壞一份副本的 fixture 路徑時的 +輸出: + +``` +fmp-test-http (broken) search [E] + Expected: empty + Actual: [ 'search: expected success, got UnexpectedError: …', + 'search: request #1 (GET https://api.fmp.test/search?access_key=***&keyword=tone&page=1) does not match fixtures/search/001.json (GET https://api.fmp.test/find?access_key=***&keyword=tone&page=1)', + 'search: fixtures/search/001.json was not requested' ] +``` diff --git a/.trellis/tasks/archive/2026-09/09-30-plugin-contract/task.json b/.trellis/tasks/archive/2026-09/09-30-plugin-contract/task.json new file mode 100644 index 00000000..7dc7e5bc --- /dev/null +++ b/.trellis/tasks/archive/2026-09/09-30-plugin-contract/task.json @@ -0,0 +1,26 @@ +{ + "id": "plugin-contract", + "name": "plugin-contract", + "title": "插件契約執行器與 fixture 重播", + "description": "M1 PR 9b: fixture record/replay adapter, checks.json, packages/plugin_contract contract runner, CI step", + "status": "completed", + "dev_type": null, + "scope": null, + "package": "app", + "priority": "P2", + "creator": "1morr", + "assignee": "1morr", + "createdAt": "2026-09-30", + "completedAt": "2026-09-30", + "branch": "feat/plugin-contract", + "base_branch": "main", + "worktree_path": null, + "commit": null, + "pr_url": null, + "subtasks": [], + "children": [], + "parent": "09-28-m1-skeleton-tracer", + "relatedFiles": [], + "notes": "", + "meta": {} +} \ No newline at end of file diff --git a/app/AGENTS.md b/app/AGENTS.md index 67c7d31e..8f664d49 100644 --- a/app/AGENTS.md +++ b/app/AGENTS.md @@ -26,6 +26,19 @@ 經 `test/support/quickjs.dart` 先以絕對路徑載入 flutter_js 內附的原生庫(Windows、Linux), 不必先建置桌面版;插件的背景 isolate 以檔名開到同一份(整個行程共用)。找不到就拋錯,CI 不會 默默跳過。macOS 不支援。看門狗的測試會留下一條忙著的執行緒,到那個測試檔的行程結束為止。 +- 插件契約執行器(`test/plugins/contract/`,ADR 0015 §決定 6)就在裸 `flutter test` 裡: + `contract_test.dart` 以 fixture 重播跑 `test/fixtures/plugins/` 底下每個插件目錄的 + `checks.json`,CI 沒有另外的步驟。 +- 對 `app/` 以外的插件目錄跑契約檢查(插件庫的 CI 以固定的 FMP ref 這樣跑): + `FMP_PLUGIN_DIR=<絕對路徑> flutter test test/plugins/contract/contract_test.dart` + (PowerShell:`$env:FMP_PLUGIN_DIR='<絕對路徑>'; flutter test test/plugins/contract/contract_test.dart; Remove-Item Env:FMP_PLUGIN_DIR`; + 不刪的話它留在整個 session,之後裸 `flutter test` 的契約測試也改跑那個目錄)。 + 路徑是一個插件目錄,或每個子目錄都是插件目錄的目錄;插件目錄的格式見 § 插件。 +- 錄 fixture(真實連線,ADR 0027 §決定 2;只限不需要登入的案例): + `FMP_PLUGIN_DIR=<絕對路徑> flutter test --run-skipped --tags live test/plugins/contract/record_test.dart`。 + 每個案例的 fixture 整組重寫;有 `meta.edited` 的案例略過(手寫的錯誤案例不被蓋掉);結果不符 + checks.json 期望的案例(例如連線失敗)什麼都不寫、原本的檔案不動。閘門:`record_test.dart`。 + 錄完同一個測試以重播再跑一次。 - 插件執行環境的實機量測:`flutter test integration_test/plugin_runtime_benchmark_test.dart -d <裝置>` (dev flavor;結果是 `FMP_BENCH` 開頭的行)。數字與方法在 `.trellis/tasks/archive/2026-09/09-30-js-runtime/research/notes.md` §4。 @@ -269,7 +282,26 @@ lint 的範圍是整個 `lib/platform/`,組裝點以外的平台層檔案、 確認(ADR 0014 §決定 6),參數與環境變數都能由別的程式帶入。`devPluginPath` 在 prod 一律回 `null`。閘門:`plugin_installer_test.dart` 的 `development entry`(含 `prod reads neither…`)。 - 測試插件 `test/fixtures/plugins/test_plugin/`(`fmp-test`)只以 dev flavor 的 asset 打包,串流指向 - 同目錄的 `tone.wav`(`asset:///…`)。prod 的建置只留下空目錄,沒有檔案。 + 同目錄的 `tone.wav`(`asset:///…`)。prod 的建置只留下空目錄,沒有檔案。第二個測試插件 + `http_test_plugin/`(`fmp-test-http`)會發請求(`*.fmp.test`),只給契約執行器,不打包。 +- 插件目錄(契約檢查的單位):剛好一個 `.js` 安裝檔、`checks.json`(鍵是能力名稱,所以每能力最多 + 一條;只收 `SourcePlugin` 已有方法的能力)、`fixtures/<能力>/*.json`(依檔名是請求順序)。格式 + 寫在 `fmp-plugin.d.ts` 的 `FmpChecks`、`FmpFixture`。執行器對每個案例各開一份資料庫、log 與 + client,檢查:能力與匯出一致、DTO 驗證、案例期望(成功的筆數與非空欄位,或失敗的 `AppError` + 類別與 `Unavailable` 原因)、沒試著連清單外的網域、串流 headers 不帶憑證、log 與 fixture 都遮蔽 + 過。閘門:`test/plugins/contract/contract_runner_test.dart`(每種違反一個會紅的變異,另有改無關 + 處不紅的案例)、`checks_test.dart`。 +- 重播:第 n 個請求對第 n 個 fixture,比 method 與網址(實際網址先經 `Redactor`;query 不分順序; + fixture 裡值為 `***` 的 query 參數與路徑段不比值)。對不上或用完就讓那次請求失敗、不送出;沒用 + 到的 fixture 也算違反。header 與 body 不比。閘門:`contract_runner_test.dart`、 + `fixture_scan_test.dart` 的 `replay matching`。 +- fixture 寫檔前經 `Redactor`(`HttpFixture.redacted`:網址與文字 body 用 `redact`、header 與 + `jsonBody` 用 `redactValue`,`set-cookie` 只遮值、留名稱與屬性,重播時才解析得了);不是 + UTF-8 的 body 不錄。`app/` 內每個 fixture 都要「再遮一次不變」且名單上的欄位值是 `***`。閘門: + `fixture_scan_test.dart`(`every fixture in app/ is redacted`,兩道檢查各有會紅與不紅的案例)、 + `record_test.dart`(假上游錄出的檔案沒有假憑證、重播通過)。 +- 執行器看不到的:插件接住並吞掉「網域不在清單」的錯誤(網路層照樣不送出);插件自己拋的 + `ParseError` 與 DTO 驗證失敗的 `ParseError` 分不出來。沒有閘門,已知限制。 ## 設定 diff --git a/app/lib/plugins/types/fmp-plugin.d.ts b/app/lib/plugins/types/fmp-plugin.d.ts index 1cad0f4c..62dc3036 100644 --- a/app/lib/plugins/types/fmp-plugin.d.ts +++ b/app/lib/plugins/types/fmp-plugin.d.ts @@ -9,7 +9,8 @@ // 可以省略或給 null。 // // 與宿主的 Dart 端一致:interface 的欄位對 `manifestShapes`、`sourceDtoShapes`、 -// `hostApiShapes`,FmpHost 對 `js_prelude.dart` 實際建出的 `fmp` +// `hostApiShapes`,契約檢查的格式對契約執行器的 `checkShapes`、`fixtureShapes` +// (app/test/plugins/contract/),FmpHost 對 `js_prelude.dart` 實際建出的 `fmp` // (app/test/plugins/type_definitions_test.dart 比對)。 // ---------------------------------------------------------------- manifest @@ -248,6 +249,95 @@ export interface FmpPluginExports { resolveStream?(request: StreamRequest): StreamResult | Promise; } +// ---------------------------------------------------------------- 契約檢查 +// +// 插件目錄(插件庫的一個插件、app/test/fixtures/plugins/ 的測試插件): +// +// <目錄>/<名稱>.js 安裝檔,只能有一個 +// <目錄>/checks.json FmpChecks +// <目錄>/fixtures/<能力>/<序號>.json FmpFixture,依檔名排序就是請求的順序 +// +// 契約執行器以 fixture 重播每條案例,檢查能力與匯出一致、回傳值通過 DTO 驗證、 +// 案例的期望、錯誤類別、只連 allowedHosts、串流 headers 不帶憑證、log 與 +// fixture 都遮蔽過。錄製模式真的連網跑同一份 checks.json,遮蔽後寫出 +// fixture(只限不需要登入的案例)。指令見 app/AGENTS.md § 驗證。 + +/** checks.json:每個能力最多一條案例,鍵就是能力名稱。目前能寫案例的只有這兩個。 */ +export interface FmpChecks { + search?: FmpSearchCheck | null; + resolveStream?: FmpResolveStreamCheck | null; +} + +export interface FmpSearchCheck { + input: SearchQuery; + expect: FmpExpectSuccess | FmpExpectError; +} + +export interface FmpResolveStreamCheck { + input: StreamRequest; + expect: FmpExpectSuccess | FmpExpectError; +} + +/** 成功;回傳的清單(search 的 items、resolveStream 的 candidates)符合條件。 */ +export interface FmpExpectSuccess { + /** 至少幾筆,預設 0。 */ + minItems?: number | null; + /** + * 每一筆都要有值的欄位,名稱同 TrackSummary/StreamCandidate:不是 null、 + * 空白字串、空陣列或空物件。 + */ + nonEmpty?: string[] | null; +} + +/** 以這個錯誤失敗。 */ +export interface FmpExpectError { + error: FmpErrorName; + /** 只有 Unavailable 能給。 */ + reason?: FmpUnavailableReason | null; +} + +/** + * 一次請求與它的回應,一個檔案。欄位名稱沿用 WireMock 的 stub mapping。 + * 錄製時寫檔前一律經宿主的遮蔽函式;手寫的也必須是遮過的樣子(值換成 `***`), + * 否則契約檢查失敗。 + */ +export interface FmpFixture { + meta: FmpFixtureMeta; + request: FmpFixtureRequest; + response: FmpFixtureResponse; +} + +export interface FmpFixtureMeta { + /** 錄製時間(ISO 8601,UTC);手寫的沒有。 */ + recordedAt?: string | null; + /** 手寫或手改的理由(例如錯誤案例)。有它的案例,錄製模式不覆蓋。 */ + edited?: string | null; +} + +export interface FmpFixtureRequest { + /** 與實際請求比對。 */ + method: string; + /** + * 與實際請求比對(實際的網址先經同一個遮蔽函式):scheme、host、port、路徑 + * 與 query,query 不分順序;值是 `***` 的 query 參數與路徑段不比對值。 + */ + url: string; + /** 只供閱讀,不比對。名稱小寫。 */ + headers?: Record | null; + /** 只供閱讀,不比對。 */ + body?: string | null; +} + +export interface FmpFixtureResponse { + status: number; + /** 名稱小寫。錄製時不留 content-length、content-encoding、transfer-encoding。 */ + headers?: Record | null; + /** 文字 body;與 jsonBody 最多給一個。 */ + body?: string | null; + /** JSON 物件或陣列的 body,重播時編碼成緊湊的 JSON 文字。 */ + jsonBody?: unknown; +} + declare global { const fmp: FmpHost; } diff --git a/app/test/fixtures/plugins/http_test_plugin/README.md b/app/test/fixtures/plugins/http_test_plugin/README.md new file mode 100644 index 00000000..c93fc83b --- /dev/null +++ b/app/test/fixtures/plugins/http_test_plugin/README.md @@ -0,0 +1,12 @@ +# 測試插件 `fmp-test-http` + +契約執行器(`test/plugins/contract/`)的第二個測試插件:會發 HTTP 請求,讓重播、 +網域與遮蔽的檢查真的被執行到。網域是 RFC 2606 保留的 `.test`,回應全部由 +`fixtures/` 重播,不會真的連網;也不打包進 App。 + +- `http_test_plugin.js`:安裝檔,能力 `search`、`resolveStream`。請求帶一個假的 + `access_key`,fixture 與 log 裡只能看到 `***`;manifest 追加遮蔽鍵名 + `demo_session`。 +- `checks.json`:`search` 期望成功;`resolveStream` 期望 `Unavailable` + (`copyright`),示範錯誤案例。 +- `fixtures/`:手寫,每個檔的 `meta.edited` 寫明理由,錄製模式不會覆蓋。 diff --git a/app/test/fixtures/plugins/http_test_plugin/checks.json b/app/test/fixtures/plugins/http_test_plugin/checks.json new file mode 100644 index 00000000..e69af999 --- /dev/null +++ b/app/test/fixtures/plugins/http_test_plugin/checks.json @@ -0,0 +1,17 @@ +{ + "search": { + "input": { "keyword": "tone", "page": 1 }, + "expect": { + "minItems": 2, + "nonEmpty": ["sourceId", "title", "uploader", "durationMs"] + } + }, + "resolveStream": { + "input": { + "sourceId": "a1", + "purpose": "playback", + "formats": [{ "container": "mp4", "codec": "aac" }] + }, + "expect": { "error": "Unavailable", "reason": "copyright" } + } +} diff --git a/app/test/fixtures/plugins/http_test_plugin/fixtures/resolveStream/001.json b/app/test/fixtures/plugins/http_test_plugin/fixtures/resolveStream/001.json new file mode 100644 index 00000000..041f3a5f --- /dev/null +++ b/app/test/fixtures/plugins/http_test_plugin/fixtures/resolveStream/001.json @@ -0,0 +1,16 @@ +{ + "meta": { + "edited": "手寫的錯誤案例:上游對版權受限的曲目回 code 403" + }, + "request": { + "method": "GET", + "url": "https://api.fmp.test/stream?id=a1" + }, + "response": { + "status": 200, + "headers": { + "content-type": ["application/json; charset=utf-8"] + }, + "jsonBody": { "code": 403 } + } +} diff --git a/app/test/fixtures/plugins/http_test_plugin/fixtures/search/001.json b/app/test/fixtures/plugins/http_test_plugin/fixtures/search/001.json new file mode 100644 index 00000000..b8bdd576 --- /dev/null +++ b/app/test/fixtures/plugins/http_test_plugin/fixtures/search/001.json @@ -0,0 +1,27 @@ +{ + "meta": { + "edited": "手寫的合成資料,不是錄的" + }, + "request": { + "method": "GET", + "url": "https://api.fmp.test/search?access_key=***&keyword=tone&page=1", + "headers": { + "referer": "https://www.fmp.test/" + } + }, + "response": { + "status": 200, + "headers": { + "content-type": ["application/json; charset=utf-8"], + "set-cookie": ["demo_session=***; Path=/; Secure"] + }, + "jsonBody": { + "demo_session": "***", + "list": [ + { "id": "a1", "name": "Tone A", "owner": "FMP", "ms": 2000 }, + { "id": "a2", "name": "Tone B", "owner": "FMP", "ms": 3000 } + ], + "more": false + } + } +} diff --git a/app/test/fixtures/plugins/http_test_plugin/http_test_plugin.js b/app/test/fixtures/plugins/http_test_plugin/http_test_plugin.js new file mode 100644 index 00000000..3364fbaa --- /dev/null +++ b/app/test/fixtures/plugins/http_test_plugin/http_test_plugin.js @@ -0,0 +1,71 @@ +/* ==FMP Plugin== +{ + "id": "fmp-test-http", + "name": "FMP HTTP Test Plugin", + "version": "1.0.0", + "author": "FMP", + "apiVersion": 1, + "capabilities": ["search", "resolveStream"], + "allowedHosts": ["api.fmp.test", "media.fmp.test"], + "redaction": { "keyNames": ["demo_session"] } +} +==/FMP Plugin== */ + +// 契約執行器的第二個測試插件:會發 HTTP 請求,網域是 RFC 2606 保留的 .test, +// 回應由 fixtures/ 重播,不會真的連網。讓重播比對、網域與遮蔽的檢查真的被 +// 執行到。 + +const API = 'https://api.fmp.test'; +const REFERER = 'https://www.fmp.test/'; +// 假的憑證:請求帶著它;fixture、log 與網路紀錄裡只能看到 ***。 +const ACCESS_KEY = 'FAKE_ACCESS_KEY_0001'; + +export async function search({ keyword, page }) { + // query 的順序刻意與 fixture 不同:重播比對不看順序。 + const url = + `${API}/search?page=${page}&keyword=${encodeURIComponent(keyword)}` + + `&access_key=${ACCESS_KEY}`; + fmp.log.debug(`search ${url}`); + const response = await fmp.http.request({ url, headers: { Referer: REFERER } }); + if (response.status === 429) throw { fmpError: 'RateLimited' }; + if (response.status !== 200) { + throw { fmpError: 'ParseError', message: `HTTP ${response.status}` }; + } + const json = JSON.parse(response.body); + fmp.log.info('search results', { + count: json.list.length, + demo_session: json.demo_session, + }); + return { + items: json.list.map((item) => ({ + sourceId: item.id, + title: item.name, + uploader: item.owner, + durationMs: item.ms, + })), + hasMore: json.more, + }; +} + +export async function resolveStream({ sourceId }) { + const response = await fmp.http.request({ + url: `${API}/stream?id=${encodeURIComponent(sourceId)}`, + }); + if (response.status === 404) throw { fmpError: 'NotFound' }; + if (response.status !== 200) { + throw { fmpError: 'ParseError', message: `HTTP ${response.status}` }; + } + const json = JSON.parse(response.body); + if (json.code === 403) throw { fmpError: 'Unavailable', reason: 'copyright' }; + if (json.code !== 0) throw { fmpError: 'NotFound', message: `code ${json.code}` }; + return { + candidates: [ + { + url: json.url, + headers: { Referer: REFERER }, + container: 'mp4', + codec: 'aac', + }, + ], + }; +} diff --git a/app/test/fixtures/plugins/test_plugin/README.md b/app/test/fixtures/plugins/test_plugin/README.md index ea54e0cb..08fd8a6d 100644 --- a/app/test/fixtures/plugins/test_plugin/README.md +++ b/app/test/fixtures/plugins/test_plugin/README.md @@ -18,3 +18,9 @@ ffmpeg -f lavfi -i "sine=frequency=440:duration=2:sample_rate=16000" \ ``` 以 [CC0 1.0](https://creativecommons.org/publicdomain/zero/1.0/) 釋出至公有領域。 + +## 契約檢查 + +`checks.json` 是契約執行器(`test/plugins/contract/`)跑的案例:`search` 與 +`resolveStream` 都期望成功。這個插件不發請求,所以沒有 `fixtures/`。`checks.json` +也會隨目錄打包進 dev flavor 的 asset(`flutter.assets` 是整個目錄),App 不讀它。 diff --git a/app/test/fixtures/plugins/test_plugin/checks.json b/app/test/fixtures/plugins/test_plugin/checks.json new file mode 100644 index 00000000..26524727 --- /dev/null +++ b/app/test/fixtures/plugins/test_plugin/checks.json @@ -0,0 +1,17 @@ +{ + "search": { + "input": { "keyword": "tone", "page": 1 }, + "expect": { + "minItems": 2, + "nonEmpty": ["sourceId", "title", "uploader", "durationMs"] + } + }, + "resolveStream": { + "input": { + "sourceId": "tone-440", + "purpose": "playback", + "formats": [{ "container": "wav", "codec": "pcm_s16le" }] + }, + "expect": { "minItems": 1, "nonEmpty": ["url", "container", "codec"] } + } +} diff --git a/app/test/plugins/contract/checks.dart b/app/test/plugins/contract/checks.dart new file mode 100644 index 00000000..c01cbbd9 --- /dev/null +++ b/app/test/plugins/contract/checks.dart @@ -0,0 +1,276 @@ +import 'dart:convert'; + +import 'package:fmp/core/errors/app_error.dart'; +import 'package:fmp/plugins/json_shape.dart'; +import 'package:fmp/plugins/manifest/plugin_manifest.dart'; +import 'package:fmp/plugins/runtime/script_errors.dart'; +import 'package:fmp/plugins/source_dto.dart'; +import 'package:fmp/plugins/source_plugin.dart'; + +// checks.json:每插件每能力最多一條檢查案例(ADR 0015 §決定 4)。檔案是一個 +// 物件,鍵是能力名稱,所以「最多一條」由格式本身保證。能寫案例的能力只有 +// SourcePlugin 已有方法的那些(M1 是 search、resolveStream);其他能力的鍵是 +// 不認得的欄位,整個檔案拒收。 +// +// 形狀與 `lib/plugins/types/fmp-plugin.d.ts` 的 `FmpChecks` 等一致 +// (test/plugins/type_definitions_test.dart 比對)。 + +/// checks.json 的欄位表,鍵是 `fmp-plugin.d.ts` 裡的 interface 名稱。 +const checkShapes = { + 'FmpChecks': {'search': false, 'resolveStream': false}, + 'FmpSearchCheck': {'input': true, 'expect': true}, + 'FmpResolveStreamCheck': {'input': true, 'expect': true}, + 'FmpExpectSuccess': {'minItems': false, 'nonEmpty': false}, + 'FmpExpectError': {'error': true, 'reason': false}, +}; + +/// 每個能力的案例形狀與回傳清單裡一筆的 DTO。 +const _capabilities = { + PluginCapability.search: (check: 'FmpSearchCheck', item: 'TrackSummary'), + PluginCapability.resolveStream: ( + check: 'FmpResolveStreamCheck', + item: 'StreamCandidate', + ), +}; + +/// 一條檢查案例。 +final class PluginCheck { + const PluginCheck(this.capability, this.input, this.expectation); + + final PluginCapability capability; + + /// [SearchQuery] 或 [StreamRequest]。 + final Object input; + final CheckExpectation expectation; + + /// 以 [plugin] 執行這個案例。 + Future run(SourcePlugin plugin) => switch (input) { + final SearchQuery query => plugin.search(query), + final StreamRequest request => plugin.resolveStream(request), + _ => throw StateError('No host method for ${capability.wireName}'), + }; +} + +/// 解析 checks.json;格式不對拋 [FormatException]。 +List parseChecks(String text) { + final Object? json; + try { + json = jsonDecode(text); + } on FormatException catch (error) { + throw FormatException('checks.json: not JSON (${error.message})'); + } + final fields = JsonFields(json, checkShapes['FmpChecks']!, path: 'checks'); + return [ + for (final MapEntry(key: capability, value: (:check, :item)) + in _capabilities.entries) + if (fields.has(capability.wireName)) + _check( + capability, + JsonFields( + fields.raw(capability.wireName), + checkShapes[check]!, + path: 'checks.${capability.wireName}', + ), + item, + ), + ]; +} + +PluginCheck _check( + PluginCapability capability, + JsonFields fields, + String item, +) { + final path = fields.path; + try { + return PluginCheck(capability, switch (capability) { + PluginCapability.search => _searchQuery(fields.raw('input'), path), + _ => _streamRequest(fields.raw('input'), path), + }, _expectation(fields.raw('expect'), '$path.expect', item)); + } on ArgumentError catch (error) { + throw FormatException('$path.input: ${error.message}'); + } +} + +SearchQuery _searchQuery(Object? json, String path) { + final fields = JsonFields( + json, + sourceDtoShapes['SearchQuery']!, + path: '$path.input', + ); + return SearchQuery( + keyword: fields.string('keyword'), + page: fields.integer('page', min: 1), + ); +} + +StreamRequest _streamRequest(Object? json, String path) { + final fields = JsonFields( + json, + sourceDtoShapes['StreamRequest']!, + path: '$path.input', + ); + final purpose = fields.string('purpose'); + return StreamRequest( + sourceId: fields.string('sourceId'), + cid: fields.optionalInteger('cid'), + purpose: StreamPurpose.values.firstWhere( + (value) => value.wireName == purpose, + orElse: () => + throw FormatException('$path.input.purpose: unknown "$purpose"'), + ), + formats: [ + for (final (index, format) in fields.list('formats').indexed) + _streamFormat(format, '$path.input.formats[$index]'), + ], + ); +} + +StreamFormat _streamFormat(Object? json, String path) { + final fields = JsonFields(json, sourceDtoShapes['StreamFormat']!, path: path); + return StreamFormat( + container: fields.string('container'), + codec: fields.string('codec'), + ); +} + +CheckExpectation _expectation(Object? json, String path, String item) { + if (json case {'error': _}) { + final fields = JsonFields(json, checkShapes['FmpExpectError']!, path: path); + final error = fields.string('error'); + // 認得的名稱轉出來就是同名的類別;不認得的會變成 UnexpectedError。 + if (structuredScriptError( + pluginId: 'checks', + fmpError: error, + reason: 'region', + ).typeName != + error) { + throw FormatException('$path.error: unknown error "$error"'); + } + final reason = fields.optionalString('reason'); + if (reason != null && error != 'Unavailable') { + throw FormatException('$path.reason: only Unavailable has a reason'); + } + return ExpectError( + error, + reason: reason == null + ? null + : UnavailableReason.values.firstWhere( + (value) => unavailableReasonWireName(value) == reason, + orElse: () => + throw FormatException('$path.reason: unknown "$reason"'), + ), + ); + } + final fields = JsonFields(json, checkShapes['FmpExpectSuccess']!, path: path); + final nonEmpty = fields.optionalStringList('nonEmpty') ?? const []; + for (final name in nonEmpty) { + if (!sourceDtoShapes[item]!.containsKey(name)) { + throw FormatException('$path.nonEmpty: $item has no field "$name"'); + } + } + return ExpectSuccess( + minItems: fields.optionalInteger('minItems') ?? 0, + nonEmpty: nonEmpty, + ); +} + +/// 案例的期望。 +sealed class CheckExpectation { + const CheckExpectation(); + + /// 比對結果:成功時 [result] 是能力的回傳值,失敗時 [error] 是丟出的錯誤, + /// [describe] 把錯誤寫成一行(含遮蔽過的原因)。回傳不符之處。 + List evaluate({ + Object? result, + AppError? error, + required String Function(AppError error) describe, + }); +} + +/// 成功,回傳的清單(search 的 `items`、resolveStream 的 `candidates`)至少 +/// [minItems] 筆,每一筆的 [nonEmpty] 欄位都有值。 +final class ExpectSuccess extends CheckExpectation { + const ExpectSuccess({required this.minItems, required this.nonEmpty}); + + final int minItems; + final List nonEmpty; + + @override + List evaluate({ + Object? result, + AppError? error, + required String Function(AppError error) describe, + }) { + if (error != null) return ['expected success, got ${describe(error)}']; + final items = checkItems(result!); + return [ + if (items.length < minItems) + 'expected at least $minItems items, got ${items.length}', + for (final (index, item) in items.indexed) + for (final name in nonEmpty) + if (_isEmpty(item[name])) 'item $index: "$name" is empty', + ]; + } + + static bool _isEmpty(Object? value) => switch (value) { + null => true, + final String text => text.trim().isEmpty, + final Iterable items => items.isEmpty, + final Map map => map.isEmpty, + _ => false, + }; +} + +/// 以 [error] 這個 `AppError` 類別失敗;`Unavailable` 可以另外指定 [reason]。 +final class ExpectError extends CheckExpectation { + const ExpectError(this.error, {this.reason}); + + final String error; + final UnavailableReason? reason; + + @override + List evaluate({ + Object? result, + AppError? error, + required String Function(AppError error) describe, + }) { + final expected = reason == null + ? this.error + : '${this.error} (${unavailableReasonWireName(reason!)})'; + if (error == null) return ['expected $expected, but the call succeeded']; + final matches = + error.typeName == this.error && + (reason == null || error is Unavailable && error.reason == reason); + return matches ? const [] : ['expected $expected, got ${describe(error)}']; + } +} + +/// 回傳值裡要檢查的清單,每一筆以 `fmp-plugin.d.ts` 的欄位名稱表示。 +List> checkItems(Object result) => switch (result) { + final SearchPage page => [for (final track in page.items) trackFields(track)], + final List candidates => [ + for (final candidate in candidates) candidateFields(candidate), + ], + _ => throw ArgumentError.value(result, 'result'), +}; + +/// [TrackSummary] 的欄位,名稱同 `sourceDtoShapes['TrackSummary']`。 +Map trackFields(TrackSummary track) => { + 'sourceId': track.sourceId, + 'cid': track.cid, + 'title': track.title, + 'uploader': track.uploader, + 'durationMs': track.duration?.inMilliseconds, + 'artwork': track.artwork, +}; + +/// [StreamCandidate] 的欄位,名稱同 `sourceDtoShapes['StreamCandidate']`。 +Map candidateFields(StreamCandidate candidate) => { + 'url': candidate.url.toString(), + 'headers': candidate.headers, + 'container': candidate.container, + 'codec': candidate.codec, + 'bitrate': candidate.bitrate, + 'expiresAt': candidate.expiresAt?.millisecondsSinceEpoch, +}; diff --git a/app/test/plugins/contract/checks_test.dart b/app/test/plugins/contract/checks_test.dart new file mode 100644 index 00000000..be056cd4 --- /dev/null +++ b/app/test/plugins/contract/checks_test.dart @@ -0,0 +1,91 @@ +import 'package:flutter_test/flutter_test.dart'; +import 'package:fmp/core/errors/app_error.dart'; +import 'package:fmp/plugins/source_dto.dart'; + +import 'checks.dart'; + +String _check(String capability, String input, String expect) => + '{"$capability": {"input": $input, "expect": $expect}}'; + +const _search = '{"keyword": "k", "page": 1}'; + +void main() { + test('the item fields cover the DTO shapes', () { + // nonEmpty 以 d.ts 的欄位名稱檢查:DTO 多了欄位,這兩張表也要多。 + expect( + trackFields( + const TrackSummary(sourceTypeId: 'p', sourceId: 's', title: 't'), + ).keys.toSet(), + sourceDtoShapes['TrackSummary']!.keys.toSet(), + ); + expect( + candidateFields(StreamCandidate(url: Uri.parse('https://a.test/'))).keys + .toSet(), + sourceDtoShapes['StreamCandidate']!.keys.toSet(), + ); + }); + + group('parseChecks', () { + test('reads both kinds of expectation', () { + final checks = parseChecks( + '{"search": {"input": $_search, "expect": {"minItems": 1}}, ' + '"resolveStream": {"input": {"sourceId": "a", "purpose": "playback", ' + '"formats": [{"container": "mp4", "codec": "aac"}]}, ' + '"expect": {"error": "Unavailable", "reason": "region"}}}', + ); + + expect(checks.map((check) => check.capability.wireName), [ + 'search', + 'resolveStream', + ]); + expect(checks.first.input, isA()); + expect( + checks.last.expectation, + isA() + .having((e) => e.error, 'error', 'Unavailable') + .having((e) => e.reason, 'reason', UnavailableReason.region), + ); + }); + + for (final (name, text, message) in [ + ( + 'a capability the host cannot call yet', + _check('charts', '{}', '{}'), + 'unknown field "charts"', + ), + ( + 'an unknown error name', + _check('search', _search, '{"error": "Timeout"}'), + 'unknown error "Timeout"', + ), + ( + 'a reason on another error', + _check('search', _search, '{"error": "NotFound", "reason": "age"}'), + 'only Unavailable has a reason', + ), + ( + 'a nonEmpty field the item does not have', + _check('search', _search, '{"nonEmpty": ["name"]}'), + 'TrackSummary has no field "name"', + ), + ( + 'an input the DTO rejects', + _check('search', '{"keyword": " ", "page": 1}', '{}'), + 'checks.search.input: must not be empty', + ), + ]) { + test('rejects $name', () { + expect( + () => parseChecks(text), + throwsA( + isA().having( + (e) => e.message, + 'message', + contains(message), + ), + ), + ); + }); + } + }); +} diff --git a/app/test/plugins/contract/contract_runner.dart b/app/test/plugins/contract/contract_runner.dart new file mode 100644 index 00000000..b3f825fa --- /dev/null +++ b/app/test/plugins/contract/contract_runner.dart @@ -0,0 +1,492 @@ +import 'dart:convert'; +import 'dart:io'; +import 'dart:math' as math; + +import 'package:dio/dio.dart'; +import 'package:drift/drift.dart' show DatabaseConnection; +import 'package:drift/native.dart'; +import 'package:fmp/core/errors/app_error.dart'; +import 'package:fmp/core/logging/log.dart'; +import 'package:fmp/core/logging/log_record.dart'; +import 'package:fmp/core/network/allowed_hosts.dart'; +import 'package:fmp/core/network/source_http_client.dart'; +import 'package:fmp/core/redaction/redactor.dart'; +import 'package:fmp/data/database/app_database.dart'; +import 'package:fmp/data/repositories/plugin_repository.dart'; +import 'package:fmp/data/repositories/plugin_storage_repository.dart'; +import 'package:fmp/plugins/manifest/plugin_file.dart'; +import 'package:fmp/plugins/manifest/plugin_manifest.dart'; +import 'package:fmp/plugins/script_source_plugin.dart'; +import 'package:fmp/plugins/source_dto.dart'; +import 'package:path/path.dart' as p; + +import 'checks.dart'; +import 'credential_scan.dart'; +import 'fixture.dart'; +import 'fixture_adapters.dart'; + +// 插件契約執行器(ADR 0015 §決定 6)。對一個插件目錄: +// +// <目錄>/<任意名稱>.js 安裝檔(只能有一個) +// <目錄>/checks.json 檢查案例(checks.dart) +// <目錄>/fixtures/<能力>/*.json 該案例依序的請求與回應(fixture.dart) +// +// 以重播執行每個案例,回傳違反契約之處(空的就是通過)。每個案例各自一份 +// 記憶體資料庫、log、遮蔽函式與 HTTP client,案例之間不共用 storage 與 +// cookie。入口是 contract_test.dart(重播)與 record_test.dart(錄製)。 + +/// 插件目錄的清單:[root] 本身有 `.js` 檔就是一個插件目錄,否則是它底下有 +/// `.js` 檔的子目錄(依名稱排序)。 +List pluginDirectories(Directory root) { + if (_scripts(root).isNotEmpty) return [root]; + return [ + for (final entry + in root.listSync()..sort((a, b) => a.path.compareTo(b.path))) + if (entry is Directory && _scripts(entry).isNotEmpty) entry, + ]; +} + +List _scripts(Directory directory) => [ + for (final entry in directory.listSync()) + if (entry is File && entry.path.endsWith('.js')) entry, +]; + +/// 讀好的插件目錄。讀的時候發現的問題在 [problems],其他部分盡量讀。 +final class PluginDirectory { + PluginDirectory._(this.directory); + + /// 讀 [directory]:安裝檔、checks.json、fixture(格式、網域、憑證)。 + factory PluginDirectory.read(Directory directory) { + final plugin = PluginDirectory._(directory); + final scripts = _scripts(directory); + if (scripts.length != 1) { + plugin.problems.add( + 'expected one .js install file, found ${scripts.length}', + ); + return plugin; + } + try { + plugin.file = PluginFile.parse(scripts.single.readAsStringSync()); + } on AppError catch (error) { + plugin.problems.add('install file: ${describeError(error)}'); + return plugin; + } + plugin + .._readChecks() + .._readFixtures(); + return plugin; + } + + final Directory directory; + PluginFile? file; + final checks = []; + + /// 依能力名稱(`fixtures/` 底下的目錄名),依檔名排序。 + final fixtures = >{}; + final problems = []; + + PluginManifest get manifest => file!.manifest; + + /// 測試名稱用。 + String get label => switch (file) { + null => p.basename(directory.path), + final file => '${file.manifest.id} (${p.basename(directory.path)})', + }; + + void _readChecks() { + final checksFile = File(p.join(directory.path, 'checks.json')); + if (!checksFile.existsSync()) { + problems.add('checks.json is missing'); + return; + } + try { + checks.addAll(parseChecks(checksFile.readAsStringSync())); + } on FormatException catch (error) { + problems.add('checks.json: ${error.message}'); + return; + } + for (final check in checks) { + if (!manifest.capabilities.contains(check.capability)) { + problems.add( + 'checks.json: ${check.capability.wireName} is not a declared ' + 'capability', + ); + } + } + } + + void _readFixtures() { + final root = Directory(p.join(directory.path, 'fixtures')); + if (!root.existsSync()) return; + final checked = {for (final check in checks) check.capability.wireName}; + final allowedHosts = AllowedHosts(manifest.allowedHosts); + final redactor = redactorFor(manifest); + final names = CredentialNames.of(manifest); + for (final entry in root.listSync()) { + final name = p.basename(entry.path); + if (entry is! Directory) { + problems.add('fixtures/$name: fixtures go in fixtures//'); + continue; + } + if (!checked.contains(name)) { + problems.add('fixtures/$name: checks.json has no $name check'); + } + final files = [ + for (final file in entry.listSync()) + if (file is File && file.path.endsWith('.json')) file, + ]..sort((a, b) => p.basename(a.path).compareTo(p.basename(b.path))); + fixtures[name] = [ + for (final file in files) + if (_readFixture(file, 'fixtures/$name/${p.basename(file.path)}') + case final fixture?) + (name: 'fixtures/$name/${p.basename(file.path)}', fixture: fixture), + ]; + for (final (:name, :fixture) in fixtures[name]!) { + final url = Uri.parse(fixture.request.url); + if (!allowedHosts.allows(url)) { + problems.add( + '$name: ${url.host} is outside allowedHosts, or not https', + ); + } + problems.addAll(scanFixture(name, fixture, redactor, names)); + } + } + } + + HttpFixture? _readFixture(File file, String name) { + try { + return HttpFixture.fromJson(jsonDecode(file.readAsStringSync())); + } on FormatException catch (error) { + problems.add('$name: ${error.message}'); + return null; + } + } +} + +/// 加上 [manifest] 追加名單的遮蔽函式(載入插件時 `ScriptPluginLoader` 也這樣 +/// 加)。 +Redactor redactorFor(PluginManifest manifest) => Redactor() + ..addRules( + headerNames: manifest.redaction.headerNames, + keyNames: manifest.redaction.keyNames, + mediaCdns: manifest.redaction.mediaCdns, + ); + +/// 把 [error] 寫成一行:類別名與遮蔽過的原因(原因只有 `log.report` 讀得到)。 +String describeError(AppError error, [Log? log]) { + final target = log ?? Log(redactor: Redactor(), minimumLevel: LogLevel.debug); + target.report('Contract check', error, tag: 'contract'); + final cause = target.history.last.error; + return cause == null ? error.typeName : '${error.typeName}: $cause'; +} + +/// 整個插件目錄:[PluginDirectory.read]、[checkPluginDirectory]、每個案例的 +/// [runCheck]。回傳所有違反之處。 +Future> runContract(Directory directory) async { + final plugin = PluginDirectory.read(directory); + return [ + ...await checkPluginDirectory(plugin), + for (final check in plugin.checks) ...await runCheck(plugin, check), + ]; +} + +/// 讀取時的問題,加上載入一次(能力與匯出一致、腳本讀得懂)。 +Future> checkPluginDirectory(PluginDirectory plugin) async { + final file = plugin.file; + if (file == null) return plugin.problems; + final environment = _Environment( + (redactor) => ReplayAdapter( + const [], + redactor, + AllowedHosts(file.manifest.allowedHosts), + ), + ); + try { + (await environment.load(file)).close(); + return plugin.problems; + } on AppError catch (error) { + return [ + ...plugin.problems, + 'load: ${describeError(error, environment.log)}', + ]; + } finally { + await environment.close(); + } +} + +/// 以重播執行 [check]:[_execute] 的檢查,加上每個請求都對上 fixture、每個 +/// fixture 都被用到。 +/// +/// [log] 換掉這個案例的 log(預設是以案例的遮蔽函式建的門面):執行器自己的 +/// 測試以它造一個沒接上插件遮蔽名單的 log,證明檢查看得到插件寫的 log。 +Future> runCheck( + PluginDirectory plugin, + PluginCheck check, { + Log Function(Redactor redactor) log = _facade, +}) async { + final name = check.capability.wireName; + late final ReplayAdapter replay; + final environment = _Environment( + (redactor) => replay = ReplayAdapter( + plugin.fixtures[name] ?? const [], + redactor, + AllowedHosts(plugin.manifest.allowedHosts), + ), + log: log, + ); + final (:problems, matched: _) = await _execute(plugin, check, environment); + if (!environment.loaded) return problems; + return [ + ...problems, + for (final problem in replay.problems) '$name: $problem', + for (final unused in replay.unused) '$name: $unused was not requested', + ]; +} + +/// 錄製(prd 擁有者決定 8):以 [network] 真的送出每個案例的請求,遮蔽後寫進 +/// `fixtures/<能力>/001.json`…(先刪掉那個能力原本的 fixture)。回傳違反之處 +/// 與略過的案例。 +/// +/// 只有案例的結果符合 checks.json 的期望才寫;不符(例如連線失敗是 +/// `NetworkError`)就那個案例什麼都不寫,原本的 fixture 不動,並回報原因。 +/// 網路層重試成功的請求照樣寫:失敗的那幾次沒有回應,不會被錄到。 +/// +/// 只錄得了不需要登入的案例:認證來源是 `NoCredentials`,要登入的請求會以 +/// `AuthRequired` 失敗。`meta.edited` 的 fixture 是手寫或手改的,那個案例略過 +/// 不錄。重試照真的時間等(不像重播立刻重送);[wait] 換掉等待,給執行器自己 +/// 的測試用。 +Future<({List problems, List skipped})> recordContract( + Directory directory, { + required HttpClientAdapter Function() network, + DateTime Function() now = DateTime.now, + Future Function(Duration delay)? wait, +}) async { + final plugin = PluginDirectory.read(directory); + if (plugin.file == null) { + return (problems: plugin.problems, skipped: const []); + } + final problems = []; + final skipped = []; + for (final check in plugin.checks) { + final name = check.capability.wireName; + final existing = plugin.fixtures[name] ?? const []; + if (existing.any((named) => named.fixture.edited != null)) { + skipped.add('$name: has hand-edited fixtures'); + continue; + } + late final RecordingAdapter recorder; + final environment = _Environment( + (redactor) => recorder = RecordingAdapter(network(), redactor, now: now), + wait: wait ?? _delay, + ); + final outcome = await _execute(plugin, check, environment); + problems.addAll(outcome.problems); + if (!environment.loaded) continue; + if (recorder.problems.isNotEmpty) { + problems.addAll([ + for (final problem in recorder.problems) '$name: $problem', + ]); + continue; + } + if (!outcome.matched) { + problems.add( + '$name: fixtures not written: the outcome does not match checks.json', + ); + continue; + } + _writeFixtures( + Directory(p.join(directory.path, 'fixtures', name)), + recorder.recorded, + ); + } + return (problems: problems, skipped: skipped); +} + +void _writeFixtures(Directory target, List fixtures) { + if (target.existsSync()) { + for (final entry in target.listSync()) { + if (entry is File && entry.path.endsWith('.json')) entry.deleteSync(); + } + } + if (fixtures.isEmpty) { + if (target.existsSync() && target.listSync().isEmpty) target.deleteSync(); + return; + } + target.createSync(recursive: true); + for (final (index, fixture) in fixtures.indexed) { + File(p.join(target.path, '${'${index + 1}'.padLeft(3, '0')}.json')) + .writeAsStringSync(fixture.encode()); + } +} + +/// 載入插件、執行 [check],檢查:錯誤都是 `AppError`、案例的期望(DTO 驗證 +/// 失敗是 `ParseError`,期望成功時就不符)、沒有試著連清單外的網域、串流 +/// headers 不帶憑證、log 都遮蔽過。[matched]:結果符合案例的期望。 +Future<({List problems, bool matched})> _execute( + PluginDirectory plugin, + PluginCheck check, + _Environment environment, +) async { + try { + return await _executeLoaded(plugin, check, environment); + } finally { + await environment.close(); + } +} + +Future<({List problems, bool matched})> _executeLoaded( + PluginDirectory plugin, + PluginCheck check, + _Environment environment, +) async { + final name = check.capability.wireName; + final ScriptSourcePlugin source; + try { + source = await environment.load(plugin.file!); + } on AppError catch (error) { + return ( + problems: ['$name: load: ${describeError(error, environment.log)}'], + matched: false, + ); + } + Object? result; + AppError? error; + final problems = []; + try { + result = await check.run(source); + } on AppError catch (thrown) { + error = thrown; + } on Object catch (thrown) { + problems.add('$name: threw ${thrown.runtimeType}, not an AppError'); + } finally { + source.close(); + } + // 每個錯誤只 report 一次。 + final descriptions = Map.identity(); + String describe(AppError error) => descriptions.putIfAbsent( + error, + () => describeError(error, environment.log), + ); + final names = CredentialNames.of(plugin.manifest); + if (error != null) { + final description = describe(error); + // 網路層不送出清單外的請求,丟 Unsupported(原因 `Host not allowed`、 + // `Redirect to a host not allowed`)。插件自己接住吞掉的看不到。 + if (error is Unsupported && description.contains('not allowed')) { + problems.add( + '$name: tried to reach a host outside allowedHosts ($description)', + ); + } + } + // 沒丟 AppError 以外的東西、沒試著出網域,才比對期望。 + final mismatches = problems.isEmpty + ? check.expectation.evaluate( + result: result, + error: error, + describe: describe, + ) + : null; + problems.addAll([ + for (final problem in mismatches ?? const []) '$name: $problem', + ]); + if (result case final List candidates) { + problems.addAll([ + for (final problem in mediaHeaderProblems( + candidates, + environment.redactor, + names, + )) + '$name: $problem', + ]); + } + problems.addAll([ + for (final problem in inspectLog( + environment.log.history, + environment.redactor, + names, + )) + '$name: $problem', + ]); + return (problems: problems, matched: mismatches?.isEmpty ?? false); +} + +/// 一個案例的執行環境。 +final class _Environment { + _Environment( + this._adapter, { + this._wait = _noWait, + Log Function(Redactor redactor) log = _facade, + }) : _createLog = log; + + final HttpClientAdapter Function(Redactor redactor) _adapter; + + /// 重試與限流的等待:重播不等,錄製照真的時間等。 + final Future Function(Duration delay) _wait; + final Log Function(Redactor redactor) _createLog; + final redactor = Redactor(); + + /// 插件與宿主寫的 log;檢查的是它的歷史。 + late final Log log = _createLog(redactor); + + /// 插件載入成功過。 + bool loaded = false; + + /// 每個案例一份記憶體資料庫,案例結束就關(drift 在 debug 下會對同時開著的 + /// 多個 `AppDatabase` 警告)。 + AppDatabase? _database; + + Future close() async => _database?.close(); + + Future load(PluginFile file) async { + final database = _database = AppDatabase( + DatabaseConnection( + NativeDatabase.memory(), + closeStreamsSynchronously: true, + ), + ); + // storage 有外鍵:先放一列(內容不重要)。 + await PluginRepository(database).install( + InstalledPlugin( + id: file.manifest.id, + version: file.manifest.version, + manifestJson: file.manifestJson, + script: '', + installedAt: DateTime.utc(2026), + ), + ); + final loader = ScriptPluginLoader( + log: log, + redactor: redactor, + httpClients: SourceHttpClientFactory( + log: log, + createAdapter: () => _adapter(redactor), + wait: _wait, + random: math.Random(7), + ), + storage: PluginStorageRepository(database), + ); + final plugin = await loader.load(file); + loaded = true; + return plugin; + } +} + +Future _noWait(Duration _) async {} + +Future _delay(Duration delay) => Future.delayed(delay); + +/// 案例的 log:經 [redactor] 的門面,debug 以上都留。 +Log _facade(Redactor redactor) => + Log(redactor: redactor, minimumLevel: LogLevel.debug); + +/// 找不到任何插件目錄時的說明。 +String noPluginDirectories(Directory root) => + 'no plugin directory in ${root.absolute.path}: a plugin directory holds ' + 'exactly one .js install file'; + +/// `FMP_PLUGIN_DIR` 指的目錄;沒設就是 `null`。 +Directory? pluginDirectoryFromEnvironment() => + switch (Platform.environment['FMP_PLUGIN_DIR']) { + null || '' => null, + final path => Directory(path), + }; diff --git a/app/test/plugins/contract/contract_runner_test.dart b/app/test/plugins/contract/contract_runner_test.dart new file mode 100644 index 00000000..35bbe4f6 --- /dev/null +++ b/app/test/plugins/contract/contract_runner_test.dart @@ -0,0 +1,459 @@ +import 'dart:convert'; +import 'dart:io'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:fmp/core/logging/log.dart'; +import 'package:fmp/core/logging/log_record.dart'; +import 'package:fmp/core/redaction/redactor.dart'; +import 'package:fmp/plugins/manifest/plugin_manifest.dart'; +import 'package:path/path.dart' as p; + +import 'contract_runner.dart'; +import 'credential_scan.dart'; +import 'plugin_copy.dart'; + +// 契約執行器自己的測試:每一種違反都造一個讓它紅,也改一個無關處證明它不會 +// 誤紅(雙向變異)。變異都在 test/fixtures/plugins/ 的副本上做。 + +const _script = 'http_test_plugin.js'; +const _checks = 'checks.json'; +final _searchFixture = p.join('fixtures', 'search', '001.json'); +final _streamFixture = p.join('fixtures', 'resolveStream', '001.json'); + +/// 讓 `fmp-test-http` 的 resolveStream 成功(原本是錯誤案例)。 +void resolveStreamSucceeds(Directory directory) { + edit( + directory, + _streamFixture, + '"jsonBody": { "code": 403 }', + '"jsonBody": { "code": 0, "url": "https://media.fmp.test/a1.m4a" }', + ); + edit( + directory, + _checks, + '"expect": { "error": "Unavailable", "reason": "copyright" }', + '"expect": { "minItems": 1, "nonEmpty": ["url", "headers"] }', + ); +} + +/// 讓 `fmp-test-http` 的 search 把一個假的 demo_session 寫進 log 的欄位 +/// (manifest 追加的遮蔽鍵名)。 +void logDemoSession(Directory directory) { + edit( + directory, + _script, + 'demo_session: json.demo_session,', + "demo_session: 'FAKE_DEMO_SESSION_0001',", + ); +} + +void main() { + group('stays green', () { + test('on both test plugins', () async { + expect(await runContract(copyPlugin('http_test_plugin')), isEmpty); + expect(await runContract(copyPlugin('test_plugin')), isEmpty); + }); + + test('when the install file is renamed', () async { + final directory = copyPlugin('http_test_plugin'); + File(p.join(directory.path, _script)) + .renameSync(p.join(directory.path, 'plugin.js')); + + expect(await runContract(directory), isEmpty); + }); + + test('when checks.json and fixtures are reformatted', () async { + final directory = copyPlugin('http_test_plugin'); + for (final relative in [_checks, _searchFixture, _streamFixture]) { + final file = File(p.join(directory.path, relative)); + final json = jsonDecode(file.readAsStringSync()); + file.writeAsStringSync( + const JsonEncoder.withIndent('\t').convert(json), + ); + } + + expect(await runContract(directory), isEmpty); + }); + + test('when the fixture lists the query in another order', () async { + final directory = copyPlugin('http_test_plugin'); + edit( + directory, + _searchFixture, + '?access_key=***&keyword=tone&page=1', + '?page=1&access_key=***&keyword=tone', + ); + + expect(await runContract(directory), isEmpty); + }); + + test('when request headers in a fixture change (not matched)', () async { + final directory = copyPlugin('http_test_plugin'); + edit( + directory, + _searchFixture, + '"referer": "https://www.fmp.test/"', + '"referer": "https://other.fmp.test/", "x-trace": "1"', + ); + + expect(await runContract(directory), isEmpty); + }); + + test('when the plugin logs a credential the log facade redacts', () async { + final directory = copyPlugin('http_test_plugin'); + logDemoSession(directory); + + expect(await runContract(directory), isEmpty); + }); + + test('when stream headers hold only media headers', () async { + final directory = copyPlugin('http_test_plugin'); + resolveStreamSucceeds(directory); + + expect(await runContract(directory), isEmpty); + }); + }); + + group('turns red on', () { + test('a request to a host outside allowedHosts', () async { + final directory = copyPlugin('http_test_plugin'); + edit( + directory, + _script, + "const API = 'https://api.fmp.test';", + "const API = 'https://api.evil.test';", + ); + + final problems = await runContract(directory); + + expect( + problems, + contains( + startsWith('search: tried to reach a host outside allowedHosts'), + ), + ); + }); + + test('a fixture on a host outside allowedHosts', () async { + final directory = copyPlugin('http_test_plugin'); + edit( + directory, + _searchFixture, + 'https://api.fmp.test/', + 'https://api.evil.test/', + ); + + expect( + await runContract(directory), + contains(contains('api.evil.test is outside allowedHosts')), + ); + }); + + test('a request that matches no fixture', () async { + final directory = copyPlugin('http_test_plugin'); + edit(directory, _searchFixture, '/search?', '/find?'); + + expect( + await runContract(directory), + contains( + allOf(startsWith('search: request #1'), contains('does not match')), + ), + ); + }); + + test('a request after the fixtures ran out', () async { + final directory = copyPlugin('http_test_plugin'); + File(p.join(directory.path, _streamFixture)).deleteSync(); + + expect( + await runContract(directory), + contains( + allOf( + startsWith('resolveStream: request #1'), + contains('no fixture'), + ), + ), + ); + }); + + test('a fixture that is never requested', () async { + final directory = copyPlugin('http_test_plugin'); + File(p.join(directory.path, _searchFixture)) + .copySync(p.join(directory.path, 'fixtures', 'search', '002.json')); + + expect( + await runContract(directory), + contains('search: fixtures/search/002.json was not requested'), + ); + }); + + test('fixtures for a capability without a check', () async { + final directory = copyPlugin('http_test_plugin'); + Directory(p.join(directory.path, 'fixtures', 'charts')).createSync(); + + expect( + await runContract(directory), + contains('fixtures/charts: checks.json has no charts check'), + ); + }); + + test('too few items', () async { + final directory = copyPlugin('http_test_plugin'); + edit(directory, _checks, '"minItems": 2', '"minItems": 5'); + + expect( + await runContract(directory), + contains('search: expected at least 5 items, got 2'), + ); + }); + + test('an empty field', () async { + final directory = copyPlugin('http_test_plugin'); + edit(directory, _searchFixture, '"owner": "FMP"', '"owner": " "'); + + expect( + await runContract(directory), + contains('search: item 0: "uploader" is empty'), + ); + }); + + test('another error than expected', () async { + final directory = copyPlugin('http_test_plugin'); + edit(directory, _checks, '"reason": "copyright"', '"reason": "region"'); + + expect( + await runContract(directory), + contains( + startsWith( + 'resolveStream: expected Unavailable (region), got ' + 'Unavailable', + ), + ), + ); + }); + + test('success where an error is expected', () async { + final directory = copyPlugin('http_test_plugin'); + edit( + directory, + _streamFixture, + '"jsonBody": { "code": 403 }', + '"jsonBody": { "code": 0, "url": "https://media.fmp.test/a1.m4a" }', + ); + + expect( + await runContract(directory), + contains( + 'resolveStream: expected Unavailable (copyright), but the call ' + 'succeeded', + ), + ); + }); + + test('a return value that fails DTO validation', () async { + final directory = copyPlugin('http_test_plugin'); + edit(directory, _searchFixture, '"name": "Tone A"', '"name": 5'); + + expect( + await runContract(directory), + contains( + allOf( + startsWith('search: expected success, got ParseError'), + contains('SearchPage.items[0].title'), + ), + ), + ); + }); + + test('a declared capability that is not exported', () async { + final directory = copyPlugin('http_test_plugin'); + edit( + directory, + _script, + '"capabilities": ["search", "resolveStream"]', + '"capabilities": ["search", "resolveStream", "charts"]', + ); + + expect( + await runContract(directory), + contains( + allOf( + startsWith('load: Unsupported'), + contains('declared but not exported: charts'), + ), + ), + ); + }); + + test('an exported capability that is not declared', () async { + final directory = copyPlugin('http_test_plugin'); + edit( + directory, + _script, + 'export async function resolveStream', + 'export function charts() {}\nexport async function resolveStream', + ); + + expect( + await runContract(directory), + contains(contains('exported but not declared: charts')), + ); + }); + + test('a check for an undeclared capability', () async { + final directory = copyPlugin('test_plugin'); + edit( + directory, + 'test_plugin.js', + '"capabilities": ["search", "resolveStream"]', + '"capabilities": ["search"]', + ); + + expect( + await runContract(directory), + contains('checks.json: resolveStream is not a declared capability'), + ); + }); + + test('an unknown field in checks.json', () async { + final directory = copyPlugin('http_test_plugin'); + edit(directory, _checks, '"minItems": 2', '"minItems": 2, "maxItems": 3'); + + expect( + await runContract(directory), + contains('checks.json: checks.search.expect: unknown field "maxItems"'), + ); + }); + + test('a missing checks.json', () async { + final directory = copyPlugin('http_test_plugin'); + File(p.join(directory.path, _checks)).deleteSync(); + + expect(await runContract(directory), contains('checks.json is missing')); + }); + + test('a fixture with an unredacted credential', () async { + final directory = copyPlugin('http_test_plugin'); + edit( + directory, + _searchFixture, + 'access_key=***', + 'access_key=FAKE_ACCESS_KEY_0001', + ); + + expect( + await runContract(directory), + contains( + 'fixtures/search/001.json: request.url has an unredacted ' + '"access_key"', + ), + ); + }); + + test('a plugin log that skipped redaction', () async { + // 案例的 log 沒接上插件追加的遮蔽名單(demo_session):插件寫進 log 的 + // 值沒遮,執行器要從插件實際寫的 log 看出來。 + final directory = copyPlugin('http_test_plugin'); + logDemoSession(directory); + final plugin = PluginDirectory.read(directory); + final search = plugin.checks.firstWhere( + (check) => check.capability == PluginCapability.search, + ); + + expect( + await runCheck( + plugin, + search, + log: (_) => Log(redactor: Redactor(), minimumLevel: LogLevel.debug), + ), + contains( + allOf( + startsWith('search: log record'), + contains('fields.demo_session is not redacted'), + ), + ), + ); + }); + + test('stream headers that carry a credential', () async { + final directory = copyPlugin('http_test_plugin'); + resolveStreamSucceeds(directory); + edit( + directory, + _script, + 'headers: { Referer: REFERER },', + "headers: { Cookie: 'SESSDATA=FAKE_SESSDATA_0001' },", + ); + + expect( + await runContract(directory), + contains( + startsWith( + 'resolveStream: candidate #1: header "Cookie" carries a ' + 'credential', + ), + ), + ); + }); + }); + + group('the log inspection', () { + final names = CredentialNames(); + LogRecord record( + String message, { + Map fields = const {}, + }) => LogRecord( + time: DateTime.utc(2026, 9, 30), + level: LogLevel.info, + tag: 'fmp-test-http', + message: message, + fields: fields, + ); + + test('reports a record that skipped redaction', () { + final problems = inspectLog( + [ + record( + 'search https://api.fmp.test/search?access_key=FAKE_ACCESS_KEY_0001', + ), + record('ok', fields: {'token': 'FAKE_TOKEN_0001'}), + ], + Redactor(), + names, + ); + + expect(problems, [ + 'log record #1 (fmp-test-http): message changes when redacted again', + 'log record #1 (fmp-test-http): message has an unredacted "access_key"', + 'log record #2 (fmp-test-http): fields change when redacted again', + 'log record #2 (fmp-test-http): fields.token is not redacted', + ]); + }); + + test('passes what the log facade wrote', () { + final redactor = Redactor(); + final log = Log(redactor: redactor, minimumLevel: LogLevel.debug) + ..info( + 'search https://api.fmp.test/search?access_key=FAKE_ACCESS_KEY_0001', + tag: 'fmp-test-http', + fields: {'token': 'FAKE_TOKEN_0001'}, + ) + ..info('{"token":["FAKE_TOKEN_0001"]}', tag: 'fmp-test-http'); + + expect(inspectLog(log.history, redactor, names), isEmpty); + }); + + test('passes look-alike names that are not credentials', () { + expect( + inspectLog( + [ + record('no token found', fields: {'tokens_used': 3, 'bvid': 'BV1'}), + ], + Redactor(), + names, + ), + isEmpty, + ); + }); + }); +} diff --git a/app/test/plugins/contract/contract_test.dart b/app/test/plugins/contract/contract_test.dart new file mode 100644 index 00000000..dc24c4c2 --- /dev/null +++ b/app/test/plugins/contract/contract_test.dart @@ -0,0 +1,37 @@ +import 'dart:io'; + +import 'package:flutter_test/flutter_test.dart'; + +import 'contract_runner.dart'; + +// 插件契約執行器的入口(ADR 0015 §決定 6):以重播跑插件目錄裡每條檢查案例。 +// +// - 不設 `FMP_PLUGIN_DIR`:跑 `test/fixtures/plugins/` 底下的測試插件,裸 +// `flutter test` 就包含它(CI 的 `app` job)。 +// - 設了:只跑它指的目錄(一個插件目錄,或底下的每個插件目錄)。插件庫的 CI +// 以固定的 FMP 版本這樣跑,指令見 app/AGENTS.md § 驗證。 +void main() { + final root = + pluginDirectoryFromEnvironment() ?? Directory('test/fixtures/plugins'); + final directories = root.existsSync() + ? pluginDirectories(root) + : const []; + + test('finds plugin directories', () { + expect(directories, isNotEmpty, reason: noPluginDirectories(root)); + }); + + for (final directory in directories) { + final plugin = PluginDirectory.read(directory); + group(plugin.label, () { + test('install file, checks.json and fixtures', () async { + expect(await checkPluginDirectory(plugin), isEmpty); + }); + for (final check in plugin.checks) { + test(check.capability.wireName, () async { + expect(await runCheck(plugin, check), isEmpty); + }); + } + }); + } +} diff --git a/app/test/plugins/contract/credential_scan.dart b/app/test/plugins/contract/credential_scan.dart new file mode 100644 index 00000000..80dc5f4a --- /dev/null +++ b/app/test/plugins/contract/credential_scan.dart @@ -0,0 +1,201 @@ +import 'dart:convert'; + +import 'package:fmp/core/logging/log_record.dart'; +import 'package:fmp/core/redaction/redaction_lists.dart'; +import 'package:fmp/core/redaction/redactor.dart'; +import 'package:fmp/plugins/manifest/plugin_manifest.dart'; +import 'package:fmp/plugins/source_dto.dart'; + +import 'fixture.dart'; + +// 憑證檢查:fixture、插件的 log、串流 headers 裡不得有未遮蔽的憑證 +// (ADR 0015 §如何確認、ADR 0011、ADR 0012)。兩道獨立的檢查: +// +// 1. 以正式的遮蔽函式再遮一次,結果必須不變(已經遮過); +// 2. 依名單直接看欄位:名單上的 header 與鍵名,值必須是 `***`(或空、null)。 +// 不經 Redactor 的正規式,Redactor 本身漏遮時由這一道抓到。 +// +// 名單就是 Redactor 的名單(`redaction_lists.dart` 加 manifest 追加的),比對 +// 方式也相同:header 名稱整個相同、鍵名以它結尾,都不分大小寫。 + +/// 憑證的 header 與鍵名。 +final class CredentialNames { + CredentialNames({ + Iterable headerNames = const [], + Iterable keyNames = const [], + }) : _headers = { + for (final name in [...builtInHeaderNames, ...headerNames]) + name.toLowerCase(), + }, + _keys = { + for (final name in [...builtInKeyNames, ...keyNames]) + name.toLowerCase(), + } { + // 名稱、結尾引號、`:` 或 `=`、開頭引號或 `[`、值(到分隔符為止)。header + // 名稱前面不是字元或 `-`;鍵名前面不設邊界(以它結尾就算),同 Redactor。 + // `[` 開頭的值 Redactor 遮成 `[***]`(多值的 header、JSON 陣列),不跳過 + // `[` 就會把遮好的值當成沒遮。 + const separator = r'''\\?["']?[ \t]*[:=][ \t]*\[?[ \t]*\\?["']?'''; + const value = r'''([^"'&;\s,}\])\\]*)'''; + _text = RegExp( + '(?:(? CredentialNames( + headerNames: manifest.redaction.headerNames, + keyNames: manifest.redaction.keyNames, + ); + + final Set _headers; + final Set _keys; + late final RegExp _text; + + bool isSensitive(String name) { + final lower = name.toLowerCase(); + return _headers.contains(lower) || _keys.any(lower.endsWith); + } + + /// 文字裡 `名稱: 值`、`名稱=值`、`"名稱": "值"` 的值沒遮的,回傳名稱。 + List unredactedInText(String text) => [ + for (final match in _text.allMatches(text)) + if (!_isRedacted(match[3]!)) match[1] ?? match[2]!, + ]; + + /// JSON 值(物件、陣列、字串)裡沒遮的憑證,回傳它們的路徑。 + List unredactedInJson(Object? value, String path) => switch (value) { + final Map map => [ + for (final MapEntry(:key, value: entry) in map.entries) + if (isSensitive('$key') && entry != null && entry != redactedValue) + '$path.$key' + else + ...unredactedInJson(entry, '$path.$key'), + ], + final List items => [ + for (final (index, item) in items.indexed) + ...unredactedInJson(item, '$path[$index]'), + ], + final String text => [ + for (final name in unredactedInText(text)) '$path ("$name")', + ], + _ => const [], + }; + + static bool _isRedacted(String value) => + value.isEmpty || value == redactedValue || value == 'null'; + + static String _alternation(Iterable names) => + (names.toList()..sort((a, b) => b.length.compareTo(a.length))) + .map(RegExp.escape) + .join('|'); +} + +/// fixture 裡沒遮的憑證(兩道檢查)。[name] 放在訊息開頭。 +List scanFixture( + String name, + HttpFixture fixture, + Redactor redactor, + CredentialNames names, +) { + final original = fixture.toJson(); + final again = fixture.redacted(redactor).toJson(); + final problems = [ + for (final part in ['request', 'response']) + for (final key in { + ...(original[part]! as Map).keys, + ...(again[part]! as Map).keys, + }) + if (jsonEncode((original[part]! as Map)[key]) != + jsonEncode((again[part]! as Map)[key])) + '$name: $part.$key changes when redacted again', + ]; + final request = fixture.request; + final response = fixture.response; + problems.addAll([ + for (final key in names.unredactedInText(request.url)) + '$name: request.url has an unredacted "$key"', + ..._headerProblems(name, 'request', { + for (final MapEntry(:key, :value) in request.headers.entries) + key: [value], + }, names), + ..._headerProblems(name, 'response', response.headers, names), + for (final path in [ + ...names.unredactedInJson(request.body, 'request.body'), + ...names.unredactedInJson(response.body, 'response.body'), + ...names.unredactedInJson(response.jsonBody, 'response.jsonBody'), + ]) + '$name: $path is not redacted', + ]); + return problems; +} + +List _headerProblems( + String name, + String part, + Map> headers, + CredentialNames names, +) => [ + for (final MapEntry(key: header, value: values) in headers.entries) + for (final value in values) + if (header == 'set-cookie' + ? !_cookieValueRedacted(value) + : names.isSensitive(header) + ? value != redactedValue + : names.unredactedInText(value).isNotEmpty) + '$name: $part.headers.$header is not redacted', +]; + +/// `名稱=***; 屬性`(`redactSetCookie` 的輸出)或整個 `***`。 +bool _cookieValueRedacted(String value) { + if (value == redactedValue) return true; + final pair = value.split(';').first; + final equals = pair.indexOf('='); + return equals > 0 && pair.substring(equals + 1).trim() == redactedValue; +} + +/// log 裡沒遮的憑證(兩道檢查):訊息、error、stackTrace 與結構化欄位。 +List inspectLog( + Iterable records, + Redactor redactor, + CredentialNames names, +) => [ + for (final (index, record) in records.indexed) + for (final problem in [ + for (final (part, text) in [ + ('message', record.message), + ('error', record.error), + ('stackTrace', record.stackTrace), + ]) + if (text != null) ...[ + if (redactor.redact(text) != text) + '$part changes when redacted again', + for (final key in names.unredactedInText(text)) + '$part has an unredacted "$key"', + ], + if (jsonEncode(redactor.redactValue(record.fields)) != + jsonEncode(record.fields)) + 'fields change when redacted again', + for (final path in names.unredactedInJson(record.fields, 'fields')) + '$path is not redacted', + ]) + 'log record #${index + 1} (${record.tag}): $problem', +]; + +/// 串流候選的 headers 不得帶憑證(ADR 0012:媒體請求不帶憑證)。M1 還沒有 +/// 媒體 client,所以直接看插件回傳的 headers:名單上的 header,或值裡有 +/// 會被遮蔽的內容。 +List mediaHeaderProblems( + List candidates, + Redactor redactor, + CredentialNames names, +) => [ + for (final (index, candidate) in candidates.indexed) + for (final MapEntry(key: name, :value) in candidate.headers.entries) + if (names.isSensitive(name) || + redactor.redact('$name: $value') != '$name: $value') + 'candidate #${index + 1}: header "$name" carries a credential; media ' + 'requests must not (ADR 0012)', +]; diff --git a/app/test/plugins/contract/fixture.dart b/app/test/plugins/contract/fixture.dart new file mode 100644 index 00000000..3d95ea67 --- /dev/null +++ b/app/test/plugins/contract/fixture.dart @@ -0,0 +1,327 @@ +import 'dart:convert'; + +import 'package:dio/dio.dart'; +import 'package:fmp/core/redaction/redactor.dart'; +import 'package:fmp/plugins/json_shape.dart'; + +// fixture:一次 HTTP 請求與它的回應,一個 JSON 檔(ADR 0015 §決定 5)。欄位名稱 +// 沿用 WireMock stub mapping(`request.method`/`url`、`response.status`/ +// `headers`/`body`/`jsonBody`),外層加 `meta`。檔案在插件目錄的 +// `fixtures/<能力>/`,依檔名排序就是請求的順序。 +// +// 形狀與 `lib/plugins/types/fmp-plugin.d.ts` 的 `FmpFixture*` 一致 +// (test/plugins/type_definitions_test.dart 比對)。 + +/// fixture 的欄位表,鍵是 `fmp-plugin.d.ts` 裡的 interface 名稱。 +const fixtureShapes = { + 'FmpFixture': {'meta': true, 'request': true, 'response': true}, + 'FmpFixtureMeta': {'recordedAt': false, 'edited': false}, + 'FmpFixtureRequest': { + 'method': true, + 'url': true, + 'headers': false, + 'body': false, + }, + 'FmpFixtureResponse': { + 'status': true, + 'headers': false, + 'body': false, + 'jsonBody': false, + }, +}; + +/// 錄製時不留的回應 header(名稱小寫):body 已經解碼、遮蔽後重新編碼,長度 +/// 與編碼都不再是原本的。 +const droppedResponseHeaders = { + 'content-length', + 'content-encoding', + 'transfer-encoding', +}; + +/// 一個 fixture 檔。 +final class HttpFixture { + const HttpFixture({ + this.recordedAt, + this.edited, + required this.request, + required this.response, + }); + + /// 解碼一個 fixture 檔的 JSON;形狀不對拋 [FormatException]。 + factory HttpFixture.fromJson(Object? json) { + final fields = JsonFields( + json, + fixtureShapes['FmpFixture']!, + path: 'fixture', + ); + final meta = JsonFields( + fields.raw('meta'), + fixtureShapes['FmpFixtureMeta']!, + path: 'fixture.meta', + ); + final recordedAt = meta.optionalString('recordedAt'); + if (recordedAt != null && DateTime.tryParse(recordedAt) == null) { + throw const FormatException('fixture.meta.recordedAt: not ISO 8601'); + } + return HttpFixture( + recordedAt: recordedAt, + edited: meta.optionalString('edited'), + request: FixtureRequest._fromJson(fields.raw('request')), + response: FixtureResponse._fromJson(fields.raw('response')), + ); + } + + /// 錄製時間(ISO 8601,UTC);手寫的是 `null`。 + final String? recordedAt; + + /// 手寫或手改的理由。有它的案例,錄製模式不覆蓋。 + final String? edited; + + final FixtureRequest request; + final FixtureResponse response; + + Map toJson() => { + 'meta': {'recordedAt': ?recordedAt, 'edited': ?edited}, + 'request': request.toJson(), + 'response': response.toJson(), + }; + + /// 寫檔的內容:兩格縮排、結尾換行。 + String encode() => + '${const JsonEncoder.withIndent(' ').convert(toJson())}\n'; + + /// 經正式的遮蔽函式(ADR 0011 §決定 3)後的 fixture。錄製寫檔前一律經過它; + /// fixture 掃描以「再遮一次結果不變」確認檔案已經遮過。 + /// + /// - 網址、request body、文字的 response body:`Redactor.redact`; + /// - header:`Redactor.redactValue({名稱: 值})`,名單上的 header 整個換成 + /// `***`;`set-cookie` 例外,只換每個 cookie 的值(`名稱=***; 屬性`),重播 + /// 時 cookie 管理才解析得了; + /// - `jsonBody`:`Redactor.redactValue`,鍵在名單上的值整個換成 `***`,結果 + /// 仍是合法的 JSON。 + HttpFixture redacted(Redactor redactor) => HttpFixture( + recordedAt: recordedAt, + edited: edited, + request: FixtureRequest( + method: request.method, + url: redactor.redact(request.url), + headers: { + for (final MapEntry(:key, :value) in request.headers.entries) + key: _redactHeader(redactor, key, value), + }, + body: switch (request.body) { + null => null, + final body => redactor.redact(body), + }, + ), + response: FixtureResponse( + status: response.status, + headers: { + for (final MapEntry(:key, :value) in response.headers.entries) + key: [for (final item in value) _redactHeader(redactor, key, item)], + }, + body: switch (response.body) { + null => null, + final body => redactor.redact(body), + }, + jsonBody: switch (response.jsonBody) { + null => null, + final json => redactor.redactValue(json), + }, + ), + ); +} + +String _redactHeader(Redactor redactor, String name, String value) { + if (name == 'set-cookie') return redactSetCookie(redactor, value); + final redacted = redactor.redactValue({name: value})! as Map; + return redacted.values.single as String; +} + +/// `名稱=值; 屬性` 的值換成 `***`,名稱與屬性經 `Redactor.redact`。 +String redactSetCookie(Redactor redactor, String value) { + final end = value.indexOf(';'); + final pair = end < 0 ? value : value.substring(0, end); + final attributes = end < 0 ? '' : value.substring(end); + final equals = pair.indexOf('='); + final name = equals < 0 ? '' : pair.substring(0, equals).trim(); + if (name.isEmpty) return redactedValue; + return '${redactor.redact(name)}=$redactedValue' + '${redactor.redact(attributes)}'; +} + +/// fixture 的請求。 +final class FixtureRequest { + const FixtureRequest({ + required this.method, + required this.url, + this.headers = const {}, + this.body, + }); + + factory FixtureRequest._fromJson(Object? json) { + final fields = JsonFields( + json, + fixtureShapes['FmpFixtureRequest']!, + path: 'fixture.request', + ); + final url = fields.string('url'); + final uri = Uri.tryParse(url); + if (uri == null || !uri.hasAuthority) { + throw const FormatException('fixture.request.url: not an absolute URL'); + } + return FixtureRequest( + method: fields.nonEmptyString('method'), + url: url, + headers: { + for (final MapEntry(:key, :value) + in (fields.optionalStringMap('headers') ?? {}).entries) + key.toLowerCase(): value, + }, + body: fields.optionalString('body'), + ); + } + + final String method; + + /// 送出的網址(遮蔽過)。比對見 [fixtureUrlMatches]。 + final String url; + + /// 只供閱讀,不參與比對。名稱小寫。 + final Map headers; + + /// 只供閱讀,不參與比對。 + final String? body; + + Map toJson() => { + 'method': method, + 'url': url, + if (headers.isNotEmpty) 'headers': _sorted(headers), + 'body': ?body, + }; +} + +/// fixture 的回應。 +final class FixtureResponse { + const FixtureResponse({ + required this.status, + this.headers = const {}, + this.body, + this.jsonBody, + }) : assert(body == null || jsonBody == null); + + factory FixtureResponse._fromJson(Object? json) { + final fields = JsonFields( + json, + fixtureShapes['FmpFixtureResponse']!, + path: 'fixture.response', + ); + final jsonBody = fields.raw('jsonBody'); + if (jsonBody != null && jsonBody is! Map && jsonBody is! List) { + throw const FormatException( + 'fixture.response.jsonBody: expected an object or an array', + ); + } + final body = fields.optionalString('body'); + if (body != null && jsonBody != null) { + throw const FormatException( + 'fixture.response: body and jsonBody are mutually exclusive', + ); + } + return FixtureResponse( + status: fields.integer('status', min: 100), + headers: _headerLists(fields.optionalObject('headers') ?? {}), + body: body, + jsonBody: jsonBody, + ); + } + + final int status; + + /// 名稱小寫。 + final Map> headers; + + /// 文字的 body。 + final String? body; + + /// JSON 的 body(物件或陣列),重播時編碼成緊湊的 JSON 文字。 + final Object? jsonBody; + + /// 交給 dio 的回應。 + ResponseBody toResponseBody() => ResponseBody.fromString( + switch (jsonBody) { + null => body ?? '', + final json => jsonEncode(json), + }, + status, + headers: headers, + ); + + Map toJson() => { + 'status': status, + if (headers.isNotEmpty) 'headers': _sorted(headers), + 'body': ?body, + 'jsonBody': ?jsonBody, + }; +} + +Map> _headerLists(Map json) => { + for (final MapEntry(:key, :value) in json.entries) + key.toLowerCase(): switch (value) { + final List items when items.every((item) => item is String) => + items.cast(), + _ => throw FormatException( + 'fixture.response.headers.$key: expected an array of strings', + ), + }, +}; + +Map _sorted(Map map) => { + for (final key in map.keys.toList()..sort()) key: map[key] as T, +}; + +/// 重播的比對(ADR 0015 §決定 5):scheme、host、port、路徑與 query 相同; +/// query 不分順序(Polly.js、VCR 的做法)。[fixture] 裡值是 `***` 的 query +/// 參數與路徑段是遮蔽過的欄位,不比對值,只要求它存在。 +/// +/// 呼叫端先以同一個遮蔽函式遮過 [actual]:遮蔽會拿掉的簽名參數兩邊都不會有。 +bool fixtureUrlMatches(Uri fixture, Uri actual) { + if (fixture.scheme.toLowerCase() != actual.scheme.toLowerCase() || + fixture.host.toLowerCase() != actual.host.toLowerCase() || + fixture.port != actual.port) { + return false; + } + final expectedPath = fixture.pathSegments; + final actualPath = actual.pathSegments; + if (expectedPath.length != actualPath.length) return false; + for (var i = 0; i < expectedPath.length; i++) { + if (expectedPath[i] != redactedValue && expectedPath[i] != actualPath[i]) { + return false; + } + } + final expectedQuery = fixture.queryParametersAll; + final actualQuery = actual.queryParametersAll; + if (expectedQuery.length != actualQuery.length) return false; + for (final MapEntry(key: name, value: expected) in expectedQuery.entries) { + final values = [...?actualQuery[name]]; + if (values.length != expected.length) return false; + for (final value in expected) { + if (value == redactedValue) continue; + if (!values.remove(value)) return false; + } + } + return true; +} + +/// 錯誤訊息用的網址寫法:query 解碼後依名稱、值排序(只給人看,不是合法的 +/// 網址)。 +String canonicalUrl(Uri uri) { + final pairs = [ + for (final MapEntry(key: name, value: values) + in uri.queryParametersAll.entries) + for (final value in values) (name, value), + ]..sort((a, b) => a.$1 == b.$1 ? a.$2.compareTo(b.$2) : a.$1.compareTo(b.$1)); + final query = [for (final (name, value) in pairs) '$name=$value'].join('&'); + return '${uri.scheme}://${uri.authority}${uri.path}' + '${query.isEmpty ? '' : '?$query'}'; +} diff --git a/app/test/plugins/contract/fixture_adapters.dart b/app/test/plugins/contract/fixture_adapters.dart new file mode 100644 index 00000000..16285675 --- /dev/null +++ b/app/test/plugins/contract/fixture_adapters.dart @@ -0,0 +1,166 @@ +import 'dart:async'; +import 'dart:convert'; +import 'dart:typed_data'; + +import 'package:dio/dio.dart'; +import 'package:fmp/core/network/allowed_hosts.dart'; +import 'package:fmp/core/redaction/redactor.dart'; + +import 'fixture.dart'; + +// 錄製與重播接在網路層最底部的 dio `HttpClientAdapter` +// (`SourceHttpClientFactory` 的 `createAdapter`,ADR 0015 §決定 5):攔截器 +// 全部照常執行,只有真正送出這一步換掉。 + +/// fixture 檔與它的檔名(錯誤訊息用)。 +typedef NamedFixture = ({String name, HttpFixture fixture}); + +/// 重播:第 n 個請求必須對上第 n 個 fixture(method 與 [fixtureUrlMatches]), +/// 對上就回它的回應;對不上、或 fixture 用完了,記進 [problems] 並讓這次請求 +/// 失敗,不發真實請求。 +final class ReplayAdapter implements HttpClientAdapter { + ReplayAdapter(this._fixtures, this._redactor, this._allowedHosts); + + final List _fixtures; + final Redactor _redactor; + final AllowedHosts _allowedHosts; + int _next = 0; + + /// 比對不到的請求、送到網域清單外的請求。 + final problems = {}; + + /// 沒被請求用到的 fixture。 + List get unused => [ + for (final (:name, fixture: _) in _fixtures.skip(_next)) name, + ]; + + @override + Future fetch( + RequestOptions options, + Stream? requestStream, + Future? cancelFuture, + ) async { + final number = _next + 1; + // 網路層已經擋過;這裡再看一次,擋不住的話契約測試會說出來。 + if (!_allowedHosts.allows(options.uri)) { + problems.add( + 'request #$number went to ${options.uri.host}, outside ' + 'allowedHosts', + ); + } + final actual = Uri.parse(_redactor.redact(options.uri.toString())); + final described = '${options.method} ${canonicalUrl(actual)}'; + if (_next >= _fixtures.length) { + return _mismatch('request #$number ($described) has no fixture left'); + } + final (:name, :fixture) = _fixtures[_next]; + final expected = Uri.parse(fixture.request.url); + if (fixture.request.method != options.method || + !fixtureUrlMatches(expected, actual)) { + return _mismatch( + 'request #$number ($described) does not match $name ' + '(${fixture.request.method} ${canonicalUrl(expected)})', + ); + } + _next++; + return fixture.response.toResponseBody(); + } + + Never _mismatch(String problem) { + problems.add(problem); + // 不是 IOException:網路層把它當成非傳輸錯誤(UnexpectedError),不重試。 + throw StateError('No fixture: $problem'); + } + + @override + void close({bool force = false}) {} +} + +/// 錄製:請求交給 [_network](真實連線或測試的假 adapter),回應原樣交回給 +/// 插件,同時把遮蔽過的一份存進 [recorded]。 +/// +/// body 不是 UTF-8 時不錄(fixture 沒有二進位 body),記進 [problems]。 +final class RecordingAdapter implements HttpClientAdapter { + RecordingAdapter(this._network, this._redactor, {required this.now}); + + final HttpClientAdapter _network; + final Redactor _redactor; + final DateTime Function() now; + + final recorded = []; + final problems = []; + + @override + Future fetch( + RequestOptions options, + Stream? requestStream, + Future? cancelFuture, + ) async { + final response = await _network.fetch(options, requestStream, cancelFuture); + final builder = BytesBuilder(copy: false); + await response.stream.forEach(builder.add); + final bytes = builder.takeBytes(); + final String text; + try { + text = const Utf8Decoder().convert(bytes); + } on FormatException { + problems.add( + '${options.method} ${options.uri.host}${options.uri.path} returned a ' + 'body that is not UTF-8; fixtures only hold text', + ); + return _copy(response, bytes); + } + final json = _json(text); + recorded.add( + HttpFixture( + recordedAt: now().toUtc().toIso8601String(), + request: FixtureRequest( + method: options.method, + url: options.uri.toString(), + headers: { + for (final MapEntry(:key, :value) in options.headers.entries) + if (value != null) key.toLowerCase(): '$value', + }, + body: switch (options.data) { + final String body => body, + _ => null, + }, + ), + response: FixtureResponse( + status: response.statusCode, + headers: { + for (final MapEntry(:key, :value) in response.headers.entries) + if (!droppedResponseHeaders.contains(key.toLowerCase())) + key.toLowerCase(): value, + }, + body: json == null ? text : null, + jsonBody: json, + ), + ).redacted(_redactor), + ); + return _copy(response, bytes); + } + + /// 物件或陣列的 JSON;其他文字是 `null`(存成 `body`)。 + static Object? _json(String text) { + final trimmed = text.trimLeft(); + if (!trimmed.startsWith('{') && !trimmed.startsWith('[')) return null; + try { + return jsonDecode(text); + } on FormatException { + return null; + } + } + + static ResponseBody _copy(ResponseBody response, Uint8List bytes) => + ResponseBody.fromBytes( + bytes, + response.statusCode, + statusMessage: response.statusMessage, + isRedirect: response.isRedirect, + headers: response.headers, + ); + + @override + void close({bool force = false}) => _network.close(force: force); +} diff --git a/app/test/plugins/contract/fixture_scan_test.dart b/app/test/plugins/contract/fixture_scan_test.dart new file mode 100644 index 00000000..22a91ab2 --- /dev/null +++ b/app/test/plugins/contract/fixture_scan_test.dart @@ -0,0 +1,289 @@ +import 'dart:convert'; +import 'dart:io'; + +import 'package:dio/dio.dart'; +import 'package:flutter_test/flutter_test.dart'; +import 'package:fmp/core/network/allowed_hosts.dart'; +import 'package:fmp/core/redaction/redactor.dart'; + +import 'contract_runner.dart'; +import 'credential_scan.dart'; +import 'fixture.dart'; +import 'fixture_adapters.dart'; + +// fixture 掃描(ADR 0015 §如何確認:掃描所有 fixture 不得有未遮蔽憑證)與重播 +// 的網址比對。掃描的兩道檢查各自有會紅與不該紅的案例。 + +HttpFixture _fixture({ + String url = 'https://api.fmp.test/search?page=1', + Map requestHeaders = const {}, + Map> responseHeaders = const {}, + String? body, + Object? jsonBody, +}) => HttpFixture( + request: FixtureRequest(method: 'GET', url: url, headers: requestHeaders), + response: FixtureResponse( + status: 200, + headers: responseHeaders, + body: body, + jsonBody: jsonBody, + ), +); + +List _scan(HttpFixture fixture) => + scanFixture('001.json', fixture, Redactor(), CredentialNames()); + +const _redactedAgain = '001.json: request.url changes when redacted again'; + +void main() { + test('every fixture in app/ is redacted', () { + var scanned = 0; + for (final directory in pluginDirectories( + Directory('test/fixtures/plugins'), + )) { + final plugin = PluginDirectory.read(directory); + final redactor = redactorFor(plugin.manifest); + final names = CredentialNames.of(plugin.manifest); + for (final list in plugin.fixtures.values) { + for (final (:name, :fixture) in list) { + expect(scanFixture(name, fixture, redactor, names), isEmpty); + scanned++; + } + } + } + // 至少掃到 fmp-test-http 的兩個:否則這個測試什麼也沒證明。 + expect(scanned, greaterThanOrEqualTo(2)); + }); + + group('the scan reports', () { + test('a credential in the query (both checks)', () { + expect(_scan(_fixture(url: 'https://api.fmp.test/s?access_key=abc12')), [ + _redactedAgain, + '001.json: request.url has an unredacted "access_key"', + ]); + }); + + test('a signed media URL (only the redaction function knows it)', () { + expect( + _scan( + _fixture( + url: + 'https://upos-sz.bilivideo.com/a.m4s?upsig=0123abcd&deadline=1', + ), + ), + [_redactedAgain], + ); + }); + + test('a credential header', () { + expect( + _scan(_fixture(requestHeaders: {'cookie': 'SESSDATA=abc12'})), + containsAll([ + '001.json: request.headers changes when redacted again', + '001.json: request.headers.cookie is not redacted', + ]), + ); + }); + + test('a Set-Cookie value', () { + expect( + _scan( + _fixture( + responseHeaders: { + 'set-cookie': ['buvid3=abc12; Path=/'], + }, + ), + ), + containsAll([ + '001.json: response.headers changes when redacted again', + '001.json: response.headers.set-cookie is not redacted', + ]), + ); + }); + + test('a credential in a JSON body, however deep', () { + expect( + _scan( + _fixture( + jsonBody: { + 'data': [ + {'access_token': 'abc12'}, + ], + }, + ), + ), + [ + '001.json: response.jsonBody changes when redacted again', + '001.json: response.jsonBody.data[0].access_token is not redacted', + ], + ); + }); + + test('a credential in a text body', () { + expect( + _scan(_fixture(body: 'ok; MUSIC_U=abc12; x=1')), + contains('001.json: response.body ("MUSIC_U") is not redacted'), + ); + }); + + test('a credential in a list value', () { + expect( + _scan(_fixture(body: 'token=[abc12]; cb({"MUSIC_U":["abc12"]})')), + containsAll([ + '001.json: response.body ("token") is not redacted', + '001.json: response.body ("MUSIC_U") is not redacted', + ]), + ); + }); + + test('a credential the redaction function was not told about', () { + // 名單比對不經 Redactor:Redactor 沒登記這個鍵名(例如插件的追加名單沒接 + // 上)時,只有依名單直接看的那一道抓得到。 + expect( + scanFixture( + '001.json', + _fixture(body: 'demo_session=abc12'), + Redactor(), + CredentialNames(keyNames: ['demo_session']), + ), + ['001.json: response.body ("demo_session") is not redacted'], + ); + }); + }); + + group('the scan passes', () { + test('redacted values', () { + expect( + _scan( + _fixture( + url: 'https://api.fmp.test/s?access_key=***', + requestHeaders: {'cookie': '***'}, + responseHeaders: { + 'set-cookie': ['buvid3=***; Path=/'], + }, + jsonBody: {'token': '***', 'list': []}, + ), + ), + isEmpty, + ); + }); + + test('list values the redaction function redacted', () { + // Redactor 把 `[` 開頭的值遮成 `[***]`(多值的 header、JSON 陣列)。 + final body = Redactor().redact('cb({"token":["abc12"]}); Cookie: [a=1]'); + + expect(body, 'cb({"token":[***]}); Cookie: [***]'); + expect(_scan(_fixture(body: body)), isEmpty); + }); + + test('names that only look like credentials', () { + expect( + _scan( + _fixture( + url: 'https://api.fmp.test/s?bvid=BV1xx&tokens_used=3', + body: 'no token here; tokenizer: whitespace', + jsonBody: null, + ), + ), + isEmpty, + ); + expect( + _scan(_fixture(jsonBody: {'tokens_used': 3, 'bvid': 'BV1xx'})), + isEmpty, + ); + }); + + test('a fixture reformatted and re-read', () { + final fixture = _fixture( + url: 'https://api.fmp.test/s?access_key=***', + jsonBody: {'b': 1, 'a': '***'}, + ); + final reread = HttpFixture.fromJson( + jsonDecode( + const JsonEncoder.withIndent('\t').convert(fixture.toJson()), + ), + ); + + expect(_scan(reread), isEmpty); + }); + }); + + group('replay matching', () { + Future replay(String recorded, String actual) async { + final redactor = Redactor(); + final url = Uri.parse(recorded); + final adapter = ReplayAdapter( + [ + ( + name: '001.json', + fixture: HttpFixture( + request: FixtureRequest( + method: 'GET', + url: redactor.redact(recorded), + ), + response: const FixtureResponse(status: 200), + ), + ), + ], + redactor, + AllowedHosts([if (!url.host.contains('evil')) url.host]), + ); + await adapter.fetch(RequestOptions(path: actual), null, null); + return adapter; + } + + test('redacts the actual URL before comparing', () async { + // 錄製時遮蔽拿掉的簽名參數(媒體 CDN),實際的請求一定帶著:比對前要 + // 以同一個遮蔽函式遮過實際的網址,兩邊才一樣。 + const signed = + 'https://upos-sz.bilivideo.com/a.m4s?upsig=0123abcd&deadline=1&x=1'; + final adapter = await replay(signed, signed); + + expect(adapter.problems, isEmpty); + expect(adapter.unused, isEmpty); + }); + + test('reports a request outside allowedHosts that got through', () async { + // 網路層本來就擋;adapter 再看一次,擋不住時契約測試會說出來。 + const url = 'https://api.evil.test/p'; + final adapter = await replay(url, url); + + expect(adapter.problems, [ + 'request #1 went to api.evil.test, outside allowedHosts', + ]); + }); + + bool matches(String fixture, String actual) => + fixtureUrlMatches(Uri.parse(fixture), Uri.parse(actual)); + + test('ignores the order of the query', () { + expect( + matches('https://a.test/p?x=1&y=2', 'https://a.test/p?y=2&x=1'), + isTrue, + ); + }); + + test('skips redacted values but not their names', () { + expect( + matches('https://a.test/p?k=***&x=1', 'https://a.test/p?x=1&k=9'), + isTrue, + ); + expect( + matches('https://a.test/***/f.m4a', 'https://a.test/abc/f.m4a'), + isTrue, + ); + expect(matches('https://a.test/p?k=***', 'https://a.test/p'), isFalse); + }); + + test('compares everything else', () { + expect(matches('https://a.test/p?x=1', 'https://a.test/p?x=2'), isFalse); + expect( + matches('https://a.test/p?x=1', 'https://a.test/p?x=1&y=2'), + isFalse, + ); + expect(matches('https://a.test/p', 'https://b.test/p'), isFalse); + expect(matches('https://a.test/p', 'https://a.test/q'), isFalse); + expect(matches('https://a.test/p', 'https://a.test:8443/p'), isFalse); + }); + }); +} diff --git a/app/test/plugins/contract/plugin_copy.dart b/app/test/plugins/contract/plugin_copy.dart new file mode 100644 index 00000000..e18ea22d --- /dev/null +++ b/app/test/plugins/contract/plugin_copy.dart @@ -0,0 +1,34 @@ +import 'dart:io'; + +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; + +// 契約執行器的測試用:在 test/fixtures/plugins/ 的副本上做變異。 + +/// 把測試插件 [name] 複製到暫存目錄。 +Directory copyPlugin(String name) { + final target = Directory.systemTemp.createTempSync('fmp_contract_'); + addTearDown(() => target.deleteSync(recursive: true)); + final source = Directory(p.join('test', 'fixtures', 'plugins', name)); + for (final entry in source.listSync(recursive: true)) { + final relative = p.relative(entry.path, from: source.path); + if (entry is Directory) { + Directory(p.join(target.path, relative)).createSync(recursive: true); + } else if (entry is File) { + final copy = File(p.join(target.path, relative)); + copy.parent.createSync(recursive: true); + entry.copySync(copy.path); + } + } + return target; +} + +/// 把 [directory] 裡 [relative] 的第一個 [from] 換成 [to];沒換到就讓測試失敗 +/// (變異沒生效的測試什麼也沒證明)。 +void edit(Directory directory, String relative, String from, String to) { + final file = File(p.join(directory.path, relative)); + final before = file.readAsStringSync(); + final after = before.replaceFirst(from, to); + expect(after, isNot(before), reason: '"$from" is not in $relative'); + file.writeAsStringSync(after); +} diff --git a/app/test/plugins/contract/record_test.dart b/app/test/plugins/contract/record_test.dart new file mode 100644 index 00000000..43ab9b64 --- /dev/null +++ b/app/test/plugins/contract/record_test.dart @@ -0,0 +1,251 @@ +import 'dart:convert'; +import 'dart:io'; + +import 'package:dio/dio.dart'; +import 'package:dio/io.dart'; +import 'package:flutter_test/flutter_test.dart'; +import 'package:path/path.dart' as p; + +import '../../support/fake_http_adapter.dart'; +import 'contract_runner.dart'; +import 'plugin_copy.dart'; + +// 錄製模式(prd 擁有者決定 8)。會真的連網的只有最後一個 `live` 測試: +// dart_test.yaml 讓它預設跳過,要錄時明確解除(app/AGENTS.md § 驗證)。其他 +// 測試用假的 adapter 代替網路。 + +/// 上游回給 `fmp-test-http` 的內容,含假的憑證。 +ResponseBody _upstream(RequestOptions options) => switch (options.uri.path) { + '/search' => reply( + 200, + body: jsonEncode({ + 'demo_session': 'FAKE_DEMO_SESSION_0001', + 'list': [ + {'id': 'a1', 'name': 'Tone A', 'owner': 'FMP', 'ms': 2000}, + {'id': 'a2', 'name': 'Tone B', 'owner': 'FMP', 'ms': 3000}, + ], + 'more': false, + }), + headers: { + 'Content-Type': 'application/json', + 'Content-Length': '999', + 'Set-Cookie': 'demo_session=FAKE_DEMO_SESSION_0001; Path=/', + }, + ), + '/stream' => reply(200, body: '{"code":403}'), + _ => reply(404), +}; + +final _searchFixture = p.join('fixtures', 'search', '001.json'); + +/// 插件目錄裡每個檔案的內容,鍵是相對路徑。 +Map> _bytes(Directory directory) => { + for (final file in directory.listSync(recursive: true).whereType()) + p.relative(file.path, from: directory.path): file.readAsBytesSync(), +}; + +Map _read(Directory directory, String capability) => + jsonDecode( + File(p.join(directory.path, 'fixtures', capability, '001.json')) + .readAsStringSync(), + ) as Map; + +void main() { + test('records redacted fixtures that replay', () async { + final directory = copyPlugin('http_test_plugin'); + Directory(p.join(directory.path, 'fixtures')).deleteSync(recursive: true); + final upstream = FakeHttpAdapter(_upstream); + + final recording = await recordContract( + directory, + network: () => upstream, + now: () => DateTime.utc(2026, 9, 30), + ); + + expect(recording.problems, isEmpty); + expect(recording.skipped, isEmpty); + // 插件送出的是真的值;寫進檔案的是遮過的。 + expect( + upstream.requests.first.uri.queryParameters['access_key'], + 'FAKE_ACCESS_KEY_0001', + ); + final files = Directory(p.join(directory.path, 'fixtures')) + .listSync(recursive: true) + .whereType() + .toList(); + expect(files, hasLength(2)); + for (final file in files) { + expect(file.readAsStringSync(), isNot(contains('FAKE_'))); + } + final search = _read(directory, 'search'); + expect(search['meta'], {'recordedAt': '2026-09-30T00:00:00.000Z'}); + expect( + (search['request']! as Map)['url'], + 'https://api.fmp.test/search?page=1&keyword=tone&access_key=***', + ); + final response = search['response']! as Map; + expect(response['headers'], { + 'content-type': ['application/json'], + 'set-cookie': ['demo_session=***; Path=/'], + }); + expect((response['jsonBody']! as Map)['demo_session'], '***'); + expect( + (_read(directory, 'resolveStream')['response']! as Map)['jsonBody'], + {'code': 403}, + ); + + expect(await runContract(directory), isEmpty); + }); + + test('leaves hand-edited fixtures alone', () async { + final directory = copyPlugin('http_test_plugin'); + final before = _read(directory, 'search'); + final upstream = FakeHttpAdapter(_upstream); + + final recording = await recordContract(directory, network: () => upstream); + + expect(recording.skipped, [ + 'search: has hand-edited fixtures', + 'resolveStream: has hand-edited fixtures', + ]); + expect(upstream.requests, isEmpty); + expect(_read(directory, 'search'), before); + }); + + test('writes nothing for a case whose outcome misses the check', () async { + // 連線失敗:案例以 NetworkError 結束,不是 checks.json 期望的成功。原本的 + // fixture 一個位元組都不動。 + final directory = copyPlugin('http_test_plugin'); + edit( + directory, + _searchFixture, + '"edited": "手寫的合成資料,不是錄的"', + '"recordedAt": "2026-09-01T00:00:00.000Z"', + ); + final before = _bytes(directory); + + final recording = await recordContract( + directory, + network: () => FakeHttpAdapter( + (options) => throw DioException.connectionError( + requestOptions: options, + reason: 'offline', + ), + ), + wait: (_) async {}, + ); + + expect( + recording.problems, + containsAll([ + startsWith('search: expected success, got NetworkError'), + 'search: fixtures not written: the outcome does not match ' + 'checks.json', + ]), + ); + expect(_bytes(directory), before); + }); + + test('records the final exchange of a request retried to success', () async { + // 第一次送出連線失敗、網路層重試後成功:失敗的那次沒有回應,不寫;寫下 + // 的一組重播得了。 + final directory = copyPlugin('http_test_plugin'); + Directory(p.join(directory.path, 'fixtures')).deleteSync(recursive: true); + var failed = false; + final upstream = FakeHttpAdapter((options) { + if (!failed) { + failed = true; + throw DioException.connectionError( + requestOptions: options, + reason: 'reset', + ); + } + return _upstream(options); + }); + + final recording = await recordContract( + directory, + network: () => upstream, + wait: (_) async {}, + ); + + expect(recording.problems, isEmpty); + expect( + upstream.requests.where((r) => r.uri.path == '/search'), + hasLength(2), + ); + expect( + Directory(p.join(directory.path, 'fixtures', 'search')).listSync(), + hasLength(1), + ); + expect(await runContract(directory), isEmpty); + }); + + test('does not record a body that is not UTF-8', () async { + final directory = copyPlugin('http_test_plugin'); + Directory(p.join(directory.path, 'fixtures')).deleteSync(recursive: true); + + final recording = await recordContract( + directory, + network: () => FakeHttpAdapter( + (_) => ResponseBody.fromBytes([0xff, 0xfe, 0x00], 200), + ), + ); + + expect( + recording.problems, + contains(contains('returned a body that is not UTF-8')), + ); + expect(Directory(p.join(directory.path, 'fixtures')).existsSync(), isFalse); + }); + + test('sends real requests from the zone it runs in', () async { + // 下面的 live 測試靠呼叫端 zone 的 HttpOverrides 放行真實連線。這裡換成一個 + // 只計數、不連網的 HttpOverrides:它被用到,代表插件的請求(從背景 isolate + // 轉回主 isolate)確實在那個 zone 裡建立 HttpClient。 + final directory = copyPlugin('http_test_plugin'); + Directory(p.join(directory.path, 'fixtures')).deleteSync(recursive: true); + final overrides = _CountingOverrides(); + + await HttpOverrides.runWithHttpOverrides( + () => recordContract(directory, network: IOHttpClientAdapter.new), + overrides, + ); + + expect(overrides.created, greaterThan(0)); + }); + + // 真的連網錄製。只有 `flutter test --run-skipped --tags live` 會跑;插件目錄 + // 由 FMP_PLUGIN_DIR 指定(一個插件目錄,或底下的每個插件目錄)。 + test('records FMP_PLUGIN_DIR from the real network', () async { + final root = pluginDirectoryFromEnvironment(); + expect(root, isNotNull, reason: 'set FMP_PLUGIN_DIR to a plugin directory'); + final directories = pluginDirectories(root!); + expect(directories, isNotEmpty, reason: noPluginDirectories(root)); + for (final directory in directories) { + final recording = await HttpOverrides.runWithHttpOverrides( + () => recordContract(directory, network: IOHttpClientAdapter.new), + _AllowNetwork(), + ); + for (final skipped in recording.skipped) { + stdout.writeln('${directory.path}: skipped $skipped'); + } + expect(recording.problems, isEmpty, reason: directory.path); + expect(await runContract(directory), isEmpty, reason: directory.path); + } + }, tags: 'live'); +} + +/// 放行真實連線(只在錄製的 zone 內)。 +final class _AllowNetwork extends HttpOverrides {} + +/// 建立 HttpClient 時計數並拋錯,不連網。 +final class _CountingOverrides extends HttpOverrides { + int created = 0; + + @override + HttpClient createHttpClient(SecurityContext? context) { + created++; + throw StateError('HttpClient requested from the recording zone'); + } +} diff --git a/app/test/plugins/type_definitions_test.dart b/app/test/plugins/type_definitions_test.dart index e0e55b2e..c6cfc22e 100644 --- a/app/test/plugins/type_definitions_test.dart +++ b/app/test/plugins/type_definitions_test.dart @@ -8,6 +8,8 @@ import 'package:fmp/plugins/runtime/plugin_host.dart'; import 'package:fmp/plugins/runtime/script_errors.dart'; import 'package:fmp/plugins/source_dto.dart'; +import 'contract/checks.dart'; +import 'contract/fixture.dart'; import 'plugin_harness.dart'; // `lib/plugins/types/fmp-plugin.d.ts` 是給插件作者的介面說明。這裡以一個只 @@ -19,11 +21,14 @@ final _dts = File('lib/plugins/types/fmp-plugin.d.ts') .readAsStringSync() .replaceAll('\r\n', '\n'); -/// Dart 端有欄位表的 interface。 +/// Dart 端有欄位表的 interface。契約檢查的格式(checks.json、fixture)由 +/// 契約執行器解碼。 final _shapes = { ...manifestShapes, ...sourceDtoShapes, ...hostApiShapes, + ...checkShapes, + ...fixtureShapes, }; /// 沒有 JSON 欄位表的 interface:宿主 API 由執行中的 runtime 比對,其他是 diff --git a/docs/adr/0015-testing-gates-and-dev-environment.md b/docs/adr/0015-testing-gates-and-dev-environment.md index cf831c74..72cffb8d 100644 --- a/docs/adr/0015-testing-gates-and-dev-environment.md +++ b/docs/adr/0015-testing-gates-and-dev-environment.md @@ -63,8 +63,10 @@ log 只經門面、禁止空 catch、音源 id 不得出現在 UI 與 service、 6. **契約執行器**:一個通用套件以重播執行每個案例,斷言能力與匯出一致、DTO 驗證、案例期望、錯誤對到 `AppError` 類別(0013)、 媒體請求不帶憑證(0012)、只連 manifest 網域、log 經遮蔽(0011)。插件庫 CI 以固定的 FMP 版本執行;`app/` 的 CI 對 `app/test/fixtures/plugins/` 內的測試插件執行(合成資料、播放本機音檔),不依賴官方插件庫;同一個測試插件也供開發版離線開發。 + 更正(2026-09-30,M1):執行器不是獨立套件,放在 `app/test/plugins/contract/`,以 `flutter test` 加環境變數 `FMP_PLUGIN_DIR` 對任一插件目錄執行。QuickJS 與零聯網的準備在 `app/test/`,獨立套件會與 `fmp` 互相依賴,錄製與重播的 adapter 放進 `lib/` 也違反分層;插件庫 CI 一樣是 checkout 固定版本的 FMP 再執行。 7. **App 內插件開發工具**:開發者模式下從資料夾載入插件(先限桌面平台,由平台層宣告)、重新載入、跑案例看 log、 以 App 內登入錄 fixture、每插件切換真實/錄製/重播。命令列只負責重播。版面見 ADR 0025。 + 更正(2026-09-30,M1):命令列的契約執行器也能錄製,限不需要登入的案例,打 `live` tag 手動執行、寫檔前經遮蔽;需要登入的錄製仍只在 App 內。理由:App 內工具在 M3,M1 的 B 站 fixture 要能先錄、過時能重錄。 8. **開發版**:flavor `dev`/`prod`,`pubspec.yaml` 設 `default-flavor: dev`,發版明確帶 `--flavor prod`。 dev 的 Android `applicationIdSuffix ".dev"`、Windows AppUserModelID `com.personal.fmp.dev`、名稱「FMP Dev」與標記圖示、 資料目錄/單一實例鎖/secure storage 命名空間加 `-dev`;prod 維持 ADR 0008 的身分。開發版資料預設空白,