diff --git a/docs/PLAN.md b/docs/PLAN.md index 8b32d79..4a92eeb 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -1971,3 +1971,37 @@ Next: freeze shared features and normalization for detection warm-up, add AI-rev source-train instances and class-specific confusers, then use a predeclared acceptance rule requiring both detection improvement and preserved scene agreement. Do not extend the same run or use source test / Tao-Hsin test for iteration. + +### 10.26 GCS source-train intake and reviewed expansion — 2026-09-16 + +GCS access restored. Added bounded, generation-pinned intake for `gs://studioa` from +existing train cameras only: 18 clips / 453,678,490 bytes, nine Kaohsiung/Taichung cameras, +two dates/time slots. Recorded generations, MD5, SHA-256 and extraction offsets. This +bucket's timestamp format differs from studioa-recording; old UTC assumptions are not +reused. Camera views were compared with legacy images. Initial systemd PATH failure +was diagnosed from logs and resumed with the ffprobe path; pinned worker unchanged. + +36 extracted frames -> 16 visually selected frames after redundancy screening. +SAM3 produced 504 tentative positives and 553 unresolved candidates; assistant inspected +183 individual masks in context and accepted 98 observations. One phone proposal was +corrected to a physical tablet. Depictions, device parts, mats, headphones, remotes, +grouped packages and contaminated masks were deferred. Repeated observations are not +claimed as distinct physical objects; unselected proposals do not create negatives. + +Instances v3 now has 27 train frames / nine cameras / 129 positives; source val remains +six frames / 31 positives. Semantic extension contains 137 frames, Tao-Hsin fold +62 train / 33 val / 26 test, with 16 source-test frames still excluded. All 121 prior +semantic mask files and all 17 prior instance images/targets/negative masks are +byte-identical; every prior split role is unchanged. No test inference occurred. + +59 related tests passed. GPU smoke covered all 27 train frames in 14 updates with finite +loss/gradients; all 2,366,580 checked unknown logits had zero gradient. 297 frozen inputs +verified, all three completed services exited 0, no checkpoint written. Tool commits: +`c7ebd13`, `460213d`, `bafe929`. Unrelated `:memory:.ses` remains untouched. + +[Counts, visual gallery, methods and remaining gaps](STUDIOA_GCS_EXPANSION.md), +[tracked report](reviews/studioa_gcs_expansion_20260916.json). +Cardboard boxes remain at three train observations; class-specific hard negatives and +stronger validation remain pending. Next: detector warm-up with shared scene features +and normalization frozen, then predeclared source-val acceptance checks. Data expansion +is complete for this batch, not evidence of a successful model upgrade. diff --git a/docs/STUDIOA_GCS_EXPANSION.md b/docs/STUDIOA_GCS_EXPANSION.md new file mode 100644 index 0000000..bd2fb22 --- /dev/null +++ b/docs/STUDIOA_GCS_EXPANSION.md @@ -0,0 +1,82 @@ +# StudioA GCS 補資料 — 2026-09-16 + +已完成第一批 GCS 下載、抽幀、AI 覆核及訓練包擴充。偵測 train 從 **11 張/31 個 +實例/7 支鏡頭**增加到 **27 張/129 個實例/9 支鏡頭**。本輪沒有重訓或替換模型。 + +## 來源與選樣 + +`gs://studioa` 的現有來源訓練鏡頭:高雄 4、7、8;台中 3、4、5、8、9、11。 +依既有 Tao-Hsin fold 限制取得資料,沒有把 val/test 鏡頭改成 train。 + +- 2026-09-10 約 12:00、2026-09-12 約 18:50,18 段影片,共 453,678,490 bytes。 +- 下載上限 1 GB;每段記錄 GCS generation、MD5、大小、SHA-256 及 ffprobe 資訊。 +- 每段抽取第 30、330 秒,共 36 幀;36 幀皆非既有影像的逐像素完全重複。 +- 助理比對既有相機視角及抽幀原圖,保留 16 張,另外 20 張視為近似或相鄰冗餘。 +- 此 bucket 的檔名格式不同於 `studioa-recording`,舊腳本的 UTC 轉換不可直接套用。 + 正式批次燒錄時間約為 12:00/12:05、18:50/18:55;另一次探測片段發現時差, + 因此檔名只用於抽樣,不用作已校準的精確事件時間。 + +下載由持久化 systemd 工作執行。初次服務未繼承 ffprobe 路徑而失敗;補上 PATH 後 +從同一份版本鎖定清單續跑,已下載影片核對 MD5 後重用,原始錯誤日誌保留。 + +## AI 覆核與訓練資料 + +SAM3 固定 revision `3c879f39826c281e95690f02c7821c4de09afae7`,使用本機快取。 +16 張影像產生 504 個初步正向候選與 553 個未決候選。助理從中逐項查看 183 個 +候選的原圖脈絡與隔離遮罩,接受 **98 個新增實例標註**;其他候選不作正負真值。 +這是影像內的實例觀測數,並非 98 件不同實物;跨日期可能再次拍到同一商品。 + +排除廣告/包裝中的手機與人物、桌墊誤認平板、筆電螢幕誤認平板、耳機誤認喇叭、 +遙控器誤認手機、展示桌誤認椅子、多盒合併及遮罩污染。另將一個 phone 候選更正 +為畫面中可確認的實體平板,保留原提案類別與更正理由。仍有疑義的類別直接保留未知。 + +| 偵測類別 | 原 train | 新增 | 現 train | 原 val,保持不變 | +| --- | ---: | ---: | ---: | ---: | +| 筆電 | 3 | 6 | 9 | 5 | +| 手機 | 4 | 13 | 17 | 4 | +| 平板 | 4 | 11 | 15 | 1 | +| 盒裝商品 | 3 | 7 | 10 | 1 | +| 紙箱 | 3 | 0 | 3 | 2 | +| 喇叭 | 2 | 7 | 9 | 1 | +| 海報 | 2 | 18 | 20 | 2 | +| 消防設備 | 2 | 6 | 8 | 1 | +| 椅子 | 2 | 13 | 15 | 5 | +| 人物 | 6 | 17 | 23 | 9 | +| 合計 | 31 | 98 | 129 | 31 | + +新影像只提供已覆核正向遮罩;其餘像素為 ignore 255,沒有補成背景。未選取的 +teacher proposals 保留在 `unreviewed_proposals` 供追查,不會自動升格成監督訊號。 +負樣本仍為原先六塊 train 空地板區域;本批沒有新增類別專屬的混淆負樣本。 + +- 語意包:`runs/studioa_gcs_training_extension_20260916_v1/semantic/`,共 137 張。 + Tao-Hsin fold 為 62 train/33 val/26 test,另 16 張來源 test 仍排除。 +- 實例包:`runs/studioa_partial_instances_20260916_v3/`,27 train/6 val。 +- 原本 **121 張語意遮罩逐檔完全相同**,所有 fold 指派不變。 +- 原本 **17 張實例影像、正框/類別 target、空白區 mask 逐檔完全相同**。 +- 本輪沒有執行 test 推論或計算新準確率。 + +[16 張標註總覽](../runs/studioa_gcs_expansion_20260916_v1/reviewed_contact.jpg) · +[逐圖瀏覽](../runs/studioa_gcs_expansion_20260916_v1/reviewed_gallery.html) · +[物件覆核決定](reviews/studioa_gcs_object_decisions_20260916.json) · +[完整實例匯出決定](reviews/studioa_gcs_merged_instance_decisions_20260916.json) + +## 驗證與限制 + +59 項相關測試通過,包含新來源時間解析、拒絕 val/test 鏡頭、來源 hash、AI 選樣、 +類別修正、未選候選忽略,以及擴充前後原標籤/分割保留;lint、型別及提交檢查通過。 + +`runs/studioa_gcs_instance_smoke_20260916_v1/` 凍結 297 個檔案,GPU 對全數 27 張 +train 執行 14 次更新,512×896、batch 2、bf16。loss/梯度有限,檢查 2,366,580 個 +未知分類輸出,非零梯度數為 **0**。分類器更新範數 0.0140317,沒有寫出 checkpoint。 +下載、AI 標註及 GPU 檢查服務最後皆正常退出,exit 0。這只證明資料接線有效。 + +資料量增加仍不能證明模型可靠。紙箱只有 3 個 train 標註;val 僅 31 個實例, +多類只有 1 個。相同鏡頭/實物的跨日期觀測也不是獨立樣本。所有標註由 AI 產生及 +覆核,不能解讀為人工真值、獨立準確率、完整 3D 或顧客分析能力。 + +下一步固定共享場景特徵及 normalization,暖身偵測 head,同時補紙箱與明確的 +類別混淆負樣本;之後才比較來源 val 的偵測改善與語意保持。不要直接延長前次 +已失敗的完全聯合訓練設定。 + +[可追溯結果與 hash](reviews/studioa_gcs_expansion_20260916.json)。資料取得工具 commit +`c7ebd13`,raw-media AI freeze `460213d`,覆核擴充與 GPU 檢查 `bafe929`。 diff --git a/docs/STUDIOA_PARTIAL_DETECTION.md b/docs/STUDIOA_PARTIAL_DETECTION.md index 68f1c4d..20ffbba 100644 --- a/docs/STUDIOA_PARTIAL_DETECTION.md +++ b/docs/STUDIOA_PARTIAL_DETECTION.md @@ -182,3 +182,157 @@ normalization 漂移可能影響語意表現,但本輪尚未隔離因果,不 型別 ratchet 與提交 hook 通過。645 個凍結輸入及 6 個輸出 hash 核對,checkpoint job 身分/有限 tensor/初始權重保留皆通過。另有低解析度 CPU 全流程檢查, 只驗證程式連接,沒有把其分數當作模型效果。 + +## 2026-09-16:固定場景的偵測頭暖身契約 + +GCS 擴充後 instances v3 有 27 張 train/129 個物件觀測、6 張 val/31 個觀測。 +來源為 `studioa_gcs_training_extension_20260916_v1/semantic`;舊 val 與 test 分配 +不變。同一物品可能跨日期重複出現,129 不代表獨立物品數。 + +`studioa_train.py prepare --detector-warmup --instances … --initial-checkpoint …` +從 scene pilot v2 的 best 初始化,只讓 `det_head.*` 參數可訓練。所有其他模組 +固定 eval,包含 BatchNorm running statistics 與 dropout,並在每次儲存前驗證 +凍結 tensor 完全相同。模型 train/eval 切換及 checkpoint 重載均保留此契約。 +scene dataset 設為 `validation_only`,不建立它的 train loader;metadata 明記 +train_size=0。全部 33 張來源 val 的 float32 scene logits 在訓練前和選定模型 +載入後計算 SHA256,必須完全相同。續跑也重新對照原始 scene checkpoint。 + +預先固定:60 epochs 上限、batch 2、lr 2e-4、26 steps warmup、20 輪無改善停止、 +bf16 training、deterministic、關閉 TF32/cuDNN benchmark、EMA 關閉。 +best 選擇最大 reviewed-positive recall(沿用 score >0.20/IoU ≥0.50),同分 +保留較早 epoch。第 0 輪仍參與選擇。完成後只有 recall 嚴格改善、覆核空白區 +誤報不增加且場景完全一致,才記為通過此次暖身;這不是部署驗收。 +未知區預測持續獨立報告,不視為正確或錯誤;本輪不讀取 test 進行推論。 + +紙箱仍只有 3 個 train 觀測;負樣本仍僅已覆核空地板,尚無物品間類別混淆的 +負向監督。暖身結果只能回答「固定既有場景特徵,這批部分監督能否學出偵測」, +不能回答真實整店精度、3D 尺寸或顧客行為是否正確。 + +## 暖身結果:場景保留成功,偵測仍不足以使用 + +`runs/studioa_detector_warmup_20260916_v1/` 完成,程式版本 `562532d`; +第 40 輪因連續 20 輪無嚴格改善停止,共 520 次偵測 optimizer 更新。 +選定第 20 輪(260 次更新),systemd 正常退出,test 未推論。 + +| 檢查項目 | 結果 | +| --- | --- | +| 已覆核正實例召回 | 初始 0/31 → 選定 7/31(22.58%) | +| 命中類別 | person 5/9、phone 1/4、poster 1/2;其他七類 0 | +| 覆核空白區誤報 | 0 → 0;僅覆蓋 51,761 個輸入像素 | +| 未配對、真偽未定預測 | 593;六張影像每張均達 100 框上限 | +| 凍結參數及 buffers | 194 個 tensors;每次儲存皆核對,best/last 與原模型完全相同 | +| 33 張來源 val 場景 logits | 訓練前後 SHA256 完全相同 | +| 場景 AI 標籤 mIoU | 40 輪均為 0.2506537344807518 | + +通過的是「固定場景後,偵測頭得到有限學習」這項暖身契約。重疊框與類別混淆 +仍明顯,593 個未知預測不能當作真陽性,也不能據此計算 precision。相比前次 +聯合 pilot,本輪同時改了資料量與訓練隔離,不能把改善全歸因於其中一項。 +本輪關閉 TF32 並固定 deterministic 設定,mIoU 與較早 pilot 的微小差異也不能 +當成場景改善;直接證據是相同設定下完整 logits 和原場景 tensors 不變。 + +[六張逐圖對照](../runs/studioa_detector_warmup_20260916_v1/review/selected_val_review.jpg) +左側是 AI 覆核子集,右側只顯示前 20 框;評估仍用固定上限 100 框。 +[逐框 JSON](../runs/studioa_detector_warmup_20260916_v1/review/selected_diagnostics.json) +可完整重算並精確重現所有偵測驗證指標。 +[可追溯結果](reviews/studioa_detector_warmup_20260916.json) 記錄設定、檢查和 hashes。 + +757 個凍結輸入、8 個正式輸出 hash 核對通過;best/last 無非有限 tensor, +job 身分正確,原始 scene checkpoint 沒有改寫。主要測試批次 171 項通過, +資料與契約追加批次 20 項通過(兩批有重疊);lint、型別與提交 hook 通過。 +metadata 的 dirty 警示來自既有無關未追蹤檔,訓練程式使用已提交且逐檔凍結 +的 snapshot;語意資料記為 train_size=0、val_size=33。 + +下一步優先補逐類混淆負樣本的資料契約:例如確認價牌不是手機,只否定 phone +通道,不能將實際物品整片畫成十類共同背景。紙箱三個 train 觀測全部來自同一 +張影像與同一鏡頭,應由其他允許的 source-train 鏡頭補充。所有新增覆核由 AI +執行,val/test 分配及 score/NMS 門檻保持固定。尚不推進 Stage 2–4 的效果宣稱。 + +## 2026-09-16:逐類負樣本與第二支紙箱鏡頭 + +instances v4 保留 v3 的 33 張影像、正框、空白區 masks、companions 與 split +逐檔一致;新增 `0058-Kaohsiung-cam08` 中可分離的左側紙箱。相鄰兩箱合併的 +proposal 被拒絕,只接受單箱遮罩。train 現為 28 張/130 個觀測;紙箱 4 個、 +2 支鏡頭,仍十分稀少。val 保持 6 張/31 個正例,語意資料完全不改。 + +AI 重新查看 62 張來源 train 總覽及候選原圖,在 5 張既有影像選定 19 個保守 +區域、47 項逐類負向決定:桌墊與遙控器不是手機、包裝印刷的平板不是實體平板、 +桌上型螢幕不是筆電、海報上的 HomePod 不是實體喇叭、木椅不是紙箱。 +這批資料重用已下載且分割合法的影像,不需要新增 GCS 下載。 + +`class_negative_rects` 必須由綁定影像/companion hash 的 AI 覆核提供,且只准 +用於 train。匯出為逐類 0/1 masks;沒有明確否定的類別仍未知。dataset 對每個 +通道共同做縮放、裁切、翻轉與 padding,再組成 `[C,H,W]`,padding 255 忽略。 +FCOS 在對應 grid point 只加入被否定通道的 focal loss,不改 regression 或 +centerness。相同類別的正框在所有金字塔尺度都有優先權;匯出時也拒絕矛盾覆核。 +既有全類空白 mask 與部分驗證指標契約不變。無此欄位的舊資料維持原行為。 + +[覆核決定](reviews/studioa_class_negative_decisions_20260916.json) · +[逐類負樣本總覽](../runs/studioa_confusers_20260916_v1/class_negative_contact.jpg) · +[新增單箱遮罩](../runs/studioa_confusers_20260916_v1/vlm-unknown-0016.jpg)。 + +重訓沿用固定場景 warmup 設定與原始 scene 初始化,score/NMS/max_det 不調整。 +本輪完整資料變更與前版 7/31 比較;不把資料增補與 loss 修改當成單因素因果實驗。 +未知預測數減少不等於 precision 改善;僅通過隨機初始化基準也不等於超過前版。 + +### 第二輪結果:不升級模型 + +`runs/studioa_detector_warmup_20260916_v2/` 使用 commit `c274cb2`,第 25 輪 +觸發 20 輪無改善停止(350 次更新),選定第 5 輪(70 次更新)。 + +| 同一來源 val、同一評估契約 | 前版 v1 | 本輪 v2 | +| --- | ---: | ---: | +| 覆核正例召回 | 7/31(22.58%) | 6/31(19.35%) | +| person / phone / poster 命中 | 5 / 1 / 1 | 5 / 0 / 1 | +| 覆核空白區誤報 | 0 | 0 | +| 未配對、真偽未定預測 | 593 | 594 | +| 場景輸出 | 與原模型一致 | 與原模型一致 | + +因此 **保留 v1 作為較佳實驗基準,v2 不升級**。worker `warmup.accepted=true` +只代表超過第 0 輪隨機偵測頭,不代表超過 v1;明確決定另見 +[promotion_decision.json](../runs/studioa_detector_warmup_20260916_v2/promotion_decision.json)。 +兩版都未達部署條件;新程式與 AI 資料保留,沒有改寫任何舊 checkpoint。 + +五張已訓練負樣本影像中的 3,543 個 grid/class 對,score >0.20 的數量由 +1,776 降至 15,說明選定模型在這些訓練區域的反應較低。這是訓練區診斷, +且兩版選定 epoch 不同,不能當作驗證 precision 改善或單因素因果證明。 +val 六張影像仍全數達 100 框上限,尚未改善其他鏡頭的混淆。 + +固定門檻下的解碼前手機診斷進一步區分:v2 四個 val 手機中,兩個完全沒有 +IoU ≥0.50 的原始候選框;另外兩個雖有定位候選,phone score 最高只有約 +0.024–0.025,低於原定 0.20。這四個漏檢不能單靠提高 max_det 解決。 +下一步先量化每類正負 loss 比例、限制逐類負項影響,再進行預先設定的比較, +並補充小物件的定位正例;不利用這組 val 調門檻掩蓋問題。 + +79 項測試、lint、型別、提交 hook 通過。GPU smoke 對 28 張 train 做 14 次 +更新,2,457,821 個未知分類輸出梯度均為 0。319 個 smoke 輸入、779 個訓練 +輸入及 8 個正式輸出 hashes 核對;best/last 有限、job 身分正確,194 個共享 +tensors 與原模型相同。33 張 scene logits SHA256 完全一致,所有 test 未推論。 +兩個 systemd 工作正常退出。儲存的逐框結果能完整重算官方偵測指標。 + +[本輪逐圖對照](../runs/studioa_detector_warmup_20260916_v2/review/selected_val_review.jpg) · +[解碼前手機診斷](../runs/studioa_detector_warmup_20260916_v2/review/phone_candidate_probe.json) · +[完整可追溯結果](reviews/studioa_class_negative_warmup_20260916.json)。 + +## 2026-09-16:逐類負項正規化的固定比較 + +先對 28 張 source train、不增強、float32、逐張影像做分類 loss 與 logit 梯度 +分解。v1 模型的 phone 正向梯度 L1 合計約 0.0271,新增逐類負項約 9.631; +其中一張沒有 phone 正例的桌面影像,610 個 phone 負向點貢獻約 7.168。 +這說明新增負項會受到覆核區面積與每張正例數影響,但不是原先 batch 2 增強 +訓練的梯度重播,也不是模型參數梯度;不能單憑比例就判定退步原因。 +130 個 train 正框均有可分配的特徵點;「無原始框達 IoU 0.50」和「無訓練 +指派點」是不同問題,仍需另外衡量小物件框回歸品質。 + +只比較一個變因 `class_negative_normalization`:`sum` 保留原行為; +`positive_budget` 將每個 batch/class 新增負項權重總和限制在 +`max(該類正向指派點數, 1)`,每個負點權重最多 1。已有正項、全類空白負項、 +regression、centerness 與未知通道均不改;缺少該類正例的 batch 仍可提供 +最多一個等效負向點。這限制的是監督權重總量,不是梯度範數。 + +兩組從同一 scene checkpoint、同 seed、同 instances v4 開始;所有訓練與 +驗證條件沿用 v2。固定 score >0.20/IoU ≥0.50/NMS 0.6/每圖 100 框, +不得看結果後調門檻。只有候選超過 v1 的 7/31、人物至少 5、手機至少 1、 +海報至少 1、覆核空白誤報不增加且 scene logits 完全一致,才更新實驗基準。 +本次只有這兩組,不依此 val 分數追加參數搜尋。 + +[執行前比較計畫與診斷](reviews/studioa_negative_balance_plan_20260916.json)。 diff --git a/docs/reviews/studioa_class_negative_decisions_20260916.json b/docs/reviews/studioa_class_negative_decisions_20260916.json new file mode 100644 index 0000000..f028958 --- /dev/null +++ b/docs/reviews/studioa_class_negative_decisions_20260916.json @@ -0,0 +1,1843 @@ +{ + "reviewer_kind": "ai", + "reviewer": "assistant_visual_analysis", + "held_out": "Tao-Hsin", + "source_manifest_sha256": "0139eaccc00c6cf2fd841c7cfbd3a231c9b744a624b9c8ec95f82bc6e7a7b1ee", + "scope": "AI visual review of original source-train images only; per-class absence, other channels unknown. Existing positives and val unchanged; one additional unambiguous cardboard instance.", + "frames": [ + { + "frame_id": "0050-Kaohsiung-cam04", + "image_sha256": "7b1e1be2cf234ab9d881fef37b49789f4f1b64eff655758b2e674d6819ee6bac", + "companion_sha256": "0423e2002e1c61c16d815172dd247c61c3f4a21d982f6c64d88ad962b16656e0", + "positives": [ + { + "id": "ai-0005", + "entity": "laptop", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0007", + "entity": "cardboard_box", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0008", + "entity": "cardboard_box", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0010", + "entity": "cardboard_box", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0017", + "entity": "laptop", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 1650, + 400, + 1800, + 650 + ], + "reason": "Assistant inspected source image: empty floor patch, no visible object from the declared ten detection classes; not inferred from missing labels." + } + ] + }, + { + "frame_id": "0030-Taichung-cam08", + "image_sha256": "e95523bd8f92ab2fbb7ed5fa672e25b7cf01d531ed94711a016572351d1b1ea4", + "companion_sha256": "e452f0356246f096cc9bcf3c8a16b403c24a87983900fb9fc509a76397a5d720", + "positives": [ + { + "id": "ai-0003", + "entity": "tablet", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0004", + "entity": "tablet", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0005", + "entity": "tablet", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 1000, + 960, + 1500, + 1050 + ], + "reason": "Assistant inspected source image: empty floor patch, no visible object from the declared ten detection classes; not inferred from missing labels." + } + ] + }, + { + "frame_id": "0112-Taichung-cam11", + "image_sha256": "3353277f188293dba730eefd1f3f42ddbd4e6b50ee87e38d4c954f22d8495355", + "companion_sha256": "051477d05602005a21aef67b3af839ec4956f1c3511be724e006eafb1ff30eb9", + "positives": [ + { + "id": "ai-0003", + "entity": "speaker", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 1100, + 650, + 1250, + 850 + ], + "reason": "Assistant inspected source image: empty floor patch, no visible object from the declared ten detection classes; not inferred from missing labels." + } + ] + }, + { + "frame_id": "0078-Taichung-cam09", + "image_sha256": "30918a1a840d0e54a4d88dd8c34bb2907a0b7931f25ce60f7f4e878b5528f6b2", + "companion_sha256": "bedfde8148dae9581b2fa7322caa9afb173a9448e394227f68887d27baa4ffc0", + "positives": [ + { + "id": "ai-0002", + "entity": "fire_equipment", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0003", + "entity": "fire_equipment", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 1500, + 850, + 1800, + 1000 + ], + "reason": "Assistant inspected source image: empty floor patch, no visible object from the declared ten detection classes; not inferred from missing labels." + } + ] + }, + { + "frame_id": "0025-Taichung-cam05", + "image_sha256": "08b398cb45b48e99021b07d5247539c8138805cfb043c7d86421c6bfd2fe24a4", + "companion_sha256": "9dd17564938c9007cd40050ce79aceaebfedbad2b9aa1f1970174557ac462e87", + "positives": [ + { + "id": "ai-0009", + "entity": "phone", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 1730, + 760, + 1810, + 930 + ], + "reason": "Assistant inspected source image: empty floor patch, no visible object from the declared ten detection classes; not inferred from missing labels." + } + ] + }, + { + "frame_id": "0031-Taichung-cam08", + "image_sha256": "ddd3f30a15b4b527c8dc5a025b7e9a5c5cf21561936b3e98fa17bbcd8d411b4c", + "companion_sha256": "51097af5eae2df30092a13a14afc6ebaf964454f62d0800d9291c31bbca40b53", + "positives": [ + { + "id": "ai-0005", + "entity": "phone", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0007", + "entity": "phone", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0008", + "entity": "phone", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 200, + 920, + 450, + 1030 + ], + "reason": "Assistant inspected source image: empty floor patch, no visible object from the declared ten detection classes; not inferred from missing labels." + } + ] + }, + { + "frame_id": "0008-Kaohsiung-cam07", + "image_sha256": "6acd71ce695fa11235c63cf3d83cea62bb39017bfbfd2eba47e198449ad48a28", + "companion_sha256": "901ae7a38b80cd42c036e8e6f594dc668096a8646132bb76102ce3ac494ad632", + "positives": [ + { + "id": "ai-0028", + "entity": "boxed_stock", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0002", + "entity": "boxed_stock", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [] + }, + { + "frame_id": "0002-Kaohsiung-cam04", + "image_sha256": "a7339928d2e72363cefb35a8f6e25d8b1784ceaf747f87b815293dfdc53257bb", + "companion_sha256": "3dfce18c1f9d25ef717683209d6396964fcaa024ba1477777078ebf0de796436", + "positives": [ + { + "id": "ai-0004", + "entity": "laptop", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0005", + "entity": "boxed_stock", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0013", + "entity": "poster", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [] + }, + { + "frame_id": "0033-Taichung-cam09", + "image_sha256": "c5b1f42f5025b4479295913c3c41c9cfa324fbc5b4200bc6a2994efa52220b3d", + "companion_sha256": "276fbc3a765a4b33f48c00a1bf2edbb7dd9ed8fc5da0a71df6f3a85f56824137", + "positives": [ + { + "id": "ai-0011", + "entity": "poster", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [] + }, + { + "frame_id": "0022-Taichung-cam04", + "image_sha256": "ad8f01fb6860896c9bbd1e3ff9bf86c10515013c23f4e8c2de87802fcea2d07b", + "companion_sha256": "f094e3b5cf5919a27ee9155700bc98cc57af081e5750f4c8ebe3f96da235691f", + "positives": [ + { + "id": "ai-0010", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0013", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0003", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [] + }, + { + "frame_id": "0082-Taichung-cam11", + "image_sha256": "4c85c1f28791ebc81eb3f687cc55f913899381ac90cb6a16ef223aeea997df1d", + "companion_sha256": "26b1250e73b68449e4b501b9509702909c3021e44a506f1108e1f8cad4c36492", + "positives": [ + { + "id": "ai-0010", + "entity": "tablet", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0003", + "entity": "speaker", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0002", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [] + }, + { + "frame_id": "0007-Kaohsiung-cam06", + "image_sha256": "8340acd371f970f294282695828b7fdd2d27e08b9529575f7f275290b38b50f8", + "companion_sha256": "3a26496740076a5e3a58e36c63fae01393ed39b344a66d0b0fe67440b2f1bc8f", + "positives": [ + { + "id": "ai-0005", + "entity": "laptop", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0011", + "entity": "laptop", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0010", + "entity": "tablet", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0007", + "entity": "speaker", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0016", + "entity": "poster", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0002", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 150, + 800, + 400, + 950 + ], + "reason": "Assistant inspected original context with this rectangle: clear floor without a visible object from the ten detection classes." + } + ] + }, + { + "frame_id": "0053-Kaohsiung-cam05", + "image_sha256": "68440d495ad4d190d7b68b046d64707339105fa7c5290b36b2c46abd3a1f4e8a", + "companion_sha256": "a9565f0ac21f98f711d0a0ff3c91608767fdc023a91557974fc41903ded832ab", + "positives": [ + { + "id": "ai-0004", + "entity": "boxed_stock", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0002", + "entity": "cardboard_box", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0006", + "entity": "cardboard_box", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0005", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 850, + 500, + 1250, + 800 + ], + "reason": "Assistant inspected original context with this rectangle: clear floor without a visible object from the ten detection classes." + } + ] + }, + { + "frame_id": "0016-Taichung-cam01", + "image_sha256": "fb3086c1ff8ed11a485226d69bacacb33ea4d534d0708096c80526f348230784", + "companion_sha256": "b30aed64d3c57dba492dec341442706b77458556b8afb6991fe1b2f6741d7bee", + "positives": [ + { + "id": "ai-0002", + "entity": "laptop", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0021", + "entity": "fire_equipment", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0004", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0005", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 600, + 700, + 700, + 900 + ], + "reason": "Assistant inspected original context with this rectangle: clear floor without a visible object from the ten detection classes." + } + ] + }, + { + "frame_id": "0076-Taichung-cam07", + "image_sha256": "a7e42db0b7ac8ebae37bc90da01b090b7de169b9cf0bf1b06088d049add3f83e", + "companion_sha256": "bf42c51b3bb9657e8bd99379eaabba2d1e53e4d4559bae34f2ceae1959b11dd0", + "positives": [ + { + "id": "ai-0005", + "entity": "laptop", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0006", + "entity": "poster", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0001", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0004", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0002", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 900, + 800, + 1200, + 1000 + ], + "reason": "Assistant inspected original context with this rectangle: clear floor without a visible object from the ten detection classes." + } + ] + }, + { + "frame_id": "0034-Taichung-cam10", + "image_sha256": "28c5c9e5a09f55a473bda61d0d84de4a8f679ebc3db43f892358a6a56d0c57fd", + "companion_sha256": "602aa819d8944d6dadc76bedecfa8b2609641066dd2dc6a046c52cbebf2846d3", + "positives": [ + { + "id": "ai-0002", + "entity": "laptop", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0007", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0005", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0010", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [] + }, + { + "frame_id": "0013-Kaohsiung-cam11", + "image_sha256": "4c19e17cad632f5551e2280b514e76884e74658d62841354e3ddfcc1c73a6f19", + "companion_sha256": "06b6e65f9c04d603749965cb03a5fc7eb6ee2c8419638baeb6cb93bba69f5f0a", + "positives": [ + { + "id": "vlm-unknown-0004", + "entity": "phone", + "reason": "Assistant inspected context and isolated mask: real tethered smartphone on foreground circular display table; excludes phone depicted on TV advertisement." + }, + { + "id": "vlm-unknown-0006", + "entity": "phone", + "reason": "Assistant inspected context and isolated mask: real tethered smartphone on foreground circular display table; excludes phone depicted on TV advertisement." + }, + { + "id": "vlm-unknown-0007", + "entity": "phone", + "reason": "Assistant inspected context and isolated mask: real tethered smartphone on foreground circular display table; excludes phone depicted on TV advertisement." + }, + { + "id": "vlm-unknown-0008", + "entity": "phone", + "reason": "Assistant inspected context and isolated mask: real tethered smartphone on foreground circular display table; excludes phone depicted on TV advertisement." + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Kaohsiung-cam04-t330", + "image_sha256": "a479c28dc3688ca122efa93ce4288707fffa4eb2031ca942d860fae2397c0add", + "companion_sha256": "8cbb99661c64303eae9bd97cf37953079b614235f7f39622385d8b70302abc89", + "positives": [ + { + "id": "ai-0005", + "entity": "laptop", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 0 + }, + { + "id": "ai-0011", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 3 + }, + { + "id": "ai-0019", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 4 + }, + { + "id": "ai-0009", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 5 + }, + { + "id": "ai-0006", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 6 + }, + { + "id": "ai-0031", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 12 + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 13 + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 14 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Kaohsiung-cam07-t030", + "image_sha256": "43a4ec3d39087331280da0a4b868ffc47b98cebb4410a304a2f1d14adf9fdc9d", + "companion_sha256": "18d91aac7c1cbb4a2e29a8a6f3ef9df12f4e7ef96d967e94e7175ff509d06e53", + "positives": [ + { + "id": "ai-0080", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 19 + }, + { + "id": "ai-0017", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 21 + } + ], + "negative_rects": [], + "class_negative_rects": [ + { + "entity": "phone", + "xyxy": [ + 290, + 385, + 465, + 480 + ], + "reason": "Printed device/case pictures on hanging retail packaging; no physical device." + }, + { + "entity": "tablet", + "xyxy": [ + 290, + 385, + 465, + 480 + ], + "reason": "Printed device/case pictures on hanging retail packaging; no physical device." + }, + { + "entity": "laptop", + "xyxy": [ + 290, + 385, + 465, + 480 + ], + "reason": "Printed device/case pictures on hanging retail packaging; no physical device." + }, + { + "entity": "phone", + "xyxy": [ + 665, + 374, + 818, + 429 + ], + "reason": "Printed tablet/case faces on sealed retail packs." + }, + { + "entity": "tablet", + "xyxy": [ + 665, + 374, + 818, + 429 + ], + "reason": "Printed tablet/case faces on sealed retail packs." + }, + { + "entity": "laptop", + "xyxy": [ + 665, + 374, + 818, + 429 + ], + "reason": "Printed tablet/case faces on sealed retail packs." + }, + { + "entity": "phone", + "xyxy": [ + 938, + 250, + 1050, + 325 + ], + "reason": "Device pictures on hanging white case packages." + }, + { + "entity": "tablet", + "xyxy": [ + 938, + 250, + 1050, + 325 + ], + "reason": "Device pictures on hanging white case packages." + }, + { + "entity": "laptop", + "xyxy": [ + 938, + 250, + 1050, + 325 + ], + "reason": "Device pictures on hanging white case packages." + }, + { + "entity": "person", + "xyxy": [ + 1375, + 408, + 1440, + 483 + ], + "reason": "Printed advertising figures on retail packaging, not a physical person." + } + ] + }, + { + "frame_id": "gcs-20260910-Kaohsiung-cam08-t030", + "image_sha256": "facae65b929ba40c46bf9bb671e1b158c3e9edd392734e6cdea2b4fb69717462", + "companion_sha256": "97cf772c23ef3a6058ae6aac87f2cf207b5391d34336713254257157b839e66f", + "positives": [ + { + "id": "ai-0023", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 27 + }, + { + "id": "ai-0025", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 28 + }, + { + "id": "ai-0018", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 29 + }, + { + "id": "ai-0042", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 31 + } + ], + "negative_rects": [], + "class_negative_rects": [ + { + "entity": "laptop", + "xyxy": [ + 880, + 175, + 960, + 220 + ], + "reason": "Desktop monitor on stand, distinct from tablet/laptop." + }, + { + "entity": "tablet", + "xyxy": [ + 880, + 175, + 960, + 220 + ], + "reason": "Desktop monitor on stand, distinct from tablet/laptop." + }, + { + "entity": "phone", + "xyxy": [ + 880, + 175, + 960, + 220 + ], + "reason": "Desktop monitor on stand, distinct from tablet/laptop." + }, + { + "entity": "laptop", + "xyxy": [ + 1092, + 183, + 1174, + 235 + ], + "reason": "Desktop monitor panel, not a portable device." + }, + { + "entity": "tablet", + "xyxy": [ + 1092, + 183, + 1174, + 235 + ], + "reason": "Desktop monitor panel, not a portable device." + }, + { + "entity": "phone", + "xyxy": [ + 1092, + 183, + 1174, + 235 + ], + "reason": "Desktop monitor panel, not a portable device." + }, + { + "entity": "laptop", + "xyxy": [ + 1374, + 220, + 1444, + 260 + ], + "reason": "Desktop monitor panel on wall-side display table." + }, + { + "entity": "tablet", + "xyxy": [ + 1374, + 220, + 1444, + 260 + ], + "reason": "Desktop monitor panel on wall-side display table." + }, + { + "entity": "phone", + "xyxy": [ + 1374, + 220, + 1444, + 260 + ], + "reason": "Desktop monitor panel on wall-side display table." + } + ] + }, + { + "frame_id": "gcs-20260910-Taichung-cam03-t030", + "image_sha256": "e4dc5e1e322ad6b43c802169a816c06fd0b1f6cb8c399e87d4215cd5056e1f82", + "companion_sha256": "5ed3466365e1b23c8810968530385e6203616aeb487dae6f0c77e31e10043c6a", + "positives": [ + { + "id": "ai-0005", + "entity": "laptop", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 32 + }, + { + "id": "ai-0006", + "entity": "laptop", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 33 + }, + { + "id": "ai-0014", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 35 + }, + { + "id": "ai-0011", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 36 + }, + { + "id": "ai-0008", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 37 + }, + { + "id": "ai-0020", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 40 + }, + { + "id": "ai-0057", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 41 + }, + { + "id": "ai-0010", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 42 + }, + { + "id": "ai-0003", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 43 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Taichung-cam04-t330", + "image_sha256": "7daaff94e19ec7f2eb63738d779c358cbbe59406841f5499f8269376befbcad1", + "companion_sha256": "503fb3c85237741465e9b3d8c3702535e9651206267e325c69634c81540e62f8", + "positives": [ + { + "id": "ai-0054", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 50 + }, + { + "id": "ai-0057", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 51 + }, + { + "id": "ai-0050", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 53 + }, + { + "id": "ai-0009", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 55 + }, + { + "id": "ai-0028", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 56 + }, + { + "id": "ai-0016", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 58 + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 59 + }, + { + "id": "ai-0012", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 60 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Taichung-cam05-t030", + "image_sha256": "8a7a6e129561f819a598241bbe50107322192a61ca450510ae97c957672686ad", + "companion_sha256": "dd4ebfadebb5c9b9a4777c130b124aa1cac9cb3610d20783acf610056b056e4e", + "positives": [ + { + "id": "ai-0036", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 64 + }, + { + "id": "ai-0005", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 66 + }, + { + "id": "ai-0061", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 68 + }, + { + "id": "ai-0014", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 69 + }, + { + "id": "ai-0011", + "entity": "fire_equipment", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 70 + }, + { + "id": "ai-0009", + "entity": "fire_equipment", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 71 + }, + { + "id": "ai-0055", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 73 + } + ], + "negative_rects": [], + "class_negative_rects": [ + { + "entity": "speaker", + "xyxy": [ + 1030, + 515, + 1100, + 600 + ], + "reason": "Printed HomePod advertising panel, not a physical speaker/device; excludes real speaker below left." + }, + { + "entity": "phone", + "xyxy": [ + 1030, + 515, + 1100, + 600 + ], + "reason": "Printed HomePod advertising panel, not a physical speaker/device; excludes real speaker below left." + }, + { + "entity": "tablet", + "xyxy": [ + 1030, + 515, + 1100, + 600 + ], + "reason": "Printed HomePod advertising panel, not a physical speaker/device; excludes real speaker below left." + }, + { + "entity": "laptop", + "xyxy": [ + 1095, + 199, + 1230, + 275 + ], + "reason": "Desktop monitor panel on stand, not a laptop or handheld device." + }, + { + "entity": "tablet", + "xyxy": [ + 1095, + 199, + 1230, + 275 + ], + "reason": "Desktop monitor panel on stand, not a laptop or handheld device." + }, + { + "entity": "phone", + "xyxy": [ + 1095, + 199, + 1230, + 275 + ], + "reason": "Desktop monitor panel on stand, not a laptop or handheld device." + }, + { + "entity": "laptop", + "xyxy": [ + 949, + 242, + 1030, + 292 + ], + "reason": "Second desktop monitor panel; keyboard and stand establish desktop form." + }, + { + "entity": "tablet", + "xyxy": [ + 949, + 242, + 1030, + 292 + ], + "reason": "Second desktop monitor panel; keyboard and stand establish desktop form." + }, + { + "entity": "phone", + "xyxy": [ + 949, + 242, + 1030, + 292 + ], + "reason": "Second desktop monitor panel; keyboard and stand establish desktop form." + } + ] + }, + { + "frame_id": "gcs-20260910-Taichung-cam08-t030", + "image_sha256": "8f15fd481aeb8f63aa50b25dfa79637535b45defcecae4b5f99b9a5e547b5daa", + "companion_sha256": "8a5a47e8f624dde013d5b8bfddfc02b5b6f309196da9a60c94325871d1c28c2d", + "positives": [ + { + "id": "ai-0002", + "entity": "tablet", + "reason": "Assistant corrected phone proposal to tablet: full-size physical iPad with tether and stand on tabletop, compared with the surrounding counter and power sockets; not the adjacent black desk mats.", + "review_index": 76 + } + ], + "negative_rects": [], + "class_negative_rects": [ + { + "entity": "phone", + "xyxy": [ + 730, + 280, + 1010, + 500 + ], + "reason": "Black work mat with cable, no device inside reviewed rectangle." + }, + { + "entity": "tablet", + "xyxy": [ + 730, + 280, + 1010, + 500 + ], + "reason": "Black work mat with cable, no device inside reviewed rectangle." + }, + { + "entity": "laptop", + "xyxy": [ + 730, + 280, + 1010, + 500 + ], + "reason": "Black work mat with cable, no device inside reviewed rectangle." + }, + { + "entity": "phone", + "xyxy": [ + 1190, + 280, + 1450, + 505 + ], + "reason": "Black work mat is not a tablet or laptop screen." + }, + { + "entity": "tablet", + "xyxy": [ + 1190, + 280, + 1450, + 505 + ], + "reason": "Black work mat is not a tablet or laptop screen." + }, + { + "entity": "laptop", + "xyxy": [ + 1190, + 280, + 1450, + 505 + ], + "reason": "Black work mat is not a tablet or laptop screen." + }, + { + "entity": "phone", + "xyxy": [ + 405, + 575, + 505, + 616 + ], + "reason": "Remote control in visible cradle, not a phone." + }, + { + "entity": "phone", + "xyxy": [ + 413, + 646, + 500, + 687 + ], + "reason": "Remote control with white label, not a phone." + }, + { + "entity": "phone", + "xyxy": [ + 415, + 701, + 507, + 740 + ], + "reason": "Remote control in cradle, not a phone." + } + ] + }, + { + "frame_id": "gcs-20260910-Taichung-cam09-t030", + "image_sha256": "c989bfe1f6a7998834c79588b000dd24f869b96bfc93c14055b9a2b8d2995828", + "companion_sha256": "5f7a89f5e2cc3d58f472a954b8fd41992c5615be016b97956911f9bac6aead07", + "positives": [ + { + "id": "ai-0066", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 82 + }, + { + "id": "ai-0028", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 83 + }, + { + "id": "ai-0026", + "entity": "fire_equipment", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 84 + }, + { + "id": "ai-0002", + "entity": "fire_equipment", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 85 + }, + { + "id": "ai-0005", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 86 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Taichung-cam11-t330", + "image_sha256": "272d31978a93c47d1a1aaa04f8581cd0c47135abdd60aab94d0650ef49186d7d", + "companion_sha256": "b5d5d582256ff8a0ba2a0f87bedde01d97739e6e1eb9ba8ffdb1e885b9251965", + "positives": [ + { + "id": "ai-0017", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 91 + }, + { + "id": "ai-0030", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 92 + }, + { + "id": "ai-0011", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 93 + }, + { + "id": "ai-0028", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 95 + }, + { + "id": "ai-0002", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 96 + }, + { + "id": "ai-0006", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 97 + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 98 + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 99 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260912-Kaohsiung-cam04-t030", + "image_sha256": "8f2455851cd3d7d609931dfe0c7681801df689eb20633bb42af56f06837d0730", + "companion_sha256": "e57e043fc7a2e8f7a9fce6db43bd41c0758fd9a997cfc803ed49aafdf71a8d43", + "positives": [ + { + "id": "ai-0006", + "entity": "laptop", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 100 + }, + { + "id": "ai-0039", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 102 + }, + { + "id": "ai-0010", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 103 + }, + { + "id": "ai-0020", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 105 + }, + { + "id": "ai-0008", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 106 + }, + { + "id": "ai-0074", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 113 + }, + { + "id": "ai-0068", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 114 + }, + { + "id": "ai-0002", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 115 + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 116 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260912-Taichung-cam03-t330", + "image_sha256": "6f696165a74ce3d833e6fc59a868213f3f261d6b904e2964553abd3107513c52", + "companion_sha256": "ffbc822f24b384d131a9e6255a56762405c5c9999e140ebcb54a53edb5ffda85", + "positives": [ + { + "id": "ai-0011", + "entity": "laptop", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 117 + }, + { + "id": "ai-0010", + "entity": "laptop", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 118 + }, + { + "id": "ai-0017", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 120 + }, + { + "id": "ai-0019", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 121 + }, + { + "id": "ai-0015", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 125 + }, + { + "id": "ai-0005", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 126 + }, + { + "id": "ai-0006", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 127 + }, + { + "id": "ai-0008", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 129 + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 130 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260912-Taichung-cam04-t030", + "image_sha256": "e528fceb02e0402a1b98840d1e13cd342b8e823ee2bd9f7367353ab9be0bb4cb", + "companion_sha256": "d21485f131d60f65e634d033b0a5b58c18c26e78dc4bd3edb89083f882076e8e", + "positives": [ + { + "id": "ai-0016", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 139 + }, + { + "id": "ai-0033", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 140 + }, + { + "id": "ai-0020", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 141 + }, + { + "id": "ai-0038", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 142 + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 143 + }, + { + "id": "ai-0003", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 144 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260912-Taichung-cam05-t330", + "image_sha256": "f0757ff6f43fadc72a211aee58b1132ac29afe242e35a07ef55c41a2b5bf4209", + "companion_sha256": "8b39a80107cfbf5bfd72aed896d486f1579b4a4939f7f72d54696dd98ab1c20e", + "positives": [ + { + "id": "ai-0014", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 151 + }, + { + "id": "ai-0081", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 154 + }, + { + "id": "ai-0020", + "entity": "fire_equipment", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 155 + }, + { + "id": "ai-0019", + "entity": "fire_equipment", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 156 + }, + { + "id": "ai-0010", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 157 + }, + { + "id": "ai-0024", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 158 + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 159 + }, + { + "id": "ai-0004", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 160 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260912-Taichung-cam08-t030", + "image_sha256": "04aae73f6f1184bb6b1fdd4f2f924243c68f6a423b794ca7d806e4fc03cad2ad", + "companion_sha256": "d9d462189ede768927dc8b63bf925e6bd141076a6adbc7ba9c2318505e54d331", + "positives": [ + { + "id": "ai-0002", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 161 + }, + { + "id": "ai-0003", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 162 + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 164 + }, + { + "id": "ai-0005", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 165 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260912-Taichung-cam08-t330", + "image_sha256": "c834c7362415b2fa11d32470166b44b0a80519929e258ab8b228bdb5a1a79578", + "companion_sha256": "2c5ec8ec5dfa4e68f5d6fa34e4cfd1ff0f3203233738a149bd975c8e6a0787cf", + "positives": [ + { + "id": "ai-0007", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 167 + }, + { + "id": "ai-0001", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 168 + }, + { + "id": "ai-0002", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 169 + }, + { + "id": "ai-0005", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 173 + }, + { + "id": "ai-0004", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 174 + } + ], + "negative_rects": [], + "class_negative_rects": [ + { + "entity": "phone", + "xyxy": [ + 800, + 270, + 978, + 475 + ], + "reason": "Visible work mat and cable, excluding actual phone to its left." + }, + { + "entity": "tablet", + "xyxy": [ + 800, + 270, + 978, + 475 + ], + "reason": "Visible work mat and cable, excluding actual phone to its left." + }, + { + "entity": "laptop", + "xyxy": [ + 800, + 270, + 978, + 475 + ], + "reason": "Visible work mat and cable, excluding actual phone to its left." + }, + { + "entity": "phone", + "xyxy": [ + 760, + 692, + 935, + 825 + ], + "reason": "Unoccupied mat interior below the watch." + }, + { + "entity": "tablet", + "xyxy": [ + 760, + 692, + 935, + 825 + ], + "reason": "Unoccupied mat interior below the watch." + }, + { + "entity": "laptop", + "xyxy": [ + 760, + 692, + 935, + 825 + ], + "reason": "Unoccupied mat interior below the watch." + }, + { + "entity": "phone", + "xyxy": [ + 1300, + 790, + 1480, + 832 + ], + "reason": "Visible mat interior below umbrella, no device." + }, + { + "entity": "tablet", + "xyxy": [ + 1300, + 790, + 1480, + 832 + ], + "reason": "Visible mat interior below umbrella, no device." + }, + { + "entity": "laptop", + "xyxy": [ + 1300, + 790, + 1480, + 832 + ], + "reason": "Visible mat interior below umbrella, no device." + }, + { + "entity": "cardboard_box", + "xyxy": [ + 1225, + 85, + 1390, + 221 + ], + "reason": "Wood-grain stool seat with cutout handle, not a cardboard carton." + } + ] + }, + { + "frame_id": "gcs-20260912-Taichung-cam11-t330", + "image_sha256": "f3360b01d4e4d82e1fc8c552714e287efadaeedd015cc3ab0440ee01603ab19a", + "companion_sha256": "adba1791cd74f51dab852defec9e743ed3b20045454f100b59a93cb1617b370c", + "positives": [ + { + "id": "ai-0003", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 176 + }, + { + "id": "ai-0016", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 178 + }, + { + "id": "ai-0000", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 179 + }, + { + "id": "ai-0011", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 180 + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 181 + } + ], + "negative_rects": [] + }, + { + "frame_id": "0058-Kaohsiung-cam08", + "image_sha256": "58dbfa809ad611de31ccc2bd1e2ba07b00377974702513e0bde6cef7e8cacd06", + "companion_sha256": "5b183748ae1474ca2acefe6d2e92f666181b5d0e6af96309e1434745d3a8f066", + "positives": [ + { + "id": "vlm-unknown-0016", + "entity": "cardboard_box", + "reason": "Assistant inspected original frame and isolated mask. Left plain brown closed shipping carton held by a person; mask separates this single carton from the adjacent one. Overrides earlier VLM abstention; combined two-carton proposal vlm-unknown-0013 rejected." + } + ], + "negative_rects": [] + } + ], + "deferred_candidates": [ + { + "frame_id": "0008-Kaohsiung-cam07", + "id": "ai-0035", + "reason": "Multiple packaged objects in one mask" + }, + { + "frame_id": "0008-Kaohsiung-cam07", + "id": "ai-0036", + "reason": "Disconnected mask contaminates bounding box" + }, + { + "frame_id": "0002-Kaohsiung-cam04", + "id": "ai-0007", + "reason": "Ambiguous payment terminal or calculator" + }, + { + "frame_id": "0033-Taichung-cam09", + "id": "ai-0007", + "reason": "Multiple overlapping packages" + }, + { + "frame_id": "0033-Taichung-cam09", + "id": "ai-0008", + "reason": "Multiple overlapping packages" + }, + { + "frame_id": "0082-Taichung-cam11", + "id": "ai-0001", + "reason": "Mask includes floor or disconnected background" + }, + { + "frame_id": "0007-Kaohsiung-cam06", + "id": "ai-0008", + "reason": "Printed price placard, not physical phone" + }, + { + "frame_id": "0007-Kaohsiung-cam06", + "id": "ai-0018", + "reason": "Headphones, not speaker" + }, + { + "frame_id": "0053-Kaohsiung-cam05", + "id": "ai-0003", + "reason": "Phone versus payment terminal unresolved" + }, + { + "frame_id": "0016-Taichung-cam01", + "id": "ai-0014", + "reason": "Disconnected mask contaminates bounding box" + }, + { + "frame_id": "0016-Taichung-cam01", + "id": "ai-0015", + "reason": "Printed placard, not boxed stock" + }, + { + "frame_id": "0016-Taichung-cam01", + "id": "ai-0016", + "reason": "Printed placard, not boxed stock" + }, + { + "frame_id": "0034-Taichung-cam10", + "id": "ai-0020", + "reason": "Laptop versus detachable tablet unresolved" + }, + { + "frame_id": "0034-Taichung-cam10", + "id": "ai-0014", + "reason": "Physical phone versus printed price card unresolved" + }, + { + "frame_id": "0013-Kaohsiung-cam11", + "id": "ai-0004", + "reason": "Phone depicted inside TV advertisement, not a physical scene instance" + } + ], + "parent_review_sha256": "15081f1fe3dddddbff13d505e3d0f4ed7b237cdcda33dace406f7fcf76e88de9", + "added_review_sha256": "9f4c858c78cc83a28f40769d859f8f373572f8b8ee1f5058e0297c3df2615b55" +} diff --git a/docs/reviews/studioa_class_negative_warmup_20260916.json b/docs/reviews/studioa_class_negative_warmup_20260916.json new file mode 100644 index 0000000..d4b87ce --- /dev/null +++ b/docs/reviews/studioa_class_negative_warmup_20260916.json @@ -0,0 +1,404 @@ +{ + "status": "completed AI-reviewed data/loss extension; candidate not promoted", + "git_commit": "c274cb2fadb44df8d31e1731ff224a1b42ed842b", + "data": { + "root": "runs/studioa_partial_instances_20260916_v4", + "source_train_frames_visually_screened": 62, + "class_negative_frames": 5, + "reviewed_image_regions": 19, + "class_negative_decisions": 47, + "class_negative_pixels": { + "phone": 263044, + "tablet": 251789, + "laptop": 245839, + "person": 4875, + "speaker": 5950, + "cardboard_box": 22440 + }, + "train_frames": 28, + "train_positive_observations": 130, + "train_by_class": { + "laptop": 9, + "cardboard_box": 4, + "tablet": 15, + "speaker": 9, + "fire_equipment": 8, + "phone": 17, + "boxed_stock": 10, + "poster": 20, + "person": 23, + "chair": 15 + }, + "cardboard_train_cameras": 2, + "new_cardboard_frame": "0058-Kaohsiung-cam08", + "new_cardboard_source_id": "vlm-unknown-0016", + "combined_two_box_proposal_rejected": "vlm-unknown-0013", + "new_gcs_download_bytes": 0, + "old_images_targets_empty_masks_companions_preserved": 33, + "val_frames": 6, + "val_positive_observations": 31, + "semantic_data_unchanged": true, + "all_old_split_assignments_unchanged": true + }, + "validation": { + "tests_passed": 79, + "ruff_ty_and_commit_hooks_passed": true, + "gpu_smoke_frames": 28, + "gpu_smoke_steps": 14, + "unknown_gradient_channels_checked": 2457821, + "unknown_gradient_nonzero": 0, + "smoke_frozen_files_verified": 319, + "training_frozen_files_verified": 779, + "training_outputs_verified": 8, + "best_last_shared_tensors_equal_original": true, + "checkpoint_tensors_finite": true, + "per_image_predictions_reproduce_official_metrics": true, + "both_services_exit_status": 0 + }, + "training": { + "run": "runs/studioa_detector_warmup_20260916_v2", + "last_epoch": 25, + "best_epoch": 5, + "optimizer_steps": 350, + "best_optimizer_steps": 70, + "early_stop_patience": 20, + "scene_invariance": { + "frozen_state_tensors": 194, + "frozen_state_equal": true, + "scene_before": { + "frames": 33, + "float32_logits_sha256": "72b0bc0688abbc2e2a8cbafb9275219300f36dc87ef620cb34b9ffda4acd8867" + }, + "scene_after": { + "frames": 33, + "float32_logits_sha256": "72b0bc0688abbc2e2a8cbafb9275219300f36dc87ef620cb34b9ffda4acd8867" + }, + "scene_logits_equal": true, + "baseline_recall": 0.0, + "selected_recall": 0.1935483870967742, + "baseline_empty_fp": 0.0, + "selected_empty_fp": 0.0, + "accepted": true + }, + "worker_accepted_means": "random-initialization baseline only; see promotion decision for comparison to previous trained detector" + }, + "comparison": { + "reference_run": "runs/studioa_detector_warmup_20260916_v1", + "candidate_run": "runs/studioa_detector_warmup_20260916_v2", + "reference_recall": 0.22580645161290322, + "candidate_recall": 0.1935483870967742, + "accepted_for_upgrade": false, + "reason": "Candidate recall 6/31 is below reference 7/31; zero empty-region FP and unchanged scene do not compensate.", + "retained_reference_checkpoint": "runs/studioa_detector_warmup_20260916_v1/model/best.pt", + "worker_acceptance_scope": "report.json warmup.accepted is relative to random detector epoch 0 only, not this reference comparison.", + "deployment_ready": false + }, + "selected_validation": { + "matched": 6, + "support": 31, + "recall": 0.1935483870967742, + "empty_fp": 0, + "empty_pixels": 51761, + "unknown_predictions": 594, + "every_image_hits_100_prediction_cap": true, + "per_class": { + "laptop": { + "support": 5, + "matched": 0 + }, + "phone": { + "support": 4, + "matched": 0 + }, + "tablet": { + "support": 1, + "matched": 0 + }, + "boxed_stock": { + "support": 1, + "matched": 0 + }, + "cardboard_box": { + "support": 2, + "matched": 0 + }, + "speaker": { + "support": 1, + "matched": 0 + }, + "poster": { + "support": 2, + "matched": 1 + }, + "fire_equipment": { + "support": 1, + "matched": 0 + }, + "chair": { + "support": 5, + "matched": 0 + }, + "person": { + "support": 9, + "matched": 5 + } + } + }, + "train_negative_response_probe": { + "scope": "five training images; pre-NMS grid-class scores in explicitly reviewed negative regions; not held-out accuracy, not object false positive count", + "frames": [ + "gcs-20260910-Kaohsiung-cam07-t030", + "gcs-20260910-Kaohsiung-cam08-t030", + "gcs-20260910-Taichung-cam05-t030", + "gcs-20260910-Taichung-cam08-t030", + "gcs-20260912-Taichung-cam08-t330" + ], + "results": { + "v1": { + "checkpoint_sha256": "8381ff3ab6d97a86d9c2896568867971991805288c9f847d6f09d3928aa2c3f5", + "reviewed_grid_class_pairs": 3543, + "score_over_020": 1776, + "mean_cls_times_centerness": 0.2296547035626476, + "per_class": { + "laptop": { + "grid_points": 1092, + "score_over_020": 357, + "mean_score": 0.17110549139742476 + }, + "phone": { + "grid_points": 1174, + "score_over_020": 763, + "mean_score": 0.2740117663944384 + }, + "tablet": { + "grid_points": 1119, + "score_over_020": 600, + "mean_score": 0.24573806704582116 + }, + "cardboard_box": { + "grid_points": 106, + "score_over_020": 15, + "mean_score": 0.11761251121829704 + }, + "speaker": { + "grid_points": 27, + "score_over_020": 20, + "mean_score": 0.3011369218842851 + }, + "person": { + "grid_points": 25, + "score_over_020": 21, + "mean_score": 0.3820433706045151 + } + } + }, + "v2": { + "checkpoint_sha256": "7c1d23f263bc1a1b18936c46e3226dba9c666c4585ac9ad931b5bb5560a9642a", + "reviewed_grid_class_pairs": 3543, + "score_over_020": 15, + "mean_cls_times_centerness": 0.030960009247288024, + "per_class": { + "laptop": { + "grid_points": 1092, + "score_over_020": 1, + "mean_score": 0.022852739354598955 + }, + "phone": { + "grid_points": 1174, + "score_over_020": 0, + "mean_score": 0.02824464823167733 + }, + "tablet": { + "grid_points": 1119, + "score_over_020": 1, + "mean_score": 0.03571344299077109 + }, + "cardboard_box": { + "grid_points": 106, + "score_over_020": 0, + "mean_score": 0.037260681531339324 + }, + "speaker": { + "grid_points": 27, + "score_over_020": 0, + "mean_score": 0.10272153511781383 + }, + "person": { + "grid_points": 25, + "score_over_020": 13, + "mean_score": 0.19561791867017747 + } + } + } + } + }, + "phone_failure_diagnosis": { + "scope": "source-val phone geometric diagnostics at existing score >0.20 and IoU >=0.50; no tuning or test inference; not one-to-one recall", + "results": { + "v1": [ + { + "frame_id": "0013-Kaohsiung-cam11", + "box": [ + 540.86669921875, + 399.26666259765625, + 566.066650390625, + 439.8666687011719 + ], + "localized_grid_candidates": 11, + "localized_phone_score_over_020": 10, + "eligible_before_nms_and_cap": 5, + "retained": true, + "max_phone_score_on_localized_boxes": 0.40896254777908325, + "reason": "retained" + }, + { + "frame_id": "0013-Kaohsiung-cam11", + "box": [ + 494.6666564941406, + 403.0, + 516.5999755859375, + 445.0 + ], + "localized_grid_candidates": 0, + "localized_phone_score_over_020": 0, + "eligible_before_nms_and_cap": 0, + "retained": false, + "max_phone_score_on_localized_boxes": null, + "reason": "no_box_localized_at_iou_050" + }, + { + "frame_id": "0013-Kaohsiung-cam11", + "box": [ + 572.5999755859375, + 375.0, + 600.5999755859375, + 412.79998779296875 + ], + "localized_grid_candidates": 7, + "localized_phone_score_over_020": 6, + "eligible_before_nms_and_cap": 4, + "retained": false, + "max_phone_score_on_localized_boxes": 0.3057223856449127, + "reason": "removed_by_nms_or_top100" + }, + { + "frame_id": "0013-Kaohsiung-cam11", + "box": [ + 457.3333435058594, + 385.73333740234375, + 482.0666809082031, + 425.8666687011719 + ], + "localized_grid_candidates": 0, + "localized_phone_score_over_020": 0, + "eligible_before_nms_and_cap": 0, + "retained": false, + "max_phone_score_on_localized_boxes": null, + "reason": "no_box_localized_at_iou_050" + } + ], + "v2": [ + { + "frame_id": "0013-Kaohsiung-cam11", + "box": [ + 540.86669921875, + 399.26666259765625, + 566.066650390625, + 439.8666687011719 + ], + "localized_grid_candidates": 1, + "localized_phone_score_over_020": 0, + "eligible_before_nms_and_cap": 0, + "retained": false, + "max_phone_score_on_localized_boxes": 0.02521689049899578, + "reason": "phone_score_below_threshold" + }, + { + "frame_id": "0013-Kaohsiung-cam11", + "box": [ + 494.6666564941406, + 403.0, + 516.5999755859375, + 445.0 + ], + "localized_grid_candidates": 0, + "localized_phone_score_over_020": 0, + "eligible_before_nms_and_cap": 0, + "retained": false, + "max_phone_score_on_localized_boxes": null, + "reason": "no_box_localized_at_iou_050" + }, + { + "frame_id": "0013-Kaohsiung-cam11", + "box": [ + 572.5999755859375, + 375.0, + 600.5999755859375, + 412.79998779296875 + ], + "localized_grid_candidates": 5, + "localized_phone_score_over_020": 0, + "eligible_before_nms_and_cap": 0, + "retained": false, + "max_phone_score_on_localized_boxes": 0.023917578160762787, + "reason": "phone_score_below_threshold" + }, + { + "frame_id": "0013-Kaohsiung-cam11", + "box": [ + 457.3333435058594, + 385.73333740234375, + 482.0666809082031, + 425.8666687011719 + ], + "localized_grid_candidates": 0, + "localized_phone_score_over_020": 0, + "eligible_before_nms_and_cap": 0, + "retained": false, + "max_phone_score_on_localized_boxes": null, + "reason": "no_box_localized_at_iou_050" + } + ] + } + }, + "limits": [ + "All labels and reviews are AI-produced; no independent accuracy claim.", + "Negative-region probe is on training images and different selected epochs, not evidence of held-out precision or isolated causal improvement.", + "V2 phone: two reviewed boxes have no raw candidate at IoU >=0.50; two have localized candidates but max phone score approximately 0.024-0.025, below unchanged 0.20 threshold. Raising max_det alone cannot repair those four misses.", + "Same validation set was used for checkpoint selection; it is not a fresh independent test." + ], + "next_step": "Measure per-class positive/negative loss balance and bound class-negative influence before another predeclared comparison; improve small-object positive localization support. Preserve fixed thresholds and original test isolation.", + "test_evaluated": false, + "independent_accuracy": false, + "deployment_ready": false, + "artifacts": { + "runs/studioa_detector_warmup_20260916_v2/job.json": "c042b7471bbc3f608725404948f53f670505913b7c816ad17aba3abd5d12f07e", + "runs/studioa_detector_warmup_20260916_v2/report.json": "bda6ba29c40ba2f71fc3228755aae58ebe3cb0bb7d315d839d83befde6adc228", + "runs/studioa_detector_warmup_20260916_v2/promotion_decision.json": "1224a16aa6f37c4f47f74abdec814554e7deac9bd20f49cdda3982e8cc0e3eba", + "runs/studioa_detector_warmup_20260916_v2/status.json": "8d5d32f512ef2b1d787f6cb713204c17daf773748812261a3dd736258adb33ef", + "runs/studioa_class_negative_smoke_20260916_v1/job.json": "ba72688fec6f61819486fafb4cb09f81feebb7d97dae5386bcad2180dae4947d", + "runs/studioa_class_negative_smoke_20260916_v1/report.json": "ba4e6894d3e0f767434549214fc9229b14330ce889e26ff283144babc12d58a8", + "runs/studioa_class_negative_smoke_20260916_v1/status.json": "3214ab26ed45b60457e62581fb64b8ee09bfc5f5b85a9ea6c6bbbca928062729", + "runs/studioa_partial_instances_20260916_v4/manifest.json": "e12339ea28db97a8a59c839ad095bf58d961f92af46b0758d6343ebc83475fc1", + "runs/studioa_partial_instances_20260916_v4/report.json": "2c17a48f9772030eee957fd24027830a711b768eaa64cf964408d646c6529dd6", + "docs/reviews/studioa_class_negative_decisions_20260916.json": "c647a5e13f533363af44465da22b424a8441c6b89e385703acac402b51b1e932", + "runs/studioa_confusers_20260916_v1/class_negative_contact.jpg": "9c5657ca1b5298a269336646fa67634e2694d64e847d881cb6336e50ce1ae55e", + "runs/studioa_confusers_20260916_v1/vlm-unknown-0016.jpg": "4c596e0c887859948e47fb2cc55c95fd1e64bf743364081b965f4535ba7198b1", + "runs/studioa_detector_warmup_20260916_v2/model/best.pt": "7c1d23f263bc1a1b18936c46e3226dba9c666c4585ac9ad931b5bb5560a9642a", + "runs/studioa_detector_warmup_20260916_v2/model/last.pt": "cc9213a6996d915bed67d1bdccf8dfb1b956719de150d509c427b8ecc10cef39", + "runs/studioa_detector_warmup_20260916_v2/validation.json": "6950b3e1e5ac11d506131bc7e33b6f40bd6e14051ce705dd4ce1f40ca58fdf97", + "runs/studioa_detector_warmup_20260916_v2/baseline_val.json": "445010c860040e87a677a984948f275e120f3f0f9bd6c85c070820efe7831962", + "runs/studioa_detector_warmup_20260916_v2/model/metrics.jsonl": "e9c2012e73bb114c4faea57117851039b26c649a05976a079aa64da60bc220ae", + "runs/studioa_detector_warmup_20260916_v2/validation_scene_preview.jpg": "e1798a2a38d0cab2eb1b06b75d18d29a8d66ad69cecb3082cc7c9bad25b78b57", + "runs/studioa_detector_warmup_20260916_v2/scene_before.json": "d7c9e18e014bb4e47b0a6acbae3e56ead9448a44e7dffe79097eeb4221d527a4", + "runs/studioa_detector_warmup_20260916_v2/warmup_evidence.json": "2e90f6e5b4e97f471fd2a99f1de143153801a87a1262024410fefeec30002c3c", + "runs/studioa_detector_warmup_20260916_v2/review/probe_phone_candidates.py": "3047e7c151d125504dd6c49a0cc8ac28ff80b043572fda8bf73ba58c01284161", + "runs/studioa_detector_warmup_20260916_v2/review/inspect_selected.py": "eae0fb3f94338de83abfe586b430b753a028abed44564355a95a82bdd5462e91", + "runs/studioa_detector_warmup_20260916_v2/review/probe_train_negatives.py": "c8de43945061d229f7ec3e30c3f10a8ee988014418d4c1f525b69302b496181d", + "runs/studioa_detector_warmup_20260916_v2/review/train_negative_probe.json": "a05f1b19ad42e34344ad341d05764a040f987ce13ecb3aaef931270bff260dc3", + "runs/studioa_detector_warmup_20260916_v2/review/phone_candidate_probe.json": "632416d4446caa8d1314b532dc2c49fdbf3d10810caa5d5f98df714ad904b454", + "runs/studioa_detector_warmup_20260916_v2/review/retained_prediction_error_analysis.json": "07e66d8c4e6431a144b7e90565d56da092fcc589216d8ec4ed2e85c4d36edd20", + "runs/studioa_detector_warmup_20260916_v2/review/selected_diagnostics.json": "cd3cf1c69d5de3e29339691a7e80a567a886a29a337da69b71d140c11b92c739", + "runs/studioa_detector_warmup_20260916_v2/review/selected_val_review.jpg": "14c2e2db2da562358270df87d313ed6f08a33c1f1ec9189c5796ce5bfcc48b17" + } +} diff --git a/docs/reviews/studioa_detector_warmup_20260916.json b/docs/reviews/studioa_detector_warmup_20260916.json new file mode 100644 index 0000000..175db39 --- /dev/null +++ b/docs/reviews/studioa_detector_warmup_20260916.json @@ -0,0 +1,158 @@ +{ + "status": "completed; frozen-scene detector warmup accepted; detector not deployment-ready", + "run": "runs/studioa_detector_warmup_20260916_v1", + "git_commit": "562532d9c72ead3e7c78ce162c7b66c9666701ae", + "job_sha256": "be9412eb64d286d129ef79c794c6da36b6df7d2b163851f12bc8f686ff66cbe6", + "training": { + "train_frames": 27, + "train_observations": 129, + "train_cameras": 9, + "scene_train_frames_used": 0, + "max_epochs": 60, + "last_epoch": 40, + "best_epoch": 20, + "optimizer_steps": 520, + "best_optimizer_steps": 260, + "stop_reason": "20 validations without strict recall improvement", + "systemd_unit": "studioa-detector-warmup-20260916-v1.service", + "service_result": "success", + "exit_status": 0 + }, + "scene_invariance": { + "frozen_state_tensors": 194, + "frozen_state_equal": true, + "scene_before": { + "frames": 33, + "float32_logits_sha256": "72b0bc0688abbc2e2a8cbafb9275219300f36dc87ef620cb34b9ffda4acd8867" + }, + "scene_after": { + "frames": 33, + "float32_logits_sha256": "72b0bc0688abbc2e2a8cbafb9275219300f36dc87ef620cb34b9ffda4acd8867" + }, + "scene_logits_equal": true, + "baseline_recall": 0.0, + "selected_recall": 0.22580645161290322, + "baseline_empty_fp": 0.0, + "selected_empty_fp": 0.0, + "accepted": true + }, + "evaluation": { + "split": "source_val_only", + "frames": 6, + "reviewed_positives": 31, + "matched": 7, + "recall": 0.22580645161290322, + "reviewed_empty_false_alarms": 0, + "reviewed_empty_input_pixels": 51761, + "unmatched_truth_unknown_predictions": 593, + "each_frame_reached_max_det_100": true, + "per_class": { + "laptop": { + "train_observations": 9, + "train_camera_count": 2, + "val_support": 5, + "val_matched": 0 + }, + "phone": { + "train_observations": 17, + "train_camera_count": 4, + "val_support": 4, + "val_matched": 1 + }, + "tablet": { + "train_observations": 15, + "train_camera_count": 5, + "val_support": 1, + "val_matched": 0 + }, + "boxed_stock": { + "train_observations": 10, + "train_camera_count": 5, + "val_support": 1, + "val_matched": 0 + }, + "cardboard_box": { + "train_observations": 3, + "train_camera_count": 1, + "val_support": 2, + "val_matched": 0 + }, + "speaker": { + "train_observations": 9, + "train_camera_count": 3, + "val_support": 1, + "val_matched": 0 + }, + "poster": { + "train_observations": 20, + "train_camera_count": 7, + "val_support": 2, + "val_matched": 1 + }, + "fire_equipment": { + "train_observations": 8, + "train_camera_count": 2, + "val_support": 1, + "val_matched": 0 + }, + "chair": { + "train_observations": 15, + "train_camera_count": 5, + "val_support": 5, + "val_matched": 0 + }, + "person": { + "train_observations": 23, + "train_camera_count": 7, + "val_support": 9, + "val_matched": 5 + } + } + }, + "verification": { + "frozen_input_hashes_verified": 757, + "output_hashes_verified": 8, + "best_and_last_shared_state_equal_original": true, + "checkpoint_tensors_finite": true, + "scene_miou_all_40_epochs": 0.2506537344807518, + "saved_per_image_predictions_reproduce_all_detection_metrics": true, + "main_test_batch_passed": 171, + "data_contract_followup_test_batch_passed": 20, + "test_batches_overlap": true, + "changed_file_ruff_ty_and_commit_hooks_passed": true, + "test_evaluated": false + }, + "interpretation": [ + "Acceptance applies only to preserving scene outputs while improving partial validation recall, not general detection quality.", + "All six images reach the fixed 100-prediction cap; 593 unmatched predictions remain unknown under partial supervision and cannot define precision.", + "Seven classes have zero matched validation positives.", + "Cardboard has only three training observations, all in one frame from one camera.", + "Compare scene mIoU to this run baseline: deterministic TF32-disabled settings differ from earlier runs.", + "Data quantity and training isolation changed together relative to the joint pilot; no single-factor causal attribution." + ], + "next_step": [ + "AI-review source-train confusers and add explicit per-class negative supervision; do not mark real objects as all-class background.", + "Seek cardboard in other allowed source-train cameras; preserve val/test assignments.", + "Keep score/NMS/max_det fixed; do not tune thresholds on this val split." + ], + "independent_accuracy": false, + "deployment_ready": false, + "artifacts": { + "runs/studioa_detector_warmup_20260916_v1/job.json": "be9412eb64d286d129ef79c794c6da36b6df7d2b163851f12bc8f686ff66cbe6", + "runs/studioa_detector_warmup_20260916_v1/report.json": "e4af368a4221673df7cf6440a52d9eae32bfcabba3f7bac52bb975a17a385921", + "runs/studioa_detector_warmup_20260916_v1/status.json": "8d448ee2341364ce4a3fe174d5dff529c8be0b2b29e3f458c12a13951544f614", + "runs/studioa_detector_warmup_20260916_v1/service.log": "a717ed6aa1976c5afabb27a61d6fa7e7f9e410c7bba8304d3572a88db9bd73b6", + "runs/studioa_detector_warmup_20260916_v1/config.json": "55080088b611308124460203c1a869c736eb0a6d332cc8977a9e084bc778689a", + "runs/studioa_detector_warmup_20260916_v1/review/inspect_selected.py": "ceab1f30d937928152d46b946d84a444231f29598f60d487988836d762fe0092", + "runs/studioa_detector_warmup_20260916_v1/review/selected_val_review.jpg": "8ce9cee0423fc81d708947c300d75c0e0d55c9e8103ebab0b3165bdbc1d5cd8c", + "runs/studioa_detector_warmup_20260916_v1/review/selected_diagnostics.json": "f51ee51464278bdd6193b71390018c7f98d6fa7551cdbffba7583c5f6444a9fa", + "runs/studioa_detector_warmup_20260916_v1/model/best.pt": "8381ff3ab6d97a86d9c2896568867971991805288c9f847d6f09d3928aa2c3f5", + "runs/studioa_detector_warmup_20260916_v1/model/last.pt": "c9337234b942d950591c9f2ab49b35c0976bed05294ab432918b033ed54d68a2", + "runs/studioa_detector_warmup_20260916_v1/validation.json": "a9e3b29410192b1f6bf8d49f583bd8b641f583c09f5b5558767cfd4a9171fa5a", + "runs/studioa_detector_warmup_20260916_v1/baseline_val.json": "445010c860040e87a677a984948f275e120f3f0f9bd6c85c070820efe7831962", + "runs/studioa_detector_warmup_20260916_v1/model/metrics.jsonl": "00b7e9492f55f7bd5ede40f3fcbba1a114d13e401c0d324126f234148026e846", + "runs/studioa_detector_warmup_20260916_v1/validation_scene_preview.jpg": "e1798a2a38d0cab2eb1b06b75d18d29a8d66ad69cecb3082cc7c9bad25b78b57", + "runs/studioa_detector_warmup_20260916_v1/scene_before.json": "d7c9e18e014bb4e47b0a6acbae3e56ead9448a44e7dffe79097eeb4221d527a4", + "runs/studioa_detector_warmup_20260916_v1/warmup_evidence.json": "93b264b01dfe739bd1ca301060719fb7dbb3dd89d1a33cc27c9a1a8975064641" + } +} diff --git a/docs/reviews/studioa_gcs_expansion_20260916.json b/docs/reviews/studioa_gcs_expansion_20260916.json new file mode 100644 index 0000000..4d64d33 --- /dev/null +++ b/docs/reviews/studioa_gcs_expansion_20260916.json @@ -0,0 +1,119 @@ +{ + "status": "completed AI-reviewed data expansion; not a trained model upgrade", + "intake": { + "bucket": "gs://studioa", + "source_stores": [ + "Kaohsiung", + "Taichung" + ], + "source_train_cameras": 9, + "date_slots": [ + "2026-09-10 12:00", + "2026-09-12 18:50" + ], + "filename_time_is_approximate": true, + "clips": 18, + "download_bytes": 453678490, + "download_budget_bytes": 1000000000, + "gcs_generation_and_md5_verified": true, + "candidate_frames": 36, + "selected_frames": 16, + "excluded_redundant_frames": 20, + "excluded_exact_pixel_duplicates": 0 + }, + "annotation": { + "teacher": "facebook/sam3", + "source_ai_entities": 504, + "source_unresolved_candidates": 553, + "candidate_masks_visually_inspected": 183, + "new_accepted_instance_observations": 98, + "manually_class_corrected_by_assistant": 1, + "unselected_proposals": "unknown; no negative/background claim", + "human_reviewed": false, + "unique_physical_object_count": "not measured; same physical object may recur across dates" + }, + "instance_training_package": "runs/studioa_partial_instances_20260916_v3", + "semantic_training_package": "runs/studioa_gcs_training_extension_20260916_v1/semantic", + "counts": { + "train": { + "frames": 27, + "cameras": 9, + "instances": 129, + "by_class": { + "laptop": 9, + "cardboard_box": 3, + "tablet": 15, + "speaker": 9, + "fire_equipment": 8, + "phone": 17, + "boxed_stock": 10, + "poster": 20, + "person": 23, + "chair": 15 + }, + "negative_pixels": 198600 + }, + "val": { + "frames": 6, + "cameras": 6, + "instances": 31, + "by_class": { + "laptop": 5, + "tablet": 1, + "speaker": 1, + "poster": 2, + "person": 9, + "boxed_stock": 1, + "cardboard_box": 2, + "fire_equipment": 1, + "chair": 5, + "phone": 4 + }, + "negative_pixels": 237500 + } + }, + "preservation": { + "old_semantic_masks_byte_identical": 121, + "old_instance_images_targets_negative_masks_byte_identical": 17, + "all_prior_fold_assignments_unchanged": true, + "test_evaluated": false + }, + "validation": { + "related_tests_passed": 59, + "intake_output_hashes_verified": 57, + "ai_report_output_hashes_verified": 17, + "smoke_frozen_files_verified": 297, + "gpu_steps": 14, + "gpu_frames": 27, + "unknown_logit_channels_checked": 2366580, + "unknown_nonzero_gradients": 0, + "classifier_update_norm": 0.014031676575541496, + "checkpoint_written": false, + "services_final_result": "success; exit 0", + "initial_intake_failure": "ffprobe missing from systemd PATH; resumed same pinned job with explicit PATH; logs retained" + }, + "remaining_gaps": [ + "Cardboard box still only 3 training observations; no new unambiguous boxes accepted.", + "Object-specific hard negatives have not been added; reviewed-empty regions remain the original floor patches.", + "Only 31 reviewed validation positives; this is not sufficient for deployment/generalization claims.", + "Same-camera repeated physical objects do not establish independent sample diversity." + ], + "next_step": "Freeze shared scene features and normalization for detector warm-up; add reviewed class-specific confusers and cardboard boxes; do not repeat the failed unrestricted joint recipe.", + "independent_accuracy": false, + "deployment_ready": false, + "artifacts": { + "runs/studioa_gcs_intake_20260916_v1/plan.json": "1b4d177c7b0b01f142395cd0cf325deee222e8224ee89876650cd128598033e9", + "runs/studioa_gcs_intake_20260916_v1/manifest.json": "a59a572a41e29dc42e6b8113c00f3a8a581abee341993aff964fe3a82fa59d0d", + "runs/studioa_gcs_intake_20260916_v1/report.json": "f925f7e6c20a89539432d18ff9d54aa093c425479b9398d696bab844fdf49f0a", + "runs/studioa_gcs_ai_20260916_v1/job.json": "2c48f8697a173d8bc4460d2960806d6e620ee98c5871a19ef4eff7006984b13d", + "runs/studioa_gcs_ai_20260916_v1/report.json": "68f3b1c535b01cb1c1dc7524539cd99cee2d988e2c3c7b12253dd98b7bad9d62", + "runs/studioa_gcs_training_extension_20260916_v1/extension_report.json": "55b66c5e6414f6001c366b2003a6ca132886a8fd4af91d19d223d48cf5519438", + "runs/studioa_gcs_training_extension_20260916_v1/semantic/manifest.json": "0139eaccc00c6cf2fd841c7cfbd3a231c9b744a624b9c8ec95f82bc6e7a7b1ee", + "runs/studioa_partial_instances_20260916_v3/manifest.json": "f74f99103a841bd4553d41cc8ce85e1257486de0cb065a108dbc077dfb643367", + "runs/studioa_partial_instances_20260916_v3/report.json": "43892b57c74c50a441952949c74bfde145a72fd5c4922743feea6ec199d342f7", + "runs/studioa_gcs_instance_smoke_20260916_v1/job.json": "d336b04d781e597f6a1edbbe0bf14fd0e75a11679e2691bb4eb7e4154aa69dfa", + "runs/studioa_gcs_instance_smoke_20260916_v1/report.json": "757be212a1abd9c1a8301e5fe02ba7f3eadfcfc337fa32de97043b8d1d411c55", + "runs/studioa_gcs_expansion_20260916_v1/reviewed_contact.jpg": "c9479bae04651c97b281d4347506f828d09e9b512813c0eeef23f011779c928c", + "runs/studioa_gcs_expansion_20260916_v1/reviewed_gallery.html": "215ff833266288f2e60c1131fdb3f77a6ada9e97d068d0e20337dc20e55cadb7" + } +} diff --git a/docs/reviews/studioa_gcs_merged_instance_decisions_20260916.json b/docs/reviews/studioa_gcs_merged_instance_decisions_20260916.json new file mode 100644 index 0000000..df9b402 --- /dev/null +++ b/docs/reviews/studioa_gcs_merged_instance_decisions_20260916.json @@ -0,0 +1,1350 @@ +{ + "reviewer_kind": "ai", + "reviewer": "assistant_visual_analysis", + "held_out": "Tao-Hsin", + "source_manifest_sha256": "0139eaccc00c6cf2fd841c7cfbd3a231c9b744a624b9c8ec95f82bc6e7a7b1ee", + "scope": "Existing reviewed train/val subset plus 16 AI-reviewed GCS source-train frames; unknown pixels are not negative. Original val positives/boxes/negative masks preserved.", + "frames": [ + { + "frame_id": "0050-Kaohsiung-cam04", + "image_sha256": "7b1e1be2cf234ab9d881fef37b49789f4f1b64eff655758b2e674d6819ee6bac", + "companion_sha256": "0423e2002e1c61c16d815172dd247c61c3f4a21d982f6c64d88ad962b16656e0", + "positives": [ + { + "id": "ai-0005", + "entity": "laptop", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0007", + "entity": "cardboard_box", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0008", + "entity": "cardboard_box", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0010", + "entity": "cardboard_box", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0017", + "entity": "laptop", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 1650, + 400, + 1800, + 650 + ], + "reason": "Assistant inspected source image: empty floor patch, no visible object from the declared ten detection classes; not inferred from missing labels." + } + ] + }, + { + "frame_id": "0030-Taichung-cam08", + "image_sha256": "e95523bd8f92ab2fbb7ed5fa672e25b7cf01d531ed94711a016572351d1b1ea4", + "companion_sha256": "e452f0356246f096cc9bcf3c8a16b403c24a87983900fb9fc509a76397a5d720", + "positives": [ + { + "id": "ai-0003", + "entity": "tablet", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0004", + "entity": "tablet", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0005", + "entity": "tablet", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 1000, + 960, + 1500, + 1050 + ], + "reason": "Assistant inspected source image: empty floor patch, no visible object from the declared ten detection classes; not inferred from missing labels." + } + ] + }, + { + "frame_id": "0112-Taichung-cam11", + "image_sha256": "3353277f188293dba730eefd1f3f42ddbd4e6b50ee87e38d4c954f22d8495355", + "companion_sha256": "051477d05602005a21aef67b3af839ec4956f1c3511be724e006eafb1ff30eb9", + "positives": [ + { + "id": "ai-0003", + "entity": "speaker", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 1100, + 650, + 1250, + 850 + ], + "reason": "Assistant inspected source image: empty floor patch, no visible object from the declared ten detection classes; not inferred from missing labels." + } + ] + }, + { + "frame_id": "0078-Taichung-cam09", + "image_sha256": "30918a1a840d0e54a4d88dd8c34bb2907a0b7931f25ce60f7f4e878b5528f6b2", + "companion_sha256": "bedfde8148dae9581b2fa7322caa9afb173a9448e394227f68887d27baa4ffc0", + "positives": [ + { + "id": "ai-0002", + "entity": "fire_equipment", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0003", + "entity": "fire_equipment", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 1500, + 850, + 1800, + 1000 + ], + "reason": "Assistant inspected source image: empty floor patch, no visible object from the declared ten detection classes; not inferred from missing labels." + } + ] + }, + { + "frame_id": "0025-Taichung-cam05", + "image_sha256": "08b398cb45b48e99021b07d5247539c8138805cfb043c7d86421c6bfd2fe24a4", + "companion_sha256": "9dd17564938c9007cd40050ce79aceaebfedbad2b9aa1f1970174557ac462e87", + "positives": [ + { + "id": "ai-0009", + "entity": "phone", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 1730, + 760, + 1810, + 930 + ], + "reason": "Assistant inspected source image: empty floor patch, no visible object from the declared ten detection classes; not inferred from missing labels." + } + ] + }, + { + "frame_id": "0031-Taichung-cam08", + "image_sha256": "ddd3f30a15b4b527c8dc5a025b7e9a5c5cf21561936b3e98fa17bbcd8d411b4c", + "companion_sha256": "51097af5eae2df30092a13a14afc6ebaf964454f62d0800d9291c31bbca40b53", + "positives": [ + { + "id": "ai-0005", + "entity": "phone", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0007", + "entity": "phone", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + }, + { + "id": "ai-0008", + "entity": "phone", + "reason": "Assistant inspected original scene and isolated candidate mask; visible single object supports this class. Partial review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 200, + 920, + 450, + 1030 + ], + "reason": "Assistant inspected source image: empty floor patch, no visible object from the declared ten detection classes; not inferred from missing labels." + } + ] + }, + { + "frame_id": "0008-Kaohsiung-cam07", + "image_sha256": "6acd71ce695fa11235c63cf3d83cea62bb39017bfbfd2eba47e198449ad48a28", + "companion_sha256": "901ae7a38b80cd42c036e8e6f594dc668096a8646132bb76102ce3ac494ad632", + "positives": [ + { + "id": "ai-0028", + "entity": "boxed_stock", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0002", + "entity": "boxed_stock", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [] + }, + { + "frame_id": "0002-Kaohsiung-cam04", + "image_sha256": "a7339928d2e72363cefb35a8f6e25d8b1784ceaf747f87b815293dfdc53257bb", + "companion_sha256": "3dfce18c1f9d25ef717683209d6396964fcaa024ba1477777078ebf0de796436", + "positives": [ + { + "id": "ai-0004", + "entity": "laptop", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0005", + "entity": "boxed_stock", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0013", + "entity": "poster", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [] + }, + { + "frame_id": "0033-Taichung-cam09", + "image_sha256": "c5b1f42f5025b4479295913c3c41c9cfa324fbc5b4200bc6a2994efa52220b3d", + "companion_sha256": "276fbc3a765a4b33f48c00a1bf2edbb7dd9ed8fc5da0a71df6f3a85f56824137", + "positives": [ + { + "id": "ai-0011", + "entity": "poster", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [] + }, + { + "frame_id": "0022-Taichung-cam04", + "image_sha256": "ad8f01fb6860896c9bbd1e3ff9bf86c10515013c23f4e8c2de87802fcea2d07b", + "companion_sha256": "f094e3b5cf5919a27ee9155700bc98cc57af081e5750f4c8ebe3f96da235691f", + "positives": [ + { + "id": "ai-0010", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0013", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0003", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [] + }, + { + "frame_id": "0082-Taichung-cam11", + "image_sha256": "4c85c1f28791ebc81eb3f687cc55f913899381ac90cb6a16ef223aeea997df1d", + "companion_sha256": "26b1250e73b68449e4b501b9509702909c3021e44a506f1108e1f8cad4c36492", + "positives": [ + { + "id": "ai-0010", + "entity": "tablet", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0003", + "entity": "speaker", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0002", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [] + }, + { + "frame_id": "0007-Kaohsiung-cam06", + "image_sha256": "8340acd371f970f294282695828b7fdd2d27e08b9529575f7f275290b38b50f8", + "companion_sha256": "3a26496740076a5e3a58e36c63fae01393ed39b344a66d0b0fe67440b2f1bc8f", + "positives": [ + { + "id": "ai-0005", + "entity": "laptop", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0011", + "entity": "laptop", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0010", + "entity": "tablet", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0007", + "entity": "speaker", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0016", + "entity": "poster", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0002", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 150, + 800, + 400, + 950 + ], + "reason": "Assistant inspected original context with this rectangle: clear floor without a visible object from the ten detection classes." + } + ] + }, + { + "frame_id": "0053-Kaohsiung-cam05", + "image_sha256": "68440d495ad4d190d7b68b046d64707339105fa7c5290b36b2c46abd3a1f4e8a", + "companion_sha256": "a9565f0ac21f98f711d0a0ff3c91608767fdc023a91557974fc41903ded832ab", + "positives": [ + { + "id": "ai-0004", + "entity": "boxed_stock", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0002", + "entity": "cardboard_box", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0006", + "entity": "cardboard_box", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0005", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 850, + 500, + 1250, + 800 + ], + "reason": "Assistant inspected original context with this rectangle: clear floor without a visible object from the ten detection classes." + } + ] + }, + { + "frame_id": "0016-Taichung-cam01", + "image_sha256": "fb3086c1ff8ed11a485226d69bacacb33ea4d534d0708096c80526f348230784", + "companion_sha256": "b30aed64d3c57dba492dec341442706b77458556b8afb6991fe1b2f6741d7bee", + "positives": [ + { + "id": "ai-0002", + "entity": "laptop", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0021", + "entity": "fire_equipment", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0004", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0005", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 600, + 700, + 700, + 900 + ], + "reason": "Assistant inspected original context with this rectangle: clear floor without a visible object from the ten detection classes." + } + ] + }, + { + "frame_id": "0076-Taichung-cam07", + "image_sha256": "a7e42db0b7ac8ebae37bc90da01b090b7de169b9cf0bf1b06088d049add3f83e", + "companion_sha256": "bf42c51b3bb9657e8bd99379eaabba2d1e53e4d4559bae34f2ceae1959b11dd0", + "positives": [ + { + "id": "ai-0005", + "entity": "laptop", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0006", + "entity": "poster", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0001", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0004", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0002", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [ + { + "xyxy": [ + 900, + 800, + 1200, + 1000 + ], + "reason": "Assistant inspected original context with this rectangle: clear floor without a visible object from the ten detection classes." + } + ] + }, + { + "frame_id": "0034-Taichung-cam10", + "image_sha256": "28c5c9e5a09f55a473bda61d0d84de4a8f679ebc3db43f892358a6a56d0c57fd", + "companion_sha256": "602aa819d8944d6dadc76bedecfa8b2609641066dd2dc6a046c52cbebf2846d3", + "positives": [ + { + "id": "ai-0002", + "entity": "laptop", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0007", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0005", + "entity": "chair", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + }, + { + "id": "ai-0010", + "entity": "person", + "reason": "Assistant inspected original context and isolated mask; a visible single physical object supports this class. Partial AI review only." + } + ], + "negative_rects": [] + }, + { + "frame_id": "0013-Kaohsiung-cam11", + "image_sha256": "4c19e17cad632f5551e2280b514e76884e74658d62841354e3ddfcc1c73a6f19", + "companion_sha256": "06b6e65f9c04d603749965cb03a5fc7eb6ee2c8419638baeb6cb93bba69f5f0a", + "positives": [ + { + "id": "vlm-unknown-0004", + "entity": "phone", + "reason": "Assistant inspected context and isolated mask: real tethered smartphone on foreground circular display table; excludes phone depicted on TV advertisement." + }, + { + "id": "vlm-unknown-0006", + "entity": "phone", + "reason": "Assistant inspected context and isolated mask: real tethered smartphone on foreground circular display table; excludes phone depicted on TV advertisement." + }, + { + "id": "vlm-unknown-0007", + "entity": "phone", + "reason": "Assistant inspected context and isolated mask: real tethered smartphone on foreground circular display table; excludes phone depicted on TV advertisement." + }, + { + "id": "vlm-unknown-0008", + "entity": "phone", + "reason": "Assistant inspected context and isolated mask: real tethered smartphone on foreground circular display table; excludes phone depicted on TV advertisement." + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Kaohsiung-cam04-t330", + "image_sha256": "a479c28dc3688ca122efa93ce4288707fffa4eb2031ca942d860fae2397c0add", + "companion_sha256": "8cbb99661c64303eae9bd97cf37953079b614235f7f39622385d8b70302abc89", + "positives": [ + { + "id": "ai-0005", + "entity": "laptop", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 0 + }, + { + "id": "ai-0011", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 3 + }, + { + "id": "ai-0019", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 4 + }, + { + "id": "ai-0009", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 5 + }, + { + "id": "ai-0006", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 6 + }, + { + "id": "ai-0031", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 12 + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 13 + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 14 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Kaohsiung-cam07-t030", + "image_sha256": "43a4ec3d39087331280da0a4b868ffc47b98cebb4410a304a2f1d14adf9fdc9d", + "companion_sha256": "18d91aac7c1cbb4a2e29a8a6f3ef9df12f4e7ef96d967e94e7175ff509d06e53", + "positives": [ + { + "id": "ai-0080", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 19 + }, + { + "id": "ai-0017", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 21 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Kaohsiung-cam08-t030", + "image_sha256": "facae65b929ba40c46bf9bb671e1b158c3e9edd392734e6cdea2b4fb69717462", + "companion_sha256": "97cf772c23ef3a6058ae6aac87f2cf207b5391d34336713254257157b839e66f", + "positives": [ + { + "id": "ai-0023", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 27 + }, + { + "id": "ai-0025", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 28 + }, + { + "id": "ai-0018", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 29 + }, + { + "id": "ai-0042", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 31 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Taichung-cam03-t030", + "image_sha256": "e4dc5e1e322ad6b43c802169a816c06fd0b1f6cb8c399e87d4215cd5056e1f82", + "companion_sha256": "5ed3466365e1b23c8810968530385e6203616aeb487dae6f0c77e31e10043c6a", + "positives": [ + { + "id": "ai-0005", + "entity": "laptop", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 32 + }, + { + "id": "ai-0006", + "entity": "laptop", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 33 + }, + { + "id": "ai-0014", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 35 + }, + { + "id": "ai-0011", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 36 + }, + { + "id": "ai-0008", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 37 + }, + { + "id": "ai-0020", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 40 + }, + { + "id": "ai-0057", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 41 + }, + { + "id": "ai-0010", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 42 + }, + { + "id": "ai-0003", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 43 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Taichung-cam04-t330", + "image_sha256": "7daaff94e19ec7f2eb63738d779c358cbbe59406841f5499f8269376befbcad1", + "companion_sha256": "503fb3c85237741465e9b3d8c3702535e9651206267e325c69634c81540e62f8", + "positives": [ + { + "id": "ai-0054", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 50 + }, + { + "id": "ai-0057", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 51 + }, + { + "id": "ai-0050", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 53 + }, + { + "id": "ai-0009", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 55 + }, + { + "id": "ai-0028", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 56 + }, + { + "id": "ai-0016", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 58 + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 59 + }, + { + "id": "ai-0012", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 60 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Taichung-cam05-t030", + "image_sha256": "8a7a6e129561f819a598241bbe50107322192a61ca450510ae97c957672686ad", + "companion_sha256": "dd4ebfadebb5c9b9a4777c130b124aa1cac9cb3610d20783acf610056b056e4e", + "positives": [ + { + "id": "ai-0036", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 64 + }, + { + "id": "ai-0005", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 66 + }, + { + "id": "ai-0061", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 68 + }, + { + "id": "ai-0014", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 69 + }, + { + "id": "ai-0011", + "entity": "fire_equipment", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 70 + }, + { + "id": "ai-0009", + "entity": "fire_equipment", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 71 + }, + { + "id": "ai-0055", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 73 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Taichung-cam08-t030", + "image_sha256": "8f15fd481aeb8f63aa50b25dfa79637535b45defcecae4b5f99b9a5e547b5daa", + "companion_sha256": "8a5a47e8f624dde013d5b8bfddfc02b5b6f309196da9a60c94325871d1c28c2d", + "positives": [ + { + "id": "ai-0002", + "entity": "tablet", + "reason": "Assistant corrected phone proposal to tablet: full-size physical iPad with tether and stand on tabletop, compared with the surrounding counter and power sockets; not the adjacent black desk mats.", + "review_index": 76 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Taichung-cam09-t030", + "image_sha256": "c989bfe1f6a7998834c79588b000dd24f869b96bfc93c14055b9a2b8d2995828", + "companion_sha256": "5f7a89f5e2cc3d58f472a954b8fd41992c5615be016b97956911f9bac6aead07", + "positives": [ + { + "id": "ai-0066", + "entity": "boxed_stock", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 82 + }, + { + "id": "ai-0028", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 83 + }, + { + "id": "ai-0026", + "entity": "fire_equipment", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 84 + }, + { + "id": "ai-0002", + "entity": "fire_equipment", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 85 + }, + { + "id": "ai-0005", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 86 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260910-Taichung-cam11-t330", + "image_sha256": "272d31978a93c47d1a1aaa04f8581cd0c47135abdd60aab94d0650ef49186d7d", + "companion_sha256": "b5d5d582256ff8a0ba2a0f87bedde01d97739e6e1eb9ba8ffdb1e885b9251965", + "positives": [ + { + "id": "ai-0017", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 91 + }, + { + "id": "ai-0030", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 92 + }, + { + "id": "ai-0011", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 93 + }, + { + "id": "ai-0028", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 95 + }, + { + "id": "ai-0002", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 96 + }, + { + "id": "ai-0006", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 97 + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 98 + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 99 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260912-Kaohsiung-cam04-t030", + "image_sha256": "8f2455851cd3d7d609931dfe0c7681801df689eb20633bb42af56f06837d0730", + "companion_sha256": "e57e043fc7a2e8f7a9fce6db43bd41c0758fd9a997cfc803ed49aafdf71a8d43", + "positives": [ + { + "id": "ai-0006", + "entity": "laptop", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 100 + }, + { + "id": "ai-0039", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 102 + }, + { + "id": "ai-0010", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 103 + }, + { + "id": "ai-0020", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 105 + }, + { + "id": "ai-0008", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 106 + }, + { + "id": "ai-0074", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 113 + }, + { + "id": "ai-0068", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 114 + }, + { + "id": "ai-0002", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 115 + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 116 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260912-Taichung-cam03-t330", + "image_sha256": "6f696165a74ce3d833e6fc59a868213f3f261d6b904e2964553abd3107513c52", + "companion_sha256": "ffbc822f24b384d131a9e6255a56762405c5c9999e140ebcb54a53edb5ffda85", + "positives": [ + { + "id": "ai-0011", + "entity": "laptop", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 117 + }, + { + "id": "ai-0010", + "entity": "laptop", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 118 + }, + { + "id": "ai-0017", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 120 + }, + { + "id": "ai-0019", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 121 + }, + { + "id": "ai-0015", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 125 + }, + { + "id": "ai-0005", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 126 + }, + { + "id": "ai-0006", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 127 + }, + { + "id": "ai-0008", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 129 + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 130 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260912-Taichung-cam04-t030", + "image_sha256": "e528fceb02e0402a1b98840d1e13cd342b8e823ee2bd9f7367353ab9be0bb4cb", + "companion_sha256": "d21485f131d60f65e634d033b0a5b58c18c26e78dc4bd3edb89083f882076e8e", + "positives": [ + { + "id": "ai-0016", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 139 + }, + { + "id": "ai-0033", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 140 + }, + { + "id": "ai-0020", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 141 + }, + { + "id": "ai-0038", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 142 + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 143 + }, + { + "id": "ai-0003", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 144 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260912-Taichung-cam05-t330", + "image_sha256": "f0757ff6f43fadc72a211aee58b1132ac29afe242e35a07ef55c41a2b5bf4209", + "companion_sha256": "8b39a80107cfbf5bfd72aed896d486f1579b4a4939f7f72d54696dd98ab1c20e", + "positives": [ + { + "id": "ai-0014", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 151 + }, + { + "id": "ai-0081", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 154 + }, + { + "id": "ai-0020", + "entity": "fire_equipment", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 155 + }, + { + "id": "ai-0019", + "entity": "fire_equipment", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 156 + }, + { + "id": "ai-0010", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 157 + }, + { + "id": "ai-0024", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 158 + }, + { + "id": "ai-0000", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 159 + }, + { + "id": "ai-0004", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 160 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260912-Taichung-cam08-t030", + "image_sha256": "04aae73f6f1184bb6b1fdd4f2f924243c68f6a423b794ca7d806e4fc03cad2ad", + "companion_sha256": "d9d462189ede768927dc8b63bf925e6bd141076a6adbc7ba9c2318505e54d331", + "positives": [ + { + "id": "ai-0002", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 161 + }, + { + "id": "ai-0003", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 162 + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 164 + }, + { + "id": "ai-0005", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 165 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260912-Taichung-cam08-t330", + "image_sha256": "c834c7362415b2fa11d32470166b44b0a80519929e258ab8b228bdb5a1a79578", + "companion_sha256": "2c5ec8ec5dfa4e68f5d6fa34e4cfd1ff0f3203233738a149bd975c8e6a0787cf", + "positives": [ + { + "id": "ai-0007", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 167 + }, + { + "id": "ai-0001", + "entity": "phone", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 168 + }, + { + "id": "ai-0002", + "entity": "tablet", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 169 + }, + { + "id": "ai-0005", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 173 + }, + { + "id": "ai-0004", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 174 + } + ], + "negative_rects": [] + }, + { + "frame_id": "gcs-20260912-Taichung-cam11-t330", + "image_sha256": "f3360b01d4e4d82e1fc8c552714e287efadaeedd015cc3ab0440ee01603ab19a", + "companion_sha256": "adba1791cd74f51dab852defec9e743ed3b20045454f100b59a93cb1617b370c", + "positives": [ + { + "id": "ai-0003", + "entity": "speaker", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 176 + }, + { + "id": "ai-0016", + "entity": "poster", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 178 + }, + { + "id": "ai-0000", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 179 + }, + { + "id": "ai-0011", + "entity": "chair", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 180 + }, + { + "id": "ai-0001", + "entity": "person", + "reason": "Assistant inspected full camera context and isolated mask: one visible physical instance supports this class; partial AI review, not exhaustive labels.", + "review_index": 181 + } + ], + "negative_rects": [] + } + ], + "deferred_candidates": [ + { + "frame_id": "0008-Kaohsiung-cam07", + "id": "ai-0035", + "reason": "Multiple packaged objects in one mask" + }, + { + "frame_id": "0008-Kaohsiung-cam07", + "id": "ai-0036", + "reason": "Disconnected mask contaminates bounding box" + }, + { + "frame_id": "0002-Kaohsiung-cam04", + "id": "ai-0007", + "reason": "Ambiguous payment terminal or calculator" + }, + { + "frame_id": "0033-Taichung-cam09", + "id": "ai-0007", + "reason": "Multiple overlapping packages" + }, + { + "frame_id": "0033-Taichung-cam09", + "id": "ai-0008", + "reason": "Multiple overlapping packages" + }, + { + "frame_id": "0082-Taichung-cam11", + "id": "ai-0001", + "reason": "Mask includes floor or disconnected background" + }, + { + "frame_id": "0007-Kaohsiung-cam06", + "id": "ai-0008", + "reason": "Printed price placard, not physical phone" + }, + { + "frame_id": "0007-Kaohsiung-cam06", + "id": "ai-0018", + "reason": "Headphones, not speaker" + }, + { + "frame_id": "0053-Kaohsiung-cam05", + "id": "ai-0003", + "reason": "Phone versus payment terminal unresolved" + }, + { + "frame_id": "0016-Taichung-cam01", + "id": "ai-0014", + "reason": "Disconnected mask contaminates bounding box" + }, + { + "frame_id": "0016-Taichung-cam01", + "id": "ai-0015", + "reason": "Printed placard, not boxed stock" + }, + { + "frame_id": "0016-Taichung-cam01", + "id": "ai-0016", + "reason": "Printed placard, not boxed stock" + }, + { + "frame_id": "0034-Taichung-cam10", + "id": "ai-0020", + "reason": "Laptop versus detachable tablet unresolved" + }, + { + "frame_id": "0034-Taichung-cam10", + "id": "ai-0014", + "reason": "Physical phone versus printed price card unresolved" + }, + { + "frame_id": "0013-Kaohsiung-cam11", + "id": "ai-0004", + "reason": "Phone depicted inside TV advertisement, not a physical scene instance" + } + ], + "parent_review_sha256": "8f7d9a491442c8482d5e392b7558c6c579d37609457c16d940013e6fa18cb59f", + "added_review_sha256": "9f4c858c78cc83a28f40769d859f8f373572f8b8ee1f5058e0297c3df2615b55" +} diff --git a/docs/reviews/studioa_negative_balance_plan_20260916.json b/docs/reviews/studioa_negative_balance_plan_20260916.json new file mode 100644 index 0000000..880e4fb --- /dev/null +++ b/docs/reviews/studioa_negative_balance_plan_20260916.json @@ -0,0 +1,202 @@ +{ + "status": "predeclared before either comparison arm runs", + "question": "Does capping the added per-class negative mask mass preserve source-val positive recall while scene outputs remain fixed?", + "data": "runs/studioa_partial_instances_20260916_v4", + "data_manifest_sha256": "e12339ea28db97a8a59c839ad095bf58d961f92af46b0758d6343ebc83475fc1", + "initial_checkpoint": "runs/studioa_scene_pilot_20260916_v2/model/best.pt", + "arms": { + "control": "sum", + "candidate": "positive_budget" + }, + "only_training_change": "model.heads.detection.loss.class_negative_normalization", + "normalization": "For each batch/class, newly supervised negative points exclude existing positive/empty supervision; multiply by min(1, max(positive_count,1)/max(new_negative_count,1)). Counts independent of logits. This bounds effective mask mass, not gradient norm.", + "fixed": { + "seed": 42, + "input_size": [ + 512, + 896 + ], + "batch_size": 2, + "lr": 0.0002, + "warmup_iters": 26, + "epochs": 60, + "early_stop_patience": 20, + "detector_only": true, + "scene_eval_frames": 33, + "detector_val_frames": 6, + "detector_val_positives": 31, + "score_strictly_greater_than": 0.2, + "iou_at_least": 0.5, + "nms": 0.6, + "max_det": 100, + "test_inference": false, + "no_threshold_sweep": true + }, + "upgrade_criteria": { + "reference_run": "runs/studioa_detector_warmup_20260916_v1", + "total_matched_strictly_greater_than": 7, + "matched_person_at_least": 5, + "matched_phone_at_least": 1, + "matched_poster_at_least": 1, + "reviewed_empty_fp_at_most": 0, + "all_33_scene_logits_byte_identical": true, + "deployment_claim": false + }, + "diagnostic_scope": "28 source-train images, deterministic no augmentation, single image normalization, float32; logit gradient L1 is not model parameter gradient or deployment accuracy", + "diagnostic_summary": { + "v1": { + "laptop": { + "positive": 0.012918037129566073, + "empty": 0.0003741828104466549, + "class_negative": 3.4755482971668243, + "positive_points": 442, + "class_negative_points": 1092 + }, + "phone": { + "positive": 0.027144994597620098, + "empty": 0.0011384378412913065, + "class_negative": 9.630970358848572, + "positive_points": 249, + "class_negative_points": 1174 + }, + "tablet": { + "positive": 0.009524137155267454, + "empty": 0.0015294674449251033, + "class_negative": 7.666859656572342, + "positive_points": 578, + "class_negative_points": 1119 + }, + "boxed_stock": { + "positive": 0.007861402924618233, + "empty": 0.0006191605880303541, + "class_negative": 0.0, + "positive_points": 290, + "class_negative_points": 0 + }, + "cardboard_box": { + "positive": 0.10530798323452473, + "empty": 0.00014953502613934688, + "class_negative": 0.1007998138666153, + "positive_points": 160, + "class_negative_points": 106 + }, + "speaker": { + "positive": 0.007214011835458223, + "empty": 0.00023646760564588476, + "class_negative": 0.13119319081306458, + "positive_points": 143, + "class_negative_points": 27 + }, + "poster": { + "positive": 0.07255605278413668, + "empty": 0.0036417511128092883, + "class_negative": 0.0, + "positive_points": 901, + "class_negative_points": 0 + }, + "fire_equipment": { + "positive": 0.010544058675591828, + "empty": 0.0023294030424949597, + "class_negative": 0.0, + "positive_points": 115, + "class_negative_points": 0 + }, + "chair": { + "positive": 0.024498611608578358, + "empty": 0.0010922043438768014, + "class_negative": 0.0, + "positive_points": 350, + "class_negative_points": 0 + }, + "person": { + "positive": 0.0603497300617164, + "empty": 0.008183579477190506, + "class_negative": 0.22365543246269226, + "positive_points": 720, + "class_negative_points": 25 + } + }, + "v2": { + "laptop": { + "positive": 0.21216800808906555, + "empty": 0.00010892083173530409, + "class_negative": 0.03900669398717582, + "positive_points": 442, + "class_negative_points": 1092 + }, + "phone": { + "positive": 0.5927745196968317, + "empty": 0.00010077359479510051, + "class_negative": 0.06103876978158951, + "positive_points": 249, + "class_negative_points": 1174 + }, + "tablet": { + "positive": 0.4205053150653839, + "empty": 0.0001715062435323489, + "class_negative": 0.10881199990399182, + "positive_points": 578, + "class_negative_points": 1119 + }, + "boxed_stock": { + "positive": 0.2133826856734231, + "empty": 0.0001516974407422822, + "class_negative": 0.0, + "positive_points": 290, + "class_negative_points": 0 + }, + "cardboard_box": { + "positive": 0.17267386615276337, + "empty": 0.00024281461901409784, + "class_negative": 0.0030626498628407717, + "positive_points": 160, + "class_negative_points": 106 + }, + "speaker": { + "positive": 0.25613438803702593, + "empty": 6.87338163061213e-05, + "class_negative": 0.011610515415668488, + "positive_points": 143, + "class_negative_points": 27 + }, + "poster": { + "positive": 0.4980439506471157, + "empty": 0.00047163556337181944, + "class_negative": 0.0, + "positive_points": 901, + "class_negative_points": 0 + }, + "fire_equipment": { + "positive": 0.11960204120259732, + "empty": 0.0004438365522219101, + "class_negative": 0.0, + "positive_points": 115, + "class_negative_points": 0 + }, + "chair": { + "positive": 0.2698142668232322, + "empty": 0.00014771780251976452, + "class_negative": 0.0, + "positive_points": 350, + "class_negative_points": 0 + }, + "person": { + "positive": 0.32032438833266497, + "empty": 0.0020131336023041513, + "class_negative": 0.07633097469806671, + "positive_points": 720, + "class_negative_points": 25 + } + } + }, + "diagnostic_limits": [ + "Per-image float32 no-augmentation logit-gradient L1 is not the historical batch-2 augmented training gradient.", + "A well-fit positive can have a small focal gradient, so a large negative/positive ratio alone does not prove the cause of the validation regression.", + "All 130 train boxes have at least one assignable grid point in the unaugmented inputs; regression quality may still be insufficient." + ], + "diagnostic_artifacts": { + "runs/studioa_negative_balance_20260916_v1/diagnose.py": "8b7823c9ec02abe2cdda9cf852b2cedcf1ba14325c8cdc21f2dc44fda05bd371", + "runs/studioa_negative_balance_20260916_v1/loss_diagnosis.json": "3c5fe8da9fb69ae9f03a4d81828c5ff46417a66e9dd8500a560f5202ed30a53b", + "runs/studioa_negative_balance_20260916_v1/diagnose.log": "43a23f0fe14ec609efb9525229b6d76e1a842ec20155e9b23da5e188b160a551" + } +} diff --git a/scripts/campaign_site30k.py b/scripts/campaign_site30k.py index 7acaf22..3ba4398 100644 --- a/scripts/campaign_site30k.py +++ b/scripts/campaign_site30k.py @@ -59,6 +59,7 @@ import argparse import json +import os import sys import time from collections import Counter, defaultdict @@ -70,6 +71,7 @@ from PIL import Image HERE = Path(__file__).resolve().parent +ROOT = Path(os.environ.get("SYNCAI_ROOT", HERE.parent)) sys.path.insert(0, str(HERE)) # The teachers live in the wheel as of 2026-08-20 (52eb141 SAM 3, 556582b Grounding @@ -158,13 +160,13 @@ CONSENSUS = 0.9 # the measured setting from sam3_prelabel NMS_IOU = 0.55 # same as teachers.boxes.nms -THIRD_OPINION_RUN = HERE.parent / "runs/hydranet_retail_security_b03_gdino" +THIRD_OPINION_RUN = ROOT / "runs/hydranet_retail_security_b03_gdino" # b03 terrain head: 0 void, 1 floor, 2 wall, 3 column, 4 fixture, 5 person. # Veto map: site id -> the b03 id that counts as agreement. `display_table` and # `shelf` are absent on purpose -- the model cannot arbitrate a split it never saw. VETO_SITE_TO_B03 = {1: 1, 2: 2, 3: 3, 6: 5} -PROGRESS_LOG = HERE.parent / "runs/site30k_qa/progress.log" +PROGRESS_LOG = ROOT / "runs/site30k_qa/progress.log" def log_progress(msg: str) -> None: @@ -351,9 +353,9 @@ def __init__( unproject, ) - root = Path(calib_root) if calib_root is not None else HERE.parent / "runs/onboard01" + root = Path(calib_root) if calib_root is not None else ROOT / "runs/onboard01" calib = json.loads((root / f"{camera}.calib.json").read_text()) - zones = json.loads((HERE.parent / f"runs/zones01/{camera}.zones.json").read_text()) + zones = json.loads((ROOT / f"runs/zones01/{camera}.zones.json").read_text()) if zones.get("units") != "m" or calib.get("scale") is None: raise SystemExit(f"{camera}: zones/calib not metric; v2 floor recipe needs both") poly = next(p for p in zones["proposals"] if p["name_suggestion"] == "walkable_floor") @@ -650,13 +652,11 @@ def cmd_annotate(args) -> int: geom_cache[camera] = GeomTeacher(camera, third) geom = geom_cache[camera] slot = Path(clip).stem.split("_")[1] - cand = HERE.parent / f"datasets/studioa_static/{camera}/plate_{slot}.png" + cand = ROOT / f"datasets/studioa_static/{camera}/plate_{slot}.png" if cand.exists(): plate_path = str(cand) # the SAME clip's own median -- crispest diff else: - calib_p = json.loads( - (HERE.parent / f"runs/onboard01/{camera}.calib.json").read_text() - ) + calib_p = json.loads((ROOT / f"runs/onboard01/{camera}.calib.json").read_text()) plate_path = calib_p["plate_used"] plate_floor, dirty, plate_rgb = geom.plate_labeling(plate_path) # Per-pixel noise floor of |frame - plate|, static_plates style: a LOW @@ -984,7 +984,7 @@ def cmd_floor_diag(args) -> int: report = {} for camera in args.cameras: geom = GeomTeacher(camera, third) - calib = json.loads((HERE.parent / f"runs/onboard01/{camera}.calib.json").read_text()) + calib = json.loads((ROOT / f"runs/onboard01/{camera}.calib.json").read_text()) plate = ( Image.open(calib["plate_used"]) .convert("RGB") @@ -1408,7 +1408,7 @@ def build_parser() -> argparse.ArgumentParser: an.add_argument("--sample-fps", type=float, default=1.0) an.add_argument( "--split-json", - default=str(HERE.parent / "datasets/retail_objects_batch03/split.json"), + default=str(ROOT / "datasets/retail_objects_batch03/split.json"), help="camera split assignments to INHERIT (R1/R2)", ) an.add_argument( diff --git a/scripts/onboard_camera.py b/scripts/onboard_camera.py index d7e712b..16d105f 100644 --- a/scripts/onboard_camera.py +++ b/scripts/onboard_camera.py @@ -75,6 +75,51 @@ PLATE_MODEL_CKPT = for_terrain() K1_FLEET = pc.K1_FLEET # the fleet lens; defined once in syncai_bev3d.plate_calibration VFOV_PRIMARY = 70.4 # likewise: pinned on cam01, a fleet assumption for the rest +#: The one camera whose vfov and k1 were measured off its tile grid (calib01), as data. +#: Until 2026-09-11 this was `camera == "Taichung-cam01"` inside the sweep, so a vfov +#: measured on any other camera -- `floor_calibrate.py` writes them to a pins file -- +#: had nowhere to go: the primary row was the 70.4 constant whatever the instrument said. +BUILTIN_PINS: dict[str, dict] = { + "Taichung-cam01": { + "vfov_deg": 70.4, + "source": "tile_grid_pinned", + "k1_source": "tile_grid_measured", + } +} + + +def load_pins(path: Path | None) -> dict[str, dict]: + """`floor_calibrate.py --pins-out` rows (`{camera: {vfov_deg, source, ...}}`) over + the built-in tile pin; a file pin for the same camera wins, being the later measurement.""" + pins = {k: dict(v) for k, v in BUILTIN_PINS.items()} + if path is not None: + for cam, row in json.loads(Path(path).read_text()).items(): + if "vfov_deg" not in row: + raise ValueError(f"{path}: pin for {cam} has no vfov_deg") + pins[cam] = {"source": "pinned", **row} + return pins + + +def pin_for(camera: str, pins: dict[str, dict] | None) -> dict | None: + return (pins if pins is not None else BUILTIN_PINS).get(camera) + + +def source_size_from_index(plates_root: Path, camera: str, slot: str) -> list[int] | None: + """The clip's native (w, h) that `static_plates.py` recorded for this slot, if any.""" + index = Path(plates_root) / "index.json" + if not index.exists(): + return None + slots = json.loads(index.read_text()).get("cameras", {}).get(camera, {}).get("slots", {}) + wh = (slots.get(slot) or {}).get("source_wh") + return [int(wh[0]), int(wh[1])] if wh else None + + +def primary_row(by_vfov: list[dict], vfov: float) -> dict | None: + """The sweep row at the camera's own vfov; None when that row has no fitted floor.""" + row = next((r for r in by_vfov if abs(float(r["vfov_deg"]) - vfov) < 1e-9), None) + return row if row is not None and "pitch_deg" in row else None + + MIN_HEIGHTS = 10 # min samples for the person-height statistic to emit a number (task spec) DIRTY_PLATE_FRAC = 0.05 # person share above this marks a dirty plate (cam04's 8.6% is above) # calib01: the gap between the person-height scale and the tile-grid anchor (pose bias, @@ -167,21 +212,31 @@ def onboard_one( meter: PlatePersonMeter | None, plates_root: Path, person_anns: Path = PERSON_ANNS, + utc_offset: int = pc.DEFAULT_UTC_OFFSET_HOURS, + pins: dict[str, dict] | None = None, + daytime: tuple[int, int] = pc.DEFAULT_DAYTIME_HOURS_LOCAL, ) -> dict: now = _dt.date.today().isoformat() - is_pinned = camera == "Taichung-cam01" # tile grid: k1 and vfov both measured + pin = pin_for(camera, pins) + is_pinned = pin is not None + vfov_primary = float(pin["vfov_deg"]) if pin else VFOV_PRIMARY + k1_source = (pin or {}).get("k1_source", "fleet_hardware_assumed") flags: list[str] = [] if not is_pinned: - flags += ["vfov_fleet_assumed", "k1_fleet_assumed"] + flags.append("vfov_fleet_assumed") + if k1_source == "fleet_hardware_assumed": + flags.append("k1_fleet_assumed") + # the sweep always contains the camera's own vfov, so the primary row exists + vfovs = sorted(set(vfovs) | {vfov_primary}) result: dict = { "schema": SCHEMA, "camera": camera, "generated": now, - "vfov_assumed_deg": VFOV_PRIMARY, - "vfov_source": "tile_grid_pinned" if is_pinned else "fleet_hardware_assumed", + "vfov_assumed_deg": vfov_primary, + "vfov_source": pin["source"] if pin else "fleet_hardware_assumed", "k1_division_model": k1, - "k1_source": "tile_grid_measured" if is_pinned else "fleet_hardware_assumed", + "k1_source": k1_source, # calib02's visual priors (door height / floor tiles) cannot be automated: the # fields are kept, the values left empty, and the SOP's manual step back-fills # them. This is "not measured", not "measured out as null". @@ -217,12 +272,17 @@ def onboard_one( ) return result - slot = pc.pick_daytime_slot(cam_dir) + slot = pc.pick_daytime_slot(cam_dir, utc_offset, daytime) plate_path = cam_dir / f"plate_{slot}.png" rgb = np.asarray(Image.open(plate_path).convert("RGB")) h, w = rgb.shape[:2] result.update( - {"plate_used": str(plate_path), "plate_slot_utc": slot, "frame_hw_px": [h, w]} + { + "plate_used": str(plate_path), + "plate_slot_utc": slot, + "frame_hw_px": [h, w], + "source_size_px": source_size_from_index(plates_root, camera, slot), + } ) # Plate dirty region (the raw plate, not the undistorted one -- stable_infer @@ -273,8 +333,8 @@ def onboard_one( result["by_vfov"] = by_vfov ok_rows = [r for r in by_vfov if "pitch_deg" in r] - primary = next((r for r in by_vfov if r["vfov_deg"] == VFOV_PRIMARY), None) - if primary is None or "pitch_deg" not in primary: + primary = primary_row(by_vfov, vfov_primary) + if primary is None: primary = ok_rows[0] if ok_rows else None if primary is not None: flags.append("primary_vfov_failed_using_fallback") @@ -542,6 +602,13 @@ def main(argv=None) -> int: ap.add_argument("--plates-root", type=Path, default=pc.PLATES) ap.add_argument("--k1", type=float, default=K1_FLEET) ap.add_argument("--vfovs", default="55,70.4,85") + ap.add_argument( + "--vfov-pins", + type=Path, + default=None, + help="per-camera measured vfov, `floor_calibrate.py --pins-out`'s file; the " + "camera's row becomes its primary and joins the sweep", + ) ap.add_argument("--skip-person-frac", action="store_true") ap.add_argument("--cameras-json", type=Path, default=CAMERAS_JSON) ap.add_argument( @@ -555,6 +622,9 @@ def main(argv=None) -> int: fleet = selling_floor_cameras(args.cameras_json) cameras = args.camera or fleet + site = json.loads(Path(args.cameras_json).read_text()) + utc_offset = pc.utc_offset_hours(site) + daytime = pc.daytime_hours_local(site) args.out.mkdir(parents=True, exist_ok=True) if args.report_only: @@ -566,11 +636,22 @@ def main(argv=None) -> int: if not args.skip_person_frac: meter = PlatePersonMeter(PLATE_MODEL_CONFIG, PLATE_MODEL_CKPT) vfovs = [float(x) for x in args.vfovs.split(",")] + pins = load_pins(args.vfov_pins) for i, cam in enumerate(cameras, 1): print(f"[{i}/{len(cameras)}] {cam}") try: - result = onboard_one(cam, vfovs, args.k1, meter, args.plates_root, args.person_anns) + result = onboard_one( + cam, + vfovs, + args.k1, + meter, + args.plates_root, + args.person_anns, + utc_offset, + pins, + daytime=daytime, + ) except Exception: traceback.print_exc() result = { diff --git a/scripts/propose_zones.py b/scripts/propose_zones.py index 758a274..d33a6bd 100644 --- a/scripts/propose_zones.py +++ b/scripts/propose_zones.py @@ -46,6 +46,7 @@ import hashlib import json import math +import os import sys from pathlib import Path @@ -54,6 +55,7 @@ from scipy import ndimage HERE = Path(__file__).resolve().parent +ROOT = Path(os.environ.get("SYNCAI_ROOT", HERE.parent)) sys.path.insert(0, str(HERE)) from syncai_bev3d.floorplan import ( # noqa: E402 diff --git a/scripts/static_plates.py b/scripts/static_plates.py index 4e9f3ae..ae409a9 100644 --- a/scripts/static_plates.py +++ b/scripts/static_plates.py @@ -73,6 +73,8 @@ import numpy as np from PIL import Image +from syncai_bev3d.plate_calibration import daytime_hours_local, utc_offset_hours + # How many of a pixel's OWN noise floors it must move to count as dynamic. # # **Per pixel, not per frame, and that is the whole point.** A single floor for the frame @@ -198,6 +200,32 @@ def plate_and_mask(frames: np.ndarray) -> tuple[np.ndarray, np.ndarray, dict]: ) +def source_size(path: Path) -> tuple[int, int] | None: + """The clip's native (width, height), before `WORK_W x WORK_H`: what a consumer of the + calibration will receive boxes in. None when ffprobe cannot say.""" + proc = subprocess.run( + [ + "ffprobe", + "-v", + "error", + "-select_streams", + "v:0", + "-show_entries", + "stream=width,height", + "-of", + "csv=p=0", + str(path), + ], + capture_output=True, + text=True, + ) + try: + w, h = proc.stdout.strip().split(",")[:2] + return int(w), int(h) + except ValueError: + return None + + def slot_of(path: Path) -> str: """`archive_20260816-113024_...` -> `20260816-113024`, which is UTC. See the header.""" return path.stem.split("_")[1] @@ -216,7 +244,10 @@ def main(argv: list[str] | None = None) -> int: ) args = ap.parse_args(argv) - roles = json.loads((args.cameras or args.root / "cameras.json").read_text())["cameras"] + site = json.loads((args.cameras or args.root / "cameras.json").read_text()) + roles = site["cameras"] + utc_offset = utc_offset_hours(site) + daytime = daytime_hours_local(site) names = args.only or sorted( k for k, v in roles.items() if args.include_dead or v.get("role") != "dead" ) @@ -244,7 +275,11 @@ def main(argv: list[str] | None = None) -> int: Image.fromarray((static * 255).astype(np.uint8)).save( cam_out / f"static_{slot}.png" ) - slots[slot] = {**stats, "role": roles.get(cam, {}).get("role")} + slots[slot] = { + **stats, + "role": roles.get(cam, {}).get("role"), + "source_wh": source_size(clip), + } every = static if every is None else (every & static) if every is not None: Image.fromarray((every * 255).astype(np.uint8)).save(cam_out / "static_all.png") @@ -264,7 +299,9 @@ def main(argv: list[str] | None = None) -> int: json.dumps( { "measured": "static plates per camera per time slot", - "note": "slot keys are UTC; the burned-in timestamp is store-local (+8)", + "note": "slot keys are UTC; the burned-in timestamp is store-local", + "utc_offset_hours": utc_offset, + "daytime_hours_local": list(daytime), "sample_fps": SAMPLE_FPS, "work_size": [WORK_W, WORK_H], "dynamic_mult": DYNAMIC_MULT, diff --git a/src/syncai_bev3d/commissioning.py b/src/syncai_bev3d/commissioning.py index 6d36172..bf0652d 100644 --- a/src/syncai_bev3d/commissioning.py +++ b/src/syncai_bev3d/commissioning.py @@ -81,6 +81,11 @@ def from_onboard_calib(path: str | Path) -> CameraFile: radius_px=math.hypot(h, w) / 2.0, ), plate_file=raw.get("plate_used"), + source_size_px=( + (int(raw["source_size_px"][0]), int(raw["source_size_px"][1])) + if raw.get("source_size_px") + else None + ), commissioned_at=raw.get("generated"), teachers=_teachers_of(raw), ) diff --git a/src/syncai_bev3d/plate_calibration.py b/src/syncai_bev3d/plate_calibration.py index 46777ef..cec3703 100644 --- a/src/syncai_bev3d/plate_calibration.py +++ b/src/syncai_bev3d/plate_calibration.py @@ -79,13 +79,48 @@ # plate handling -def pick_daytime_slot(cam_dir: Path) -> str: - """Brightest plate among the daytime slots (slot keys are UTC; store-local is +8).""" +#: The offset every site carried until 2026-09-11, when a US-Central factory's plates +#: passed the daytime gate by coincidence (its 10 UTC slot read as 18 local under +8). +#: A site states its own offset in `cameras.json`; this is only what an old file means. +DEFAULT_UTC_OFFSET_HOURS = 8 + + +def utc_offset_hours(cameras_json: dict) -> int: + """The site's `utc_offset_hours` from a `cameras.json` payload, +8 when it says nothing.""" + return int(cameras_json.get("utc_offset_hours", DEFAULT_UTC_OFFSET_HOURS)) + + +#: A shop's lit hours. A factory on night shifts is lit at 05:00 local (FTI, 2026-09-16: +#: its only capture fell there and the gate refused it), so a site states its own window. +DEFAULT_DAYTIME_HOURS_LOCAL = (8, 18) + + +def daytime_hours_local(cameras_json: dict) -> tuple[int, int]: + """The site's `daytime_hours_local` `[start, end]` from `cameras.json`, 08-18 when absent. + + Inclusive local hours; `start > end` is a window across midnight (`[20, 6]`).""" + start, end = cameras_json.get("daytime_hours_local", DEFAULT_DAYTIME_HOURS_LOCAL) + return int(start), int(end) + + +def in_daytime(hour_local: int, daytime: tuple[int, int] = DEFAULT_DAYTIME_HOURS_LOCAL) -> bool: + start, end = daytime + if start <= end: + return start <= hour_local <= end + return hour_local >= start or hour_local <= end + + +def pick_daytime_slot( + cam_dir: Path, + utc_offset: int = DEFAULT_UTC_OFFSET_HOURS, + daytime: tuple[int, int] = DEFAULT_DAYTIME_HOURS_LOCAL, +) -> str: + """Brightest plate among the daytime slots (slot keys are UTC; local = UTC + offset).""" best, best_luma = None, -1.0 for p in sorted(cam_dir.glob("plate_*.png")): slot = p.stem.split("_", 1)[1] - hour_local = (int(slot[9:11]) + 8) % 24 - if not (8 <= hour_local <= 18): + hour_local = (int(slot[9:11]) + utc_offset) % 24 + if not in_daytime(hour_local, daytime): continue luma = float(np.asarray(Image.open(p).convert("L"), dtype=float).mean()) if luma > best_luma: diff --git a/src/syncai_bev3d/rulers.py b/src/syncai_bev3d/rulers.py index 1506375..326c265 100644 --- a/src/syncai_bev3d/rulers.py +++ b/src/syncai_bev3d/rulers.py @@ -64,10 +64,27 @@ class Ruler: factor: float # multiply the camera's current metres by this sigma: float # fractional uncertainty of `factor` note: str = "" - - -def person_ruler(calib: dict) -> Ruler: - """The calibration's own person-prior scale: factor 1, uncertainty from its record.""" + #: Whether the reference length came from outside the camera's own metres. Every + #: ruler *reads* through those metres; what separates a witness from an echo is where + #: its reference came from. The 1.70 m prior and a known object are anchored. A + #: catalogue tile chosen because the store's cameras agree on it is not: two cameras + #: through one over-scaled depth model agree with each other, not with the floor -- + #: FTI's 600 mm raised floor read 0.85 / 0.74 m and the catalogue picked 0.80 + #: (2026-09-10). A store-median counter is relative by construction. + anchored: bool = True + + +def person_ruler(calib: dict) -> Ruler | None: + """The calibration's own person-prior scale: factor 1, uncertainty from its record. + + None when the calibration says its scale was never measured -- `unmeasured`, or a + bootstrap on the depth model's raw metres: factor 1 there would anchor the other + rulers to the very reading they are meant to check. A record with no `scale_source` + at all is read as measured; absence is not a declaration. + """ + source = str(calib.get("scale_source") or "") + if source.startswith("unmeasured") or "bootstrap" in source: + return None u = calib.get("uncertainty") or {} stat = float(u.get("scale_frac_stat_person_mad") or 0.09) sys_ = float(u.get("scale_frac_sys_person_prior") or 0.11) @@ -93,11 +110,19 @@ def standard_tile(periods_m: dict[str, float]) -> tuple[float | None, float]: return best, best_err -def tile_ruler(period_m: float | None, tile_m: float | None) -> Ruler | None: +def tile_ruler( + period_m: float | None, tile_m: float | None, *, anchored: bool = True +) -> Ruler | None: + """`anchored=False` when `tile_m` is `standard_tile`'s consensus pick rather than a + size known independently of the cameras' readings.""" if not period_m or not tile_m: return None return Ruler( - "tile", tile_m / period_m, TILE_SIGMA, f"pitch {period_m:.2f} m vs {tile_m:.2f} m" + "tile", + tile_m / period_m, + TILE_SIGMA, + f"pitch {period_m:.2f} m vs {tile_m:.2f} m", + anchored=anchored, ) @@ -108,7 +133,11 @@ def table_ruler(table_h_m: float | None, store_table_h_m: list[float]) -> Ruler return None ref = float(np.median(store_table_h_m)) return Ruler( - "table", ref / table_h_m, TABLE_SIGMA, f"{table_h_m:.2f} m vs store {ref:.2f} m" + "table", + ref / table_h_m, + TABLE_SIGMA, + f"{table_h_m:.2f} m vs store {ref:.2f} m", + anchored=False, ) @@ -124,6 +153,17 @@ def combine(rulers: list[Ruler]) -> Verdict: rs = tuple(r for r in rulers if r is not None) if not rs: return Verdict(1.0, False, "no ruler", ()) + if not any(r.anchored for r in rs): + # A verdict here would be an echo: the rulers can agree with each other and all be + # wrong by the same factor. Say so rather than "within 10%". + names = ", ".join(r.name for r in rs) + return Verdict( + 1.0, + False, + f"unanchored: {names} take their reference from the same metres they read; " + "needs a person prior or a size known independently", + rs, + ) logs = np.array([math.log(r.factor) for r in rs]) w = np.array([1.0 / (r.sigma**2) for r in rs]) order = np.argsort(logs) diff --git a/src/syncai_hydranet/analytics/world.py b/src/syncai_hydranet/analytics/world.py index 31479c8..1027bc6 100644 --- a/src/syncai_hydranet/analytics/world.py +++ b/src/syncai_hydranet/analytics/world.py @@ -353,6 +353,24 @@ def as_rows(frame: WorldFrame) -> list[dict]: return rows +def panel_box( + box, source_size_px: tuple[int, int], panel_size_px: tuple[int, int] +) -> tuple[float, float, float, float]: + """A source-frame box drawn on a panel of another size: scaled by each axis' ratio. + + `demo_video.py` drew every track box with `/ 2.0` -- a 1920-wide source over a 960-wide + panel, written as a constant. On a 1280-wide clip the boxes sat at two thirds of the + people; on a 3840-wide one they drew at twice their size (FTI, 2026-09-11). The blur + boxes beside them were scaled by the probed source size and were right, which is what + made the two disagree on the same panel. The ratio is a property of the two frames, + computed here and nowhere else. + """ + x0, y0, x1, y1 = (float(v) for v in box) + sx = panel_size_px[0] / source_size_px[0] + sy = panel_size_px[1] / source_size_px[1] + return (x0 * sx, y0 * sy, x1 * sx, y1 * sy) + + def _finite(value: float | None) -> float | None: if value is None: return None diff --git a/src/syncai_hydranet/config_schema.py b/src/syncai_hydranet/config_schema.py index 2d24d92..db6eab1 100644 --- a/src/syncai_hydranet/config_schema.py +++ b/src/syncai_hydranet/config_schema.py @@ -66,6 +66,7 @@ class ConfigError(ValueError): } MODEL = { + "detection_only_training": Spec((bool,)), "backbone": Spec((dict,), required=True), "neck": Spec((dict,), required=True), "heads": Spec((dict,), required=True), @@ -161,6 +162,7 @@ class ConfigError(ValueError): "cls_weight": Spec(NUMBER), "reg_weight": Spec(NUMBER), "centerness_weight": Spec(NUMBER), + "class_negative_normalization": Spec((str,), choices=("sum", "positive_budget")), }, "pose_p3": {}, "depth_fpn": { @@ -199,6 +201,7 @@ class ConfigError(ValueError): } DATASET = { + "validation_only": Spec((bool,)), "name": Spec((str,), required=True), "type": Spec( (str,), @@ -685,6 +688,8 @@ def _check_datasets(rep: _Report, dcfg: dict, head_names: set[str]) -> None: _check_section(rep, ds, DATASET, path) if not isinstance(ds, dict): continue + if ds.get("validation_only") and not ds.get("split_val"): + rep.errors.append(f"{path}: validation_only requires split_val") name = ds.get("name") if name in seen: rep.errors.append(f"{path}.name: {name!r} is used by an earlier dataset") diff --git a/src/syncai_hydranet/data/datasets.py b/src/syncai_hydranet/data/datasets.py index 44d9928..b9ed1ed 100644 --- a/src/syncai_hydranet/data/datasets.py +++ b/src/syncai_hydranet/data/datasets.py @@ -552,6 +552,8 @@ def split_leaks(datasets: list[dict]) -> list[tuple[str, str, str, list[str]]]: out: list[tuple[str, str, str, list[str]]] = [] segs = [d for d in datasets if d.get("root")] for a in segs: + if a.get("validation_only", False): + continue trained = _configured_cameras(a, a.get("split_train", "train")) if not trained: continue diff --git a/src/syncai_hydranet/data/studioa_instances.py b/src/syncai_hydranet/data/studioa_instances.py index 73a5abc..d364cb4 100644 --- a/src/syncai_hydranet/data/studioa_instances.py +++ b/src/syncai_hydranet/data/studioa_instances.py @@ -8,6 +8,7 @@ from pathlib import Path import numpy as np +import torch from PIL import Image from torch.utils.data import Dataset @@ -32,6 +33,38 @@ ) +def class_negative_regions(item: dict, size, boxes, labels, split: str) -> dict: + """Only explicit, image-bound AI decisions can negate an individual class.""" + regions = item.get("class_negative_rects", []) + if regions and split != "train": + raise ValueError("class negative additions are train-only") + w, h = size + masks = {} + for region in regions: + entity, rect = region.get("entity"), region.get("xyxy", []) + if ( + entity not in CLASSES + or not region.get("reason") + or len(rect) != 4 + or any(type(v) is not int for v in rect) + ): + raise ValueError("class negative region needs class, integer bounds and reason") + x0, y0, x1, y1 = rect + if not (0 <= x0 < x1 <= w and 0 <= y0 < y1 <= h): + raise ValueError("class negative region outside image") + label = CLASSES.index(entity) + for box, positive_label in zip(boxes, labels, strict=True): + if ( + label == positive_label + and min(x1, box[2]) > max(x0, box[0]) + and min(y1, box[3]) > max(y0, box[1]) + ): + raise ValueError("class negative contradicts a reviewed positive box") + mask = masks.setdefault(entity, np.zeros((h, w), dtype=np.uint8)) + mask[y0:y1, x0:x1] = 1 + return masks + + def export_instances(source: Path, reviews: Path, out: Path) -> dict: """Export only source train/val frames explicitly inspected by the AI reviewer.""" manifest = check_supervision(source) @@ -90,7 +123,8 @@ def export_instances(source: Path, reviews: Path, out: Path) -> dict: negative[y0:y1, x0:x1] = 1 if (occupied & (negative == 1)).any(): raise ValueError("reviewed empty region overlaps a reviewed object") - prepared.append((frame, boxes, labels, identities, negative)) + class_negatives = class_negative_regions(item, (w, h), boxes, labels, assignments[fid]) + prepared.append((frame, boxes, labels, identities, negative, class_negatives)) if not prepared: raise ValueError("empty instance review") out.mkdir(parents=True, exist_ok=False) @@ -100,7 +134,7 @@ def export_instances(source: Path, reviews: Path, out: Path) -> dict: shutil.copyfile(__file__, out / "producer.py") frames = [] counts = Counter(dict.fromkeys(CLASSES, 0)) - for frame, boxes, labels, identities, negative in prepared: + for frame, boxes, labels, identities, negative, class_negatives in prepared: fid = frame["id"] shutil.copyfile(source / frame["image"], out / frame["image"]) shutil.copyfile(source / frame["companion"], out / frame["companion"]) @@ -109,6 +143,11 @@ def export_instances(source: Path, reviews: Path, out: Path) -> dict: out / target_name, {"boxes": boxes, "labels": labels, "source_ids": identities} ) Image.fromarray(negative).save(out / negative_name) + class_files = {} + for entity, mask in class_negatives.items(): + name = f"annotations/{fid}.negative-{entity}.png" + Image.fromarray(mask).save(out / name) + class_files[entity] = name counts.update(CLASSES[i] for i in labels) frames.append( { @@ -129,6 +168,8 @@ def export_instances(source: Path, reviews: Path, out: Path) -> dict: "targets": target_name, "negative_mask": negative_name, "negative_pixels": int(negative.sum()), + "class_negative_masks": class_files, + "class_negative_pixels": {k: int(v.sum()) for k, v in class_negatives.items()}, "instances": len(labels), } ) @@ -142,10 +183,12 @@ def export_instances(source: Path, reviews: Path, out: Path) -> dict: "review_sha256": digest(reviews), "instances_by_class": dict(counts), "policy": ( - "positive assigned channel only; negatives only in AI-reviewed empty regions; " + "positive assigned channel; negatives in AI-reviewed empty regions or explicit " + "per-class reviewed absence; " "unknown and padding ignored" ), "exhaustive_labels": False, + "class_negative_supervision": any(row[-1] for row in prepared), "independent_accuracy": False, } write_json(out / "manifest.json", result) @@ -184,6 +227,22 @@ def check_instances(root: Path) -> dict: f["store"], f["original_split"], m["held_out"] ): raise ValueError("instance split role changed") + for entity, name in f.get("class_negative_masks", {}).items(): + if ( + split != "train" + or entity not in CLASSES + or not m.get("class_negative_supervision") + ): + raise ValueError("invalid class negative split/class contract") + if name not in report["outputs"]: + raise ValueError("unbound class negative mask") + with Image.open(_relative(root, name)) as image: + mask = np.array(image) + if ( + mask.shape != tuple(reversed(f["image_size_px"])) + or not np.isin(mask, [0, 1]).all() + ): + raise ValueError("invalid class negative mask shape/values") for mapping, key in ((cameras, f["camera"]), (pixels, f["pixel_sha256"])): if mapping.setdefault(key, split) != split: raise ValueError("instance camera/content split leak") @@ -212,6 +271,7 @@ def __init__( ): raise ValueError("invalid instance fold/split/augmentation") self.root = root + self.class_negative_supervision = bool(m.get("class_negative_supervision", False)) if partial_eval not in (None, "reviewed_regions_v1"): raise ValueError("unsupported partial evaluation protocol") self.partial_detection_evaluation = partial_eval @@ -236,14 +296,30 @@ def __getitem__(self, index): rgb = image.convert("RGB") with Image.open(self.root / f["negative_mask"]) as image: negative = np.array(image) + masks = {"det_negative_mask": negative} + if self.class_negative_supervision: + for entity in CLASSES: + name = f.get("class_negative_masks", {}).get(entity) + if name: + with Image.open(self.root / name) as image: + masks[f"negative_{entity}"] = np.array(image) + else: + masks[f"negative_{entity}"] = np.zeros_like(negative) sample = self.transform( Sample( image=rgb, - masks={"det_negative_mask": negative}, + masks=masks, boxes=np.asarray(targets["boxes"], dtype=np.float32).reshape(-1, 4), labels=np.asarray(targets["labels"], dtype=np.int64), ) ) + if self.class_negative_supervision: + sample["masks"]["det_class_negative_mask"] = torch.stack( + [ + sample["masks"].pop(f"negative_{entity}").to(torch.uint8) + for entity in CLASSES + ] + ) return { "image": sample["image"], "supervises": self.supervises, diff --git a/src/syncai_hydranet/engine/trainer.py b/src/syncai_hydranet/engine/trainer.py index 670e6d2..9c1a8ab 100644 --- a/src/syncai_hydranet/engine/trainer.py +++ b/src/syncai_hydranet/engine/trainer.py @@ -81,7 +81,12 @@ def _build_datasets(dcfg, input_size): aug = dcfg.get("augment") train_sets, val_sets, names, ratios, val_names = [], [], [], [], [] for ds in dcfg["datasets"]: - train_sets.append(build_dataset(ds, input_size, "train", letterbox=lb, augment=aug)) + if not ds.get("validation_only", False): + train_sets.append(build_dataset(ds, input_size, "train", letterbox=lb, augment=aug)) + names.append(ds["name"]) + ratios.append(float(ds.get("sample_ratio", 1.0))) + elif not ds.get("split_val"): + raise ValueError("validation_only requires split_val") # A dataset may contribute training signal without joining checkpoint selection: # omit `split_val` and it is trained on but never validated on. # @@ -96,8 +101,8 @@ def _build_datasets(dcfg, input_size): # Validation never augments, so it takes no augment argument. val_sets.append(build_dataset(ds, input_size, "val", letterbox=lb)) val_names.append(ds["name"]) - names.append(ds["name"]) - ratios.append(float(ds.get("sample_ratio", 1.0))) + if not train_sets: + raise ValueError("at least one training dataset is required") if not val_sets: raise ValueError( "no dataset declares split_val, so nothing can select a checkpoint; " @@ -447,6 +452,7 @@ def _write_run_meta(self, cfg, names, train_sets, dcfg, val_size, total_iters) - to the data behind it. """ n_params = sum(p.numel() for p in self.model.parameters()) + train_sizes = {name: len(ds) for name, ds in zip(names, train_sets, strict=True)} meta = write_run_meta( self.out_dir, cfg, @@ -456,16 +462,17 @@ def _write_run_meta(self, cfg, names, train_sets, dcfg, val_size, total_iters) - parameters=n_params, datasets=[ { - "name": n, - "train_size": len(t), + "name": ds["name"], + "train_size": train_sizes.get(ds["name"], 0), + "validation_only": bool(ds.get("validation_only", False)), # None, not 0: "not validated on" and "validated on nothing" are # different facts and the run meta should not blur them. - "val_size": val_size.get(n), + "val_size": val_size.get(ds["name"]), # Datasets live outside git; without this, "which data produced # this checkpoint" has no answer six months later. **fingerprint_dataset(ds), } - for n, t, ds in zip(names, train_sets, dcfg["datasets"], strict=True) + for ds in dcfg["datasets"] ], ) _log_code_version(meta["git"], self.out_dir, self.logger) diff --git a/src/syncai_hydranet/geometry/camera_json.py b/src/syncai_hydranet/geometry/camera_json.py index 5a3d400..ff760d0 100644 --- a/src/syncai_hydranet/geometry/camera_json.py +++ b/src/syncai_hydranet/geometry/camera_json.py @@ -119,6 +119,12 @@ class CameraFile: mask_ignore: int = IGNORE plate_file: str | None = None plate_sha256: str | None = None + # (width, height) of the stream the boxes will arrive in. `image_size_px` is the frame + # the intrinsics were fitted at -- half the plate's source by convention -- and every + # consumer that rescales boxes into it assumed the source was 1920x1080. An FTI + # camera commissioned from a 1280x720 stream and another from 3840x2160 both returned + # metres, wrong by 3/2 and 1/2, and nothing said so. None: not recorded (a v3 file). + source_size_px: tuple[int, int] | None = None commissioned_at: str | None = None # ISO 8601, stamped by the writer # Which teacher models produced this file, as `{model_id: revision}`. The plate is # hashed (`plate_sha256`) and the models that read it were not, so two camera.jsons @@ -192,6 +198,11 @@ def load(cls, path: str | Path) -> CameraFile: mask_files=dict(raw.get("mask_files", {})), mask_ignore=raw.get("mask_ignore", IGNORE), plate_file=raw.get("plate_file"), + source_size_px=( + (int(raw["source_size_px"][0]), int(raw["source_size_px"][1])) + if raw.get("source_size_px") + else None + ), plate_sha256=raw.get("plate_sha256"), commissioned_at=raw.get("commissioned_at"), teachers=raw.get("teachers"), @@ -255,6 +266,15 @@ def validate(self) -> None: w, h = self.image_size_px if w <= 0 or h <= 0: problems.append(f"image_size_px {self.image_size_px} is not a size") + if self.source_size_px is not None: + sw, sh = self.source_size_px + if sw <= 0 or sh <= 0: + problems.append(f"source_size_px {self.source_size_px} is not a size") + elif w > 0 and h > 0 and abs((sw / sh) / (w / h) - 1) > 0.01: + problems.append( + f"source_size_px {sw}x{sh} and image_size_px {w}x{h} differ in aspect; " + "a box scaled by width alone lands on the wrong floor pixel" + ) if self.appearance_thr is not None and not self.appearance_thr > 0: problems.append( f"appearance_thr {self.appearance_thr!r} is not a positive distance. " diff --git a/src/syncai_hydranet/models/heads/registry.py b/src/syncai_hydranet/models/heads/registry.py index 003a701..deceac2 100644 --- a/src/syncai_hydranet/models/heads/registry.py +++ b/src/syncai_hydranet/models/heads/registry.py @@ -153,6 +153,7 @@ def loss(self, out: dict, targets: dict) -> tuple[torch.Tensor, dict]: # and trains unchanged. class_mask=targets.get("det_class_mask"), negative_mask=targets.get("det_negative_mask"), + class_negative_mask=targets.get("det_class_negative_mask"), ) def supervised_by(self, _targets: dict) -> bool: diff --git a/src/syncai_hydranet/models/hydranet.py b/src/syncai_hydranet/models/hydranet.py index f99b3bf..00e2185 100644 --- a/src/syncai_hydranet/models/hydranet.py +++ b/src/syncai_hydranet/models/hydranet.py @@ -93,6 +93,9 @@ def __init__(self, cfg): cls_weight=lcfg.get("cls_weight", 1.0), reg_weight=lcfg.get("reg_weight", 1.0), centerness_weight=lcfg.get("centerness_weight", 1.0), + class_negative_normalization=lcfg.get( + "class_negative_normalization", "sum" + ), ) elif hcfg["type"] == "depth_fpn": self.depth_heads[name] = build_depth_head(hcfg, ch) @@ -116,6 +119,23 @@ def __init__(self, cfg): else: self.balancer = FixedWeighting(mcfg.get("fixed_weights", {})) + self.detection_only_training = bool(mcfg.get("detection_only_training", False)) + if self.detection_only_training: + if self.det_head is None: + raise ValueError("detection_only_training requires a detection head") + for name, parameter in self.named_parameters(): + parameter.requires_grad_(name.startswith("det_head.")) + self.train(self.training) + + def train(self, mode: bool = True): + """A frozen scene also needs frozen normalization and disabled dropout.""" + super().train(mode) + if self.detection_only_training: + for name, module in self.named_children(): + if name != "det_head": + module.eval() + return self + def forward(self, images: torch.Tensor) -> dict: """Pure convolution graph. This is exactly what gets exported to ONNX.""" feats = self.neck(self.backbone(images)) diff --git a/src/syncai_hydranet/models/losses.py b/src/syncai_hydranet/models/losses.py index a4e2242..cba05c0 100644 --- a/src/syncai_hydranet/models/losses.py +++ b/src/syncai_hydranet/models/losses.py @@ -146,11 +146,36 @@ def giou_loss(pred_ltrb: torch.Tensor, target_ltrb: torch.Tensor) -> torch.Tenso return (1.0 - giou).sum() +def class_negative_weights(selected, positive, existing): + """Cap added negative mass per batch/class; large reviewed areas are not more labels. + + Positive and existing empty supervision retain their original weights. For a + class absent from this batch, allow one effective negative point so negative-only + examples still teach absence. Counts and scales are detached supervision, not a + function of model confidence. This bounds mask mass, not the gradient norm. + """ + novel = selected & (existing == 0) + budget = positive.float().sum(dim=(0, 1)).clamp(min=1) + count = novel.sum(dim=(0, 1)).clamp(min=1) + scale = (budget / count).clamp(max=1) + return novel.float() * scale[None, None, :] + + class FCOSLoss(nn.Module): - def __init__(self, num_classes: int, cls_weight=1.0, reg_weight=1.0, centerness_weight=1.0): + def __init__( + self, + num_classes: int, + cls_weight=1.0, + reg_weight=1.0, + centerness_weight=1.0, + class_negative_normalization="sum", + ): super().__init__() self.num_classes = num_classes self.w = (cls_weight, reg_weight, centerness_weight) + if class_negative_normalization not in ("sum", "positive_budget"): + raise ValueError("unsupported class_negative_normalization") + self.class_negative_normalization = class_negative_normalization def forward( self, @@ -162,6 +187,7 @@ def forward( labels_list, class_mask=None, negative_mask=None, + class_negative_mask=None, ): """``class_mask`` is [B, C] or [C]: which channels this batch's dataset can label. @@ -170,6 +196,8 @@ def forward( Only the assigned positive channel is supervised at a labelled box; other channels and unlabelled locations are unknown, not false negatives. None preserves exhaustive-box training, including its ordinary background loss. + ``class_negative_mask`` is optional [B,C,H,W] reviewed absence per class; + it never turns the other channels into background. """ device = cls_out[0].device shapes = [c.shape[-2:] for c in cls_out] @@ -189,6 +217,8 @@ def forward( # is why it survived every seg-only run. onehot[pos] = F.one_hot(cls_t[pos], self.num_classes).to(onehot.dtype) partial_mask = None + if class_negative_mask is not None and negative_mask is None: + raise ValueError("class_negative_mask requires partial negative_mask") if negative_mask is not None: if negative_mask.ndim != 3 or negative_mask.shape[0] != flat_cls.shape[0]: raise ValueError("partial detection negative_mask must be [B,H,W]") @@ -216,6 +246,34 @@ def forward( ).any(dim=1) negative[b] &= ~in_box partial_mask = torch.maximum(onehot, negative[..., None].to(onehot.dtype)) + if class_negative_mask is not None: + expected = (flat_cls.shape[0], self.num_classes, h, w) + if tuple(class_negative_mask.shape) != expected: + raise ValueError("class_negative_mask must be [B,C,H,W] aligned with image") + cm = class_negative_mask.to(device) + if not ((cm == 0) | (cm == 1) | (cm == 255)).all(): + raise ValueError("class_negative_mask supports only 0/1/255") + selected = ( + cm[:, :, xy[:, 1].clamp(max=h - 1), xy[:, 0].clamp(max=w - 1)] == 1 + ).permute(0, 2, 1) + selected &= inside[None, :, None] + for b, (boxes, labels) in enumerate(zip(boxes_list, labels_list, strict=True)): + # Reviewed positives take precedence on every pyramid level, + # including levels to which their regression was not assigned. + for box, label in zip(boxes, labels, strict=True): + inside_box = ( + (points[:, 0] >= box[0]) + & (points[:, 0] <= box[2]) + & (points[:, 1] >= box[1]) + & (points[:, 1] <= box[3]) + ) + selected[b, inside_box, label] = False + weight = ( + class_negative_weights(selected, onehot, partial_mask) + if self.class_negative_normalization == "positive_budget" + else selected.to(onehot.dtype) + ) + partial_mask = torch.maximum(partial_mask, weight) if class_mask is not None: # [B, C] -> [B, 1, C] against flat_cls's [B, points, C]; a [C] mask # broadcasts as it is. Cast rather than assume: under autocast flat_cls is diff --git a/src/syncai_hydranet/paths.py b/src/syncai_hydranet/paths.py new file mode 100644 index 0000000..5b054cd --- /dev/null +++ b/src/syncai_hydranet/paths.py @@ -0,0 +1,22 @@ +"""Where the tree is: one answer for every tool that reads `runs/` and `datasets/`. + +Every commissioning tool derived its root from its own file, so the chain could only ever +run against the checkout it lived in. Commissioning a second site (FTI, 2026-09-10) meant +copying `src/ tools/ scripts/` into a checkout-shaped directory and symlinking the +datasets the hardcoded paths named. `scene_mesh.py` already read `SYNCAI_ROOT`; now the +tools do too, through this one function, so a second site is an environment variable +rather than a second copy of the code. +""" + +from __future__ import annotations + +import os +from pathlib import Path + +ENV = "SYNCAI_ROOT" + + +def repo_root(derived: Path) -> Path: + """`$SYNCAI_ROOT` when set, else the root the caller derived from its own location.""" + override = os.environ.get(ENV) + return Path(override).resolve() if override else derived diff --git a/src/syncai_hydranet/shipped.py b/src/syncai_hydranet/shipped.py index ef05ceb..f81cbf7 100644 --- a/src/syncai_hydranet/shipped.py +++ b/src/syncai_hydranet/shipped.py @@ -44,7 +44,9 @@ from pathlib import Path -REPO = Path(__file__).resolve().parents[2] +from syncai_hydranet.paths import repo_root + +REPO = repo_root(Path(__file__).resolve().parents[2]) #: The run every tool ships from. A timestamped directory rather than a stable name on #: purpose: the name says when the weights were trained, and promoting a new run is an diff --git a/tests/test_camera_json.py b/tests/test_camera_json.py index b9965c2..10f2d70 100644 --- a/tests/test_camera_json.py +++ b/tests/test_camera_json.py @@ -285,3 +285,21 @@ def test_ground_points_rejects_a_policy_it_does_not_have(): with pytest.raises(ValueError, match="above_horizon must be"): a_camera_file().ground_points(np.array([[960.0, 900.0]]), above_horizon="warn") + + +def test_the_source_frame_survives_a_round_trip_and_is_absent_from_older_files(tmp_path): + """The boxes' own frame is part of the contract; a v3 file simply does not know it.""" + cf = dataclasses.replace(a_camera_file(), source_size_px=(1280, 720)) + path = tmp_path / "c.json" + cf.save(path) + back = CameraFile.load(path) + assert back.source_size_px == (1280, 720) + raw = json.loads(path.read_text()) + del raw["source_size_px"] + path.write_text(json.dumps(raw)) + assert CameraFile.load(path).source_size_px is None + + +def test_a_source_frame_of_another_aspect_is_refused(): + with pytest.raises(ValueError, match="aspect"): + dataclasses.replace(a_camera_file(), source_size_px=(1280, 960)).validate() diff --git a/tests/test_commissioning.py b/tests/test_commissioning.py index e3fd72e..82502b6 100644 --- a/tests/test_commissioning.py +++ b/tests/test_commissioning.py @@ -206,3 +206,11 @@ def test_height_only_update_refuses_another_camera(tmp_path): cam = _commissioned(tmp_path) with pytest.raises(ValueError, match="camera_id"): regeometry_from_calib(cam, write(tmp_path, camera="Another-camera")) + + +def test_the_streams_native_size_reaches_the_contract_when_the_scan_recorded_it(tmp_path): + """Recorded by static_plates from the clip, carried by the calib, kept by camera.json; + a scan that did not record it converts as before.""" + cf = from_onboard_calib(write(tmp_path, source_size_px=[1280, 720])) + assert cf.source_size_px == (1280, 720) + assert from_onboard_calib(write(tmp_path)).source_size_px is None diff --git a/tests/test_detector_warmup.py b/tests/test_detector_warmup.py new file mode 100644 index 0000000..65e03fb --- /dev/null +++ b/tests/test_detector_warmup.py @@ -0,0 +1,82 @@ +"""A detector update must not mutate any scene weights, buffers or logits.""" + +import importlib.util +from pathlib import Path + +import pytest +import torch + +from syncai_hydranet.models.hydranet import HydraNet + + +@pytest.fixture +def worker(): + path = Path(__file__).resolve().parents[1] / "tools/annotation/studioa_train.py" + spec = importlib.util.spec_from_file_location("warmup_worker_test", path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def config(): + return { + "model": { + "backbone": {"name": "resnet18", "pretrained": False}, + "neck": {"name": "fpn", "out_channels": 16, "num_levels": 5}, + "heads": { + "scene": { + "type": "semantic_fpn", + "num_classes": 3, + "channels": 16, + "dropout": 0.5, + }, + "detection": {"type": "fcos", "num_classes": 2, "channels": 16, "num_convs": 1}, + }, + "loss_balancing": "fixed", + "detection_only_training": True, + } + } + + +def test_detector_updates_preserve_scene_through_modes_and_reload(worker): + torch.set_num_threads(2) + model = HydraNet(config()) + reference = worker.frozen_state(model) + before_head = model.det_head.cls_pred.weight.detach().clone() + trainable = [p for p in model.parameters() if p.requires_grad] + assert {id(p) for p in trainable} == {id(p) for p in model.det_head.parameters()} + optimizer = torch.optim.AdamW(trainable, lr=1e-3) + images = torch.randn(2, 3, 64, 96) + loaders = [("studioa", [{"image": images}])] + before = worker.scene_signature(model, loaders, "cpu") + for _ in range(3): + model.eval().train() + assert model.det_head.training + output = model(images) + targets = { + "boxes": [torch.tensor([[16.0, 16.0, 48.0, 48.0]])] * 2, + "labels": [torch.tensor([1])] * 2, + "det_negative_mask": torch.zeros(2, 64, 96, dtype=torch.long), + } + loss, _ = model.compute_losses(output, targets, ["detection"]) + optimizer.zero_grad() + loss.backward() + optimizer.step() + worker.assert_frozen(model, reference) + assert not torch.equal(before_head, model.det_head.cls_pred.weight) + restored = HydraNet(config()) + restored.load_state_dict(model.state_dict()) + restored.eval().train() + worker.assert_frozen(restored, reference) + assert worker.scene_signature(restored, loaders, "cpu") == before + with torch.no_grad(): + restored.neck.state_dict()[next(iter(restored.neck.state_dict()))].add_(1) + with pytest.raises(ValueError, match="frozen scene state changed"): + worker.assert_frozen(restored, reference) + + +def test_warmup_without_detection_rejected(): + cfg = config() + del cfg["model"]["heads"]["detection"] + with pytest.raises(ValueError, match="requires a detection head"): + HydraNet(cfg) diff --git a/tests/test_onboard_camera.py b/tests/test_onboard_camera.py new file mode 100644 index 0000000..ada2a56 --- /dev/null +++ b/tests/test_onboard_camera.py @@ -0,0 +1,81 @@ +"""A measured vfov reaches the calibration: the pin is data, the primary row follows it. + +`onboard_camera.py` pinned exactly one camera by name and swept the 70.4 constant as +the primary for every other; `floor_calibrate.py`'s measured vfov had no consumer. +""" + +from __future__ import annotations + +import importlib.util +import json +import sys +from pathlib import Path + +import pytest + +REPO = Path(__file__).resolve().parent.parent + + +def _load(): + name = "_onboard_camera" + spec = importlib.util.spec_from_file_location(name, REPO / "scripts" / "onboard_camera.py") + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +ob = _load() + + +def test_the_tile_pinned_camera_is_data_not_a_name_test(): + pin = ob.pin_for("Taichung-cam01", None) + assert pin["vfov_deg"] == 70.4 and pin["source"] == "tile_grid_pinned" + assert ob.pin_for("Taichung-cam04", None) is None + + +def test_a_pins_file_adds_measured_cameras_and_a_later_pin_wins(tmp_path): + path = tmp_path / "pins.json" + path.write_text( + json.dumps( + { + "FTI-MDF-cam821": {"vfov_deg": 62.75, "source": "floor_orthogonality"}, + "Taichung-cam01": {"vfov_deg": 69.0, "source": "floor_orthogonality"}, + } + ) + ) + pins = ob.load_pins(path) + assert pins["FTI-MDF-cam821"]["vfov_deg"] == 62.75 + assert pins["Taichung-cam01"]["vfov_deg"] == 69.0 # the measurement over the built-in + assert ob.pin_for("Taichung-cam04", pins) is None + + +def test_a_pin_without_a_vfov_is_refused(tmp_path): + path = tmp_path / "pins.json" + path.write_text(json.dumps({"X-cam01": {"source": "typo"}})) + with pytest.raises(ValueError, match="vfov_deg"): + ob.load_pins(path) + + +def test_the_primary_row_is_the_cameras_own_vfov_and_a_failed_row_is_none(): + rows = [ + {"vfov_deg": 55.0, "pitch_deg": 34.9}, + {"vfov_deg": 62.75, "pitch_deg": 39.1}, + {"vfov_deg": 70.4, "failed": "no plausible floor plane"}, + ] + assert ob.primary_row(rows, 62.75)["pitch_deg"] == 39.1 + assert ob.primary_row(rows, 70.4) is None + assert ob.primary_row(rows, 85.0) is None + + +def test_the_plate_indexs_source_size_reaches_the_calib(tmp_path): + root = tmp_path / "plates" + root.mkdir() + (root / "index.json").write_text( + json.dumps( + {"cameras": {"X-cam01": {"slots": {"20260910-150000": {"source_wh": [1280, 720]}}}}} + ) + ) + assert ob.source_size_from_index(root, "X-cam01", "20260910-150000") == [1280, 720] + assert ob.source_size_from_index(root, "X-cam01", "20260910-160000") is None + assert ob.source_size_from_index(tmp_path / "nowhere", "X-cam01", "s") is None diff --git a/tests/test_partial_detection.py b/tests/test_partial_detection.py index f234a4b..990d443 100644 --- a/tests/test_partial_detection.py +++ b/tests/test_partial_detection.py @@ -7,6 +7,76 @@ from syncai_hydranet.models.losses import FCOSLoss +def test_class_negative_budget_bounds_mass_and_excludes_existing_supervision(): + from syncai_hydranet.models.losses import class_negative_weights + + positive = torch.zeros(2, 20, 3) + positive[0, :3, 0] = 1 + existing = positive.clone() + existing[:, -2:, :] = 1 # existing empty negatives must not spend the added budget + selected = torch.ones_like(positive, dtype=torch.bool) + weight = class_negative_weights(selected, positive, existing) + torch.testing.assert_close(weight.sum((0, 1)), torch.tensor([3.0, 1.0, 1.0])) + assert (weight[existing > 0] == 0).all() + assert not weight.requires_grad + + +def test_negative_only_area_replication_keeps_bounded_loss_and_total_gradient(): + results = [] + for area in (8, 16): + head, cls, reg, ctr, _, _, negative = batch() + negative.zero_() + cm = torch.zeros(1, 3, 32, 32, dtype=torch.uint8) + cm[:, 0, :area, :area] = 1 + loss, _ = FCOSLoss(3, class_negative_normalization="positive_budget")( + head, + cls, + reg, + ctr, + [torch.zeros(0, 4)], + [torch.zeros(0, dtype=torch.long)], + negative_mask=negative, + class_negative_mask=cm, + ) + loss.backward() + results.append((loss.detach(), sum(x.grad.sum() for x in cls))) + assert all((x.grad[:, 1:] == 0).all() for x in cls) + assert all((x.grad == 0).all() for x in reg + ctr) + torch.testing.assert_close(results[0], results[1]) + + +def test_budget_preserves_positive_and_empty_gradients_and_only_scales_added_negatives(): + gradients = [] + for mode in ("sum", "positive_budget"): + head, cls, reg, ctr, boxes, labels, negative = batch() + cm = torch.zeros(1, 3, 32, 32, dtype=torch.uint8) + cm[:, 0] = 1 + cm[:, 1, 8:24, 8:24] = 1 # contradictory same-class signal remains protected + cm[:, 2, 24:] = 255 + FCOSLoss(3, class_negative_normalization=mode)( + head, + cls, + reg, + ctr, + boxes, + labels, + negative_mask=negative, + class_negative_mask=cm, + )[0].backward() + gradients.append(([x.grad.clone() for x in cls], [x.grad.clone() for x in reg + ctr])) + a, b = gradients + torch.testing.assert_close(a[1], b[1]) + for x, y in zip(a[0], b[0], strict=True): + torch.testing.assert_close(x[:, 1:], y[:, 1:]) + torch.testing.assert_close(a[0][0][:, :, 0, 0], b[0][0][:, :, 0, 0]) + assert 0 < b[0][0][0, 0, 0, 1] < a[0][0][0, 0, 0, 1] + + +def test_bad_class_negative_normalization_rejected(): + with pytest.raises(ValueError, match="normalization"): + FCOSLoss(3, class_negative_normalization="invalid") + + def batch(): head = FCOSHead(8, 3, in_levels=[0, 1], channels=8, num_convs=1, strides=[8, 16]) cls = [ @@ -44,6 +114,55 @@ def test_partial_focal_unknown_channels_padding_and_regression_gradients(): assert (cls[1].grad == 0).all() # small box out of second level's range +def test_class_negative_only_affects_reviewed_channel_and_protects_positive_all_levels(): + head, cls, reg, ctr, boxes, labels, negative = batch() + negative.zero_() + per_class = torch.zeros(1, 3, 32, 32, dtype=torch.uint8) + per_class[:, 0, :8, 8:16] = 1 + per_class[:, 0, 8:16, 8:16] = 1 # explicit wrong-class negative on another positive + per_class[:, 1, 8:24, 8:24] = 1 # contradictory positive is protected at every level + per_class[:, 2, 24:] = 255 # padding is not a negative + FCOSLoss(3)( + head, + cls, + reg, + ctr, + boxes, + labels, + negative_mask=negative, + class_negative_mask=per_class, + )[0].backward() + g = cls[0].grad[0] + assert g[0, 0, 1] > 0 + assert (g[1:, 0, 1] == 0).all() + assert g[1, 1, 1] < 0 + assert g[0, 1, 1] > 0 and g[2, 1, 1] == 0 + assert (g[:, 3, :] == 0).all() + assert (cls[1].grad[:, 1:] == 0).all() + assert cls[1].grad[0, 0, 0, 0] > 0 + assert (reg[0].grad[:, :, 0, :] == 0).all() + assert (ctr[0].grad[:, :, 0, :] == 0).all() + + +@pytest.mark.parametrize( + "bad", [torch.zeros(1, 32, 32), torch.zeros(1, 2, 32, 32), torch.full((1, 3, 32, 32), 2)] +) +def test_invalid_class_negatives_rejected(bad): + head, cls, reg, ctr, boxes, labels, negative = batch() + with pytest.raises(ValueError, match="class_negative_mask"): + FCOSLoss(3)( + head, cls, reg, ctr, boxes, labels, negative_mask=negative, class_negative_mask=bad + ) + + +def test_class_negative_cannot_turn_exhaustive_training_into_partial(): + head, cls, reg, ctr, boxes, labels, _ = batch() + with pytest.raises(ValueError, match="requires partial"): + FCOSLoss(3)( + head, cls, reg, ctr, boxes, labels, class_negative_mask=torch.zeros(1, 3, 32, 32) + ) + + def test_box_never_becomes_negative_on_an_unassigned_level(): head, cls, reg, ctr, boxes, labels, negative = batch() negative.fill_(1) diff --git a/tests/test_partial_detection_eval.py b/tests/test_partial_detection_eval.py index 41de294..d93f7f2 100644 --- a/tests/test_partial_detection_eval.py +++ b/tests/test_partial_detection_eval.py @@ -121,6 +121,10 @@ def test_joint_config_never_declares_test_and_warm_start_checks_taxonomy(tmp_pat cfg = worker.joint_config(copy.deepcopy(original), tmp_path, "Tao-Hsin") assert all("split_test" not in ds for ds in cfg["data"]["datasets"]) + warmup = worker.detector_warmup_config(copy.deepcopy(cfg)) + assert warmup["model"]["detection_only_training"] + assert warmup["data"]["datasets"][0]["validation_only"] + assert warmup["train"]["primary_metric"] == worker.DET_METRIC model = HydraNet(cfg) worker.warm_start(model, {"cfg": original, "model": source.state_dict()}, cfg) torch.testing.assert_close( diff --git a/tests/test_plate_calibration.py b/tests/test_plate_calibration.py index 23db7c5..d7eee59 100644 --- a/tests/test_plate_calibration.py +++ b/tests/test_plate_calibration.py @@ -622,3 +622,48 @@ def test_a_depth_frame_with_nothing_in_it_yields_no_candidates(): assert floor_candidates(empty, cam, inlier_m=0.03) == [] plane, residual, rows = choose_floor([]) assert plane is None and residual is None and rows == [] + + +def _plate(dir_, slot, luma): + from PIL import Image + + Image.new("L", (4, 4), luma).save(dir_ / f"plate_{slot}.png") + + +def test_the_daytime_gate_uses_the_sites_offset_not_a_constant(tmp_path): + """A US-Central site (UTC-5) recorded at 15:00 UTC is 10:00 local -- daytime -- and the + same plates read under +8 pick another slot. The old constant refused the 15 UTC slot + (23 local) and passed a 10 UTC slot by luck (18 local).""" + from syncai_bev3d.plate_calibration import pick_daytime_slot + + _plate(tmp_path, "20260910-150000", 200) + _plate(tmp_path, "20260910-030000", 250) # 22:00 local at -5: brighter, but night + assert pick_daytime_slot(tmp_path, utc_offset=-5) == "20260910-150000" + assert pick_daytime_slot(tmp_path, utc_offset=8) == "20260910-030000" + + +def test_a_cameras_json_without_an_offset_still_means_the_original_site(): + from syncai_bev3d.plate_calibration import DEFAULT_UTC_OFFSET_HOURS, utc_offset_hours + + assert utc_offset_hours({"cameras": {}}) == DEFAULT_UTC_OFFSET_HOURS == 8 + assert utc_offset_hours({"utc_offset_hours": -5, "cameras": {}}) == -5 + + +def test_a_night_shift_site_states_its_own_lit_hours(tmp_path): + """FTI's only capture is 05:22 local (UTC-6, slot 11 UTC): lit, and refused by a shop's + 08-18 window. The site's cameras.json widens the window; a window across midnight + is stated as [start, end] with start > end.""" + from syncai_bev3d.plate_calibration import ( + DEFAULT_DAYTIME_HOURS_LOCAL, + daytime_hours_local, + in_daytime, + pick_daytime_slot, + ) + + _plate(tmp_path, "20260910-112151", 149) + with pytest.raises(SystemExit, match="no daytime plate"): + pick_daytime_slot(tmp_path, utc_offset=-6) + assert pick_daytime_slot(tmp_path, utc_offset=-6, daytime=(4, 23)) == "20260910-112151" + assert daytime_hours_local({"cameras": {}}) == DEFAULT_DAYTIME_HOURS_LOCAL == (8, 18) + assert daytime_hours_local({"daytime_hours_local": [4, 23], "cameras": {}}) == (4, 23) + assert in_daytime(23, (20, 6)) and in_daytime(2, (20, 6)) and not in_daytime(12, (20, 6)) diff --git a/tests/test_rulers.py b/tests/test_rulers.py index 3768a4d..ee29eb1 100644 --- a/tests/test_rulers.py +++ b/tests/test_rulers.py @@ -85,3 +85,32 @@ def test_two_rulers_on_opposite_sides_do_not_apply(): def test_no_ruler_keeps(): v = combine([]) assert v.factor == 1.0 and not v.apply + + +def test_a_bootstrap_calibration_has_no_person_ruler_to_anchor_to(): + """Factor 1 on a scale that was never measured would anchor every other ruler to the + reading they exist to check.""" + assert person_ruler({"scale_source": "unmeasured"}) is None + assert person_ruler({"scale_source": "dav2_metric_indoor_raw_bootstrap_unverified"}) is None + assert person_ruler({"scale_source": "person_height_median_vs_1.7m_prior_n38"}) is not None + + +def test_unanchored_rulers_agreeing_is_not_a_verdict(): + """FTI MDF room, 2026-09-10: a 600 mm raised floor read 0.85 and 0.74 m through DA-V2's + raw metres, the catalogue picked 0.80, and the two cameras 'agreed'. The consensus tile + and the store-median table both take their reference from the readings themselves.""" + tile = tile_ruler(0.85, 0.80, anchored=False) + table = table_ruler(0.94, [0.94, 0.90, 0.92]) + v = combine([tile, table]) + assert not v.apply + assert v.factor == 1.0 + assert "unanchored" in v.reason + + +def test_an_anchored_ruler_restores_the_two_witness_rule(): + """With the person prior present, an unanchored tile can still be the second witness.""" + person = Ruler("person", 1.0, 0.14) + tile = tile_ruler(0.85, 0.60, anchored=False) # x0.706, a real size this time + known = Ruler("folding_table", 0.74 / 0.94, 0.05, "0.94 m read vs 0.74 m") + v = combine([person, tile, known]) + assert v.apply and abs(v.factor - 0.706) < 0.1 diff --git a/tests/test_stage0_baseline.py b/tests/test_stage0_baseline.py new file mode 100644 index 0000000..9163135 --- /dev/null +++ b/tests/test_stage0_baseline.py @@ -0,0 +1,56 @@ +"""The Stage 0 freeze carries every file `capture_inputs` identifies, code included. + +`freeze()` copied `src/`, `scene_mesh.py`, `pyproject.toml` and `uv.lock` by name while +`capture_inputs` also listed `rebuild_geometry.py`; the identity check at the end of the +freeze then failed on every camera (FTI-SMT-cam912, 2026-09-16), and the runner never +reached a scene. The copy now walks the identity record, so the two cannot drift apart. +""" + +from __future__ import annotations + +import math +import runpy +from pathlib import Path + +import numpy as np +from PIL import Image + +from syncai_bev3d.scene_audit import capture_inputs +from syncai_hydranet.geometry.camera_json import CameraFile +from syncai_hydranet.geometry.ground import Camera, GroundPlane + +TOOL = Path(__file__).resolve().parents[1] / "tools/commissioning/stage0_baseline.py" + + +def _site(root: Path) -> None: + folder = root / "runs/commission01/sample/masks" + folder.mkdir(parents=True) + cache = root / "runs/site30k_qa/geometry_cache/sample.npz" + cache.parent.mkdir(parents=True) + np.savez(cache, horiz=np.ones((48, 64))) + CameraFile( + camera_id="sample", + image_size_px=(64, 48), + camera=Camera(fx=42, fy=42, cx=32, cy=24), + plane=GroundPlane(height=2.8, pitch=math.radians(25)), + mask_files={"objects": "sample/masks/objects.png"}, + ).save(root / "runs/commission01/sample.camera.json") + Image.new("L", (64, 48)).save(folder / "objects.png") + (root / "src/syncai_bev3d").mkdir(parents=True) + (root / "src/syncai_bev3d/scene.py").write_text("SCENE = 1\n") + for name in ("scene_mesh.py", "rebuild_geometry.py"): + (root / "tools/commissioning" / name).parent.mkdir(parents=True, exist_ok=True) + (root / "tools/commissioning" / name).write_text(f"# {name}\n") + (root / "pyproject.toml").write_text("[project]\nname = 'sample'\n") + (root / "uv.lock").write_text("version = 1\n") + + +def test_the_snapshot_identifies_as_the_source_it_was_frozen_from(tmp_path): + freeze = runpy.run_path(str(TOOL))["freeze"] + root, out = tmp_path / "site", tmp_path / "out" + _site(root) + out.mkdir() + snapshot = freeze(root, out, ["sample"]) + assert (snapshot / "tools/commissioning/rebuild_geometry.py").is_file() + assert capture_inputs(snapshot, "sample") == capture_inputs(root, "sample") + assert (out / "frozen-inputs.json").is_file() diff --git a/tests/test_stage0_commission.py b/tests/test_stage0_commission.py new file mode 100644 index 0000000..92c77a1 --- /dev/null +++ b/tests/test_stage0_commission.py @@ -0,0 +1,104 @@ +"""The one-camera Stage 0 chain: the README's steps, the bootstrap rule, the site root.""" + +from __future__ import annotations + +import json +import runpy +from pathlib import Path + +import pytest + +TOOL = Path(__file__).resolve().parents[1] / "tools/commissioning/stage0_commission.py" + + +@pytest.fixture(scope="module") +def tool(): + return runpy.run_path(str(TOOL)) + + +def test_the_slot_is_the_archive_name_or_the_files_utc_mtime(tool, tmp_path): + named = tmp_path / "FTI-SMT-cam912_archive_20260910-112151_fti.mp4" + named.write_bytes(b"") + assert tool["slot_from_clip"](named) == "20260910-112151" + plain = tmp_path / "nvr9_ch12.mp4" + plain.write_bytes(b"") + assert len(tool["slot_from_clip"](plain)) == len("20260910-112151") + + +def _calib(tmp_path, **fields): + p = tmp_path / "cam.calib.json" + d = { + "pitch_deg": 39.6, + "height_dav2_raw_m": 3.1, + "scale_source": "unmeasured", + "flags": ["scale_unmeasured"], + } + d.update(fields) + p.write_text(json.dumps(d)) + return p + + +def test_no_floor_is_a_refusal_and_bootstrap_needs_permission(tool, tmp_path): + with pytest.raises(SystemExit, match="2"): + tool["bootstrap_unmeasured"](_calib(tmp_path, pitch_deg=None), allow=True) + with pytest.raises(SystemExit, match="3"): + tool["bootstrap_unmeasured"](_calib(tmp_path), allow=False) + + +def test_the_bootstrap_is_flagged_and_a_measured_scale_is_left_alone(tool, tmp_path): + p = _calib(tmp_path) + assert tool["bootstrap_unmeasured"](p, allow=True).startswith( + "dav2_metric_indoor_raw_bootstrap" + ) + d = json.loads(p.read_text()) + assert d["scale"] == 1.0 and d["height_m"] == 3.1 + assert tool["BOOTSTRAP_FLAG"] in d["flags"] and "scale_unmeasured" not in d["flags"] + p = _calib(tmp_path, scale_source="person_height_median_vs_1.7m_prior_n38", flags=[]) + assert ( + tool["bootstrap_unmeasured"](p, allow=False) == "person_height_median_vs_1.7m_prior_n38" + ) + assert "scale" not in json.loads(p.read_text()) + + +def test_the_plan_is_the_readme_chain_ending_in_a_freeze(tool, tmp_path): + steps = tool["plan"]( + tmp_path, + "cam", + Path("datasets/fti_clips"), + Path("datasets/fti_static"), + tmp_path / "out", + ) + labels = [label for label, _ in steps] + assert labels == [ + "plate", + "person boxes", + "onboard", + "zones", + "review bundle", + "masks", + "extras", + "depth completion", + "scene3d", + "scene overlay", + "scene mesh", + "freeze", + ] + freeze = dict(steps)["freeze"] + bundle = dict(steps)["review bundle"] + assert bundle[-1] == "runs/commission_review/cam_batch" + stamped = dict(tool["plan"](tmp_path, "cam", Path("c"), Path("p"), None, "20260910-102225")) + assert stamped["review bundle"][-1] == "runs/commission_review/cam_20260910-102225" + assert freeze[1].endswith("stage0_baseline.py") and str(tmp_path) in freeze + assert len(tool["plan"](tmp_path, "cam", Path("c"), Path("p"), None)) == 11 + + +def test_dry_run_prints_the_chain_without_touching_the_tree( + tool, tmp_path, capsys, monkeypatch +): + clip = tmp_path / "nvr9_ch12.mp4" + clip.write_bytes(b"") + monkeypatch.setitem(tool, "ROOT", tmp_path) + assert tool["main"](["cam", "--clip", str(clip), "--dry-run"]) == 0 + out = capsys.readouterr().out + assert out.count("[") == 11 and "[scene mesh]" in out + assert not (tmp_path / "datasets").exists() diff --git a/tests/test_studioa_supervision.py b/tests/test_studioa_supervision.py index 16969cd..b225ef8 100644 --- a/tests/test_studioa_supervision.py +++ b/tests/test_studioa_supervision.py @@ -133,6 +133,9 @@ def test_reviewed_instances_export_transform_factory_and_boundaries(tmp_path): {"id": data["entities"][0]["id"], "entity": "phone", "reason": "fixture"} ], "negative_rects": [{"xyxy": [32, 0, 40, 8], "reason": "fixture empty patch"}], + "class_negative_rects": [ + {"entity": "laptop", "xyxy": [0, 0, 8, 8], "reason": "phone is not laptop"} + ], } ], } @@ -159,6 +162,10 @@ def test_reviewed_instances_export_transform_factory_and_boundaries(tmp_path): assert sample["targets"]["boxes"].tolist() == [[10.0, 0.0, 40.0, 24.0]] assert (sample["targets"]["det_negative_mask"][:8, :8] == 1).all() assert sample["targets"]["labels"].tolist() == [1] + class_mask = sample["targets"]["det_class_negative_mask"] + assert class_mask.shape == (10, 32, 40) + assert (class_mask[0, :8, 32:] == 1).all() # flipped with image and boxes + assert (class_mask[1:] == 0).all() assert fingerprint_dataset(cfg)["splits"]["train"]["frames"] == [f["id"]] assert split_leaks([cfg]) == [] with pytest.raises(ValueError, match="unchanged splits"): @@ -176,6 +183,20 @@ def test_reviewed_instances_export_transform_factory_and_boundaries(tmp_path): check_instances(out) +def test_class_negative_review_rejects_contradiction_and_val(): + from syncai_hydranet.data.studioa_instances import class_negative_regions + + item = { + "class_negative_rects": [{"entity": "phone", "xyxy": [1, 1, 4, 4], "reason": "fixture"}] + } + with pytest.raises(ValueError, match="contradicts"): + class_negative_regions(item, (10, 10), [[0, 0, 5, 5]], [1], "train") + with pytest.raises(ValueError, match="train-only"): + class_negative_regions(item, (10, 10), [], [], "val") + masks = class_negative_regions(item, (10, 10), [[0, 0, 5, 5]], [0], "train") + assert list(masks) == ["phone"] and masks["phone"].sum() == 9 + + def test_real_export_reader_preserves_folds_source_labels_and_ignore(tmp_path): source, out = tmp_path / "source", tmp_path / "out" frames = source_package(source) diff --git a/tests/test_tools_honour_syncai_root.py b/tests/test_tools_honour_syncai_root.py new file mode 100644 index 0000000..e898210 --- /dev/null +++ b/tests/test_tools_honour_syncai_root.py @@ -0,0 +1,39 @@ +"""Every tool in the commissioning chain takes its tree root from `repo_root()`. + +A tool whose root is only `Path(__file__)...` runs against the checkout it lives in and +nowhere else; commissioning a second site then means a second copy of the code +(FTI, 2026-09-10). `SYNCAI_ROOT` is the one override, read in one place. +""" + +from __future__ import annotations + +import re +from pathlib import Path + +import pytest + +REPO = Path(__file__).resolve().parents[1] +FILES = sorted( + list((REPO / "tools" / "commissioning").glob("*.py")) + + list((REPO / "tools" / "site30k").glob("*.py")) + + [REPO / "scripts" / "propose_zones.py", REPO / "scripts" / "campaign_site30k.py"] +) +DERIVED = re.compile(r"^(?:ROOT|REPO|_REPO)\s*=\s*Path\(__file__\)", re.M) # no override + + +@pytest.mark.parametrize("path", FILES, ids=lambda p: p.name) +def test_the_root_is_read_through_repo_root(path): + text = path.read_text() + assert not DERIVED.search(text), ( + f"{path.relative_to(REPO)} derives its root from its own file with no override; " + 'write ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2]))' + ) + + +def test_the_override_is_the_environment(monkeypatch, tmp_path): + from syncai_hydranet.paths import repo_root + + monkeypatch.delenv("SYNCAI_ROOT", raising=False) + assert repo_root(tmp_path) == tmp_path + monkeypatch.setenv("SYNCAI_ROOT", str(tmp_path / "site2")) + assert repo_root(tmp_path) == (tmp_path / "site2").resolve() diff --git a/tests/test_trainer.py b/tests/test_trainer.py index c041b84..f38ef8d 100644 --- a/tests/test_trainer.py +++ b/tests/test_trainer.py @@ -96,6 +96,25 @@ def _cfg(out_dir, data_root, **train): ) +def test_validation_only_dataset_never_opens_train_and_is_recorded(tmp_path, data_root): + cfg = _cfg(tmp_path / "validation_only_run", data_root) + validation = dict(cfg["data"]["datasets"][0]) + validation.update(name="scene_guard", validation_only=True, split_train="must_not_open") + cfg["data"]["datasets"].insert(0, validation) + trainer = Trainer(cfg) + try: + assert trainer.steps_per_epoch == 2 + assert [name for name, _ in trainer.val_sets] == ["scene_guard", "tiny"] + meta = json.loads((trainer.out_dir / "meta.json").read_text()) + assert meta["datasets"][0]["train_size"] == 0 + assert meta["datasets"][0]["validation_only"] + assert meta["datasets"][0]["val_size"] == 2 + assert meta["datasets"][1]["train_size"] == 4 + finally: + if trainer.tb: + trainer.tb.close() + + @pytest.fixture(scope="module") def finished_run(tmp_path_factory, data_root): out = tmp_path_factory.mktemp("run") / "exp" diff --git a/tests/test_world_frame.py b/tests/test_world_frame.py index 8a5f609..5141e68 100644 --- a/tests/test_world_frame.py +++ b/tests/test_world_frame.py @@ -504,3 +504,14 @@ def test_a_canvas_region_without_a_source_size_is_refused(): track, region = _canvas_track((0.5, 3.0)) with pytest.raises(ValueError, match="without source_size_px"): world_frame([track], _half_res_camera_file(), 0, name="person", canvas_region=region) + + +def test_a_track_box_is_drawn_by_the_panel_over_source_ratio_not_a_constant(): + """`demo_video` drew boxes with `/ 2.0`: right for 1920x1080, 2/3 off on 1280x720 and + 2x on 3840x2160. The ratio is per axis and per frame pair.""" + from syncai_hydranet.analytics.world import panel_box + + box = (640.0, 360.0, 700.0, 500.0) + assert panel_box(box, (1920, 1080), (960, 540)) == (320.0, 180.0, 350.0, 250.0) + assert panel_box(box, (1280, 720), (960, 540)) == (480.0, 270.0, 525.0, 375.0) + assert panel_box(box, (3840, 2160), (960, 540)) == (160.0, 90.0, 175.0, 125.0) diff --git a/tools/annotation/studioa_instances.py b/tools/annotation/studioa_instances.py index 140e35b..8f9673d 100644 --- a/tools/annotation/studioa_instances.py +++ b/tools/annotation/studioa_instances.py @@ -186,10 +186,17 @@ def stop(_signum, _frame): & (points[:, None, 1] >= boxes[None, :, 1]) & (points[:, None, 1] <= boxes[None, :, 3]) ).any(dim=1) - unknown = ~known_empty & ~in_any_box + unknown = ( + (~known_empty & ~in_any_box)[:, None].expand(-1, len(CLASSES)).clone() + ) + class_negative = targets.get("det_class_negative_mask") + if class_negative is not None: + reviewed = (class_negative[b, :, safe_y, safe_x] == 1).T + reviewed &= ((x < w) & (y < h))[:, None] + unknown &= ~reviewed if torch.count_nonzero(grads[b, unknown]): raise ValueError("unreviewed region produced background gradient") - unknown_channels += int(unknown.sum()) * len(CLASSES) + unknown_channels += int(unknown.sum()) optimizer.step() steps.append( { diff --git a/tools/annotation/studioa_train.py b/tools/annotation/studioa_train.py index d89f804..7afaf21 100644 --- a/tools/annotation/studioa_train.py +++ b/tools/annotation/studioa_train.py @@ -5,6 +5,7 @@ import argparse import fcntl +import hashlib import json import os import shutil @@ -23,6 +24,80 @@ from syncai_hydranet.data.studioa_supervision import CLASSES, check_supervision ROOT = Path(__file__).resolve().parents[2] +DET_METRIC = "partial_det/studioa_instances/recall50_at_020" +EMPTY_FP = "partial_det/studioa_instances/empty_fp" + + +def detector_warmup_config(config: dict) -> dict: + config["experiment"] = "studioa_detector_warmup" + config["model"]["detection_only_training"] = True + config["model"]["fixed_weights"]["detection"] = 1.0 + config["data"]["datasets"][0]["validation_only"] = True + config["train"].update( + epochs=60, + lr=2e-4, + warmup_iters=26, + early_stop_patience=20, + primary_metric=DET_METRIC, + deterministic=True, + cudnn_benchmark=False, + tf32=False, + ) + check_config(config) + return config + + +def frozen_state(model) -> dict: + return { + name: value.detach().cpu().clone() + for name, value in model.state_dict().items() + if not name.startswith("det_head.") + } + + +def assert_frozen(model, reference: dict) -> None: + import torch + + current = frozen_state(model) + if current.keys() != reference.keys(): + raise ValueError("frozen scene state keys changed") + for name, value in reference.items(): + if not torch.equal(current[name], value): + raise ValueError(f"frozen scene state changed: {name}") + for name, module in model.named_children(): + if name != "det_head" and any(m.training for m in module.modules()): + raise ValueError(f"frozen scene module in training mode: {name}") + + +def scene_signature(model, loaders, device) -> dict: + """Hash every source-validation scene logit, not just a preview or argmax.""" + import torch + + from syncai_hydranet.engine.evaluator import model_memory_format + + was_training = model.training + model.eval() + signature = hashlib.sha256() + frames = 0 + try: + with torch.inference_mode(): + for name, loader in loaders: + if name != "studioa": + continue + for batch in loader: + images = ( + batch["image"] + .to(device) + .contiguous(memory_format=model_memory_format(model)) + ) + logits = model(images)["scene"].contiguous().cpu() + signature.update(logits.numpy().tobytes()) + frames += len(images) + finally: + model.train(was_training) + if not frames: + raise ValueError("scene invariance requires source validation images") + return {"frames": frames, "float32_logits_sha256": signature.hexdigest()} def pilot_config(data: Path, out: Path, manifest: dict, held_out: str) -> dict: @@ -155,6 +230,8 @@ def prepare( held_out: str, instances: Path | None = None, initial_checkpoint: Path | None = None, + detector_warmup: bool = False, + class_negative_normalization: str = "sum", ) -> None: from syncai_hydranet.utils.visualize import terrain_palette @@ -164,6 +241,12 @@ def prepare( raise ValueError("pilot requires positive train support for every class") if (instances is None) != (initial_checkpoint is None): raise ValueError("joint pilot requires both instances and initial checkpoint") + if detector_warmup and instances is None: + raise ValueError("detector warmup requires instances and initial checkpoint") + if class_negative_normalization not in ("sum", "positive_budget"): + raise ValueError("unsupported class negative normalization") + if class_negative_normalization != "sum" and not detector_warmup: + raise ValueError("class negative normalization comparison requires detector warmup") instance_counts = {} if instances is not None: from collections import Counter @@ -217,6 +300,12 @@ def prepare( config = pilot_config(out / "data", out, manifest, held_out) if instances is not None: config = joint_config(config, out / "instances", held_out) + if detector_warmup: + config = detector_warmup_config(config) + config["model"]["heads"]["detection"]["loss"] = { + "class_negative_normalization": class_negative_normalization + } + check_config(config) write_json(out / "config.json", config) git = subprocess.run( ["git", "rev-parse", "HEAD"], cwd=ROOT, capture_output=True, text=True, check=True @@ -238,6 +327,8 @@ def prepare( if instances is not None else "studioa.semantic-pilot.v1", "joint": instances is not None, + "detector_warmup": detector_warmup, + "class_negative_normalization": class_negative_normalization, "instance_counts": instance_counts, "git_commit": git, "held_out": held_out, @@ -245,7 +336,17 @@ def prepare( "source_manifest_sha256": digest(source / "manifest.json"), "classes": CLASSES, "class_weights": "sqrt median/train frequency clipped 0.25..4", - "selection": "source-camera val mIoU on retained AI pixels only", + "selection": ( + "maximum reviewed-positive source-val recall at fixed score >0.20 / IoU >=0.50" + if detector_warmup + else "source-camera val mIoU on retained AI pixels only" + ), + "acceptance": ( + "strict recall improvement; no increase in reviewed-empty FP; frozen scene " + "parameters/buffers and all source-val float32 logits byte-identical" + if detector_warmup + else None + ), "evaluation": ( "source val only; partial recall and empty-region alarms; no test evaluation" if instances is not None @@ -301,6 +402,8 @@ def stopped(_signal, _frame): raise RuntimeError("CUDA unavailable; refusing accidental CPU pilot training") torch.set_num_threads(4) cfg = json.loads((out / "config.json").read_text()) + warmup = job.get("detector_warmup", False) + reference = None class PilotTrainer(Trainer): def state_dict(self, epoch): @@ -309,24 +412,44 @@ def state_dict(self, epoch): return result def record_epoch(self, epoch, metrics): + if warmup: + assert reference is not None + assert_frozen(self.model, reference) improved = super().record_epoch(epoch, metrics) status( "training", epoch=epoch, global_step=self.global_step, validation_teacher_miou=metrics["scene_mIoU"], - best_validation_teacher_miou=self.best_metric, + primary_metric=self.primary_metric, + best_primary_metric=self.best_metric, ) return improved last = out / "model/last.pt" trainer = PilotTrainer(cfg, resuming=last.exists()) + if warmup: + warm_start(trainer.model, load_checkpoint(out / "initial.pt"), cfg) + reference = frozen_state(trainer.model) + initial_signature = scene_signature( + trainer.model, trainer.val_loaders, trainer.device + ) + before_path = out / "scene_before.json" + if ( + before_path.exists() + and json.loads(before_path.read_text()) != initial_signature + ): + raise ValueError("resumed scene reference differs from original run") + write_json(before_path, initial_signature) if last.exists(): if load_checkpoint(last).get("studioa_job_sha256") != job_hash: raise ValueError("foreign pilot checkpoint") trainer.load(str(last), resume=True) + if warmup: + assert reference is not None + assert_frozen(trainer.model, reference) else: - if job.get("joint"): + if job.get("joint") and not warmup: warm_start(trainer.model, load_checkpoint(out / "initial.pt"), cfg) status("baseline_validation", gpu=torch.cuda.get_device_name(0)) baseline = evaluate( @@ -354,6 +477,24 @@ def record_epoch(self, epoch, metrics): if best.get("studioa_job_sha256") != job_hash: raise ValueError("foreign best checkpoint") trainer.model.load_state_dict(best["model"]) + warmup_evidence = {} + if warmup: + assert reference is not None + assert_frozen(trainer.model, reference) + final_signature = scene_signature( + trainer.model, trainer.val_loaders, trainer.device + ) + if final_signature != initial_signature: + raise ValueError( + "source validation scene logits changed during detector warmup" + ) + warmup_evidence = { + "frozen_state_tensors": len(reference), + "frozen_state_equal": True, + "scene_before": initial_signature, + "scene_after": final_signature, + "scene_logits_equal": True, + } joint = job.get("joint", False) status( "source_validation" if joint else "held_out_evaluation", best_epoch=best["epoch"] @@ -376,19 +517,43 @@ def record_epoch(self, epoch, metrics): ) evaluation_name = "validation" if joint else "test" write_json(out / f"{evaluation_name}.json", test) + if warmup: + baseline = json.loads((out / "baseline_val.json").read_text()) + warmup_evidence.update( + baseline_recall=baseline[DET_METRIC], + selected_recall=test[DET_METRIC], + baseline_empty_fp=baseline[EMPTY_FP], + selected_empty_fp=test[EMPTY_FP], + accepted=( + test[DET_METRIC] > baseline[DET_METRIC] + and test[EMPTY_FP] <= baseline[EMPTY_FP] + ), + ) + write_json(out / "warmup_evidence.json", warmup_evidence) for head, (images, predictions, targets) in samples.items(): grid = prediction_grid(images, predictions, targets, terrain_palette(list(CLASSES))) Image.fromarray(grid).save(out / f"{evaluation_name}_{head}_preview.jpg") verify(out) report = { "status": "completed", - "kind": "AI partial joint pilot" if joint else "AI partial-semantic pilot", + "kind": ( + "AI partial detector warmup" + if warmup + else "AI partial joint pilot" + if joint + else "AI partial-semantic pilot" + ), "job_sha256": job_hash, "git_commit": job["git_commit"], "counts": job["counts"], "best_epoch": best["epoch"], "last_epoch": load_checkpoint(last)["epoch"], - "best_validation_teacher_miou": best["best_metric"], + "primary_metric": trainer.primary_metric, + "best_primary_metric": best["best_metric"], + "best_validation_teacher_miou": test["scene_mIoU"] + if joint + else best["best_metric"], + "warmup": warmup_evidence, "test_evaluated": not joint, f"{evaluation_name}_teacher_miou": test["scene_mIoU"], "instance_counts": job.get("instance_counts", {}), @@ -403,6 +568,8 @@ def record_epoch(self, epoch, metrics): out / "baseline_val.json", out / "model/metrics.jsonl", *out.glob(f"{evaluation_name}_*_preview.jpg"), + *out.glob("scene_before.json"), + *out.glob("warmup_evidence.json"), ] }, } @@ -428,6 +595,10 @@ def main(): parser.add_argument("--source", type=Path) parser.add_argument("--instances", type=Path) parser.add_argument("--initial-checkpoint", type=Path) + parser.add_argument("--detector-warmup", action="store_true") + parser.add_argument( + "--class-negative-normalization", choices=("sum", "positive_budget"), default="sum" + ) parser.add_argument( "--held-out", default="Tao-Hsin", choices=("Tao-Hsin", "Taichung", "Kaohsiung") ) @@ -435,7 +606,15 @@ def main(): if args.action == "prepare": if args.source is None: parser.error("prepare requires --source") - prepare(args.source, args.out, args.held_out, args.instances, args.initial_checkpoint) + prepare( + args.source, + args.out, + args.held_out, + args.instances, + args.initial_checkpoint, + args.detector_warmup, + args.class_negative_normalization, + ) else: out = args.out.resolve() target = out / "snapshot/tools/annotation/studioa_train.py" diff --git a/tools/commissioning/appearance_calibrate.py b/tools/commissioning/appearance_calibrate.py index 2a85db7..df3365f 100644 --- a/tools/commissioning/appearance_calibrate.py +++ b/tools/commissioning/appearance_calibrate.py @@ -36,6 +36,7 @@ import argparse import json +import os import sys from itertools import pairwise from pathlib import Path @@ -44,7 +45,7 @@ import torch from PIL import Image -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) sys.path.insert(0, str(ROOT / "src")) from syncai_hydranet.analytics.appearance import torso_histogram # noqa: E402 diff --git a/tools/commissioning/cluster_rules.py b/tools/commissioning/cluster_rules.py index edb4a3c..aa46f7a 100644 --- a/tools/commissioning/cluster_rules.py +++ b/tools/commissioning/cluster_rules.py @@ -25,6 +25,7 @@ import argparse import importlib.util import json +import os from pathlib import Path import numpy as np @@ -35,7 +36,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) spec = importlib.util.spec_from_file_location("recipe", str(ROOT / "tools/site30k/recipe.py")) R = importlib.util.module_from_spec(spec) diff --git a/tools/commissioning/demo_gif.py b/tools/commissioning/demo_gif.py index 317aee4..fa6a6ae 100644 --- a/tools/commissioning/demo_gif.py +++ b/tools/commissioning/demo_gif.py @@ -45,6 +45,7 @@ import argparse import json +import os import subprocess import sys from pathlib import Path @@ -72,7 +73,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) # The run the tools ship from, named once in `syncai_hydranet.shipped`. Six files # used to carry their own copy of this string and the best run was in none of them. diff --git a/tools/commissioning/demo_video.py b/tools/commissioning/demo_video.py index 8db73d6..5159e72 100644 --- a/tools/commissioning/demo_video.py +++ b/tools/commissioning/demo_video.py @@ -93,6 +93,7 @@ track_staff, ) from syncai_hydranet.analytics.tracker import Tracker +from syncai_hydranet.analytics.world import panel_box from syncai_hydranet.config import load_config from syncai_hydranet.data.video import frames as decode_frames from syncai_hydranet.data.video import probe as probe_video @@ -113,7 +114,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) def _display_verdict(track): @@ -128,6 +129,19 @@ def _display_verdict(track): RUN = SHIPPED_RUN +def _check_source_size(cf, src_w: int, src_h: int, clip) -> None: + """Boxes are rescaled by the probed size, so a different stream is not a scaling + error any more -- it is a different lens crop, and the calibration cannot say which.""" + if cf.source_size_px and tuple(cf.source_size_px) != (src_w, src_h): + cw, ch = cf.source_size_px + print( + f" {cf.camera_id}: {Path(str(clip)).name} is {src_w}x{src_h}; the camera was " + f"commissioned from a {cw}x{ch} stream. Same aspect keeps the metres; a crop " + "does not, and only the NVR knows which this is.", + flush=True, + ) + + def _records_pass( args, clip, src_w, src_h, size, model, device, cf, person_label, rng, staff_model ): @@ -299,6 +313,7 @@ def _render_in_chunks(args, camera, clip, cf, bounds, staff_model) -> int: # The decoded size the recorded boxes are in, so the replay converts to calibrated # pixels the same way the recording pass did. src_w, src_h, _ = probe_video(str(clip)) + _check_source_size(cf, src_w, src_h, clip) t0 = time.time() provenance = capture(ROOT, camera, clip, RUN, args.checkpoint) per = math.ceil(args.frames / args.workers) @@ -706,6 +721,7 @@ def main() -> int: ] n = n_det = n_fp = n_placed = n_outside = n_blur = n_posed = 0 src_w, src_h, _ = probe_video(str(clip)) + _check_source_size(cf, src_w, src_h, clip) # The plate is this camera's own empty shop, named by its camera.json. Missing is a # refusal rather than a silent single-instrument run: the whole argument for two # instruments is that neither is trusted alone. @@ -855,7 +871,7 @@ def main() -> int: moving = 0 for t in tracks: seen_ids.add(t.track_id) - bx = np.asarray(t.box, float) / 2.0 + bx = panel_box(t.box, (src_w, src_h), view_img.size) box_col = track_colour(t, None if staff_model is None else _display_verdict) d.rectangle(list(bx), outline=box_col, width=2) d.text((bx[0] + 3, bx[1] + 2), f"#{t.track_id}", fill=box_col) diff --git a/tools/commissioning/depth_complete.py b/tools/commissioning/depth_complete.py index 75658ca..d72e294 100644 --- a/tools/commissioning/depth_complete.py +++ b/tools/commissioning/depth_complete.py @@ -18,6 +18,7 @@ Usage: uv run python tools/commissioning/depth_complete.py [...] """ +import os import sys from pathlib import Path @@ -32,7 +33,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) IGNORE = 255 TALL_M = 1.35 FLAT_M = 0.08 diff --git a/tools/commissioning/extras_pass.py b/tools/commissioning/extras_pass.py index d5d8287..253cbc4 100644 --- a/tools/commissioning/extras_pass.py +++ b/tools/commissioning/extras_pass.py @@ -16,6 +16,7 @@ """ import dataclasses +import os import sys from pathlib import Path @@ -34,7 +35,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) W, H = 1920, 1080 MIN_PX = 2000 # doors and the generic product union MIN_SUB_PX = 150 # a phone on a table is small and real diff --git a/tools/commissioning/facing_review.py b/tools/commissioning/facing_review.py index 4c3e1d8..1a774aa 100644 --- a/tools/commissioning/facing_review.py +++ b/tools/commissioning/facing_review.py @@ -7,6 +7,7 @@ import argparse import json +import os from pathlib import Path import numpy as np @@ -15,7 +16,7 @@ from syncai_bev3d.object_instances import image_digest, load_instances from syncai_hydranet.geometry.camera_json import CameraFile -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) def main(): diff --git a/tools/commissioning/floor_calibrate.py b/tools/commissioning/floor_calibrate.py index 2b512c0..def27b8 100644 --- a/tools/commissioning/floor_calibrate.py +++ b/tools/commissioning/floor_calibrate.py @@ -23,6 +23,7 @@ import itertools import json import math +import os from pathlib import Path import numpy as np @@ -33,7 +34,7 @@ from syncai_hydranet.geometry.camera_json import CameraFile from syncai_hydranet.geometry.ground import Camera, pixel_to_ground, undistort_points -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) OUT = ROOT / "runs/commission_review/floor_calib" # The sweep. 45-95 covers every lens the fleet census found (PLAN 7.12). VFOV_MIN, VFOV_MAX, VFOV_STEP = 45.0, 95.0, 2.5 diff --git a/tools/commissioning/footprints_from_masks.py b/tools/commissioning/footprints_from_masks.py index ec373d2..be7cbdd 100644 --- a/tools/commissioning/footprints_from_masks.py +++ b/tools/commissioning/footprints_from_masks.py @@ -86,6 +86,7 @@ import argparse import json +import os from pathlib import Path import numpy as np @@ -98,7 +99,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) COMMISSIONED = ROOT / "runs/commission01" OUT = ROOT / "runs/footprints01" diff --git a/tools/commissioning/fp_polygons.py b/tools/commissioning/fp_polygons.py index b8f8ab1..35c1258 100644 --- a/tools/commissioning/fp_polygons.py +++ b/tools/commissioning/fp_polygons.py @@ -13,6 +13,7 @@ """ import json +import os import sys from collections import defaultdict from pathlib import Path @@ -28,7 +29,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) ANN_DIR = ROOT / "datasets/site30k_v1/annotations" GRAY_MAX = 0.35 CELL_PX = 64 diff --git a/tools/commissioning/geometry_bench.py b/tools/commissioning/geometry_bench.py index f5d4c98..040646d 100644 --- a/tools/commissioning/geometry_bench.py +++ b/tools/commissioning/geometry_bench.py @@ -56,6 +56,7 @@ import argparse import json import math +import os import sys from dataclasses import dataclass, field from pathlib import Path @@ -72,7 +73,7 @@ unproject, ) -ROOT = Path(__file__).resolve().parent.parent.parent +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parent.parent.parent)) FRAME_H, FRAME_W = 1080, 1920 diff --git a/tools/commissioning/glazing_pass.py b/tools/commissioning/glazing_pass.py index 49de62e..97f5c45 100644 --- a/tools/commissioning/glazing_pass.py +++ b/tools/commissioning/glazing_pass.py @@ -9,6 +9,7 @@ import argparse import json +import os import shutil from pathlib import Path @@ -24,7 +25,7 @@ from syncai_hydranet.models.glazing import GlazingEncoder, GlazingHead from syncai_hydranet.utils.visualize import preprocess -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) def main(): diff --git a/tools/commissioning/heads_video.py b/tools/commissioning/heads_video.py index b8c5e8e..7149db0 100644 --- a/tools/commissioning/heads_video.py +++ b/tools/commissioning/heads_video.py @@ -76,7 +76,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) def _display_verdict(track): diff --git a/tools/commissioning/heatmap3d.py b/tools/commissioning/heatmap3d.py index 2130746..ddef1e6 100644 --- a/tools/commissioning/heatmap3d.py +++ b/tools/commissioning/heatmap3d.py @@ -15,6 +15,7 @@ import argparse import json +import os import sys import time from pathlib import Path @@ -38,7 +39,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) def main() -> int: diff --git a/tools/commissioning/map_anything_eval.py b/tools/commissioning/map_anything_eval.py index b36cfcd..78866d9 100755 --- a/tools/commissioning/map_anything_eval.py +++ b/tools/commissioning/map_anything_eval.py @@ -59,11 +59,12 @@ import argparse import json import math +import os import sys from dataclasses import dataclass, field from pathlib import Path -REPO = Path(__file__).resolve().parent.parent.parent +REPO = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parent.parent.parent)) # PLAN 7.19: pinned by tile grid on this camera and assumed on every other. ANCHOR_CAMERA = "Taichung-cam01" diff --git a/tools/commissioning/masks_diagnose.py b/tools/commissioning/masks_diagnose.py index 42acc02..f4ab94d 100644 --- a/tools/commissioning/masks_diagnose.py +++ b/tools/commissioning/masks_diagnose.py @@ -25,6 +25,7 @@ import argparse import importlib.util import json +import os from pathlib import Path import numpy as np @@ -34,7 +35,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) spec = importlib.util.spec_from_file_location("recipe", str(ROOT / "tools/site30k/recipe.py")) R = importlib.util.module_from_spec(spec) diff --git a/tools/commissioning/masks_pass.py b/tools/commissioning/masks_pass.py index 2787e88..0b81a01 100644 --- a/tools/commissioning/masks_pass.py +++ b/tools/commissioning/masks_pass.py @@ -8,6 +8,7 @@ import argparse import importlib.util +import os from pathlib import Path import numpy as np @@ -18,7 +19,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) spec = importlib.util.spec_from_file_location("recipe", str(ROOT / "tools/site30k/recipe.py")) R = importlib.util.module_from_spec(spec) diff --git a/tools/commissioning/measure_floor.py b/tools/commissioning/measure_floor.py index c16465c..05cf2c1 100644 --- a/tools/commissioning/measure_floor.py +++ b/tools/commissioning/measure_floor.py @@ -5,13 +5,14 @@ import argparse import json +import os from pathlib import Path from PIL import Image from syncai_hydranet.geometry.camera_json import CameraFile -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) PAGE = Path(__file__).with_suffix(".html").read_text() diff --git a/tools/commissioning/objects_pass.py b/tools/commissioning/objects_pass.py index da168a5..62215e3 100644 --- a/tools/commissioning/objects_pass.py +++ b/tools/commissioning/objects_pass.py @@ -5,6 +5,7 @@ """ import argparse +import os from pathlib import Path from syncai_bev3d.object_instances import OBJECT_PROMPTS @@ -12,7 +13,7 @@ from syncai_bev3d.teachers.scene_objects import run_camera from syncai_hydranet.utils.device import pick_device -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) def main(): diff --git a/tools/commissioning/scale_rulers.py b/tools/commissioning/scale_rulers.py index 27a1ab2..ff9698e 100644 --- a/tools/commissioning/scale_rulers.py +++ b/tools/commissioning/scale_rulers.py @@ -19,6 +19,7 @@ import argparse import datetime as dt import json +import os import shutil import subprocess import sys @@ -29,7 +30,7 @@ from syncai_bev3d import floor_axis, rulers, scene_mesh from syncai_hydranet.geometry.ground import pixel_to_ground, undistort_points -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) VERDICTS = ROOT / "runs/commission_review/scale" @@ -85,7 +86,8 @@ def verdicts(cameras: list[str]) -> dict[str, rulers.Verdict]: r = read[c] rs = [ rulers.person_ruler(r["calib"]), - rulers.tile_ruler(r["period"], tile), + # the catalogue size is the cameras' consensus, not an independent fact + rulers.tile_ruler(r["period"], tile, anchored=False), rulers.table_ruler(r["table_h"], tables), ] v = rulers.combine([x for x in rs if x is not None]) diff --git a/tools/commissioning/scene3d.py b/tools/commissioning/scene3d.py index 8a79ab6..d1379be 100644 --- a/tools/commissioning/scene3d.py +++ b/tools/commissioning/scene3d.py @@ -13,6 +13,7 @@ where that projection is exact. """ +import os import sys from pathlib import Path @@ -29,7 +30,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) CLASSES = {1: "floor", 2: "wall", 3: "column", 4: "display_table", 5: "display_shelf"} PALETTE = np.array( [ diff --git a/tools/commissioning/scene_mesh.py b/tools/commissioning/scene_mesh.py index f3c5713..2efa285 100644 --- a/tools/commissioning/scene_mesh.py +++ b/tools/commissioning/scene_mesh.py @@ -11,6 +11,7 @@ import argparse import json +import os import shutil import tempfile from pathlib import Path @@ -29,7 +30,7 @@ render, ) -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) def export_glb(camera, items, *, out=None): diff --git a/tools/commissioning/scene_overlay.py b/tools/commissioning/scene_overlay.py index 300510c..e3f81ce 100644 --- a/tools/commissioning/scene_overlay.py +++ b/tools/commissioning/scene_overlay.py @@ -12,6 +12,7 @@ """ import argparse +import os from pathlib import Path import numpy as np @@ -25,7 +26,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) def project(verts: np.ndarray, cf: CameraFile, scale: float) -> tuple[np.ndarray, np.ndarray]: diff --git a/tools/commissioning/service_zones.py b/tools/commissioning/service_zones.py index 567f067..fbfaeff 100644 --- a/tools/commissioning/service_zones.py +++ b/tools/commissioning/service_zones.py @@ -52,6 +52,7 @@ import argparse import dataclasses import json +import os from pathlib import Path import numpy as np @@ -71,7 +72,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) COMMISSIONED = ROOT / "runs/commission01" OUT = ROOT / "runs/service_zones01" diff --git a/tools/commissioning/social_card.py b/tools/commissioning/social_card.py index f2d9ef1..a055bf3 100644 --- a/tools/commissioning/social_card.py +++ b/tools/commissioning/social_card.py @@ -21,6 +21,7 @@ from __future__ import annotations import argparse +import os from pathlib import Path import numpy as np @@ -32,7 +33,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) CARD = (1280, 640) PANEL_W = 660 # the scene half; the text needs the rest at thumbnail size CAPTION_H = 62 # `scene_mesh.render`'s own caption band, rendered then cut diff --git a/tools/commissioning/stage0_baseline.py b/tools/commissioning/stage0_baseline.py index 2aea32b..da2e531 100644 --- a/tools/commissioning/stage0_baseline.py +++ b/tools/commissioning/stage0_baseline.py @@ -62,25 +62,27 @@ def freeze(root, out, cameras): shutil.copytree( root / "src", snapshot / "src", ignore=shutil.ignore_patterns("__pycache__", "*.pyc") ) - tool = Path("tools/commissioning/scene_mesh.py") - (snapshot / tool).parent.mkdir(parents=True) - shutil.copy2(root / tool, snapshot / tool) - shutil.copy2(Path(__file__), snapshot / "tools/commissioning/stage0_baseline.py") - for name in ("pyproject.toml", "uv.lock"): - shutil.copy2(root / name, snapshot / name) + runner = Path("tools/commissioning/stage0_baseline.py") + (snapshot / runner).parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(Path(__file__), snapshot / runner) inputs = {camera: capture_inputs(root, camera) for camera in cameras} + # Everything capture_inputs identifies travels with the snapshot -- the inputs and + # the code alike -- so the two identity records can only differ when a file changed + # under the copy, never because this list and that one drifted apart. for record in inputs.values(): - for name, digest in record["inputs"].items(): - if digest is None: - continue - relative = Path(name) - if relative.is_absolute() or ".." in relative.parts: - raise ValueError(f"snapshot requires checkout-relative input: {name}") - target = snapshot / relative - target.parent.mkdir(parents=True, exist_ok=True) - shutil.copy2(root / relative, target) - if sha256(target) != digest: - raise RuntimeError(f"input changed while copying: {name}") + for section in ("inputs", "code"): + for name, digest in record[section].items(): + if digest is None: + continue + relative = Path(name) + if relative.is_absolute() or ".." in relative.parts: + raise ValueError(f"snapshot requires checkout-relative input: {name}") + target = snapshot / relative + if not target.exists(): + target.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(root / relative, target) + if sha256(target) != digest: + raise RuntimeError(f"input changed while copying: {name}") for camera in cameras: if capture_inputs(snapshot, camera) != inputs[camera]: raise RuntimeError(f"snapshot is incomplete or source changed: {camera}") diff --git a/tools/commissioning/stage0_commission.py b/tools/commissioning/stage0_commission.py new file mode 100644 index 0000000..0c3eeda --- /dev/null +++ b/tools/commissioning/stage0_commission.py @@ -0,0 +1,263 @@ +"""Stage 0 for one camera from one clip: the README chain as one command. + + uv run python tools/commissioning/stage0_commission.py --clip \\ + [--note "nvr 10.12.31.9 ch12"] [--allow-bootstrap] \\ + [--freeze-out runs/stage0_] [--dry-run] + +Until 2026-09-16 the chain for a second site lived in a shell script under `runs/` of a +copied checkout (FTI, `batch_one.sh`), with the bootstrap rule as inline Python nobody +could test. The steps here are the README's, in the README's order, against the tree +`SYNCAI_ROOT` names: static plate, person boxes, onboard, zones, the camera.json +contract, the review bundle, masks, extras, depth completion, the three scene renders, +and finally `stage0_baseline.py`, which freezes the result as a review candidate. + +`--allow-bootstrap` is the only decision the tool takes on its own, and it says so in +the calib: a camera with no person corpus gets scale 1.0 on DA-V2's raw metres, flagged +`scale_bootstrap_dav2_raw_unverified`, so the tile ruler has a walkable mask to read +through. Nothing downstream may present those metres as measured. +""" + +from __future__ import annotations + +import argparse +import datetime as _dt +import json +import os +import re +import subprocess +import sys +from pathlib import Path + +from syncai_hydranet.paths import repo_root + +ROOT = repo_root(Path(__file__).resolve().parents[2]) +SLOT = re.compile(r"_(\d{8}-\d{6})_") +BOOTSTRAP_FLAG = "scale_bootstrap_dav2_raw_unverified" + + +def slot_from_clip(clip: Path) -> str: + """The `_YYYYMMDD-HHMMSS_` slot in an archive name, else the file's mtime in UTC.""" + m = SLOT.search(clip.name) + if m: + return m.group(1) + stamp = _dt.datetime.fromtimestamp(clip.stat().st_mtime, tz=_dt.UTC) + return stamp.strftime("%Y%m%d-%H%M%S") + + +def register_camera(cameras_json: Path, camera: str, note: str) -> None: + site = json.loads(cameras_json.read_text()) + site["cameras"].setdefault(camera, {"role": "factory", "note": note}) + cameras_json.write_text(json.dumps(site, indent=1, ensure_ascii=False) + "\n") + + +def bootstrap_unmeasured(calib_path: Path, allow: bool) -> str: + """Return the calib's scale source; substitute the bootstrap when there is none. + + No floor plane at any vfov is a refusal (exit 2): the view shows too little floor to + commission. An unmeasured scale with `allow` False is also a refusal (exit 3): the + caller has to say it wants raw metres.""" + d = json.loads(calib_path.read_text()) + if d.get("height_dav2_raw_m") is None or d.get("pitch_deg") is None: + raise SystemExit(2) + source = str(d.get("scale_source", "unmeasured")) + if not source.startswith("unmeasured"): + return source + if not allow: + raise SystemExit(3) + d.update( + { + "height_m": d["height_dav2_raw_m"], + "scale": 1.0, + "height_source": "dav2_plane_height_raw_BOOTSTRAP_unverified", + "scale_source": "dav2_metric_indoor_raw_bootstrap_unverified", + } + ) + d["flags"] = [f for f in d.get("flags", []) if f != "scale_unmeasured"] + [BOOTSTRAP_FLAG] + calib_path.write_text(json.dumps(d, indent=1) + "\n") + return d["scale_source"] + + +def plan( + root: Path, + camera: str, + clips_root: Path, + plates_root: Path, + freeze_out: Path | None, + slot: str = "", +): + """The chain's commands, each relative to `root`, in the README's order. + + The review bundle refuses to overwrite, so a rerun writes beside the last one, named + by the clip's slot rather than deleting what a person may have looked at.""" + py = sys.executable + anns = f"datasets/fti_person_gdino/{camera}/annotations/instances_all.json" + steps = [ + ( + "plate", + [ + py, + "scripts/static_plates.py", + "--root", + str(clips_root), + "--out", + str(plates_root), + "--only", + camera, + ], + ), + ( + "person boxes", + [ + py, + "scripts/gdino_person_boxes.py", + "--out", + f"datasets/fti_person_gdino/{camera}", + "--frames", + "24", + "--train-thr", + "0.35", + "--clips", + *sorted(str(p) for p in (root / clips_root / camera).glob("archive_*.mp4")), + ], + ), + ( + "onboard", + [ + py, + "scripts/onboard_camera.py", + "--camera", + camera, + "--out", + "runs/onboard01", + "--plates-root", + str(plates_root), + "--cameras-json", + str(clips_root / "cameras.json"), + "--person-anns", + anns, + "--skip-person-frac", + ], + ), + ( + "zones", + [ + py, + "scripts/propose_zones.py", + "--cameras", + camera, + "--calib-dir", + "runs/onboard01", + "--out", + "runs/zones01", + ], + ), + ( + "review bundle", + [ + py, + "tools/commissioning/commission_camera.py", + f"runs/onboard01/{camera}.calib.json", + "--out", + f"runs/commission_review/{camera}_{slot or 'batch'}", + ], + ), + ( + "masks", + [ + py, + "tools/commissioning/masks_pass.py", + camera, + "--plates-root", + str(plates_root), + "--out-root", + "runs/commission01", + "--calib-root", + "runs/onboard01", + ], + ), + ("extras", [py, "tools/commissioning/extras_pass.py", camera]), + ("depth completion", [py, "tools/commissioning/depth_complete.py", camera]), + ("scene3d", [py, "tools/commissioning/scene3d.py", camera]), + ("scene overlay", [py, "tools/commissioning/scene_overlay.py", camera]), + ("scene mesh", [py, "tools/commissioning/scene_mesh.py", camera]), + ] + if freeze_out is not None: + steps.append( + ( + "freeze", + [ + py, + "tools/commissioning/stage0_baseline.py", + "--root", + str(root), + "--out", + str(freeze_out), + camera, + ], + ) + ) + return steps + + +def main(argv=None): + ap = argparse.ArgumentParser( + description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter + ) + ap.add_argument("camera") + ap.add_argument("--clip", type=Path, required=True, help="one recorded clip of the camera") + ap.add_argument("--note", default="", help="where the stream came from, for cameras.json") + ap.add_argument("--clips-root", type=Path, default=Path("datasets/fti_clips")) + ap.add_argument("--plates-root", type=Path, default=Path("datasets/fti_static")) + ap.add_argument( + "--allow-bootstrap", + action="store_true", + help="scale 1.0 on raw DA-V2 metres when no person box; flagged in the calib", + ) + ap.add_argument( + "--freeze-out", + type=Path, + help="stage0_baseline output directory; omit to stop at the renders", + ) + ap.add_argument("--dry-run", action="store_true", help="print the commands and stop") + args = ap.parse_args(argv) + root = ROOT + clips_root, plates_root = args.clips_root, args.plates_root + clip = args.clip.resolve() + if not clip.is_file(): + ap.error(f"clip not found: {clip}") + link_dir = root / clips_root / args.camera + slot = slot_from_clip(clip) + link = link_dir / f"archive_{slot}_fti.mp4" + if args.dry_run: + print(f"link {link} -> {clip}") + for label, argv_ in plan( + root, args.camera, clips_root, plates_root, args.freeze_out, slot + ): + print(f"[{label}] {' '.join(argv_)}") + return 0 + link_dir.mkdir(parents=True, exist_ok=True) + if link.is_symlink() or link.exists(): + link.unlink() + link.symlink_to(clip) + register_camera(root / clips_root / "cameras.json", args.camera, args.note) + env = dict(os.environ, SYNCAI_ROOT=str(root), PYTHONPATH=str(root / "src")) + for label, argv_ in plan(root, args.camera, clips_root, plates_root, args.freeze_out, slot): + print(f"== {args.camera} {label}", flush=True) + subprocess.run(["nice", "-n", "10", *argv_], cwd=root, env=env, check=True) + if label == "onboard": + calib = root / f"runs/onboard01/{args.camera}.calib.json" + source = bootstrap_unmeasured(calib, args.allow_bootstrap) + print(f" scale source: {source}", flush=True) + if label == "zones": + from syncai_bev3d.commissioning import from_onboard_calib + + (root / "runs/commission01").mkdir(parents=True, exist_ok=True) + from_onboard_calib(root / f"runs/onboard01/{args.camera}.calib.json").save( + root / f"runs/commission01/{args.camera}.camera.json" + ) + print(f"STAGE0_DONE {args.camera}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/commissioning/stage0_train.py b/tools/commissioning/stage0_train.py index ccdcfe1..7cec76b 100644 --- a/tools/commissioning/stage0_train.py +++ b/tools/commissioning/stage0_train.py @@ -37,7 +37,7 @@ from syncai_hydranet.data.label_maps import get_scheme from syncai_hydranet.data.store_split import STORES, camera_store, fold_split, validate_fold -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) SOURCES = { "retail_objects_batch02": "retail_surfaces_from_objects", "retail_objects_batch03": "retail_surfaces_from_objects", diff --git a/tools/commissioning/tile_ruler.py b/tools/commissioning/tile_ruler.py index cc04fdf..97010e6 100644 --- a/tools/commissioning/tile_ruler.py +++ b/tools/commissioning/tile_ruler.py @@ -21,6 +21,7 @@ """ import argparse +import os from pathlib import Path import numpy as np @@ -29,7 +30,7 @@ from syncai_bev3d.rulers import write_scaled_root from syncai_hydranet.geometry.ground import pixel_to_ground, undistort_points -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) # Below this autocorrelation the pitch is not trusted to rescale anything. STRENGTH_MIN = 0.35 diff --git a/tools/commissioning/zone_draw.py b/tools/commissioning/zone_draw.py index e771c6c..27777bb 100644 --- a/tools/commissioning/zone_draw.py +++ b/tools/commissioning/zone_draw.py @@ -51,6 +51,7 @@ import base64 import dataclasses import json +import os from pathlib import Path import numpy as np @@ -61,7 +62,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) COMMISSIONED = ROOT / "runs/commission01" OUT = ROOT / "runs/zone_draw01" DRAWABLE_KINDS = sorted(ZONE_KINDS - {"walkable"}) diff --git a/tools/commissioning/zones_apply.py b/tools/commissioning/zones_apply.py index de5a130..7ea2c8b 100644 --- a/tools/commissioning/zones_apply.py +++ b/tools/commissioning/zones_apply.py @@ -23,6 +23,7 @@ import dataclasses import json +import os import sys from pathlib import Path @@ -35,7 +36,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) ZONES_HUMAN = ROOT / "tools/site30k/zones.json" ZONES_PROPOSED = ROOT / "runs/zones01" CLASS_FILES = { diff --git a/tools/commissioning/zones_confirm.py b/tools/commissioning/zones_confirm.py index 49dac87..ee98bb3 100644 --- a/tools/commissioning/zones_confirm.py +++ b/tools/commissioning/zones_confirm.py @@ -62,6 +62,7 @@ import argparse import dataclasses import json +import os from pathlib import Path import numpy as np @@ -74,7 +75,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) PROPOSED = ROOT / "runs/zones01" COMMISSIONED = ROOT / "runs/commission01" OUT = ROOT / "runs/zones_confirm01" diff --git a/tools/site30k/boxes.py b/tools/site30k/boxes.py index cd4915a..0bf3b40 100644 --- a/tools/site30k/boxes.py +++ b/tools/site30k/boxes.py @@ -19,6 +19,7 @@ import importlib.util import json +import os import sys from pathlib import Path @@ -30,7 +31,7 @@ # machine but the one it was typed on, and `sys.path` is the one place where that # fails before anything else can report it. `parents[2]` is the repo root -- # tools/site30k/.py. -_REPO = Path(__file__).resolve().parents[2] +_REPO = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) sys.path.insert(0, str(_REPO / "scripts")) spec = importlib.util.spec_from_file_location( "campaign", str(_REPO / "scripts/campaign_site30k.py") diff --git a/tools/site30k/plan.py b/tools/site30k/plan.py index e08cdf9..e34124c 100644 --- a/tools/site30k/plan.py +++ b/tools/site30k/plan.py @@ -33,6 +33,7 @@ import argparse import json +import os import re from collections import defaultdict from concurrent.futures import ThreadPoolExecutor @@ -48,7 +49,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) PULL = ROOT / "datasets/studioa_pull_site30k" CALIB = ROOT / "runs/onboard01" # Filenames are UTC; the store is UTC+8. 23:00-06:00 local is the night tranche. diff --git a/tools/site30k/recipe.py b/tools/site30k/recipe.py index 5a2b53d..1113517 100644 --- a/tools/site30k/recipe.py +++ b/tools/site30k/recipe.py @@ -55,7 +55,7 @@ # machine but the one it was typed on, and `sys.path` is the one place where that # fails before anything else can report it. `parents[2]` is the repo root -- # tools/site30k/.py. -_REPO = Path(__file__).resolve().parents[2] +_REPO = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) sys.path.insert(0, str(_REPO / "scripts")) import importlib.util # noqa: E402 @@ -82,7 +82,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) # The pilot ran off datasets/studioa_clips; the campaign runs off the phase-2 pull. Both # are the same layout (//.mp4), so the root is a setting rather than # a fork in the code, and the plates follow it so a campaign plate never lands in the diff --git a/tools/site30k/run_campaign.py b/tools/site30k/run_campaign.py index 0e16fc4..d59e2b7 100644 --- a/tools/site30k/run_campaign.py +++ b/tools/site30k/run_campaign.py @@ -31,7 +31,7 @@ # as an absolute path, so a second checkout ran against the first one's `runs/` and # any machine but this one failed at import with a path and no reason. Two levels up # from `tools//.py`, and `tests/test_no_absolute_sys_path.py` keeps it so. -ROOT = Path(__file__).resolve().parents[2] +ROOT = Path(os.environ.get("SYNCAI_ROOT", Path(__file__).resolve().parents[2])) RECIPE = ROOT / "tools/site30k/recipe.py" PYTHON = ROOT / ".venv/bin/python" PULL = "datasets/studioa_pull_site30k"