From 856c6d6562efe5bba917853679e229b2f4c3760f Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 20:19:57 +0000 Subject: [PATCH 01/32] Bump version to 3.0.1 Branch-opening commit for the 3.0.1 integration target. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011hqxFfwPYBmRRqt4rroHKi --- VERSION.TXT | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VERSION.TXT b/VERSION.TXT index 4a36342f..cb2b00e4 100644 --- a/VERSION.TXT +++ b/VERSION.TXT @@ -1 +1 @@ -3.0.0 +3.0.1 From 95e94c134c8b337df3bc96a142eb1cd4ef5bd514 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 4 Nov 2026 20:23:58 +0000 Subject: [PATCH 02/32] Bring the guided tour up to date with 3.0.0 The Enhancements: Categories step now names the categories as the checkboxes show them. Config: Appearance mentions the font size, and Config: Tools mentions Restore user.ini, Preview Apply and Check for Updates. Two new steps follow Tools: Settings Backup (Export / Import Settings) and Star Citizen Tools (Clear Shader Cache). The tour is now 21 steps. AI translations refreshed in all eleven languages, keeping each file's own words for theme names, font sizes and overrides. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011hqxFfwPYBmRRqt4rroHKi --- assets/tutorial.json | 20 +++++++-- languages/TRANSLATIONS.md | 12 ++++++ languages/chinese/ui.json | 40 +++++++++++++----- languages/chinese_traditional/ui.json | 40 +++++++++++++----- languages/french/ui.json | 44 ++++++++++++++------ languages/german/ui.json | 40 +++++++++++++----- languages/italian/ui.json | 40 +++++++++++++----- languages/japanese/ui.json | 40 +++++++++++++----- languages/korean/ui.json | 40 +++++++++++++----- languages/portuguese_br/ui.json | 44 ++++++++++++++------ languages/russian/ui.json | 28 +++++++++++-- languages/spanish/ui.json | 40 +++++++++++++----- languages/turkish/ui.json | 40 +++++++++++++----- src/gui/main_window.py | 2 + tests/CLAUDE.md | 3 +- tests/test_tutorial_translations.py | 58 +++++++++++++++++++++++++++ 16 files changed, 418 insertions(+), 113 deletions(-) create mode 100644 tests/test_tutorial_translations.py diff --git a/assets/tutorial.json b/assets/tutorial.json index cc7a7674..81c550ab 100644 --- a/assets/tutorial.json +++ b/assets/tutorial.json @@ -22,7 +22,7 @@ { "id": "enh_categories", "title": "3. Enhancements: Categories", - "description": "Toggle which enhancement categories run when you Generate. Ships, components, ship weapons, FPS weapons, missions, mission rewards, commodity crafting, and medical consumables can each be enabled independently.", + "description": "Toggle which enhancement categories run when you Generate: Ships, Ship Items, Gear, Missions, Commodities, Journal, Medical Consumables. Ship Items covers components, ship weapons and missiles; Gear covers FPS weapons and armor tags. Each category can be turned on or off on its own.", "preferred_side": "right" }, { @@ -88,7 +88,7 @@ { "id": "cfg_appearance", "title": "14. Config: Appearance", - "description": "Pick the app theme (Default / Light / Dark / ODW) and optionally disable the auto-tour on new versions.", + "description": "Pick the app theme (Default / Light / Dark / ODW) and the font size (Small / Medium / Large, applied after a restart), and optionally turn off the auto-tour on new versions.", "preferred_side": "right" }, { @@ -112,12 +112,24 @@ { "id": "cfg_tools", "title": "18. Config: Tools", - "description": "Import an external INI into your overrides via the conflict-resolution dialog, or reset your user.ini for the active channel (auto-backed-up before clearing).", + "description": "Import an external INI into your overrides through the conflict-resolution dialog. Reset your user.ini for the active channel (backed up first), or bring back an earlier copy with Restore user.ini. Preview Apply shows what Apply would write, and Check for Updates looks for a new version.", + "preferred_side": "right" + }, + { + "id": "cfg_backup", + "title": "19. Config: Settings Backup", + "description": "Export Settings saves your preferences, tag setups, Star Citizen folder and custom strings for every channel into one file. Import Settings brings them back, on a new PC or after a fresh install.", + "preferred_side": "right" + }, + { + "id": "cfg_sc_tools", + "title": "20. Config: Star Citizen Tools", + "description": "Clear Shader Cache lists the game's shader cache folders, one per game build. Clean deletes the shaders and keeps your graphics settings; Remove deletes the whole folder. Close Star Citizen first.", "preferred_side": "right" }, { "id": "help", - "title": "19. Help & feedback", + "title": "21. Help & feedback", "description": "The Help button opens a side-panel with the full guide, a Known Issues list, and a feedback link to the Smart Citizen Discord channel. That's everything you need to get started — good luck, citizen.", "preferred_side": "below" } diff --git a/languages/TRANSLATIONS.md b/languages/TRANSLATIONS.md index dc45501e..8a958a73 100644 --- a/languages/TRANSLATIONS.md +++ b/languages/TRANSLATIONS.md @@ -390,6 +390,18 @@ them against the new source. cihazından" (the pack uses "üretici" for a manufacturer) and "mevcut listeye", and italian says "la loro ricezione" instead of "di averli ricevuti", which made the game the receiver. +- **3.0.1 cycle (2026-11-04, Claude Opus 5.5):** the guided tour caught up + with 3.0.0. `tutorial.enh_categories`, `tutorial.cfg_appearance` and + `tutorial.cfg_tools` were reworded in `assets/tutorial.json` (the category + names as the checkboxes show them, which stay English; the font size; Restore + user.ini, Preview Apply and Check for Updates), and two steps were added + after Tools: `tutorial.cfg_backup` (Export / Import Settings) and + `tutorial.cfg_sc_tools` (Clear Shader Cache). The tour is now 21 steps, so + every title from 19 on was renumbered. AI `at` written or refreshed in all + eleven languages, none of which has a human `ht` for these keys; each + keeps its file's own words for the theme names, the font sizes and + overrides, and the new titles are built from its `config.backup_group` and + `config.sc_tools_group`. - **3.0.0 cycle (2026-10-15, Claude Opus 5.5):** #492 added `enhancements.tag_mapping_tooltip_weight` (the Edit mapping tooltip on the Tag Builder's Armor > Weight row) and reworded `enhancements.cat_desc_gear`. diff --git a/languages/chinese/ui.json b/languages/chinese/ui.json index 918b3d6b..0d4d7bec 100644 --- a/languages/chinese/ui.json +++ b/languages/chinese/ui.json @@ -3123,7 +3123,7 @@ }, "description": { "ht": "", - "at": "切换生成时运行的增强内容分类。飞船、组件、飞船武器、FPS 武器、任务、任务奖励、商品制作和医疗消耗品均可独立启用。" + "at": "选择点击生成时运行哪些增强类别:Ships, Ship Items, Gear, Missions, Commodities, Journal, Medical Consumables。Ship Items 包含组件、舰船武器和导弹;Gear 包含 FPS 武器和护甲标签。每个类别都可以单独开启或关闭。" } }, "enh_favorites": { @@ -3156,24 +3156,24 @@ "at": "构建添加到组件、导弹、飞船武器、商品和护甲上的括号名称标签。用 ▲/▼ 重新排序元素,取消勾选可排除,更改缩写长度,选择分隔符和括号,或将标签放在名称之后而不是之前。" } }, - "overlay": { + "blueprint_tracker": { "title": { "ht": "", - "at": "8. Smart Citizen 叠加层" + "at": "7. 蓝图追踪器" }, "description": { "ht": "", - "at": "在这里或用工具栏的“叠加层”按钮启动叠加层。它在游戏上方解码采矿扫描信号,打开一个抽屉来搜索你的蓝图、矿物和制作,并把你获得的蓝图标记为已拥有。请在主屏幕上以无边框窗口模式运行游戏。" + "at": "标记你拥有的蓝图。可按名称或任务搜索左侧列表,或按组件类型、等级、尺寸和品级缩小范围,然后用箭头将物品移动到你的已拥有列表。已拥有的蓝图在任务的潜在奖励中出现时会带有 [Owned] 标签,让你一眼就能看出哪些值得追求。" } }, - "blueprint_tracker": { + "overlay": { "title": { "ht": "", - "at": "7. 蓝图追踪器" + "at": "8. Smart Citizen 叠加层" }, "description": { "ht": "", - "at": "标记你拥有的蓝图。可按名称或任务搜索左侧列表,或按组件类型、等级、尺寸和品级缩小范围,然后用箭头将物品移动到你的已拥有列表。已拥有的蓝图在任务的潜在奖励中出现时会带有 [Owned] 标签,让你一眼就能看出哪些值得追求。" + "at": "在这里或用工具栏的“叠加层”按钮启动叠加层。它在游戏上方解码采矿扫描信号,打开一个抽屉来搜索你的蓝图、矿物和制作,并把你获得的蓝图标记为已拥有。请在主屏幕上以无边框窗口模式运行游戏。" } }, "edit": { @@ -3233,7 +3233,7 @@ }, "description": { "ht": "", - "at": "选择应用主题(默认 / 浅色 / 深色 / ODW),并可选择禁用新版本的自动教程。" + "at": "选择应用主题(默认 / 浅色 / 深色 / ODW)和字体大小(小 / 中 / 大,重启后生效),并可选择在新版本时关闭自动教程。" } }, "cfg_sc_install": { @@ -3273,13 +3273,33 @@ }, "description": { "ht": "", - "at": "通过冲突解决对话框将外部 INI 导入到你的覆盖项中,或重置当前频道的 user.ini(清除前会自动备份)。" + "at": "通过冲突处理对话框将外部 INI 导入你的覆盖项。重置当前频道的 user.ini(会先备份),或用“恢复 user.ini”找回之前的副本。预览会显示“应用”将写入的内容,“检查更新”会查找新版本。" + } + }, + "cfg_backup": { + "title": { + "ht": "", + "at": "19. 配置:设置备份" + }, + "description": { + "ht": "", + "at": "“导出设置”会把你的偏好、标签配置、Star Citizen 文件夹和所有频道的自定义字符串保存到一个文件中。“导入设置”可在新电脑上或全新安装后恢复它们。" + } + }, + "cfg_sc_tools": { + "title": { + "ht": "", + "at": "20. 配置:Star Citizen 工具" + }, + "description": { + "ht": "", + "at": "“清除着色器缓存”会列出游戏的着色器缓存文件夹,每个游戏版本一个。“清理”删除着色器并保留你的图形设置;“移除”删除整个文件夹。请先关闭 Star Citizen。" } }, "help": { "title": { "ht": "", - "at": "19. 帮助与反馈" + "at": "21. 帮助与反馈" }, "description": { "ht": "", diff --git a/languages/chinese_traditional/ui.json b/languages/chinese_traditional/ui.json index 83286d55..79645ac8 100644 --- a/languages/chinese_traditional/ui.json +++ b/languages/chinese_traditional/ui.json @@ -3123,7 +3123,7 @@ }, "description": { "ht": "", - "at": "切換生成時執行的增強內容分類。飛船、元件、飛船武器、FPS 武器、任務、任務獎勵、商品製作和醫療消耗品均可獨立啟用。" + "at": "選擇點擊產生時執行哪些增強類別:Ships, Ship Items, Gear, Missions, Commodities, Journal, Medical Consumables。Ship Items 包含元件、艦船武器和飛彈;Gear 包含 FPS 武器和護甲標籤。每個類別都可以單獨開啟或關閉。" } }, "enh_favorites": { @@ -3156,24 +3156,24 @@ "at": "構建新增到元件、導彈、飛船武器、商品和護甲上的括號名稱標籤。用 ▲/▼ 重新排序元素,取消勾選可排除,更改縮寫長度,選擇分隔符和括號,或將標籤放在名稱之後而不是之前。" } }, - "overlay": { + "blueprint_tracker": { "title": { "ht": "", - "at": "8. Smart Citizen 疊加層" + "at": "7. 藍圖追蹤器" }, "description": { "ht": "", - "at": "在這裡或用工具列的「疊加層」按鈕啟動疊加層。它在遊戲上方解碼採礦掃描訊號,開啟一個抽屜來搜尋你的藍圖、礦物和製作,並把你獲得的藍圖標記為已擁有。請在主螢幕上以無邊框視窗模式執行遊戲。" + "at": "標記你擁有的藍圖。可按名稱或任務搜尋左側列表,或按元件型別、等級、尺寸和品級縮小範圍,然後用箭頭將物品移動到你的已擁有列表。已擁有的藍圖在任務的潛在獎勵中出現時會帶有 [Owned] 標籤,讓你一眼就能看出哪些值得追求。" } }, - "blueprint_tracker": { + "overlay": { "title": { "ht": "", - "at": "7. 藍圖追蹤器" + "at": "8. Smart Citizen 疊加層" }, "description": { "ht": "", - "at": "標記你擁有的藍圖。可按名稱或任務搜尋左側列表,或按元件型別、等級、尺寸和品級縮小範圍,然後用箭頭將物品移動到你的已擁有列表。已擁有的藍圖在任務的潛在獎勵中出現時會帶有 [Owned] 標籤,讓你一眼就能看出哪些值得追求。" + "at": "在這裡或用工具列的「疊加層」按鈕啟動疊加層。它在遊戲上方解碼採礦掃描訊號,開啟一個抽屜來搜尋你的藍圖、礦物和製作,並把你獲得的藍圖標記為已擁有。請在主螢幕上以無邊框視窗模式執行遊戲。" } }, "edit": { @@ -3233,7 +3233,7 @@ }, "description": { "ht": "", - "at": "選擇應用主題(預設 / 淺色 / 深色 / ODW),並可選擇禁用新版本的自動教程。" + "at": "選擇應用程式主題(預設 / 淺色 / 深色 / ODW)和字型大小(小 / 中 / 大,重新啟動後生效),並可選擇在新版本時關閉自動教程。" } }, "cfg_sc_install": { @@ -3273,13 +3273,33 @@ }, "description": { "ht": "", - "at": "透過沖突解決對話方塊將外部 INI 匯入到你的覆蓋項中,或重置當前頻道的 user.ini(清除前會自動備份)。" + "at": "透過衝突處理對話框將外部 INI 匯入你的覆蓋項。重設目前頻道的 user.ini(會先備份),或用「恢復 user.ini」找回之前的副本。預覽會顯示「套用」將寫入的內容,「檢查更新」會尋找新版本。" + } + }, + "cfg_backup": { + "title": { + "ht": "", + "at": "19. 配置:設定備份" + }, + "description": { + "ht": "", + "at": "「匯出設定」會把你的偏好、標籤設定、Star Citizen 資料夾和所有頻道的自訂字串儲存到一個檔案中。「匯入設定」可在新電腦上或全新安裝後恢復它們。" + } + }, + "cfg_sc_tools": { + "title": { + "ht": "", + "at": "20. 配置:Star Citizen 工具" + }, + "description": { + "ht": "", + "at": "「清除著色器快取」會列出遊戲的著色器快取資料夾,每個遊戲版本一個。「清理」刪除著色器並保留你的圖形設定;「移除」刪除整個資料夾。請先關閉 Star Citizen。" } }, "help": { "title": { "ht": "", - "at": "19. 幫助與反饋" + "at": "21. 幫助與反饋" }, "description": { "ht": "", diff --git a/languages/french/ui.json b/languages/french/ui.json index 9e4e31c7..3f2f1169 100644 --- a/languages/french/ui.json +++ b/languages/french/ui.json @@ -3123,7 +3123,7 @@ }, "description": { "ht": "", - "at": "Choisissez les catégories d'enrichissements exécutées lors de la génération. Vaisseaux, composants, armes de vaisseau, armes FPS, missions, récompenses de mission, fabrication de marchandises et consommables médicaux peuvent être activés indépendamment." + "at": "Choisissez les catégories d'enrichissements exécutées lors de la génération : Ships, Ship Items, Gear, Missions, Commodities, Journal, Medical Consumables. Ship Items couvre les composants, les armes de vaisseau et les missiles ; Gear couvre les armes FPS et les étiquettes d'armure. Chaque catégorie s'active ou se désactive séparément." } }, "enh_favorites": { @@ -3156,6 +3156,26 @@ "at": "Composez les balises entre crochets ajoutées aux noms des composants, missiles, armes de vaisseau, marchandises et armures. Réordonnez les éléments avec ▲/▼, décochez pour exclure, changez la longueur des abréviations, choisissez le séparateur et les crochets, ou placez la balise après le nom plutôt qu'avant." } }, + "blueprint_tracker": { + "title": { + "ht": "", + "at": "7. Suivi des plans" + }, + "description": { + "ht": "", + "at": "Marquez les plans que vous possédez. Cherchez dans la liste de gauche par nom ou par mission, ou affinez par Type, Classe, Taille et Grade de composant, puis utilisez les flèches pour déplacer des éléments vers votre liste de possédés. Les plans possédés reçoivent une étiquette [Owned] partout où ils apparaissent dans les récompenses potentielles d'une mission, pour voir d'un coup d'œil ce qui vaut la peine d'être chassé." + } + }, + "overlay": { + "title": { + "ht": "", + "at": "8. Superposition Smart Citizen" + }, + "description": { + "ht": "", + "at": "Démarrez la superposition ici ou avec le bouton Superposition de la barre d'outils. Par-dessus le jeu, elle décode les signatures de scan minier, ouvre un tiroir pour chercher vos plans, minéraux et fabrications, et marque comme possédés les plans que vous obtenez. Lancez le jeu en mode fenêtré sans bordure sur votre écran principal." + } + }, "edit": { "title": { "ht": "", @@ -3213,7 +3233,7 @@ }, "description": { "ht": "", - "at": "Choisissez le thème de l'application (Défaut / Clair / Sombre / ODW) et désactivez si vous le souhaitez la visite automatique à chaque nouvelle version." + "at": "Choisissez le thème de l'application (Défaut / Clair / Sombre / ODW) et la taille du texte (Petite / Moyenne / Grande, appliquée après un redémarrage), et désactivez si besoin la visite automatique lors des nouvelles versions." } }, "cfg_sc_install": { @@ -3253,37 +3273,37 @@ }, "description": { "ht": "", - "at": "Importez un INI externe dans vos modifications via la boîte de dialogue de résolution de conflits, ou réinitialisez votre user.ini pour le canal actif (sauvegardé automatiquement avant l'effacement)." + "at": "Importez un INI externe dans vos modifications via la fenêtre de résolution des conflits. Réinitialisez le user.ini du canal actif (sauvegardé avant) ou récupérez une copie précédente avec Restaurer user.ini. L'aperçu montre ce qu'Appliquer écrirait, et Rechercher des mises à jour vérifie s'il existe une nouvelle version." } }, - "help": { + "cfg_backup": { "title": { "ht": "", - "at": "19. Aide et retours" + "at": "19. Paramètres : Sauvegarde des paramètres" }, "description": { "ht": "", - "at": "Le bouton Aide ouvre un panneau latéral avec le guide complet, la liste des problèmes connus et un lien vers le canal Discord de Smart Citizen pour vos retours. Vous avez tout ce qu'il faut pour démarrer : bonne chance, citoyen." + "at": "Exporter les paramètres enregistre vos préférences, vos réglages d'étiquettes, votre dossier Star Citizen et vos textes personnalisés de tous les canaux dans un seul fichier. Importer les paramètres les restaure, sur un nouveau PC ou après une installation propre." } }, - "overlay": { + "cfg_sc_tools": { "title": { "ht": "", - "at": "8. Superposition Smart Citizen" + "at": "20. Paramètres : Outils Star Citizen" }, "description": { "ht": "", - "at": "Démarrez la superposition ici ou avec le bouton Superposition de la barre d'outils. Par-dessus le jeu, elle décode les signatures de scan minier, ouvre un tiroir pour chercher vos plans, minéraux et fabrications, et marque comme possédés les plans que vous obtenez. Lancez le jeu en mode fenêtré sans bordure sur votre écran principal." + "at": "Vider le cache des shaders liste les dossiers de cache de shaders du jeu, un par version. Nettoyer supprime les shaders et garde vos réglages graphiques ; Supprimer efface le dossier entier. Fermez Star Citizen avant." } }, - "blueprint_tracker": { + "help": { "title": { "ht": "", - "at": "7. Suivi des plans" + "at": "21. Aide et retours" }, "description": { "ht": "", - "at": "Marquez les plans que vous possédez. Cherchez dans la liste de gauche par nom ou par mission, ou affinez par Type, Classe, Taille et Grade de composant, puis utilisez les flèches pour déplacer des éléments vers votre liste de possédés. Les plans possédés reçoivent une étiquette [Owned] partout où ils apparaissent dans les récompenses potentielles d'une mission, pour voir d'un coup d'œil ce qui vaut la peine d'être chassé." + "at": "Le bouton Aide ouvre un panneau latéral avec le guide complet, la liste des problèmes connus et un lien vers le canal Discord de Smart Citizen pour vos retours. Vous avez tout ce qu'il faut pour démarrer : bonne chance, citoyen." } } }, diff --git a/languages/german/ui.json b/languages/german/ui.json index 125cff5e..fee28878 100644 --- a/languages/german/ui.json +++ b/languages/german/ui.json @@ -3123,7 +3123,7 @@ }, "description": { "ht": "", - "at": "Schalte um, welche Erweiterungskategorien beim Erstellen ausgeführt werden. Schiffe, Komponenten, Schiffswaffen, FPS-Waffen, Missionen, Missionsbelohnungen, Rohstoffherstellung und medizinische Verbrauchsgüter können jeweils unabhängig aktiviert werden." + "at": "Schalte um, welche Erweiterungskategorien beim Erstellen laufen: Ships, Ship Items, Gear, Missions, Commodities, Journal, Medical Consumables. Ship Items umfasst Komponenten, Schiffswaffen und Raketen; Gear umfasst FPS-Waffen und Rüstungs-Tags. Jede Kategorie lässt sich einzeln ein- oder ausschalten." } }, "enh_favorites": { @@ -3156,24 +3156,24 @@ "at": "Erstelle die eingeklammerten Namens-Tags für Komponenten, Raketen, Schiffswaffen, Rohstoffe und Rüstungen. Elemente mit ▲/▼ neu anordnen, zum Ausschließen deaktivieren, Abkürzungslänge ändern, Trennzeichen und Klammern wählen oder das Tag statt davor dahinter platzieren." } }, - "overlay": { + "blueprint_tracker": { "title": { "ht": "", - "at": "8. Smart Citizen Overlay" + "at": "7. Bauplan-Tracker" }, "description": { "ht": "", - "at": "Starte das Overlay hier oder mit der Overlay-Schaltfläche in der Werkzeugleiste. Über dem Spiel entschlüsselt es Bergbau-Scan-Signaturen, öffnet eine Schublade zum Durchsuchen deiner Baupläne, Mineralien und Herstellung und markiert Baupläne, die du erhältst, als besessen. Starte das Spiel im randlosen Fenstermodus auf deinem Hauptbildschirm." + "at": "Markiere die Baupläne, die du besitzt. Durchsuche die linke Liste nach Name oder Mission, oder grenze sie nach Komponententyp, Klasse, Größe und Grad ein, und verschiebe dann mit den Pfeilen Gegenstände in deine Besessen-Liste. Besessene Baupläne erhalten ein [Owned]-Tag, wo immer sie in den möglichen Belohnungen einer Mission erscheinen, damit du auf einen Blick siehst, was sich zu verfolgen lohnt." } }, - "blueprint_tracker": { + "overlay": { "title": { "ht": "", - "at": "7. Bauplan-Tracker" + "at": "8. Smart Citizen Overlay" }, "description": { "ht": "", - "at": "Markiere die Baupläne, die du besitzt. Durchsuche die linke Liste nach Name oder Mission, oder grenze sie nach Komponententyp, Klasse, Größe und Grad ein, und verschiebe dann mit den Pfeilen Gegenstände in deine Besessen-Liste. Besessene Baupläne erhalten ein [Owned]-Tag, wo immer sie in den möglichen Belohnungen einer Mission erscheinen, damit du auf einen Blick siehst, was sich zu verfolgen lohnt." + "at": "Starte das Overlay hier oder mit der Overlay-Schaltfläche in der Werkzeugleiste. Über dem Spiel entschlüsselt es Bergbau-Scan-Signaturen, öffnet eine Schublade zum Durchsuchen deiner Baupläne, Mineralien und Herstellung und markiert Baupläne, die du erhältst, als besessen. Starte das Spiel im randlosen Fenstermodus auf deinem Hauptbildschirm." } }, "edit": { @@ -3233,7 +3233,7 @@ }, "description": { "ht": "", - "at": "Wähle das App-Design (Standard / Hell / Dunkel / ODW) und deaktiviere optional die automatische Tour bei neuen Versionen." + "at": "Wähle das App-Design (Standard / Hell / Dunkel / ODW) und die Schriftgröße (Klein / Mittel / Groß, gilt nach einem Neustart) und deaktiviere optional die automatische Tour bei neuen Versionen." } }, "cfg_sc_install": { @@ -3273,13 +3273,33 @@ }, "description": { "ht": "", - "at": "Importiere eine externe INI über den Konfliktlösungsdialog in deine Überschreibungen, oder setze deine user.ini für den aktiven Kanal zurück (wird vor dem Leeren automatisch gesichert)." + "at": "Importiere eine externe INI über den Konfliktdialog in deine Überschreibungen. Setze die user.ini des aktiven Kanals zurück (vorher gesichert) oder hole mit user.ini wiederherstellen eine frühere Kopie zurück. Die Vorschau zeigt, was Anwenden schreiben würde, und Nach Updates suchen prüft auf eine neue Version." + } + }, + "cfg_backup": { + "title": { + "ht": "", + "at": "19. Konfiguration: Einstellungssicherung" + }, + "description": { + "ht": "", + "at": "Einstellungen exportieren speichert deine Einstellungen, Tag-Konfigurationen, deinen Star Citizen-Ordner und deine eigenen Texte aller Kanäle in einer Datei. Einstellungen importieren holt sie zurück, auf einem neuen PC oder nach einer Neuinstallation." + } + }, + "cfg_sc_tools": { + "title": { + "ht": "", + "at": "20. Konfiguration: Star Citizen-Werkzeuge" + }, + "description": { + "ht": "", + "at": "Shader-Cache leeren listet die Shader-Cache-Ordner des Spiels auf, einen pro Spielversion. Bereinigen löscht die Shader und behält deine Grafikeinstellungen; Entfernen löscht den ganzen Ordner. Schließe Star Citizen vorher." } }, "help": { "title": { "ht": "", - "at": "19. Hilfe & Feedback" + "at": "21. Hilfe & Feedback" }, "description": { "ht": "", diff --git a/languages/italian/ui.json b/languages/italian/ui.json index 5bb04f1c..c16d6c00 100644 --- a/languages/italian/ui.json +++ b/languages/italian/ui.json @@ -3123,7 +3123,7 @@ }, "description": { "ht": "", - "at": "Attiva o disattiva quali categorie di miglioramenti vengono eseguite quando generi. Navi, componenti, armi delle navi, armi FPS, missioni, ricompense delle missioni, fabbricazione delle merci e beni di consumo medici possono essere abilitati ciascuno in modo indipendente." + "at": "Scegli quali categorie di miglioramenti vengono eseguite con Genera: Ships, Ship Items, Gear, Missions, Commodities, Journal, Medical Consumables. Ship Items comprende componenti, armi di nave e missili; Gear comprende armi FPS e tag delle armature. Ogni categoria si attiva o disattiva separatamente." } }, "enh_favorites": { @@ -3156,24 +3156,24 @@ "at": "Crea i tag tra parentesi aggiunti ai nomi di componenti, missili, armi delle navi, merci e armature. Riordina gli elementi con ▲/▼, deseleziona per escludere, cambia la lunghezza dell'abbreviazione, scegli il separatore e le parentesi, oppure posiziona il tag dopo il nome invece che prima." } }, - "overlay": { + "blueprint_tracker": { "title": { "ht": "", - "at": "8. Overlay di Smart Citizen" + "at": "7. Blueprint Tracker" }, "description": { "ht": "", - "at": "Avvia l'overlay qui o con il pulsante Overlay della barra degli strumenti. Sopra il gioco decodifica le firme delle scansioni minerarie, apre un cassetto per cercare blueprint, minerali e crafting, e segna come posseduti i blueprint che ottieni. Avvia il gioco in modalità finestra senza bordi sullo schermo principale." + "at": "Contrassegna i blueprint che possiedi. Cerca nell'elenco a sinistra per nome o missione, oppure restringilo per Tipo, Classe, Dimensione e Grado del componente, poi usa le frecce per spostare gli elementi nel tuo elenco posseduti. I blueprint posseduti ricevono un tag [Owned] ovunque compaiano tra le possibili ricompense di una missione, così puoi capire a colpo d'occhio cosa vale la pena inseguire." } }, - "blueprint_tracker": { + "overlay": { "title": { "ht": "", - "at": "7. Blueprint Tracker" + "at": "8. Overlay di Smart Citizen" }, "description": { "ht": "", - "at": "Contrassegna i blueprint che possiedi. Cerca nell'elenco a sinistra per nome o missione, oppure restringilo per Tipo, Classe, Dimensione e Grado del componente, poi usa le frecce per spostare gli elementi nel tuo elenco posseduti. I blueprint posseduti ricevono un tag [Owned] ovunque compaiano tra le possibili ricompense di una missione, così puoi capire a colpo d'occhio cosa vale la pena inseguire." + "at": "Avvia l'overlay qui o con il pulsante Overlay della barra degli strumenti. Sopra il gioco decodifica le firme delle scansioni minerarie, apre un cassetto per cercare blueprint, minerali e crafting, e segna come posseduti i blueprint che ottieni. Avvia il gioco in modalità finestra senza bordi sullo schermo principale." } }, "edit": { @@ -3233,7 +3233,7 @@ }, "description": { "ht": "", - "at": "Scegli il tema dell'app (Predefinito / Chiaro / Scuro / ODW) e, facoltativamente, disattiva il tour automatico per le nuove versioni." + "at": "Scegli il tema dell'app (Predefinito / Chiaro / Scuro / ODW) e la dimensione del testo (Piccola / Media / Grande, applicata dopo un riavvio) e, se vuoi, disattiva il tour automatico nelle nuove versioni." } }, "cfg_sc_install": { @@ -3273,13 +3273,33 @@ }, "description": { "ht": "", - "at": "Importa un INI esterno nelle tue sovrascritture tramite la finestra di risoluzione dei conflitti, oppure reimposta il tuo user.ini per il canale attivo (viene eseguito un backup automatico prima dello svuotamento)." + "at": "Importa un INI esterno nelle tue sovrascritture tramite la finestra di risoluzione dei conflitti. Reimposta lo user.ini del canale attivo (con backup prima) o recupera una copia precedente con Ripristina user.ini. L'anteprima mostra cosa scriverebbe Applica, e Controlla aggiornamenti verifica se c'è una nuova versione." + } + }, + "cfg_backup": { + "title": { + "ht": "", + "at": "19. Configurazione: Backup Impostazioni" + }, + "description": { + "ht": "", + "at": "Esporta impostazioni salva preferenze, configurazioni dei tag, cartella di Star Citizen e testi personalizzati di tutti i canali in un unico file. Importa impostazioni li ripristina, su un nuovo PC o dopo un'installazione pulita." + } + }, + "cfg_sc_tools": { + "title": { + "ht": "", + "at": "20. Configurazione: Strumenti di Star Citizen" + }, + "description": { + "ht": "", + "at": "Svuota cache shader elenca le cartelle della cache shader del gioco, una per versione. Pulisci elimina gli shader e mantiene le impostazioni grafiche; Rimuovi elimina l'intera cartella. Chiudi prima Star Citizen." } }, "help": { "title": { "ht": "", - "at": "19. Guida e feedback" + "at": "21. Guida e feedback" }, "description": { "ht": "", diff --git a/languages/japanese/ui.json b/languages/japanese/ui.json index 4668a06c..ae9a3b60 100644 --- a/languages/japanese/ui.json +++ b/languages/japanese/ui.json @@ -3123,7 +3123,7 @@ }, "description": { "ht": "", - "at": "「生成」時にどの拡張カテゴリーを実行するかを切り替えます。艦船、コンポーネント、艦船武器、FPS 武器、ミッション、ミッション報酬、商品クラフト、医療消耗品を、それぞれ独立して有効にできます。" + "at": "「生成」時に実行する拡張カテゴリーを切り替えます:Ships, Ship Items, Gear, Missions, Commodities, Journal, Medical Consumables。Ship Items はコンポーネント、艦船武器、ミサイル、Gear は FPS 武器とアーマータグを含みます。各カテゴリーは個別に有効・無効にできます。" } }, "enh_favorites": { @@ -3156,24 +3156,24 @@ "at": "コンポーネント、ミサイル、艦船武器、商品、アーマーに追加される角括弧付きの名称タグを組み立てます。▲/▼ で要素を並べ替え、チェックを外して除外し、省略の長さを変更し、区切り文字と括弧を選び、あるいはタグを名前の前ではなく後ろに配置できます。" } }, - "overlay": { + "blueprint_tracker": { "title": { "ht": "", - "at": "8. Smart Citizen オーバーレイ" + "at": "7. ブループリントトラッカー" }, "description": { "ht": "", - "at": "ここかツールバーのオーバーレイボタンでオーバーレイを開始します。ゲームの上で採掘スキャンのシグネチャーを解読し、ブループリント・鉱物・クラフトを検索するドロワーを開き、入手したブループリントを所有済みにします。ゲームはメイン画面のボーダーレスウィンドウモードで実行してください。" + "at": "所有しているブループリントをマークします。左のリストを名前やミッションで検索するか、コンポーネントのタイプ、クラス、サイズ、グレードで絞り込み、矢印を使って所有リストにアイテムを移動します。所有済みのブループリントは、ミッションの潜在的な報酬に表示される場所すべてで [Owned] タグが付くため、追う価値のあるものが一目でわかります。" } }, - "blueprint_tracker": { + "overlay": { "title": { "ht": "", - "at": "7. ブループリントトラッカー" + "at": "8. Smart Citizen オーバーレイ" }, "description": { "ht": "", - "at": "所有しているブループリントをマークします。左のリストを名前やミッションで検索するか、コンポーネントのタイプ、クラス、サイズ、グレードで絞り込み、矢印を使って所有リストにアイテムを移動します。所有済みのブループリントは、ミッションの潜在的な報酬に表示される場所すべてで [Owned] タグが付くため、追う価値のあるものが一目でわかります。" + "at": "ここかツールバーのオーバーレイボタンでオーバーレイを開始します。ゲームの上で採掘スキャンのシグネチャーを解読し、ブループリント・鉱物・クラフトを検索するドロワーを開き、入手したブループリントを所有済みにします。ゲームはメイン画面のボーダーレスウィンドウモードで実行してください。" } }, "edit": { @@ -3233,7 +3233,7 @@ }, "description": { "ht": "", - "at": "アプリのテーマ(Default / Light / Dark / ODW)を選び、任意で新バージョン時の自動ツアーを無効にできます。" + "at": "アプリのテーマ(Default / Light / Dark / ODW)と文字サイズ(小 / 中 / 大、再起動後に反映)を選び、任意で新バージョン時の自動ツアーを無効にできます。" } }, "cfg_sc_install": { @@ -3273,13 +3273,33 @@ }, "description": { "ht": "", - "at": "競合解決ダイアログを介して外部 INI をオーバーライドにインポートするか、アクティブなチャンネルの user.ini をリセットできます(クリア前に自動的にバックアップされます)。" + "at": "競合解決ダイアログを使って外部の INI をオーバーライドに取り込めます。アクティブチャンネルの user.ini をリセット(事前にバックアップ)したり、「user.ini を復元」で以前のコピーに戻したりできます。プレビューは「適用」で書き込まれる内容を表示し、「アップデートを確認」は新しいバージョンを探します。" + } + }, + "cfg_backup": { + "title": { + "ht": "", + "at": "19. Config:設定バックアップ" + }, + "description": { + "ht": "", + "at": "「設定をエクスポート」は、設定、タグ構成、Star Citizen フォルダー、全チャンネルのカスタム文字列を 1 つのファイルに保存します。「設定をインポート」で、新しい PC やクリーンインストール後に元に戻せます。" + } + }, + "cfg_sc_tools": { + "title": { + "ht": "", + "at": "20. Config:Star Citizen ツール" + }, + "description": { + "ht": "", + "at": "「シェーダーキャッシュを削除」は、ゲームのシェーダーキャッシュフォルダーをビルドごとに一覧表示します。「クリーン」はシェーダーを削除してグラフィック設定を残し、「削除」はフォルダー全体を削除します。先に Star Citizen を終了してください。" } }, "help": { "title": { "ht": "", - "at": "19. ヘルプとフィードバック" + "at": "21. ヘルプとフィードバック" }, "description": { "ht": "", diff --git a/languages/korean/ui.json b/languages/korean/ui.json index bbde8269..923b6824 100644 --- a/languages/korean/ui.json +++ b/languages/korean/ui.json @@ -3237,7 +3237,7 @@ }, "description": { "ht": "", - "at": "생성 시 실행할 강화 카테고리를 선택하세요. 함선, 부품, 함선 무기, FPS 무기, 임무, 임무 보상, 상품 제작, 의료 소모품을 각각 독립적으로 활성화할 수 있습니다." + "at": "생성 시 실행할 강화 카테고리를 선택합니다: Ships, Ship Items, Gear, Missions, Commodities, Journal, Medical Consumables. Ship Items는 부품, 함선 무기, 미사일을, Gear는 FPS 무기와 방어구 태그를 포함합니다. 각 카테고리는 따로 켜고 끌 수 있습니다." } }, "enh_favorites": { @@ -3270,24 +3270,24 @@ "at": "부품, 미사일, 함선 무기, 상품, 방어구에 추가되는 괄호 이름 태그를 만드세요. ▲/▼로 요소 순서를 바꾸고, 선택을 해제해 제외하고, 약자 길이를 바꾸고, 구분 기호와 괄호를 선택하고, 태그를 이름 앞이 아니라 뒤에 배치할 수 있습니다." } }, - "overlay": { + "blueprint_tracker": { "title": { "ht": "", - "at": "8. Smart Citizen 오버레이" + "at": "7. 설계도 추적기" }, "description": { "ht": "", - "at": "여기서 또는 도구 모음의 오버레이 버튼으로 오버레이를 시작하세요. 게임 위에서 채굴 스캔 신호를 해독하고, 설계도·광물·제작을 검색하는 서랍을 열며, 획득한 설계도를 보유로 표시합니다. 게임은 주 화면에서 테두리 없는 창 모드로 실행하세요." + "at": "보유한 설계도를 표시하세요. 왼쪽 목록을 이름이나 임무로 검색하거나, 부품 유형, 등급, 크기, 품급으로 좁힌 다음, 화살표로 항목을 보유 목록으로 옮기세요. 보유한 설계도는 임무의 잠재적 보상에 나타날 때마다 [보유 중] 태그를 얻어, 무엇을 노려볼 가치가 있는지 한눈에 알 수 있습니다." } }, - "blueprint_tracker": { + "overlay": { "title": { "ht": "", - "at": "7. 설계도 추적기" + "at": "8. Smart Citizen 오버레이" }, "description": { "ht": "", - "at": "보유한 설계도를 표시하세요. 왼쪽 목록을 이름이나 임무로 검색하거나, 부품 유형, 등급, 크기, 품급으로 좁힌 다음, 화살표로 항목을 보유 목록으로 옮기세요. 보유한 설계도는 임무의 잠재적 보상에 나타날 때마다 [보유 중] 태그를 얻어, 무엇을 노려볼 가치가 있는지 한눈에 알 수 있습니다." + "at": "여기서 또는 도구 모음의 오버레이 버튼으로 오버레이를 시작하세요. 게임 위에서 채굴 스캔 신호를 해독하고, 설계도·광물·제작을 검색하는 서랍을 열며, 획득한 설계도를 보유로 표시합니다. 게임은 주 화면에서 테두리 없는 창 모드로 실행하세요." } }, "edit": { @@ -3347,7 +3347,7 @@ }, "description": { "ht": "", - "at": "앱 테마(기본값 / 밝게 / 어둡게 / ODW)를 선택하고, 필요하면 새 버전에서 자동 투어를 비활성화할 수 있습니다." + "at": "앱 테마(기본값 / 밝게 / 어둡게 / ODW)와 글꼴 크기(작게 / 보통 / 크게, 재시작 후 적용)를 고르고, 원하면 새 버전에서 자동 투어를 끌 수 있습니다." } }, "cfg_sc_install": { @@ -3387,13 +3387,33 @@ }, "description": { "ht": "", - "at": "충돌 해결 대화 상자를 통해 외부 INI를 재정의 항목으로 가져오거나, 활성 채널의 user.ini를 재설정하세요(지우기 전에 자동으로 백업됨)." + "at": "충돌 해결 대화상자로 외부 INI를 재정의 항목으로 가져옵니다. 활성 채널의 user.ini를 초기화하거나(먼저 백업됨) user.ini 복원으로 이전 사본을 되돌릴 수 있습니다. 미리 보기는 적용 시 기록될 내용을 보여 주고, 업데이트 확인은 새 버전을 찾습니다." + } + }, + "cfg_backup": { + "title": { + "ht": "", + "at": "19. 설정: 설정 백업" + }, + "description": { + "ht": "", + "at": "설정 내보내기는 환경설정, 태그 구성, Star Citizen 폴더, 모든 채널의 사용자 문자열을 파일 하나에 저장합니다. 설정 가져오기로 새 PC나 새로 설치한 뒤에 다시 불러올 수 있습니다." + } + }, + "cfg_sc_tools": { + "title": { + "ht": "", + "at": "20. 설정: Star Citizen 도구" + }, + "description": { + "ht": "", + "at": "셰이더 캐시 지우기는 게임의 셰이더 캐시 폴더를 빌드마다 하나씩 보여 줍니다. 정리는 셰이더를 지우고 그래픽 설정은 남기며, 제거는 폴더 전체를 지웁니다. 먼저 Star Citizen을 종료하세요." } }, "help": { "title": { "ht": "", - "at": "19. 도움말 및 피드백" + "at": "21. 도움말 및 피드백" }, "description": { "ht": "", diff --git a/languages/portuguese_br/ui.json b/languages/portuguese_br/ui.json index ef938354..b323dacd 100644 --- a/languages/portuguese_br/ui.json +++ b/languages/portuguese_br/ui.json @@ -3123,7 +3123,7 @@ }, "description": { "ht": "", - "at": "Escolha quais categorias de aprimoramentos rodam ao Gerar. Naves, componentes, armas de nave, armas FPS, missões, recompensas de missão, fabricação de mercadorias e consumíveis médicos podem ser ativados de forma independente." + "at": "Escolha quais categorias de aprimoramentos rodam ao Gerar: Ships, Ship Items, Gear, Missions, Commodities, Journal, Medical Consumables. Ship Items inclui componentes, armas de nave e mísseis; Gear inclui armas FPS e tags de armadura. Cada categoria pode ser ligada ou desligada separadamente." } }, "enh_favorites": { @@ -3156,6 +3156,26 @@ "at": "Monte as tags entre colchetes adicionadas aos nomes de componentes, mísseis, armas de nave, commodities e armaduras. Reordene os elementos com ▲/▼, desmarque para excluir, mude o tamanho das abreviações, escolha o separador e os colchetes, ou coloque a tag depois do nome em vez de antes." } }, + "blueprint_tracker": { + "title": { + "ht": "", + "at": "7. Rastreador de Blueprints" + }, + "description": { + "ht": "", + "at": "Marque os blueprints que você possui. Busque na lista da esquerda por nome ou missão, ou refine por Tipo, Classe, Tamanho e Grau de componente, e então use as setas para mover itens para a sua lista de adquiridos. Blueprints adquiridos recebem uma tag [Owned] onde quer que apareçam nas recompensas potenciais de uma missão, para você ver de relance o que ainda vale caçar." + } + }, + "overlay": { + "title": { + "ht": "", + "at": "8. Sobreposição do Smart Citizen" + }, + "description": { + "ht": "", + "at": "Inicie a sobreposição aqui ou com o botão Sobreposição da barra de ferramentas. Sobre o jogo, ela decodifica assinaturas de escaneamento de mineração, abre uma gaveta para buscar seus blueprints, minerais e fabricação, e marca como possuídos os blueprints que você ganha. Rode o jogo em modo janela sem bordas na sua tela principal." + } + }, "edit": { "title": { "ht": "", @@ -3213,7 +3233,7 @@ }, "description": { "ht": "", - "at": "Escolha o tema do app (Padrão / Claro / Escuro / ODW) e, se quiser, desative o tour automático a cada nova versão." + "at": "Escolha o tema do app (Padrão / Claro / Escuro / ODW) e o tamanho da fonte (Pequeno / Médio / Grande, aplicado após reiniciar) e, se quiser, desative o tour automático em novas versões." } }, "cfg_sc_install": { @@ -3253,37 +3273,37 @@ }, "description": { "ht": "", - "at": "Importe um INI externo para suas personalizações pela caixa de resolução de conflitos, ou redefina seu user.ini do canal ativo (com backup automático antes de limpar)." + "at": "Importe um INI externo para suas personalizações pela janela de resolução de conflitos. Redefina o user.ini do canal ativo (com backup antes) ou recupere uma cópia anterior com Restaurar user.ini. A prévia mostra o que Aplicar escreveria, e Verificar atualizações procura uma nova versão." } }, - "help": { + "cfg_backup": { "title": { "ht": "", - "at": "19. Ajuda e feedback" + "at": "19. Config: Backup de Configurações" }, "description": { "ht": "", - "at": "O botão Ajuda abre um painel lateral com o guia completo, a lista de Problemas Conhecidos e um link para o canal do Smart Citizen no Discord. Isso é tudo o que você precisa para começar: boa sorte, cidadão." + "at": "Exportar Configurações salva suas preferências, ajustes de tags, pasta do Star Citizen e textos personalizados de todos os canais em um só arquivo. Importar Configurações os traz de volta, em um PC novo ou após uma instalação limpa." } }, - "overlay": { + "cfg_sc_tools": { "title": { "ht": "", - "at": "8. Sobreposição do Smart Citizen" + "at": "20. Config: Ferramentas do Star Citizen" }, "description": { "ht": "", - "at": "Inicie a sobreposição aqui ou com o botão Sobreposição da barra de ferramentas. Sobre o jogo, ela decodifica assinaturas de escaneamento de mineração, abre uma gaveta para buscar seus blueprints, minerais e fabricação, e marca como possuídos os blueprints que você ganha. Rode o jogo em modo janela sem bordas na sua tela principal." + "at": "Limpar cache de shaders lista as pastas de cache de shaders do jogo, uma por versão. Limpar apaga os shaders e mantém suas configurações gráficas; Remover apaga a pasta inteira. Feche o Star Citizen antes." } }, - "blueprint_tracker": { + "help": { "title": { "ht": "", - "at": "7. Rastreador de Blueprints" + "at": "21. Ajuda e feedback" }, "description": { "ht": "", - "at": "Marque os blueprints que você possui. Busque na lista da esquerda por nome ou missão, ou refine por Tipo, Classe, Tamanho e Grau de componente, e então use as setas para mover itens para a sua lista de adquiridos. Blueprints adquiridos recebem uma tag [Owned] onde quer que apareçam nas recompensas potenciais de uma missão, para você ver de relance o que ainda vale caçar." + "at": "O botão Ajuda abre um painel lateral com o guia completo, a lista de Problemas Conhecidos e um link para o canal do Smart Citizen no Discord. Isso é tudo o que você precisa para começar: boa sorte, cidadão." } } }, diff --git a/languages/russian/ui.json b/languages/russian/ui.json index aaa19211..05530ef7 100644 --- a/languages/russian/ui.json +++ b/languages/russian/ui.json @@ -3649,7 +3649,7 @@ }, "description": { "ht": "", - "at": "Выберите, какие категории улучшений обрабатываются при генерации. Корабли, компоненты, корабельное оружие, FPS-оружие, миссии, награды миссий, крафт товаров и медицинские расходники включаются независимо друг от друга." + "at": "Выберите, какие категории улучшений запускаются при генерации: Ships, Ship Items, Gear, Missions, Commodities, Journal, Medical Consumables. Ship Items включает компоненты, корабельное оружие и ракеты; Gear включает оружие FPS и метки брони. Каждую категорию можно включить или выключить отдельно." } }, "enh_favorites": { @@ -3759,7 +3759,7 @@ }, "description": { "ht": "", - "at": "Выберите тему приложения (По умолчанию / Светлая / Тёмная / ODW) и при желании отключите автоматический запуск обзора при выходе новых версий." + "at": "Выберите тему приложения (По умолчанию / Светлая / Тёмная / ODW) и размер шрифта (Мелкий / Средний / Крупный, применяется после перезапуска) и при желании отключите автоматический тур при новых версиях." } }, "cfg_sc_install": { @@ -3799,13 +3799,33 @@ }, "description": { "ht": "", - "at": "Импортируйте внешний INI в свои переопределения через диалог разрешения конфликтов или сбросьте user.ini для активного канала (перед очисткой автоматически создаётся резервная копия)." + "at": "Импортируйте внешний INI в свои переопределения через окно разрешения конфликтов. Сбросьте user.ini активного канала (с резервной копией) или верните прежнюю копию кнопкой «Восстановить user.ini». Предпросмотр показывает, что запишет «Применить», а «Проверить обновления» ищет новую версию." + } + }, + "cfg_backup": { + "title": { + "ht": "", + "at": "19. Настройки: резервное копирование настроек" + }, + "description": { + "ht": "", + "at": "«Экспорт настроек» сохраняет ваши настройки, конфигурации меток, папку Star Citizen и свои строки для всех каналов в один файл. «Импорт настроек» возвращает их, на новом ПК или после чистой установки." + } + }, + "cfg_sc_tools": { + "title": { + "ht": "", + "at": "20. Настройки: инструменты Star Citizen" + }, + "description": { + "ht": "", + "at": "«Очистить кэш шейдеров» показывает папки кэша шейдеров игры, по одной на каждую версию. «Очистить» удаляет шейдеры и сохраняет настройки графики; «Удалить» удаляет всю папку. Сначала закройте Star Citizen." } }, "help": { "title": { "ht": "", - "at": "19. Справка и обратная связь" + "at": "21. Справка и обратная связь" }, "description": { "ht": "", diff --git a/languages/spanish/ui.json b/languages/spanish/ui.json index 65759fa3..c65430dc 100644 --- a/languages/spanish/ui.json +++ b/languages/spanish/ui.json @@ -3123,7 +3123,7 @@ }, "description": { "ht": "", - "at": "Elige qué categorías de mejoras se ejecutan al Generar. Naves, componentes, armas de nave, armas FPS, misiones, recompensas de misión, fabricación de mercancías y consumibles médicos pueden activarse de forma independiente." + "at": "Elige qué categorías de mejoras se ejecutan al Generar: Ships, Ship Items, Gear, Missions, Commodities, Journal, Medical Consumables. Ship Items incluye componentes, armas de nave y misiles; Gear incluye armas FPS y etiquetas de armadura. Cada categoría se activa o desactiva por separado." } }, "enh_favorites": { @@ -3156,24 +3156,24 @@ "at": "Construye las etiquetas entre corchetes añadidas a componentes, misiles, armas de nave, mercancías y armaduras. Reordena elementos con ▲/▼, desmarca para excluir, cambia la longitud de abreviatura, elige separador y corchetes, o coloca la etiqueta después del nombre en lugar de antes." } }, - "overlay": { + "blueprint_tracker": { "title": { "ht": "", - "at": "8. Superposición de Smart Citizen" + "at": "7. Rastreador de blueprints" }, "description": { "ht": "", - "at": "Inicia la superposición aquí o con el botón Superposición de la barra de herramientas. Sobre el juego decodifica las firmas de escaneo minero, abre un cajón para buscar tus blueprints, minerales y fabricación, y marca como poseídos los blueprints que consigues. Ejecuta el juego en modo ventana sin bordes en tu pantalla principal." + "at": "Marca los blueprints que posees. Busca en la lista izquierda por nombre o misión, o acótala por Tipo, Clase, Tamaño y Grado de componente, y usa las flechas para mover objetos a tu lista de adquiridos. Los blueprints adquiridos reciben una etiqueta [Owned] allí donde aparezcan entre las recompensas potenciales de una misión, para ver de un vistazo qué merece la pena perseguir." } }, - "blueprint_tracker": { + "overlay": { "title": { "ht": "", - "at": "7. Rastreador de blueprints" + "at": "8. Superposición de Smart Citizen" }, "description": { "ht": "", - "at": "Marca los blueprints que posees. Busca en la lista izquierda por nombre o misión, o acótala por Tipo, Clase, Tamaño y Grado de componente, y usa las flechas para mover objetos a tu lista de adquiridos. Los blueprints adquiridos reciben una etiqueta [Owned] allí donde aparezcan entre las recompensas potenciales de una misión, para ver de un vistazo qué merece la pena perseguir." + "at": "Inicia la superposición aquí o con el botón Superposición de la barra de herramientas. Sobre el juego decodifica las firmas de escaneo minero, abre un cajón para buscar tus blueprints, minerales y fabricación, y marca como poseídos los blueprints que consigues. Ejecuta el juego en modo ventana sin bordes en tu pantalla principal." } }, "edit": { @@ -3233,7 +3233,7 @@ }, "description": { "ht": "", - "at": "Elige el tema de la app (Default / Light / Dark / ODW) y desactiva, si quieres, el recorrido automático en versiones nuevas." + "at": "Elige el tema de la app (Default / Light / Dark / ODW) y el tamaño de letra (Pequeño / Mediano / Grande, se aplica tras reiniciar), y desactiva, si quieres, el recorrido automático en versiones nuevas." } }, "cfg_sc_install": { @@ -3273,13 +3273,33 @@ }, "description": { "ht": "", - "at": "Importa un INI externo a tus modificaciones mediante el diálogo de resolución de conflictos, o restablece tu user.ini del canal activo (con copia de seguridad automática antes de borrarlo)." + "at": "Importa un INI externo a tus modificaciones mediante el diálogo de resolución de conflictos. Restablece tu user.ini del canal activo (con copia de seguridad previa) o recupera una copia anterior con Restaurar user.ini. Vista previa muestra lo que escribiría Aplicar, y Buscar actualizaciones comprueba si hay una versión nueva." + } + }, + "cfg_backup": { + "title": { + "ht": "", + "at": "19. Config: Copia de seguridad de ajustes" + }, + "description": { + "ht": "", + "at": "Exportar ajustes guarda tus preferencias, configuraciones de etiquetas, carpeta de Star Citizen y textos personalizados de todos los canales en un solo archivo. Importar ajustes los recupera, en un PC nuevo o tras una instalación limpia." + } + }, + "cfg_sc_tools": { + "title": { + "ht": "", + "at": "20. Config: Herramientas de Star Citizen" + }, + "description": { + "ht": "", + "at": "Borrar caché de shaders muestra las carpetas de caché de shaders del juego, una por versión. Limpiar borra los shaders y conserva tus ajustes gráficos; Eliminar borra la carpeta entera. Cierra Star Citizen antes." } }, "help": { "title": { "ht": "", - "at": "19. Ayuda y comentarios" + "at": "21. Ayuda y comentarios" }, "description": { "ht": "", diff --git a/languages/turkish/ui.json b/languages/turkish/ui.json index 8b20ae30..03306c77 100644 --- a/languages/turkish/ui.json +++ b/languages/turkish/ui.json @@ -3237,7 +3237,7 @@ }, "description": { "ht": "", - "at": "Oluştur dediğinizde hangi geliştirme kategorilerinin çalışacağını seçin. Gemiler, bileşenler, gemi silahları, FPS silahları, görevler, görev ödülleri, emtia üretimi ve tıbbi sarf malzemelerinin her biri bağımsız olarak etkinleştirilebilir." + "at": "Oluştur'a basınca hangi geliştirme kategorilerinin çalışacağını seçin: Ships, Ship Items, Gear, Missions, Commodities, Journal, Medical Consumables. Ship Items bileşenleri, gemi silahlarını ve füzeleri; Gear FPS silahlarını ve zırh etiketlerini kapsar. Her kategori ayrı ayrı açılıp kapatılabilir." } }, "enh_favorites": { @@ -3270,24 +3270,24 @@ "at": "Bileşenlere, füzelere, gemi silahlarına, emtialara ve zırhlara eklenen köşeli parantezli ad etiketlerini oluşturun. Öğeleri ▲/▼ ile yeniden sıralayın, hariç tutmak için işaretini kaldırın, kısaltma uzunluğunu değiştirin, ayırıcı ve parantez seçin veya etiketi adın önüne değil arkasına yerleştirin." } }, - "overlay": { + "blueprint_tracker": { "title": { "ht": "", - "at": "8. Smart Citizen Katmanı" + "at": "7. Plan Takipçisi" }, "description": { "ht": "", - "at": "Katmanı buradan veya araç çubuğundaki Katman düğmesiyle başlatın. Oyunun üzerinde maden tarama imzalarını çözer, planlarınızı, minerallerinizi ve üretimi aramak için bir çekmece açar ve kazandığınız planları sahip olunan olarak işaretler. Oyunu ana ekranınızda kenarlıksız pencere modunda çalıştırın." + "at": "Sahip olduğunuz planları işaretleyin. Soldaki listede ada veya göreve göre arama yapın ya da bileşen Türü, Sınıfı, Boyutu ve Derecesine göre daraltın, ardından öğeleri sahip olunanlar listenize taşımak için okları kullanın. Sahip olunan planlar, bir görevin olası ödüllerinde göründükleri her yerde [Owned] etiketi alır; böylece neyin peşine düşmeye değer olduğunu bir bakışta görebilirsiniz." } }, - "blueprint_tracker": { + "overlay": { "title": { "ht": "", - "at": "7. Plan Takipçisi" + "at": "8. Smart Citizen Katmanı" }, "description": { "ht": "", - "at": "Sahip olduğunuz planları işaretleyin. Soldaki listede ada veya göreve göre arama yapın ya da bileşen Türü, Sınıfı, Boyutu ve Derecesine göre daraltın, ardından öğeleri sahip olunanlar listenize taşımak için okları kullanın. Sahip olunan planlar, bir görevin olası ödüllerinde göründükleri her yerde [Owned] etiketi alır; böylece neyin peşine düşmeye değer olduğunu bir bakışta görebilirsiniz." + "at": "Katmanı buradan veya araç çubuğundaki Katman düğmesiyle başlatın. Oyunun üzerinde maden tarama imzalarını çözer, planlarınızı, minerallerinizi ve üretimi aramak için bir çekmece açar ve kazandığınız planları sahip olunan olarak işaretler. Oyunu ana ekranınızda kenarlıksız pencere modunda çalıştırın." } }, "edit": { @@ -3347,7 +3347,7 @@ }, "description": { "ht": "", - "at": "Uygulama temasını seçin (Varsayılan / Açık / Koyu / ODW) ve isterseniz yeni sürümlerde otomatik turu devre dışı bırakın." + "at": "Uygulama temasını (Varsayılan / Açık / Koyu / ODW) ve yazı boyutunu (Küçük / Orta / Büyük, yeniden başlatınca uygulanır) seçin; isterseniz yeni sürümlerde otomatik turu kapatın." } }, "cfg_sc_install": { @@ -3387,13 +3387,33 @@ }, "description": { "ht": "", - "at": "Çakışma çözme iletişim kutusu aracılığıyla harici bir INI dosyasını geçersiz kılmalarınıza aktarın veya etkin kanal için user.ini dosyanızı sıfırlayın (temizlenmeden önce otomatik olarak yedeklenir)." + "at": "Harici bir INI dosyasını çakışma çözme penceresiyle geçersiz kılmalarınıza aktarın. Etkin kanalın user.ini dosyasını sıfırlayın (önce yedeklenir) ya da user.ini Dosyasını Geri Yükle ile önceki bir kopyayı geri getirin. Önizleme Uygula'nın ne yazacağını gösterir, Güncellemeleri Denetle yeni sürüm arar." + } + }, + "cfg_backup": { + "title": { + "ht": "", + "at": "19. Yapılandırma: Ayar Yedeği" + }, + "description": { + "ht": "", + "at": "Ayarları Dışa Aktar tercihlerinizi, etiket ayarlarınızı, Star Citizen klasörünüzü ve tüm kanallardaki özel metinlerinizi tek bir dosyaya kaydeder. Ayarları İçe Aktar bunları yeni bir bilgisayarda ya da temiz kurulumdan sonra geri getirir." + } + }, + "cfg_sc_tools": { + "title": { + "ht": "", + "at": "20. Yapılandırma: Star Citizen Araçları" + }, + "description": { + "ht": "", + "at": "Gölgelendirici Önbelleğini Temizle, oyunun gölgelendirici önbellek klasörlerini her oyun sürümü için bir tane olacak şekilde listeler. Temizle gölgelendiricileri siler ve grafik ayarlarınızı korur; Kaldır tüm klasörü siler. Önce Star Citizen'ı kapatın." } }, "help": { "title": { "ht": "", - "at": "19. Yardım ve geri bildirim" + "at": "21. Yardım ve geri bildirim" }, "description": { "ht": "", diff --git a/src/gui/main_window.py b/src/gui/main_window.py index c61fd3d7..14119ba8 100644 --- a/src/gui/main_window.py +++ b/src/gui/main_window.py @@ -4656,6 +4656,8 @@ def _action(): "cfg_data_folder": {"target": lambda: self.config_tab._data_group, "pre_action": _switch_to(config_tab)}, "cfg_p4k_extraction": {"target": lambda: self.config_tab._p4k_group, "pre_action": _switch_to(config_tab)}, "cfg_tools": {"target": lambda: self.config_tab._tools_group, "pre_action": _switch_to(config_tab)}, + "cfg_backup": {"target": lambda: self.config_tab._backup_group, "pre_action": _switch_to(config_tab)}, + "cfg_sc_tools": {"target": lambda: self.config_tab._sc_tools_group, "pre_action": _switch_to(config_tab)}, "help": {"target": lambda: self.help_btn, "pre_action": _switch_to(strings_tab)}, } diff --git a/tests/CLAUDE.md b/tests/CLAUDE.md index c7a18134..aca0ecc7 100644 --- a/tests/CLAUDE.md +++ b/tests/CLAUDE.md @@ -86,7 +86,7 @@ Split by domain: - `test_crash_logger.py`: the real `crash_logger.show_crash_dialog` with `exec()` replaced: no dialog without a `QApplication`, and with one the error text, the dump path (or "could not write crash log"), a Copy path button that copies it, and a Close button that accepts the dialog. - `test_error_dialog.py` — `logging.ERROR`/`CRITICAL` → modal-dialog handler plus the main-thread signal hop. - `test_session_qapplication.py`: PyQt6 still marks the wrapper of a Qt-created scroll bar, menu action and status bar deleted when its parent dies. That only holds while the process's first `QApplication` lives, so on a full run it fails if a second app appeared before it ran. GUI modules that sort after it are covered by the conftest guard alone (see *Test isolation*). -- `test__activation.py` (spanish, japanese, chinese, italian, german, turkish, russian): per-language activation locks: the real `ui.json` is `{ht, at}`-shaped, covers the full English key set (no extras outside `tutorial.*`), keeps every `{placeholder}`, translates the 19-step tour, shows in the selector, maps to the right SC language id, and has a `sources.json` URL plus the four translated docs. `test_turkish_activation.py` (#404) additionally locks the **borrowed** `polish_(poland)` slot (not `turkish_(turkey)`, which the game does not accept; not `russian_(russia)`, which the game does not recognise either; not Dymerz's `german_(germany)`, which our German owns) and that `ensure_user_cfg_language` writes that slot to `user.cfg`. `test_korean_activation.py` (#367) is the AI-only, bring-your-own-file variant: every `ht` is empty and `sources.json` carries no Korean URL. It also locks that no value runs two sentences together (a `.`, `?` or `!` straight into a Hangul syllable), since Korean spaces after a sentence end. It also checks that `ensure_user_cfg_language()` with no language passed follows the stored Korean choice. `test_russian_activation.py` locks the one shared slot: Russian borrows Korean's `korean_(south_korea)` (the only in-game font with Cyrillic, approved by the owner), so one `user.cfg` walked through English, Russian and Korean in either order keeps exactly one right `g_language` line and the user's other lines, and `test_apply_already_applied.py` checks that Korean selected while Russian strings are applied reads not applied, since `user.cfg` reads the same for both. +- `test__activation.py` (spanish, japanese, chinese, italian, german, turkish, russian): per-language activation locks: the real `ui.json` is `{ht, at}`-shaped, covers the full English key set (no extras outside `tutorial.*`), keeps every `{placeholder}`, translates the 21-step tour, shows in the selector, maps to the right SC language id, and has a `sources.json` URL plus the four translated docs. `test_turkish_activation.py` (#404) additionally locks the **borrowed** `polish_(poland)` slot (not `turkish_(turkey)`, which the game does not accept; not `russian_(russia)`, which the game does not recognise either; not Dymerz's `german_(germany)`, which our German owns) and that `ensure_user_cfg_language` writes that slot to `user.cfg`. `test_korean_activation.py` (#367) is the AI-only, bring-your-own-file variant: every `ht` is empty and `sources.json` carries no Korean URL. It also locks that no value runs two sentences together (a `.`, `?` or `!` straight into a Hangul syllable), since Korean spaces after a sentence end. It also checks that `ensure_user_cfg_language()` with no language passed follows the stored Korean choice. `test_russian_activation.py` locks the one shared slot: Russian borrows Korean's `korean_(south_korea)` (the only in-game font with Cyrillic, approved by the owner), so one `user.cfg` walked through English, Russian and Korean in either order keeps exactly one right `g_language` line and the user's other lines, and `test_apply_already_applied.py` checks that Korean selected while Russian strings are applied reads not applied, since `user.cfg` reads the same for both. - `test_available_languages.py`: 2.0 language selector gate. `get_available_languages()` hides languages whose `ui.json` is a stub (only `_comment`); English is always offered. - `test_i18n.py`: 2.0 UI translation contract. `tr()` dot-path resolution, English fallback for untranslated keys, bare-key fallback when missing everywhere, kwargs interpolation degradation, `_deep_merge` scalar-over-dict tolerance, lazy English load when `set_language` was never called (workers' progress strings). - `test_language_paths.py`: 2.0 per-language cache layout. English base.ini at the cache root vs `cache/lang/{language}/` for others, enhancements dir = base.ini parent, per-language `.dataforge_stamp` isolation, both stamps written in one step and read with a short retry (a failed write keeps the old stamp, and a stamp held with no sharing for a moment is not read as missing, Windows only), the stamp setters' `folder=` and `get_dataforge_build_key(forge_dir)` (what a generation passes so its stamps go where its run started), `invalidate_enhancements_stamp` (empties the DataForge stamp only and writes nothing else, a failure only logged), base.ini URL resolution (override > bundled sources.json > ''), `SC_LANGUAGE_IDS` mapping fallback, and `get_localized_doc_path` (translated HELP/ABOUT/LEGAL win for non-English, bundled `docs/` is the fallback). @@ -131,6 +131,7 @@ Split by domain: - `test_sco_overlay.py`: SCO windows headlessly (`QT_QPA_PLATFORM=offscreen`). An autouse fixture stubs the theme colours and the default samples folder, so nothing reads real settings. The reader runs on its pool thread and its result reaches the label on the GUI thread, a tick during a read is skipped, a failing reader stops the capture, a read finishing after `stop()` is dropped, a failed sample save logs once, and the samples folder is resolved at save time. Sample saves happen only with `SAVE_SAMPLES_ENV` set, so tests that expect one take the `samples_on` fixture, and one test checks nothing is saved without it. Drawer list/search/owned-only/checkbox signal and tooltip, signature label text, scan region covers the sample readouts, capture timer only runs with a reader set, the default reader reads a real `QImage`, one saved sample per distinct reading, a one-frame misread doesn't replace the label, the label sits centred under the readout in ODW colours, the label is blacked out of the capture. The options switch the decoder and drawer off while the overlay stays running, and save-samples off saves nothing. The drawer's Minerals search groups locations by star system and finds minerals by location, the Crafting search works both ways, tooltips carry tier, signature, locations and materials, an empty catalog says how to fill it, each tab opens its own section (another switches, its own closes), the tabs overlap with the open one in front (`tab_at`), a click lands on the tab under it, the stack runs along each of the four edges, the Minerals section's watch checkboxes (only on ores with a signature, kept across a search, Clear all, mirrored without an echo), its scan decoder switch (emits, and the Overlay tab's switch mirrors back), a watched mineral's decode making the label glow (and ticking one while it shows lighting it at once), the label box sitting on its anchor with the glow room above it short of `LABEL_GAP`, the section switches hide their tabs (closing the drawer when the open one goes, and hiding the drawer with none left), and the drawer docks at each of the eight positions, on screen, against the right edge and with the handle on the panel's inner side (an unknown position falls back to middle right). File-8 review fixes: a new reading's confirming capture runs at once (and not while stopped), a failing reader emits `decoder_failed` and a restart tries again, crafting rows find their mineral through CIG's spelling differences, `prune_samples` keeps the newest crops and a session's first save prunes, and a tick (or an untick under Owned only) leaves the blueprint list's scroll position where it was. The drawer follows a language switch (`retranslate_ui`, German and back). #523: a round number that fits salvage lists salvage first and marks ROC and FPS surface only (one without salvage is left as it was); the Salvage row ends the Minerals list, after every mineral (catalog or not, under the empty-catalog message), its watch box emits, a decode with salvage sets off the label, Clear all takes it off and a mineral search hides it; the label sits centred under the pin icon and the number, not the digits alone; it holds still while the readout only jitters (boxes a few pixels apart, a new number in the same place) and follows it past `ANCHOR_DEAD_ZONE` digit heights, or afresh once hidden, and two real crops read with changed brightness and noise keep it still; with Windows display scaling (1080 px and 1600 px at 125%, 1440 px and 2160 px at 150%) captures taken as the overlay takes them, its own blackout included, keep reading with the label still (they failed 13 or 14 times in 16 before); only a clicked drawer search box takes the keyboard (every other panel widget is `NoFocus`, so opening a section focuses no search box) and it types nothing from a held key's repeats while a press types and a held Backspace deletes; a selected drawer row keeps light text with the SCLE theme's near-black highlighted text set on the app, as `apply_theme` sets it; each alert follows its switch (glow, the green text, the bigger text re-centred under its anchor, one chime only when a watched target is first read, not for a switch or a steady repeat, `play_alert_sound` patched so nothing sounds), the glow and the flash are the defaults, the flash blinks three times without changing the label's size, again only after the target was gone, and stops when switched off; the text size scales the font and an unknown size reads as medium; the chime is a short mono WAV handed to `PlaySoundW` from memory without waiting (the call recorded, not made); and the drawer hotkey, with `RegisterHotKey` and the raw input switch recorded on a stand-in user32, is held (raw input with it) only while the overlay runs with the drawer showing, follows a change and is let go on stop, a refused one is reported once per start and tried again after a restart, the hotkey opens the drawer on the section used last and activates it, and the drawer closing by the hotkey or its own tab hands focus back to the window it took it from, but not once the player has clicked away, nor after the drawer was hidden by its switch in between. A drawer the hotkey opened while Windows kept focus on another window is logged at INFO once the focus check has run, and not when the drawer got focus or was closed before the check. - `test_mining_catalog.py`: SCO drawer data. The Mining Compendium parse against the real `kraken_global_latest.ini` fixture (a community pack with tier headings) and a stock-shaped text with none (empty tier) (every named place has a star system), `star_system` rules, system grouping order, escaped or real line breaks, the CIG ore spelling alias, the `crafting_index.json` round trip plus unreadable files reading as empty, a failed index write keeping the old index and a read during the swap retried (atomic_io, #476), `material_key_for` bridging CIG's spelling differences on LIVE 4.9 names, and an exact material name beating an earlier one that only starts with the mineral's word, and `material_names` listing a material with no display names by its key. The generator side (`scan_crafting_blueprints` filling the index) is in `test_commodity_journal.py`. - `test_global_hotkey.py`: #523, the SCO drawer hotkey (`src/gui/global_hotkey.py`). `windows_hotkey` gives Windows' modifiers and virtual key for letters, digits, numpad digits, F keys and named keys, and None for an empty text, two combinations, Esc or Enter, and for a letter, digit, Space, Tab or navigation key held alone or with Shift only (an F key and Pause may stand alone). `hotkey_text`, the Overlay tab box's way back from a virtual key, round-trips every supported key with all 16 mixes of Ctrl, Alt, Shift and Win (None for the mixes that rule refuses), drops AltGr's and the number pad's own flags (a number pad arrow with NumLock off is the arrow), and gives None for a symbol key, Esc, Enter or no key. `modifiers_down_now` asks Windows about every key in `_MODIFIER_VKS`, left and right ones too. Symbol keys go through stand-in keyboard layouts (MapVirtualKeyW and VkKeyScanW answered from a table: US, German with its dead keys, Brazilian ABNT2 with its own "/" key at 0xC1, one with two backslash keys, one that knows a letter key only by its small letter): "Ctrl+/" is 0xBF on US and refused on German (Shift+7), "Ctrl+#" and "Ctrl+Ü" are found there, a symbol that needs Shift, a dead key and a symbol alone are refused, every symbol key reads back as itself with every modifier mix that has Ctrl, Alt or Win, a symbol two keys type is saved only for the one Windows finds, off Windows nothing is found, and a saved Ctrl+/ registers 0xBF on US and is refused, with nothing asked of Windows, once the layout is German. `GlobalHotkey` on a recorded stand-in for user32: registers with `MOD_NOREPEAT` and its own id, does nothing for the combination it holds, swaps for a new one, lets go for none, refuses an unusable key without calling Windows, and reports and logs a refusal. Raw input, for a game that keeps `WM_HOTKEY` away: registering opens one `STATIC` message-only window and sends the keyboard's raw input to it with `RIDEV_INPUTSINK`, letting go removes it (`RIDEV_REMOVE`) and closes the window, and a refusal closes the window and keeps the plain hotkey with an INFO line. `RawCombination` fires on the key going down with exactly the combination's modifiers, in stream order (not on a repeat, one modifier too many or too few, or another key; a left or right modifier counts), and asks Windows about a modifier held past `STALE_AFTER`, and a press of the key last seen longer ago than that is a new press (its release was lost), while one sooner is a repeat. A raw sighting waits `RAW_GRACE_MS` before it toggles, and a press seen both ways toggles once whichever comes first (`WM_HOTKEY` stops the wait at once). Each `WM_INPUT` is read once even when handed over twice, and mouse input is ignored (`GetRawInputData` faked from a table). The native filter, given built `MSG`s, consumes `WM_HOTKEY` for its id only, reads `WM_INPUT` only for its own window and leaves it to Windows' default handling. Once for real on Windows: Ctrl+Alt+Shift+F24 (no keyboard has F24) is registered, a second owner is refused by Windows itself, a `WM_HOTKEY` for its id posted to this thread comes out of the Qt event loop as `activated` and one for another id does not (each post moves the hotkey's clock past `SAME_PRESS_S`, so the one-toggle-per-press rule cannot hide a wrong id), and once let go nothing hears it and the combination is free again. Its raw input is registered for real alongside (a message-only window Windows accepts) and removed with it. +- `test_tutorial_translations.py`: every language with a `tutorial.*` section has the steps of `assets/tutorial.json` in the same order, each title numbered as the English one, and the tour has a step for the Config tab's Settings Backup box and one for its Star Citizen Tools box (Clear Shader Cache), right after Tools, wired to those boxes (3.0.1). - `test_overlay_tab.py`: Overlay tab. Every SCO option defaults on but the alerts #523 left off (color, bigger, sound), a checkbox saves its option and emits `options_changed` (a new tab reads it back), the start button asks via `overlay_toggled` while `set_running` only mirrors, the option defaults reach `profile_default_values()` (settings backups), the drawer switch greys out its section switches and the position picker, the picker saves and reads back its position (a bad stored value reads as middle right), `set_option` mirrors without saving or emitting, the watched-minerals setting round-trips (bad JSON reads as empty) and reaches settings backups, `MainWindow._on_sco_scan_decoder_toggled` (on a stub self) saves and mirrors, `MainWindow._refresh_sco_catalog` (on a stub self) re-reads its files only when one changed, and the live watch (stub selves): started only on LIVE and HOTFIX, not crashing on an install path whose `is_dir` raises, never writing the Scan's watermark, and restarted by a channel switch. The guided tour has an `overlay` step right after `blueprint_tracker`, every title numbered in order, wired to the tab's start button. #523: the alert switches and the text size picker sit under the scan decoder and grey out with it (the hotkey box goes with the drawer); the text size picker saves, reads back, reaches settings backups and reads a bad stored value as medium; the hotkey box, fed key events built as Windows delivers them (Qt's key, the modifiers and the virtual key), saves Ctrl+Alt+O when the layout made "ó" of it (the reported case) and Ctrl+Shift+1 when it made "!", refuses and explains a key Windows has no code for (Ctrl+Esc) and goes back to the saved one, takes Ctrl+/ (the layout stood in for) and refuses / alone, and the Clear button to the box's right, lit only with a hotkey saved and the drawer on, empties it; it shows held modifiers ("Ctrl+Alt+") and the saved combination again once they are let go, takes Tab with the box focused as a key, keeping the focus (refused alone, recorded with Ctrl), refuses a plain letter (nothing is typed, the saved combination stays) and records an F key alone, accepts `ShortcutOverride`, and falls back to Qt's key when a press has no virtual key; the options sit in four titled boxes, each switch in its own, with the alerts and the drawer's sections each on one line and their hints as tooltips; `MainWindow._on_sco_hotkey_problem` (on a stub self) shows a refusal on the tab and in the status bar; and `_apply_sco_options` hands the overlay its text size and hotkey. - `test_blueprint_auto_scan.py`: #386 opt-in startup auto-scan for "Scan Logs for Owned Blueprints". `AppSettings.get_/set_auto_scan_blueprints_enabled()` contract (default False); `MainWindow._maybe_auto_scan_blueprints` driven on a stub `self` (same pattern as `test_favorite_toggle_stranded`) — gated off when the setting is off (even with a valid install path, to isolate the setting itself as the gate) or the install has no LIVE or HOTFIX folder yet (silently, no warning dialog unlike the manual scan, and the same LIVE-plus-HOTFIX queue as the button whichever channel Config has selected, #446), skipped when a scan is already running, and sets `_bp_scan_silent = True` + never forces a full rescan when it does start; `MainWindow._finish_blueprint_scan_queue`'s silent branch swaps the manual scan's modal summary dialogs for a status bar message, still writes the owned set and re-weaves `[Owned]` tags, and consumes the one-shot flag whether or not anything new was found. Also covers the #386 follow-up popup opt-in: `AppSettings.get_/set_auto_scan_show_popup_enabled()` contract, and `_finish_blueprint_scan_queue`'s popup-enabled branch driven with `main_window.QMessageBox`/`_relabel_details_button` monkeypatched to fakes (same technique as `test_install_scan_finished_stale_worker.py`) — enabled brings back the exact modal dialog a manual scan shows (proven via the fake's `exec_called` + `detailed_text`), disabled stays on the status bar, and the setting only ever applies to a silent run's "found something new" case (a quiet run and a manual scan both ignore it). `TestTheScansStatusTextGoes` (#527): the scan's last progress text ("Finishing up…", shown through `_show_blueprint_scan_status`) used to stay in the status bar for the rest of the session; the run's end takes it down after a silent run that found nothing and behind a manual run's box, a silent run that found some replaces it with its own message, and a message another job showed after it is left alone. - `test_default_owned_blueprints.py`: #416, the starter blueprints every player owns (`blueprint_meta.DEFAULT_OWNED_BLUEPRINTS`). Qt-free half: the list holds exactly the eight reported items, by loc key; they join the catalogue under `DEFAULT_OWNED_MISSION_LABEL` with types from their keys, which `mission_display_name` shows in the selected language (German checked) and leaves a real mission name alone; the name follows the loaded string (a CIG rename, another language, a Tag Builder tag stripped via the stock value, any key capitalisation) with the English name as fallback; a mission-offered starter keeps its missions and gains the label; a one-shot iterator still works; and `default_owned_blueprints` names them exactly as the catalogue does, so a seeded starter leaves Available. The seed marker (`AppSettings.get_/set_default_blueprints_seed()`, JSON of the seeded loc keys and a pending flag): default none seeded, round trip, stored as sorted JSON, unreadable values (including the old single-value forms) read as none seeded, stray entries dropped, and in a settings backup: exported only once stored, like `owned_items`, kept with its owned set on a round trip, reset by an owned set from before 3.0 so the next scan adds the starters to it, left alone by a backup without an owned set (so it cannot undo the one-shot), and a damaged marker reading as none seeded and stored cleaned, so it does not travel into the next backup. `MainWindow` half on stub selves with a fake `QMessageBox` and a fake `QTimer` that holds queued calls: the first scan adds every starter and lists them in the summary (also when the logs found nothing, and in the status bar for a silent auto-scan), already-owned ones are not reported again, a profile that scanned before this release is seeded on its next scan, names follow the loaded strings, it is one-shot per item (a starter moved back stays in Available, a starter declared later reaches Owned on the next scan and only it is reported, also when that scan beats the strings load, and two keys sharing one name are both recorded so later scans do nothing), a load never seeds before the first scan, a scan before the strings load marks it pending and the next load finishes it with the loaded names before the Owned tags are woven, then reports what it added once the load is over, as a status bar message or the summary popup when the auto-scan popup option is on, and only what it added: the report is queued as the last step but one of both real load paths, followed only by the questions a job put off (#483) (checked on a MagicMock self), a modal step inside the load does not run it, it waits while a load, extraction, generation or Simple-mode run is going and goes out after the reload that ends it, and never while closing. A scan that adds blueprints runs the background already-applied check after the re-weave, but not while closing, with nothing loaded or with a load running, and a quiet scan runs nothing. Exiting from the unapplied-changes prompt settles the check again before the close is committed, while Cancel and Apply Now do not. the seed names the starters with the same stock values and Tag Builder enclosings as the metadata rebuild (so after a load every seeded name is in the catalogue and none is left in Available), and a failure is logged and left for the next scan. diff --git a/tests/test_tutorial_translations.py b/tests/test_tutorial_translations.py new file mode 100644 index 00000000..ab8c1e26 --- /dev/null +++ b/tests/test_tutorial_translations.py @@ -0,0 +1,58 @@ +"""The guided tour is the same in every language: each translated +``tutorial.*`` section has the steps of ``assets/tutorial.json`` in the same +order and with the same step numbers, and the Config tab's Settings Backup +and Star Citizen Tools boxes have steps of their own.""" +import json +import re +from pathlib import Path +from types import SimpleNamespace + +import pytest + +ROOT = Path(__file__).parent.parent +_NUMBER = re.compile(r"^(\d+)\.\s") + + +def _english_steps() -> list[dict]: + return json.loads((ROOT / "assets" / "tutorial.json").read_text(encoding="utf-8"))["steps"] + + +def _translated_languages() -> list[str]: + return sorted( + path.parent.name for path in (ROOT / "languages").glob("*/ui.json") + if "tutorial" in json.loads(path.read_text(encoding="utf-8")) + ) + + +def _number(title: str): + match = _NUMBER.match(title) + return int(match[1]) if match else None + + +def test_some_languages_translate_the_tour(): + assert len(_translated_languages()) >= 10 + + +@pytest.mark.parametrize("language", _translated_languages()) +def test_a_translated_tour_matches_the_english_one(language): + english = _english_steps() + tour = json.loads((ROOT / "languages" / language / "ui.json") + .read_text(encoding="utf-8"))["tutorial"] + assert list(tour) == [step["id"] for step in english] + for step in english: + title = tour[step["id"]]["title"]["at"] + assert _number(title) == _number(step["title"]), (step["id"], title) + + +def test_the_tour_covers_settings_backup_and_star_citizen_tools(): + from src.gui.main_window import MainWindow + + ids = [step["id"] for step in _english_steps()] + assert ids[ids.index("cfg_tools") + 1:ids.index("cfg_tools") + 3] == [ + "cfg_backup", "cfg_sc_tools"] + backup, sc_tools = object(), object() + stub = SimpleNamespace(config_tab=SimpleNamespace(_backup_group=backup, + _sc_tools_group=sc_tools)) + wiring = MainWindow._tutorial_step_wiring(stub) + assert wiring["cfg_backup"]["target"]() is backup + assert wiring["cfg_sc_tools"]["target"]() is sc_tools From d96717f35519e53af572e37784db82a012dbe612 Mon Sep 17 00:00:00 2001 From: Stealrull <93751735+Stealrull@users.noreply.github.com> Date: Fri, 6 Nov 2026 17:48:28 +1000 Subject: [PATCH 03/32] Replace the perf proposal label with #522 and #529 (#547) #522 had no number while it was a draft, so comments, docstrings, the CLAUDE.md guides and test descriptions called its work "perf proposal". #522 and #529 shipped in 3.0.0, so each of the 184 occurrences, in 26 files, now names the PR that did the work. No code changed: every changed Python file has the same AST once docstrings are taken out. Fixes #547 Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 10 +-- scripts/CLAUDE.md | 4 +- scripts/generate_enhancements_ini.py | 61 +++++++++-------- src/gui/CLAUDE.md | 10 +-- src/gui/config_tab.py | 2 +- src/gui/main_window.py | 34 +++++----- src/gui/workers.py | 20 +++--- src/utils/CLAUDE.md | 12 ++-- src/utils/pak_extractor.py | 86 ++++++++++++------------ tests/CLAUDE.md | 26 +++---- tests/event_loop.py | 4 +- tests/test_base_ini_freshness.py | 3 +- tests/test_battaglia_mineable_rs.py | 2 +- tests/test_bp_fallback_tags_language.py | 4 +- tests/test_calculated_reward_titles.py | 2 +- tests/test_clear_cache.py | 2 +- tests/test_commodity_journal.py | 4 +- tests/test_event_loop_helper.py | 2 +- tests/test_extraction_stop.py | 18 ++--- tests/test_extraction_working_folder.py | 38 +++++------ tests/test_generator_single_pass.py | 2 +- tests/test_linked_cache_folder.py | 2 +- tests/test_mission_route_title.py | 8 +-- tests/test_pending_cache_cleanup.py | 8 +-- tests/test_refused_cache_window_paths.py | 2 +- tests/test_robust_rmtree.py | 4 +- 26 files changed, 184 insertions(+), 186 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 4ea7bce7..28ce6493 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -84,7 +84,7 @@ On first run, the app sets up user data dirs, validates the SC install path, and ### DataForge extraction is a four-step pipeline The "Extract DataForge from P4K" button triggers, in order: -1. Unpack Data.p4k → entity XMLs via `src/utils/pak_extractor.py`. The bundled unp4k and unforge are .NET tools from the `odw-fast-unp4k` fork: unp4k reads Data.p4k's directory once and extracts through the archive it opened for the listing (on every core when there are several matches), and unforge reads the records on every core, leaving out the record folders the generator never reads (perf proposal; which commits the bundled exes are built from is in `src/utils/CLAUDE.md` → `pak_extractor.py`). unp4k and unforge work in a per-run folder beside the DataForge cache rather than in `%TEMP%` (#471; `%TEMP%` is only the fallback for a cache on OneDrive, a network share, a removable or network drive, or nested too deep), and closing the app stops them; see `src/utils/CLAUDE.md` → `pak_extractor.py`. +1. Unpack Data.p4k → entity XMLs via `src/utils/pak_extractor.py`. The bundled unp4k and unforge are .NET tools from the `odw-fast-unp4k` fork: unp4k reads Data.p4k's directory once and extracts through the archive it opened for the listing (on every core when there are several matches), and unforge reads the records on every core, leaving out the record folders the generator never reads (#522 and #529; which commits the bundled exes are built from is in `src/utils/CLAUDE.md` → `pak_extractor.py`). unp4k and unforge work in a per-run folder beside the DataForge cache rather than in `%TEMP%` (#471; `%TEMP%` is only the fallback for a cache on OneDrive, a network share, a removable or network drive, or nested too deep), and closing the app stops them; see `src/utils/CLAUDE.md` → `pak_extractor.py`. 2. Apply declarative patches from `patches/` via `src/utils/dataforge_patcher.py` to fix upstream CIG data bugs. Idempotent; runs on the extracted cache even when step 1 is skipped as fresh. 3. Run `scripts/generate_enhancements_ini.py` over the XMLs for every enabled category. Every file it writes is replaced in one step (`src/utils/atomic_io.py`), so a run cut off part-way leaves each file whole, old or new. Closing the app stops it at its next job boundary, never during its writes (see `src/gui/CLAUDE.md` → *Closing stops a running extraction*). Nothing hashes the cache any more: the old diff manifest, which nothing read, is gone (#495). 4. Reload all strings to refresh the table. @@ -133,8 +133,8 @@ Anchor examples already in-tree: `COL_*` constants in `src/gui/string_table_mode | User data root (portable build) | `/data/` when frozen; `/portable_data/` unfrozen | | **Per-channel data** | `{user_data_root}\{LIVE\|PTU\|EPTU\|HOTFIX\|TECH-PREVIEW}\` — 0.9.3+ nests user.ini / cache / backups / dataforge under the active channel so each SC channel is isolated. Migrator: `AppSettings.migrate_game_path_to_channel_layout()`. | | User overrides | `{user_data_root}\{active_channel}\user.ini` (legacy `overrides.ini` auto-migrated) | -| Cached sources | `{user_data_root}\{active_channel}\cache\` — only `base.ini` post-1.0 (four legacy URL sources retired in 0.7.0), with its `.base_ini_stamp` (the size of the Data.p4k it was extracted from and base.ini's own size, perf proposal); enhancement INIs live here too. Non-English languages cache their downloaded base.ini at `cache\lang\{language}\base.ini` (2.0); resolved via `AppSettings.get_base_ini_path(language)` | -| DataForge cache | `%LOCALAPPDATA%\Smart Citizen\{active_channel}\cache\dataforge\` by default (entity XMLs from Data.p4k); overrideable independently of the user-data root via the `CACHE_DIR` registry key (1.4.1+ — `AppSettings.get_cache_dir_override()` / `set_cache_dir()`, wired to the Config tab's *DataForge Cache Folder*; the cache override does *not* fall back to the user-data override). Moved out of `Documents\` in 1.x — `migrate_dataforge_cache_to_local()` relocates the legacy `…\cache\dataforge\` tree on first launch. Resolved via `AppSettings.get_dataforge_cache_dir()`, which creates it, only where the folder is meant to be made (the migrator, an extraction about to fill it or work beside it, a newly picked folder in the Config tab), and via `get_dataforge_cache_leaf()`, which creates nothing, everywhere else: every status check, Clear Cache (#479), and the enhancement generator, which writes into a cache its launcher has already checked is there. So a folder Windows refuses or a drive that is gone reads as not extracted (#494). An extraction works in a sibling `…\cache\dataforge.extracting-XXXXXXXX\` folder (up to about 4.5 GB free while it runs, checked before any tool starts (#482), deleted when it ends, or after a successful DataForge run in the background while enhancements are generated, with the old cache set aside in it (perf proposal); the global.ini extraction in `globalini.extracting-XXXXXXXX`) so the heavy writes land on the cache's drive (#471). A cache on OneDrive, a network share, a removable or network drive, or nested too deep sends the run to `%TEMP%\SmartCitizen-DataForge-XXXXXXXX\` instead (`SmartCitizen-GlobalIni-` for global.ini). A DataForge run there first makes and removes one empty `dataforge.extracting-` folder beside the cache folder, so a folder that would not let the wipe make the cache folder again is refused before any tool runs, with the old cache kept (#494). A cache folder that is itself a junction or a symbolic link is refused before an extraction runs, with the old cache and the link kept, because nothing is deleted through a link (#491). | +| Cached sources | `{user_data_root}\{active_channel}\cache\` — only `base.ini` post-1.0 (four legacy URL sources retired in 0.7.0), with its `.base_ini_stamp` (the size of the Data.p4k it was extracted from and base.ini's own size, #522); enhancement INIs live here too. Non-English languages cache their downloaded base.ini at `cache\lang\{language}\base.ini` (2.0); resolved via `AppSettings.get_base_ini_path(language)` | +| DataForge cache | `%LOCALAPPDATA%\Smart Citizen\{active_channel}\cache\dataforge\` by default (entity XMLs from Data.p4k); overrideable independently of the user-data root via the `CACHE_DIR` registry key (1.4.1+ — `AppSettings.get_cache_dir_override()` / `set_cache_dir()`, wired to the Config tab's *DataForge Cache Folder*; the cache override does *not* fall back to the user-data override). Moved out of `Documents\` in 1.x — `migrate_dataforge_cache_to_local()` relocates the legacy `…\cache\dataforge\` tree on first launch. Resolved via `AppSettings.get_dataforge_cache_dir()`, which creates it, only where the folder is meant to be made (the migrator, an extraction about to fill it or work beside it, a newly picked folder in the Config tab), and via `get_dataforge_cache_leaf()`, which creates nothing, everywhere else: every status check, Clear Cache (#479), and the enhancement generator, which writes into a cache its launcher has already checked is there. So a folder Windows refuses or a drive that is gone reads as not extracted (#494). An extraction works in a sibling `…\cache\dataforge.extracting-XXXXXXXX\` folder (up to about 4.5 GB free while it runs, checked before any tool starts (#482), deleted when it ends, or after a successful DataForge run in the background while enhancements are generated, with the old cache set aside in it (#522); the global.ini extraction in `globalini.extracting-XXXXXXXX`) so the heavy writes land on the cache's drive (#471). A cache on OneDrive, a network share, a removable or network drive, or nested too deep sends the run to `%TEMP%\SmartCitizen-DataForge-XXXXXXXX\` instead (`SmartCitizen-GlobalIni-` for global.ini). A DataForge run there first makes and removes one empty `dataforge.extracting-` folder beside the cache folder, so a folder that would not let the wipe make the cache folder again is refused before any tool runs, with the old cache kept (#494). A cache folder that is itself a junction or a symbolic link is refused before an extraction runs, with the old cache and the link kept, because nothing is deleted through a link (#491). | | Crash dumps + log exports | `{user_data_root}\logs\` (NOT per-channel — a crash can fire before the channel context is established, e.g. during startup migrators). Resolved via `AppSettings.get_logs_dir()`; created lazily on first crash or export. | | Enhancement INIs | next to the active language's base.ini via `AppSettings.get_enhancements_dir(language)`: `{user_data_root}\{active_channel}\cache\` for English, `cache\lang\{language}\` otherwise (`ships_desc_enhancements.ini`, `components_desc_enhancements.ini`, `ship_weapons_desc_enhancements.ini`, `fps_weapons_desc_enhancements.ini` (also carries the #492 armor weight tags), `mission_rewards_enhancements.ini`, `commodity_crafting_enhancements.ini`, `journal_enhancements.ini`, `missile_enhancements.ini`, `medical_consumables_enhancements.ini`; plus the `.dataforge_stamp` freshness marker and the `.tag_config_stamp` Tag Builder marker, and `crafting_index.json` for the overlay drawer's Crafting search, written whenever Commodities or Journal is generated) | | Overlay scan samples | `{user_data_root}\logs\sco_samples\` (not per-channel): a crop of each distinct signature reading while the overlay's scan decoder runs, at most 200 per session, and a session's first save prunes the folder to the newest 1000 (`sco_overlay.prune_samples`), so misreads can become reference glyphs. Tuning only: written only when the `SMART_CITIZEN_SCO_SAMPLES` environment variable (`sco_overlay.SAVE_SAMPLES_ENV`) is set when the app starts; players never get it and the app has no switch for it. Resolved via `AppSettings.get_sco_samples_dir()` at save time | @@ -166,14 +166,14 @@ Anchor examples already in-tree: `COL_*` constants in `src/gui/string_table_mode | Change backup behavior | `src/gui/main_window.py` | `manage_backups()` | | Modify P4K extraction | `src/utils/pak_extractor.py` | `extract_dataforge()` | | Change the extraction free-space check | `src/utils/pak_extractor.py` | `_require_free_space()`, `_dataforge_needs()`, `_require_room_for_new_cache()`, the #482 constants | -| Change how a folder tree is deleted (working folder, old cache, Clear Cache, queued cleanup) | `src/utils/pak_extractor.py` | `robust_rmtree()`, `_delete_files_first()` (the threaded file pass, perf proposal), `_DELETE_THREADS` | +| Change how a folder tree is deleted (working folder, old cache, Clear Cache, queued cleanup) | `src/utils/pak_extractor.py` | `robust_rmtree()`, `_delete_files_first()` (the threaded file pass, #522), `_DELETE_THREADS` | | Change enhancements generation | `scripts/generate_enhancements_ini.py` | (standalone script) | | Add performance profiling | `src/utils/perf.py` | `@timed` decorator | | Change user data paths | `src/utils/settings.py` | `AppSettings.get_user_data_dir()` | | Change DataForge freshness | `src/utils/settings.py`, `src/gui/main_window.py` | `dataforge_cache_is_fresh()` | | Change how generated files are written | `src/utils/atomic_io.py` | `atomic_writer()`, `atomic_write_text()`, `read_bytes_retrying()` (the enhancement INIs, `crafting_index.json`, both stamps, the lookup pickles and patched cache XMLs are swapped in whole, never written in place) | | Change the enhancements out-of-date check | `src/utils/settings.py`, `src/gui/main_window.py`, `src/gui/workers.py` | `enhancements_built_from_current_dataforge()` (the one definition of "current", shared by the startup check, the Generate button and the language switch), `_check_enhancements_freshness()`, `EnhancementsGeneratorWorker.run()` (empties `.dataforge_stamp` before the generator runs, writes it after) | -| Change base.ini freshness (the Extract prompt and the Extract button's red nudge) | `src/utils/pak_extractor.py` | `base_ini_is_fresh()`, `BASE_INI_STAMP` (perf proposal: Data.p4k's size, as #209 for DataForge, with base.ini's own size so the stamp speaks only for the base.ini it was written for; read by `MainWindow._check_p4k_freshness` and `ConfigTab._refresh_p4k_status`) | +| Change base.ini freshness (the Extract prompt and the Extract button's red nudge) | `src/utils/pak_extractor.py` | `base_ini_is_fresh()`, `BASE_INI_STAMP` (#522: Data.p4k's size, as #209 for DataForge, with base.ini's own size so the stamp speaks only for the base.ini it was written for; read by `MainWindow._check_p4k_freshness` and `ConfigTab._refresh_p4k_status`) | | Change Run in Background or the job lock | `src/gui/main_window.py`, `src/gui/config_tab.py`, `src/gui/enhancements_tab.py`, `src/gui/simple_mode_widget.py`, `src/gui/theme.py` | `_confirm_stop_dataforge_extraction()`, `_send_dataforge_run_to_background()`, `_confirm_stop_generation()` (the same question on the generation dialog), `_stop_generation_from_dialog()`, `_send_generation_to_background()`, `_job_running()`, `_closing_would_wait()` (what Import Settings and an update's install ask: `_job_running()` plus a background working-folder delete), `_refresh_job_lock()`, `_dataforge_run_matches_settings()`, `_ask_after_the_job()` (a startup question a job put off, asked again after the load that ends it), `ConfigTab.set_data_locations_locked()`, `EnhancementsTab.set_changes_locked()`, `SimpleModeWidget.set_busy(busy, reason)`, `theme.needs_apply_style()`, `theme.dimmed_color()` (#483) | | Change stats/favorites UI | `src/gui/enhancements_tab.py` | `setup_ui()` | | Manage Config tab UI | `src/gui/config_tab.py` | `setup_ui()`, drag-drop hierarchy | diff --git a/scripts/CLAUDE.md b/scripts/CLAUDE.md index 143ded1e..c5e86955 100644 --- a/scripts/CLAUDE.md +++ b/scripts/CLAUDE.md @@ -38,5 +38,5 @@ Closing Smart Citizen stops a running generation. `main()` takes `should_cancel` ### Mining Compendium journal — per-ore Base Resource Signature line (2.2.0) `MINEABLE_RS_VALUES` (see the Battaglia entry above for sourcing) has a second consumer: the "Mining Fundamentals #2: Where to Mine" journal entry (`Journal_General_Mining_Compendium_Content`), reformatted by `scan_crafting_blueprints`'s per-mineral block builder into `Name` / `Locations:` / `Used To Craft:` blocks. Each ore's block now gets a `Base Resource Signature: {value}` line (inserted right after the name header, before Locations) whenever `MINEABLE_RS_VALUES` has a confirmed value for it; ores without one (the journal lists 35, the table covers 26) are left unchanged, no placeholder. `_MINING_COMPENDIUM_ORE_ALIASES` bridges the one known spelling mismatch between the journal's own prose and the DataForge loc-key convention (`"Savrilium"` in-journal vs `mineabletype_primary_savrillium`). Locked by `TestMiningCompendiumRSTagging` in `tests/test_commodity_journal.py`. -### DataForge XMLs read as few times as possible, and only XML facts in the lookup pickles (perf proposal) -`generate_enhancements_ini.py` parses each XML as few times as it can, and the `.lookups` pickles it reshapes hold facts read from the XML, never text made from base.ini or the Tag Builder config, so each is keyed on the DataForge build alone (`_LOOKUP_VERSIONS` records the shapes). `scitem_lookups` (`_scitem_facts`, the ~24k scitem files) and `blueprint_pools` (`_blueprint_pool_facts`) are turned into names and `[CLASS-Sx-grade]` tags on every run (`_render_scitem_lookups`, and the name loop in `build_blueprint_pool_lookup`), so a Tag Builder change or the annotate toggle re-renders them instead of parsing every scitem XML again. Because those two hold no loc text, the English-loc fallback mark (`en_loc_cache_mark`, for a translated run whose English base.ini is missing) keys only `standings`, the one pickle still built from the loc: a fallback run and the next English run share the facts pickles, and each renders its own names from them. `_component_name_tag` and `_mining_laser_component_tag` read their XML facts through `_component_tag_facts` / `_mining_laser_name_attr` and share the rendering (`_component_name_tag_from_facts`, `_mining_laser_tag_from_name`) with the cached path. The flight controller and armour lookups (`build_controller_lookup`, `build_armor_lookup`) hold the attribute values the ship block shows (`_controller_flight_stats`, `_armor_stats`) instead of lxml trees, still formatted in `enhancements_ship_dataforge`, so an armour value `_fmt` cannot format fails only its ship, as before. Those two are built on every run (about 0.2 s on LIVE) and never pickled: DataForge patches are applied again before every generation without a re-extraction, so a pickle keyed on the DataForge build would keep a patched controller or armour XML's old values until the next extraction. A pickle must hold builtins only: the app loads this script under a module name of its own, so a pickled class of the script would not load. The scitem walk finds the magazine with lxml's own tag filter when a file is ASCII-compatible and holds no `__polymorphicType` (`_holds_no_poly_type_attr`), which is every file on LIVE today, and keeps the per-element `_poly_type` loop otherwise. `_run_gen_missions` reads each pu_missions XML once (`_scan_pu_missions`: the #165 choice between a body with and without hostiles waits until every file is read) and each contractgen XML and contract template once (`scan_contract_generators` takes `roots` and `template_lookup`; the Battaglia RS tags and the calculated-reward diagnostic run on each root just before it, their errors held and raised where their own calls raised them, and the walk is finished for them if it stops early; a failure listing the files still ends the run). `scan_crafting_blueprints` parses each blueprint once, reads a carryable only when its filename names a commodity, and lowercases the loc keys once (`_commodity_loc_keys`), for the commodity entries, the Mining Compendium and the SCO crafting index (#474) alike. The hot loops parse through `_parse_xml_file` (the bytes read in one call, then `ET.fromstring`: the same parser and errors as `ET.parse`, and faster on Windows). On a LIVE cache (`main()` with `max_workers=1`, medians of three interleaved runs of the 3.0.0 code with #474 and #477 and of the perf proposal's final code, each with its own `src/`) a regeneration went from 11.9 s to 6.3 s, a components Tag Builder change from 31.0 s to 6.2 s, and the first generation after an extraction from 33.8 s to 24.0 s, which stays bound by parsing the ~1 GB of scitem XML once; the peak working set fell from about 320 MB to 190 MB. None of this changes a byte of output: every file the generator writes (the nine INIs and `crafting_index.json`) matched the unchanged code on a real LIVE cache, cold, warm, after a components change, with non-default components, commodities and armor configs with the annotate toggle on and off, and on a run whose English base.ini is missing. Run that comparison again for any change here. Two outputs used to depend on the string hash seed, because each took the order of a set: three courier titles in `mission_rewards_enhancements.ini` (`_derive_route_fragment` took the first of a set of bodies) and the order of the materials in `crafting_index.json` (`scan_crafting_blueprints` met each blueprint's materials in set order, #474's code). Both now sort (perf proposal), so two runs of this code match whatever the seed. An endpoint some of a title's agreeing bodies write bare and others with `|Address` also takes the `|Address` form (`_agreed_endpoint_tokens`), so a route follows the tokens rather than which body's greeting sorts first; on a LIVE cache that changed one title against the sorting alone, `Covalex_courier_dc_small_title_001`, whose `Dropoff1` was bare. Against code from before those fixes, set `PYTHONHASHSEED=0` on both sides, and expect those two to differ: the titles give an endpoint their bodies write both bare and with `|Address` its `|Address` form, where the old code gave one form or the other at random, and the index holds the same JSON with its materials in a fixed order. Locked by `tests/test_generator_single_pass.py` and `tests/test_mission_route_title.py` (the route for every order of the bodies, and the `|Address` form), and by `tests/test_battaglia_mineable_rs.py`, `tests/test_calculated_reward_titles.py` and `tests/test_commodity_journal.py`, which read the RS tags, the calculated-reward titles and the blueprint material UUIDs off the code a generation runs: the separate scans they used to call were removed. +### DataForge XMLs read as few times as possible, and only XML facts in the lookup pickles (#522) +`generate_enhancements_ini.py` parses each XML as few times as it can, and the `.lookups` pickles it reshapes hold facts read from the XML, never text made from base.ini or the Tag Builder config, so each is keyed on the DataForge build alone (`_LOOKUP_VERSIONS` records the shapes). `scitem_lookups` (`_scitem_facts`, the ~24k scitem files) and `blueprint_pools` (`_blueprint_pool_facts`) are turned into names and `[CLASS-Sx-grade]` tags on every run (`_render_scitem_lookups`, and the name loop in `build_blueprint_pool_lookup`), so a Tag Builder change or the annotate toggle re-renders them instead of parsing every scitem XML again. Because those two hold no loc text, the English-loc fallback mark (`en_loc_cache_mark`, for a translated run whose English base.ini is missing) keys only `standings`, the one pickle still built from the loc: a fallback run and the next English run share the facts pickles, and each renders its own names from them. `_component_name_tag` and `_mining_laser_component_tag` read their XML facts through `_component_tag_facts` / `_mining_laser_name_attr` and share the rendering (`_component_name_tag_from_facts`, `_mining_laser_tag_from_name`) with the cached path. The flight controller and armour lookups (`build_controller_lookup`, `build_armor_lookup`) hold the attribute values the ship block shows (`_controller_flight_stats`, `_armor_stats`) instead of lxml trees, still formatted in `enhancements_ship_dataforge`, so an armour value `_fmt` cannot format fails only its ship, as before. Those two are built on every run (about 0.2 s on LIVE) and never pickled: DataForge patches are applied again before every generation without a re-extraction, so a pickle keyed on the DataForge build would keep a patched controller or armour XML's old values until the next extraction. A pickle must hold builtins only: the app loads this script under a module name of its own, so a pickled class of the script would not load. The scitem walk finds the magazine with lxml's own tag filter when a file is ASCII-compatible and holds no `__polymorphicType` (`_holds_no_poly_type_attr`), which is every file on LIVE today, and keeps the per-element `_poly_type` loop otherwise. `_run_gen_missions` reads each pu_missions XML once (`_scan_pu_missions`: the #165 choice between a body with and without hostiles waits until every file is read) and each contractgen XML and contract template once (`scan_contract_generators` takes `roots` and `template_lookup`; the Battaglia RS tags and the calculated-reward diagnostic run on each root just before it, their errors held and raised where their own calls raised them, and the walk is finished for them if it stops early; a failure listing the files still ends the run). `scan_crafting_blueprints` parses each blueprint once, reads a carryable only when its filename names a commodity, and lowercases the loc keys once (`_commodity_loc_keys`), for the commodity entries, the Mining Compendium and the SCO crafting index (#474) alike. The hot loops parse through `_parse_xml_file` (the bytes read in one call, then `ET.fromstring`: the same parser and errors as `ET.parse`, and faster on Windows). On a LIVE cache (`main()` with `max_workers=1`, medians of three interleaved runs of the 3.0.0 code with #474 and #477 and of #522's final code, each with its own `src/`) a regeneration went from 11.9 s to 6.3 s, a components Tag Builder change from 31.0 s to 6.2 s, and the first generation after an extraction from 33.8 s to 24.0 s, which stays bound by parsing the ~1 GB of scitem XML once; the peak working set fell from about 320 MB to 190 MB. None of this changes a byte of output: every file the generator writes (the nine INIs and `crafting_index.json`) matched the unchanged code on a real LIVE cache, cold, warm, after a components change, with non-default components, commodities and armor configs with the annotate toggle on and off, and on a run whose English base.ini is missing. Run that comparison again for any change here. Two outputs used to depend on the string hash seed, because each took the order of a set: three courier titles in `mission_rewards_enhancements.ini` (`_derive_route_fragment` took the first of a set of bodies) and the order of the materials in `crafting_index.json` (`scan_crafting_blueprints` met each blueprint's materials in set order, #474's code). Both now sort (#522), so two runs of this code match whatever the seed. An endpoint some of a title's agreeing bodies write bare and others with `|Address` also takes the `|Address` form (`_agreed_endpoint_tokens`), so a route follows the tokens rather than which body's greeting sorts first; on a LIVE cache that changed one title against the sorting alone, `Covalex_courier_dc_small_title_001`, whose `Dropoff1` was bare. Against code from before those fixes, set `PYTHONHASHSEED=0` on both sides, and expect those two to differ: the titles give an endpoint their bodies write both bare and with `|Address` its `|Address` form, where the old code gave one form or the other at random, and the index holds the same JSON with its materials in a fixed order. Locked by `tests/test_generator_single_pass.py` and `tests/test_mission_route_title.py` (the route for every order of the bodies, and the `|Address` form), and by `tests/test_battaglia_mineable_rs.py`, `tests/test_calculated_reward_titles.py` and `tests/test_commodity_journal.py`, which read the RS tags, the calculated-reward titles and the blueprint material UUIDs off the code a generation runs: the separate scans they used to call were removed. diff --git a/scripts/generate_enhancements_ini.py b/scripts/generate_enhancements_ini.py index 472569c6..8b16e272 100644 --- a/scripts/generate_enhancements_ini.py +++ b/scripts/generate_enhancements_ini.py @@ -426,7 +426,7 @@ def write_ini(path: Path, entries: dict[str, str]) -> None: # prefixed bp_dir/entity_dir callers compare them against and crashing # scan_crafting_blueprints's xml_file.relative_to(bp_dir) with "not in # the subpath of". Bump flushes any pre-#231 pickle. -# scitem_lookups v11 / blueprint_pools v16 (perf proposal) — both pickles +# scitem_lookups v11 / blueprint_pools v16 (#522) — both pickles # now hold only XML facts, keyed on the DataForge build alone: no loc # text, no Tag Builder config, no annotate toggle and no English-loc # fallback mark (the extra_key is gone). The names and the @@ -443,7 +443,7 @@ def write_ini(path: Path, entries: dict[str, str]) -> None: # bump, a regen against an unchanged DataForge reuses pools baked # before those fixes and the mission text stays untagged/garbled. # v15: name_fallback_tags read the English loc (see the history above). - # v16: XML facts only (perf proposal, see the history above). + # v16: XML facts only (#522, see the history above). "blueprint_pools": "v16", "scitem_lookups": "v11", "standings": "v3", @@ -482,7 +482,7 @@ def _cached_lookup(forge_dir: Path, name: str, builder, extra_key: str = ""): the version in _LOOKUP_VERSIONS; passing a different *extra_key* (used by standings for the English-loc fallback mark; scitem_lookups and blueprint_pools used it for the components Tag Builder config until - they came to hold XML facts only, perf proposal) bumps the third + they came to hold XML facts only, #522) bumps the third segment. An unreadable pickle (a truncated one from a run cut off before these writes became atomic, say) is rebuilt, with an INFO line, since a rebuild re-reads its whole subtree. The pickle is @@ -530,7 +530,7 @@ def _build_xml_path_index(records_dir: Path) -> dict[str, list[str]]: The walk stays rglob's, so the folders keep rglob's order, which every reader's file order follows. Only the relative folder key is worked out once per folder (keyed on the path text up to the last separator) rather - than with Path.relative_to for each of the ~31k files (perf proposal). + than with Path.relative_to for each of the ~31k files (#522). """ idx: dict[str, list[str]] = defaultdict(list) # type: ignore[assignment] key_by_folder: dict[str, str] = {} @@ -590,7 +590,7 @@ def _parse_xml_file(xml_file: Path) -> ET.Element: and handing them to lxml is faster on Windows than letting lxml read the file itself: on a LIVE cache the ~2.6k pu_missions XMLs parse in 0.53 s instead of 0.73 s, the ~8.5k ships scitem XMLs in 2.1 s instead of - 2.8 s (perf proposal).""" + 2.8 s (#522).""" return ET.fromstring(xml_file.read_bytes()) @@ -971,7 +971,7 @@ def _component_name_tag(desc_value: str, root: ET.Element | None = None, The XML half is read by :func:`_component_tag_facts`, so the scitem lookup can cache those facts once per DataForge build and render the tags from them with :func:`_component_name_tag_from_facts`, the one - implementation both paths share (perf proposal). + implementation both paths share (#522). """ return _component_name_tag_from_facts( desc_value, _component_tag_facts(root), config=config, component_type=component_type @@ -1238,7 +1238,7 @@ def _mining_laser_tag_from_name(desc_value: str, name_attr: str, mining_tag_cfg: "TagConfig | None") -> str | None: """:func:`_mining_laser_component_tag` for a known mining laser, from its entity class name, so the scitem lookup can render the tag from - cached facts with the same code (perf proposal).""" + cached facts with the same code (#522).""" if mining_tag_cfg is None or render_tag is None: return None size = None @@ -1561,7 +1561,7 @@ def _agreed_endpoint_tokens(per_body: list[dict]) -> dict: endpoints, invisible to a static scan; treating them as vetoes would strip routes that demonstrably resolve in-game. First body's order/tokens win, except that a var some agreeing bodies write bare and others with - ``|Address`` takes the ``|Address`` form (perf proposal): the bodies are + ``|Address`` takes the ``|Address`` form (#522): the bodies are sorted, and pooled bodies differ in their greetings, so the first one's form was down to which greeting sorts first. """ @@ -1612,7 +1612,7 @@ def _derive_route_fragment(desc_bodies: list[str], cfg=None, loc=None, expand_ca # _agreed_endpoint_tokens) and callers build the list from a set, whose # order changes with every run's string hashing. Unsorted, three courier # titles flipped between the bare and |Address forms of Pickup1 and - # Dropoff1 from one generation to the next (perf proposal); a var some + # Dropoff1 from one generation to the next (#522); a var some # bodies write bare and others with |Address now takes |Address whatever # the order. for body in sorted(b for b in desc_bodies if b): @@ -3673,7 +3673,7 @@ def build_blueprint_pool_lookup( config) is cached there per DataForge build as the ``blueprint_pools`` pickle, and only the names below are worked out per run, so a Tag Builder change or the annotate toggle no - longer re-parses the pool and blueprint XMLs (perf proposal). + longer re-parses the pool and blueprint XMLs (#522). Returns: Tuple of: @@ -3755,7 +3755,7 @@ def _builder() -> list: def _blueprint_pool_facts(pool_dir: Path, bp_dir: Path) -> list: """The XML half of :func:`build_blueprint_pool_lookup`, with no names, - tags or config, so it can be cached per DataForge build (perf proposal). + tags or config, so it can be cached per DataForge build (#522). Returns ``[(pool __ref UUID, pool filename stem, rewards), ...]`` in file order, one entry per pool record with a __ref, where *rewards* is @@ -4027,7 +4027,7 @@ def _battaglia_mineable_rs_in( *template_lookup* is :func:`_build_template_lookup` of the sibling contracttemplates folder. _run_gen_missions runs this on each root of the contractgen parse it shares with scan_contract_generators, in file - order (perf proposal: it used to be a separate walk of its own). + order (#522: it used to be a separate walk of its own). """ for contract in root.findall(".//CareerContract") + root.findall(".//Contract"): title_param = contract.find(".//ContractStringParam[@param='Title']") @@ -4142,7 +4142,7 @@ def scan_contract_generators( *template_lookup* (:func:`_build_template_lookup` of the sibling contracttemplates folder) and *roots* (the parsed contractgen XML roots, in file order) let _run_gen_missions hand over the parse it shares with - the Battaglia RS-tag and calculated-reward scans (perf proposal); each + the Battaglia RS-tag and calculated-reward scans (#522); each is worked out here when omitted, as it always was, so a call with the folder alone still works. A generation always hands both over: only the tests that call this directly (test_blueprint_pools.py, @@ -4578,7 +4578,7 @@ def _calculated_reward_titles_in(root: ET.Element, titles: set[str]) -> None: only the end-of-run diagnostic needs. _run_gen_missions runs it on each root of the contractgen parse scan_contract_generators already makes, rather than walking the files again once a zero-XP title turns up - (perf proposal). + (#522). """ for contract in root.findall(".//CareerContract") + root.findall(".//Contract"): if contract.find(".//ContractResult_CalculatedReward") is None: @@ -4692,7 +4692,7 @@ def _build_uuid_to_commodity( fname, ) # A carryable whose filename names no commodity can add nothing, - # so only the others are read (perf proposal: 658 of the 2,570 + # so only the others are read (#522: 658 of the 2,570 # carryables on a LIVE cache). if not m: continue @@ -4715,8 +4715,7 @@ def _commodity_loc_keys(loc: dict[str, str]) -> list[tuple[str, str]]: """``(key, key.lower())`` for every ``items_commodities_*`` key in *loc* (case-insensitive), in loc order: what every :func:`_discover_commodity_loc_pairs` call scans, worked out once - instead of lowercasing all ~91k loc keys for each commodity (perf - proposal).""" + instead of lowercasing all ~91k loc keys for each commodity (#522).""" keys = [] for key in loc: klow = key.lower() @@ -5123,7 +5122,7 @@ def scan_crafting_blueprints( logger.info("No crafting blueprints directory found") return {}, {} - # Step 1: Read every blueprint once (perf proposal: steps 1 and 3 each + # Step 1: Read every blueprint once (#522: steps 1 and 3 each # used to parse them all): its category, its display name and the # material UUIDs it names, collecting those UUIDs as it goes. resource_uuids: set[str] = set() @@ -5169,7 +5168,7 @@ def scan_crafting_blueprints( materials.add(uuid_names[uid]) # Sorted, so each material is first met in the same place on every # run: crafting_index.json keeps that order, and a set's order changes - # with each run's string hashing (perf proposal review). + # with each run's string hashing (#522 review). for mat in sorted(materials): commodity_items[mat].append((category, item_name)) @@ -5178,7 +5177,7 @@ def scan_crafting_blueprints( # commodity strings) — surfacing that in the log beats silently writing an # empty enhancements file. # The items_commodities_* keys every loc lookup below scans, lowercased - # once (perf proposal). + # once (#522). commodity_keys = _commodity_loc_keys(loc) if crafting_index is not None: for commodity, pairs_ in commodity_items.items(): @@ -5690,7 +5689,7 @@ def build_controller_lookup(controller_dir: Path) -> dict[str, tuple]: included so each spaceship entity can find its exact match. The values are the few attributes the ship stats read, not the XML - root (perf proposal). The lookup is built on every run, not pickled per + root (#522). The lookup is built on every run, not pickled per DataForge build: DataForge patches are applied again before every generation without a re-extraction, and a pickle would keep a patched controller's old values until the next extraction. @@ -5744,7 +5743,7 @@ def build_armor_lookup(armor_dir: Path) -> dict[str, tuple]: lowercased for case-insensitive matching. The values are the attributes _armor_stats_block formats, not the XML - root (perf proposal). The formatting stays in enhancements_ship_dataforge, + root (#522). The formatting stays in enhancements_ship_dataforge, so a value it cannot format still fails that one ship, as before. Built on every run, not pickled, for the same reason as build_controller_lookup: DataForge patches are applied again before every generation without a @@ -6136,7 +6135,7 @@ def build_scitem_lookups( ``scitem_lookups`` pickle, and the names and tags are rendered from those facts, *loc* and *tag_config* on every run (:func:`_render_scitem_lookups`). A Tag Builder change used to key the - whole pickle and re-parse all ~24k scitem files (perf proposal). + whole pickle and re-parse all ~24k scitem files (#522). """ if not scitem_dir.exists(): return {}, {}, {}, {} @@ -6178,7 +6177,7 @@ def _scitem_facts( records_dir: Path | None = None, ) -> tuple: """The XML half of :func:`build_scitem_lookups`, with no loc or Tag - Builder input, so it can be cached per DataForge build (perf proposal). + Builder input, so it can be cached per DataForge build (#522). Returns ``(mag_lookup, names, tag_candidates)``, all builtins so the pickle loads under any module name: @@ -7246,7 +7245,7 @@ def _scan_pu_missions( separator: str, ) -> _PuMissionScan: """Read every pu_missions XML once for all of _run_gen_missions' uses - (perf proposal: they were five separate passes, each parsing every + (#522: they were five separate passes, each parsing every file). Each step keeps its own error handling and its file order. *mission_enhancement(root, ambiguous_keys)* is enhancements_mission @@ -7423,7 +7422,7 @@ def _show_title_tag(_field: str) -> bool: # matched against are both English. Reading the run's own language left a # translated run's pools untagged, and a later English run reused them # from the blueprint_pools pickle, which held the tagged names until it - # came to hold XML facts only (perf proposal). + # came to hold XML facts only (#522). name_fallback_tags = ( bare_type_name_tag_lookup(ctx.get("tag_loc") or loc, _comp_cfg) if annotate_descs else {} @@ -7453,7 +7452,7 @@ def _mission_enhancement(root: ET.Element, ambiguous_keys) -> str: show_fields=_mdf, spawn_ambiguous_keys=ambiguous_keys) - # One pass over the pu_missions XMLs (perf proposal; see + # One pass over the pu_missions XMLs (#522; see # _scan_pu_missions) gives their description entries plus everything the # title passes below read from them. #165: a single mission description # is often shared by many pu_missions XMLs with DIFFERENT hostile spawns @@ -7506,7 +7505,7 @@ def _mission_enhancement(root: ET.Element, ambiguous_keys) -> str: bp_dir = records / "crafting" / "blueprints" / "crafting" # Only the XML walk is cached (per DataForge build): the pool names bake # in the English names, the components tags, placement and the annotate - # toggle, so they are worked out from it on every run (perf proposal). + # toggle, so they are worked out from it on every run (#522). # Armor tags (#492) are added later still, when the bullets are written. blueprint_pools, pool_names = build_blueprint_pool_lookup( pool_dir, bp_dir, entity_names, @@ -7520,7 +7519,7 @@ def _mission_enhancement(root: ET.Element, ambiguous_keys) -> str: # One parse of each contractgen XML (and one template lookup) feeds # scan_contract_generators, the Battaglia RS-tag scan and the # calculated-reward diagnostic, which used to walk the files three times - # (perf proposal). The other two run on each root just before + # (#522). The other two run on each root just before # scan_contract_generators sees it. Each keeps its own error handling: # a failure in either is held and raised where that scan's own call # used to raise it, and the walk is finished for them if @@ -8236,7 +8235,7 @@ def _stop_if_asked(where: str) -> None: # next English run to reuse: that run sees a different key and rebuilds. # Only standings is built from en_loc now: scitem_lookups and # blueprint_pools hold XML facts alone, and their names are resolved - # through en_loc on every run (perf proposal). + # through en_loc on every run (#522). en_loc_cache_mark = "" if english_base_ini_path is not None: english_base_ini_path = Path(english_base_ini_path) @@ -8337,7 +8336,7 @@ def _stop_if_asked(where: str) -> None: # build_scitem_lookups renders into entity_name_tags (and so into the # POTENTIAL BLUEPRINTS list entries). Only the XML facts behind them are # pickled, per DataForge build, so a config change re-renders the tags - # instead of re-parsing every scitem XML (perf proposal). + # instead of re-parsing every scitem XML (#522). _components_cfg = (tag_configs or {}).get("components") or DEFAULT_TAG_CONFIGS.get("components") def _build_scitem_pair(): diff --git a/src/gui/CLAUDE.md b/src/gui/CLAUDE.md index 28499d23..4b823122 100644 --- a/src/gui/CLAUDE.md +++ b/src/gui/CLAUDE.md @@ -4,8 +4,8 @@ GUI layer (PyQt6). See the root `CLAUDE.md` for project context and cross-cuttin ## Files -- `main_window.py` — main window with table, toolbar, filters, backup/restore, DataForge extraction trigger, worker orchestration. The largest file in the repo; runs the core workflow load→merge→edit→apply. Workers live in `workers.py`; pure-Python helpers (`validate_applied_file`, `filter_entry_indices`, `markdown_to_html`) live in their own modules and are wrapped by thin `MainWindow` methods. Owns `_stamp_frontend_version` — see *Frontend version stamp on apply* below. Less-frequent toolbar actions (restore backup, clear localization/cache, import/export INI, open loc dir) live under a **More** overflow `QMenu` (2.0). `_on_language_changed(language)` runs the language-switch pipeline (2.0, #30): cancel any in-flight loader, `i18n.set_language()` + `retranslate_ui()`, then `_apply_language_base_source()` repoints the `global` source at that language's base.ini (downloading it via `LanguageBaseDownloadWorker` when a URL is mapped; falling back to a cached copy, then to English), and `_reload_with_language_enhancements()` regenerates that language's enhancements first when the `.dataforge_stamp` is stale (skipped if no DataForge cache: prose loads without stat overlays). `retranslate_ui()` also re-renders the About, FAQ and Legal tabs (#314) and the Help panel (#540) from the new language's docs (`AppSettings.get_localized_doc_path`), which are otherwise rendered only when each is built and on a theme swap. Both go through `_render_document_browsers`, so a theme swap (`refresh_action_buttons`) and a switch cannot render different documents: each used to keep its own list, and the switch's had no Help panel. It also retitles the Help, String Editor and Test Plan panels, whose titles are set only when each is built, and each tab's `retranslate_ui` also sets again the texts its refreshes and loads write, which a switch used to leave in the old language: the Config tab reruns `_populate_channel_combo` (the channel list's not-installed tooltips and hint) and `_refresh_p4k_status`, the Blueprint Tracker renames each filter's first row (`enhancements.blueprints_facet_any`), and the Log tab rewrites its line count (#540). English goes through it too whenever its base.ini exists, because a re-extraction made while another language was selected regenerated only that language. That regeneration is the switch's launch: it clears the Generate button first, as `_run_enhancements_pipeline` does (#292), and leaves the switch's reload to the run. A run that fails still loads the language picked, with the stats it has: `_run_enhancements_generation` keeps its `if_unfinished` in `_after_unfinished_generation`, and the finished slot runs it once the run's error dialog has closed (parked in `_after_error_dialog`, which `_show_error_dialog` runs as it closes). `_run_enhancements_generation` returns whether it started, like `_run_p4k_extraction`, and a start refused because a generation is already running loads the language at once. user.cfg's `g_language` is NOT set here; that happens at window init and apply-to-game time. Its DataForge cache status and freshness checks read the cache folder without creating it, and `_run_dataforge_extraction` reports a cache folder it cannot create in one box before any worker starts (see *Status checks never create the DataForge cache folder* below, #494), as it does a cache folder that is a link (#491). More > Clear Cache (`clear_cache`, #479) deletes the active channel's cached `*.ini` and `*.txt` and, when the user agrees, the DataForge cache in use: the folder `get_dataforge_cache_leaf()` names (the default, the portable build's or a cache folder override), offered only when it holds a cache (its `.p4k_mtime` stamp or its records). It used to look in the data folder's `cache\dataforge`, where the cache lived before 1.x, so the real one was never offered; a leftover there is the queued old-cache cleanup's. A cache folder on a drive or share that is not there is named in the summary and the rest still clears, the file question is skipped when there are no files (a No to the DataForge question then changes nothing, just as a No to the file question), nothing is cleared while an extraction or a generation runs (`clear_cache_busy` in the status bar; the background delete of a finished extraction's working folder, or a sweep of the ones a crash left, does not count, perf proposal, since neither touches the other's folder), the Enhancements tab is refreshed afterwards, a failure is listed in the summary and logged once at WARNING (an ERROR record opened a second box), and every text goes through `tr()`. A channel switch and a data-folder change (`_sync_canonical_source_paths`, through `AppSettings.sync_global_source_path`) and a successful global.ini extraction (`_on_p4k_extract_finished`) keep the `global` source on the selected language's base.ini (`AppSettings.get_global_source_base_ini()`, English when this channel has none yet); they used to point every language at the English one (#502). -- `workers.py` — every `QThread` background worker plus the shared `AnimatedProgressDialog` (its `set_close_guard(guard)` asks before Esc or the title-bar X dismisses it, never on a programmatic `close()`, and only for a dialog without a cancel button, #471; its `set_progress` shows the moving busy bar for a total of 0 and for nothing counted yet, 0 of N, until the first count, #507: a bar standing at 0% looked hung, as the generator's first lookups did for seconds, and the string loader and the update download start at 0 too) and `SelectAllDelegate`. Workers: `FileLoaderWorker` (sources → `StringEntry` list + sort keys), `AppliedStateWorker` (#398 review: runs the already-applied comparison off the GUI thread; takes an injected `compute(snapshot, should_stop)` instead of importing it, because the merge it shares with Apply to Game lives in `main_window.py` and importing that here would be a gui-to-gui cycle; always emits `finished(bool)` once, False being the safe "not applied" answer), `StartupSyncWorker` (refresh URL-backed sources at startup), `EnhancementsGeneratorWorker` (run `scripts/generate_enhancements_ini.py` in-process via `importlib.util`; takes a `language` param so output lands in that language's enhancements dir while annotation paths still parse the English base.ini; captures that folder, the DataForge build key and the tag fingerprint when its run starts, empties `.dataforge_stamp` just before the generator's `main()` and writes the tag stamp and then the DataForge stamp after it returns, see *A cut-off generation is offered again at the next start* below; `requestInterruption()`, which only closing the window sends, stops it before its patches, its module load or `main()`, and inside `main()` at the next job boundary, never during the writes, records it in `stopped` and ends with `finished(False)`, no `error` and nothing at ERROR), `P4kExtractWorker` (unp4k extract of `global.ini`, working beside the DataForge cache it is handed as `scratch_near`), `DataForgeExtractWorker` (unp4k + unforge + post-extract patches; both stop on `requestInterruption()`, record it in `stopped` and then end with `finished(False)`, no `error` and nothing at ERROR, #471; a successful DataForge run keeps its working folder for `take_working_folder()`, any other outcome deletes it before `finished`, and a DataForge run first waits for the background deletes it is handed as `wait_for`, perf proposal), `WorkingFolderDeleteWorker` (deletes that folder in the background, perf proposal, see *Closing stops a running extraction*), `StaleWorkingFolderSweepWorker` (sweeps the working folders a crash or a kill left, perf proposal, same section), `LanguageBaseDownloadWorker` (2.0: fetch a non-English language's global.ini from its mapped URL into `cache\lang\{language}\base.ini`), `LiveBlueprintLogWorker` (SCO, 3.0.0: polls `blueprint_log_scanner.GameLogTail` on the active channel's `Game.log` every `POLL_MS` while the overlay runs, LIVE and HOTFIX only, and emits `blueprints_found(raw names, newest)`; read-only like the Scan, the main thread updates the owned set; stopped with `requestInterruption()` + `_reap_worker`; a lasting poll fault logs one traceback, then debug), `BlueprintLogScanWorker` (#222: scan a channel dir for "Received Blueprint" notifications via `blueprint_log_scanner.scan_channel`, watermark-aware — takes `(channel_dir, since)`, emits a `ScanResult`; stays read-only, all owned-set/watermark mutation happens on the main thread in `MainWindow._on_blueprint_log_scan_finished`). Each emits `progress` (str) and `progress_pct` (completed, total, message); `AnimatedProgressDialog.set_progress` consumes the latter. Exception: `AppUpdateCheckWorker` and `AppUpdateDownloadWorker` live in `src/utils/app_updater.py` because the workers, version comparison, and installer-asset picking belong together (#211). +- `main_window.py` — main window with table, toolbar, filters, backup/restore, DataForge extraction trigger, worker orchestration. The largest file in the repo; runs the core workflow load→merge→edit→apply. Workers live in `workers.py`; pure-Python helpers (`validate_applied_file`, `filter_entry_indices`, `markdown_to_html`) live in their own modules and are wrapped by thin `MainWindow` methods. Owns `_stamp_frontend_version` — see *Frontend version stamp on apply* below. Less-frequent toolbar actions (restore backup, clear localization/cache, import/export INI, open loc dir) live under a **More** overflow `QMenu` (2.0). `_on_language_changed(language)` runs the language-switch pipeline (2.0, #30): cancel any in-flight loader, `i18n.set_language()` + `retranslate_ui()`, then `_apply_language_base_source()` repoints the `global` source at that language's base.ini (downloading it via `LanguageBaseDownloadWorker` when a URL is mapped; falling back to a cached copy, then to English), and `_reload_with_language_enhancements()` regenerates that language's enhancements first when the `.dataforge_stamp` is stale (skipped if no DataForge cache: prose loads without stat overlays). `retranslate_ui()` also re-renders the About, FAQ and Legal tabs (#314) and the Help panel (#540) from the new language's docs (`AppSettings.get_localized_doc_path`), which are otherwise rendered only when each is built and on a theme swap. Both go through `_render_document_browsers`, so a theme swap (`refresh_action_buttons`) and a switch cannot render different documents: each used to keep its own list, and the switch's had no Help panel. It also retitles the Help, String Editor and Test Plan panels, whose titles are set only when each is built, and each tab's `retranslate_ui` also sets again the texts its refreshes and loads write, which a switch used to leave in the old language: the Config tab reruns `_populate_channel_combo` (the channel list's not-installed tooltips and hint) and `_refresh_p4k_status`, the Blueprint Tracker renames each filter's first row (`enhancements.blueprints_facet_any`), and the Log tab rewrites its line count (#540). English goes through it too whenever its base.ini exists, because a re-extraction made while another language was selected regenerated only that language. That regeneration is the switch's launch: it clears the Generate button first, as `_run_enhancements_pipeline` does (#292), and leaves the switch's reload to the run. A run that fails still loads the language picked, with the stats it has: `_run_enhancements_generation` keeps its `if_unfinished` in `_after_unfinished_generation`, and the finished slot runs it once the run's error dialog has closed (parked in `_after_error_dialog`, which `_show_error_dialog` runs as it closes). `_run_enhancements_generation` returns whether it started, like `_run_p4k_extraction`, and a start refused because a generation is already running loads the language at once. user.cfg's `g_language` is NOT set here; that happens at window init and apply-to-game time. Its DataForge cache status and freshness checks read the cache folder without creating it, and `_run_dataforge_extraction` reports a cache folder it cannot create in one box before any worker starts (see *Status checks never create the DataForge cache folder* below, #494), as it does a cache folder that is a link (#491). More > Clear Cache (`clear_cache`, #479) deletes the active channel's cached `*.ini` and `*.txt` and, when the user agrees, the DataForge cache in use: the folder `get_dataforge_cache_leaf()` names (the default, the portable build's or a cache folder override), offered only when it holds a cache (its `.p4k_mtime` stamp or its records). It used to look in the data folder's `cache\dataforge`, where the cache lived before 1.x, so the real one was never offered; a leftover there is the queued old-cache cleanup's. A cache folder on a drive or share that is not there is named in the summary and the rest still clears, the file question is skipped when there are no files (a No to the DataForge question then changes nothing, just as a No to the file question), nothing is cleared while an extraction or a generation runs (`clear_cache_busy` in the status bar; the background delete of a finished extraction's working folder, or a sweep of the ones a crash left, does not count, #522, since neither touches the other's folder), the Enhancements tab is refreshed afterwards, a failure is listed in the summary and logged once at WARNING (an ERROR record opened a second box), and every text goes through `tr()`. A channel switch and a data-folder change (`_sync_canonical_source_paths`, through `AppSettings.sync_global_source_path`) and a successful global.ini extraction (`_on_p4k_extract_finished`) keep the `global` source on the selected language's base.ini (`AppSettings.get_global_source_base_ini()`, English when this channel has none yet); they used to point every language at the English one (#502). +- `workers.py` — every `QThread` background worker plus the shared `AnimatedProgressDialog` (its `set_close_guard(guard)` asks before Esc or the title-bar X dismisses it, never on a programmatic `close()`, and only for a dialog without a cancel button, #471; its `set_progress` shows the moving busy bar for a total of 0 and for nothing counted yet, 0 of N, until the first count, #507: a bar standing at 0% looked hung, as the generator's first lookups did for seconds, and the string loader and the update download start at 0 too) and `SelectAllDelegate`. Workers: `FileLoaderWorker` (sources → `StringEntry` list + sort keys), `AppliedStateWorker` (#398 review: runs the already-applied comparison off the GUI thread; takes an injected `compute(snapshot, should_stop)` instead of importing it, because the merge it shares with Apply to Game lives in `main_window.py` and importing that here would be a gui-to-gui cycle; always emits `finished(bool)` once, False being the safe "not applied" answer), `StartupSyncWorker` (refresh URL-backed sources at startup), `EnhancementsGeneratorWorker` (run `scripts/generate_enhancements_ini.py` in-process via `importlib.util`; takes a `language` param so output lands in that language's enhancements dir while annotation paths still parse the English base.ini; captures that folder, the DataForge build key and the tag fingerprint when its run starts, empties `.dataforge_stamp` just before the generator's `main()` and writes the tag stamp and then the DataForge stamp after it returns, see *A cut-off generation is offered again at the next start* below; `requestInterruption()`, which only closing the window sends, stops it before its patches, its module load or `main()`, and inside `main()` at the next job boundary, never during the writes, records it in `stopped` and ends with `finished(False)`, no `error` and nothing at ERROR), `P4kExtractWorker` (unp4k extract of `global.ini`, working beside the DataForge cache it is handed as `scratch_near`), `DataForgeExtractWorker` (unp4k + unforge + post-extract patches; both stop on `requestInterruption()`, record it in `stopped` and then end with `finished(False)`, no `error` and nothing at ERROR, #471; a successful DataForge run keeps its working folder for `take_working_folder()`, any other outcome deletes it before `finished`, and a DataForge run first waits for the background deletes it is handed as `wait_for`, #522), `WorkingFolderDeleteWorker` (deletes that folder in the background, #522, see *Closing stops a running extraction*), `StaleWorkingFolderSweepWorker` (sweeps the working folders a crash or a kill left, #522, same section), `LanguageBaseDownloadWorker` (2.0: fetch a non-English language's global.ini from its mapped URL into `cache\lang\{language}\base.ini`), `LiveBlueprintLogWorker` (SCO, 3.0.0: polls `blueprint_log_scanner.GameLogTail` on the active channel's `Game.log` every `POLL_MS` while the overlay runs, LIVE and HOTFIX only, and emits `blueprints_found(raw names, newest)`; read-only like the Scan, the main thread updates the owned set; stopped with `requestInterruption()` + `_reap_worker`; a lasting poll fault logs one traceback, then debug), `BlueprintLogScanWorker` (#222: scan a channel dir for "Received Blueprint" notifications via `blueprint_log_scanner.scan_channel`, watermark-aware — takes `(channel_dir, since)`, emits a `ScanResult`; stays read-only, all owned-set/watermark mutation happens on the main thread in `MainWindow._on_blueprint_log_scan_finished`). Each emits `progress` (str) and `progress_pct` (completed, total, message); `AnimatedProgressDialog.set_progress` consumes the latter. Exception: `AppUpdateCheckWorker` and `AppUpdateDownloadWorker` live in `src/utils/app_updater.py` because the workers, version comparison, and installer-asset picking belong together (#211). - `markdown_renderer.py` — `markdown_to_html(text, text_color, base_color, link_color)` for About/Help. Pure-Python; the Qt caller passes palette colours so the converter has no Qt dep. Stash-and-restore code-span handling keeps `**` and `_` inside backticks literal (loc keys like `vehicle_Name*` shouldn't sprout `` tags). - `config_tab.py` — **Config Tab**: source management, drag-drop merge hierarchy, SC install path, DataForge extraction trigger. Also the language selector (2.0): emits `language_changed(str)` for `MainWindow._on_language_changed`, and hosts `LanguageSourceDialog` (the *Map Language File* button) for per-language base.ini URL overrides. Changing the data folder migrates existing data via `migrate_user_data_dir(move=True)`. A DataForge cache folder change whose cache folder cannot be made (Windows refuses it, cannot reach it or rejects it) is undone as Cancel undoes it, with one box (`_report_cache_folder_error`, #494, see below). The cache folder field saves only a folder the user typed (`_cache_dir_edited`), so a click into it and out again, or Enter with nothing typed, tries nothing. While an extraction or a generation runs, `set_data_locations_locked` locks the folders, the channel, the language, Extract, Import Settings and Check for Updates, with the reason shown (#483, see *Run in Background and the job lock* below). - `shader_cache_dialog.py` — `ShaderCacheDialog`: what the Config tab's **Clear Shader Cache** button opens. One row per build from `shader_cache.find_shader_cache_dirs()`, each with **Clean** (subfolders only; disabled and the name greyed out when `is_clean`) and **Remove** (hard-deletes the whole folder at once, no prompt), plus a single **Close** button. Each click runs `ShaderCacheWorker` (`workers.py`), which first refuses to touch anything while `StarCitizen.exe` runs, then always emits `finished(str)` once (`STATUS_DONE` / `STATUS_FAILED` / `STATUS_GAME_RUNNING`, exceptions logged at WARNING so `ErrorDialogHandler` doesn't add a second box), under an `AnimatedProgressDialog` whose close guard keeps Esc and the X from hiding it mid-delete; then the rows are re-scanned. Keyboard safety, since Remove has no prompt: row buttons have `autoDefault` off, Close is the default button and gets focus on open and after every refresh, so Enter never lands on Remove. Close is ignored while a worker runs. `ConfigTab._clear_shader_cache` just opens the dialog; the dialog's own empty state covers "no folders". @@ -57,12 +57,12 @@ Esc or the X on the DataForge progress dialog used to hide it while the extracti ### Run in Background and the job lock (#483) Run in Background dismisses the DataForge dialog for good (`_send_dataforge_run_to_background`: detached from the worker as Stop does, since a hidden `QProgressDialog` comes back while progress keeps arriving), marks the worker `in_background`, and the status bar reports. The finished slot hands that on to `_run_enhancements_generation(background=True)`, which makes no dialog and connects the worker's progress to the status bar only. If the run finished while the question was open, generation has already taken the same dialog over, so Run in Background detaches it from generation instead and the answer still holds. It was tried and dropped before 2.4.0 because the window can be used while the run goes on, which was already true of a generation whose dialog Esc had hidden. So one job runs at a time, and what a job depends on is locked while any job runs, shown or not. The jobs are the three closing stops (`_WORKERS_STOPPED_ON_CLOSE`): a global.ini extraction, a DataForge extraction and a generation. `_job_running()` says whether one runs. `_run_p4k_extraction`, `_run_dataforge_extraction`, `_run_enhancements_generation` (beside an extraction), `_run_enhancements_pipeline` and `_run_simple_apply` start nothing while one does, `_check_enhancements_freshness` asks nothing, and a refused extraction or pipeline says why in the status bar. Two startup questions come right after the startup sync, which can start a global.ini extraction, so a job can be running when they are asked or start while they are open: the post-import Apply Now (`_maybe_prompt_post_import_apply`, whose pipeline would be refused) and the OneDrive warning (`_maybe_warn_onedrive_data_dir`, whose Move would change the data folder under the job). Each is put off with `_ask_after_the_job`, before it is shown or after an answer that a job started under, and asked again after the load that ends the job: `_queue_asks_after_the_job` is the last step of both load paths, after `_queue_deferred_starter_report` and for its reason, and `_flush_asks_after_the_job` waits, like that report, while a load, a job or a Simple run is going, never runs while closing, and while any modal box is open looks again every `_ASK_AFTER_MODAL_RETRY_MS` (that report's own summary box can be open then, and so can closeEvent's unapplied-changes prompt before the close is final, where an Apply Now would start a run the exit stops). A failed or stopped job that ends in a load (the global.ini extraction's follow-up for startup, a channel switch or a data folder change) asks them after it as usual; one that ends in no load (a DataForge run, a generation with no follow-up) leaves them waiting for the next load, and each is asked at the next start anyway. Nothing is lost meanwhile: Import Settings' flag is kept (put back after a refused Apply Now), the OneDrive warning comes at every start until it is answered, and a Move put off does not keep its "don't warn again". `_refresh_job_lock()`, called wherever a job's worker is set or cleared and when a Simple run starts or ends, locks with one reason (`extract.locked_while_job_runs`): the Config tab's game, data and cache folders, channel, language and Map Language File, its global.ini Extract, the install scan (Verify Install Location, which can save a game folder), Import Settings (whose restart starts the new copy before this one closes) and Check for Updates (the installer relaunches the app), through `ConfigTab.set_data_locations_locked`, which shows the reason in a label under the tab's instructions. The install scan and the update check keep their own busy state, so unlocking leaves either button off while it runs. It also locks the Enhancements tab's Apply Category Changes, Save Tag Changes and Generate Enhancements (`set_changes_locked`: the first renames INIs in the folder a generation writes and starts a reload, the second saves the tags and asks for a regeneration one job at a time would refuse, and the third would start that refused run; the checkboxes and the Tag Builder stay free and the change waits for the button, and a change made meanwhile keeps Generate off, with the running job's own message as its tooltip when it set one) and the Simple page's button, busy with the reason as its tooltip and its fill and text dimmed (`_end_simple_run` no longer frees it while a job still runs). A lit button the lock or a running job disables (Generate Enhancements, Save Tag Changes, the Config tab's Extract) keeps a dimmed red (`theme.needs_apply_style`), so a pending change still shows without the button looking clickable. Both dim through `theme.dimmed_color` (`DISABLED_ALPHA`), because a stylesheet colour outlives `setEnabled(False)`. Nets behind the lock: the finished slot checks the run still matches the settings (`_dataforge_run_matches_settings`, the channel and the cache base the worker recorded when it started, compared without making any folder) before it drains `_cleanup_pending_old_cache()` or chains generation, and says why when not (`extract.dataforge_settings_changed`); a language switch during an extraction loads the prose and leaves generation to the extraction's own chain; `ConfigTab._save_game_path` refuses while locked, and `_use_install_root` says why before it touches the field (an install scan finishing during a job could otherwise save one through its results dialog, and its failure box blamed a disconnected drive); `ConfigTab._save_data_dir` refuses while locked and puts the field back (`change_data_dir_to`, the OneDrive warning's Move, reaches it without the locked field); a scan that finishes during a job leaves its button locked (`_refresh_locked_controls`); Import Settings and an update's install refuse while a job runs or a working folder is still being deleted (`_closing_would_wait`, see the delete paragraph below). `restart_app` (the font size) closes first, so it needs nothing. `tests/test_run_in_background.py` locks this. -A successful DataForge run's working folder is deleted after the run, in the background (perf proposal): on a re-extract beside the cache it holds the ~31,000 files of the old cache set aside there (and ~31,000 more that an unforge without `--exclude-file` wrote and the cache does not keep), and with the work in `%TEMP%` all the files unforge wrote, whose kept subtrees were copied rather than moved (see `src/utils/CLAUDE.md`). Generation used to wait for that delete. `DataForgeExtractWorker` passes `extract_dataforge` a `defer_cleanup` and keeps the `WorkingFolder` it is handed. Every outcome but `finished(True)` deletes it first (`_finish`), so a stop or a failure leaves nothing behind and closing's wait for the thread covers it. `_on_dataforge_extract_finished` takes it once the thread is done (`take_working_folder`, after `wait()`) and `_delete_working_folder_in_background` starts a `WorkingFolderDeleteWorker` on the GUI thread, before the queued old-cache cleanup and generation, kept in `_working_folder_deleters` until its `finished` slot (`_on_working_folder_deleted`, which reaps it with `quit()` and `wait()`, as every finished slot does). It runs beside the generator's thread, never inside it or inside the extraction's, and only deletes files: it parses no XML, so it is not #389's shape (lxml parsing in a thread pool nested in a QThread), and it touches only the working folder, which is beside the cache folder, or in `%TEMP%` when the run worked there, never the cache folder the generator reads. A run released by the close, or one that finishes after `_close_committed`, lets the folder's marker go (`WorkingFolder.release`) for the next extraction's sweep, as after a crash. `_stop_workers_for_close` waits for every running deleter within the same 180 s deadline, hidden, after taking the folder of a DataForge run it released (one that succeeded before its finished slot ran), and keeps a deleter still busy past it, whose marker stays held until the process ends. Clear Cache and the queued old-cache cleanup do not wait: the delete never touches a cache folder and they never touch a working folder (a queued folder that holds the working folder holds the cache in use too, which #486 keeps). +A successful DataForge run's working folder is deleted after the run, in the background (#522): on a re-extract beside the cache it holds the ~31,000 files of the old cache set aside there (and ~31,000 more that an unforge without `--exclude-file` wrote and the cache does not keep), and with the work in `%TEMP%` all the files unforge wrote, whose kept subtrees were copied rather than moved (see `src/utils/CLAUDE.md`). Generation used to wait for that delete. `DataForgeExtractWorker` passes `extract_dataforge` a `defer_cleanup` and keeps the `WorkingFolder` it is handed. Every outcome but `finished(True)` deletes it first (`_finish`), so a stop or a failure leaves nothing behind and closing's wait for the thread covers it. `_on_dataforge_extract_finished` takes it once the thread is done (`take_working_folder`, after `wait()`) and `_delete_working_folder_in_background` starts a `WorkingFolderDeleteWorker` on the GUI thread, before the queued old-cache cleanup and generation, kept in `_working_folder_deleters` until its `finished` slot (`_on_working_folder_deleted`, which reaps it with `quit()` and `wait()`, as every finished slot does). It runs beside the generator's thread, never inside it or inside the extraction's, and only deletes files: it parses no XML, so it is not #389's shape (lxml parsing in a thread pool nested in a QThread), and it touches only the working folder, which is beside the cache folder, or in `%TEMP%` when the run worked there, never the cache folder the generator reads. A run released by the close, or one that finishes after `_close_committed`, lets the folder's marker go (`WorkingFolder.release`) for the next extraction's sweep, as after a crash. `_stop_workers_for_close` waits for every running deleter within the same 180 s deadline, hidden, after taking the folder of a DataForge run it released (one that succeeded before its finished slot ran), and keeps a deleter still busy past it, whose marker stays held until the process ends. Clear Cache and the queued old-cache cleanup do not wait: the delete never touches a cache folder and they never touch a working folder (a queued folder that holds the working folder holds the cache in use too, which #486 keeps). -A DataForge extraction started while such a delete still runs waits for it (perf proposal, the user's choice): `_run_dataforge_extraction` hands `DataForgeExtractWorker` the `done` event of every delete and sweep in `_working_folder_deleters` (`wait_for`), and the run waits for them on its own thread before `extract_dataforge`, so its sweep does not skip that folder and its free-space check does not count it as used. A delete lets go of its `WorkingFolder` before it sets `done`, so after one that failed with something other than an OSError (whose marker was never closed) the marker is closed by then and that sweep takes what is left. The GUI thread never waits, the dialog shows its busy bar with the extraction's label meanwhile (nothing is counted yet), and a Stop or a close ends the wait at once, as a stop. A crash or a kill during a delete leaves the working folder, with the old cache set aside in it, and before the perf proposal it was gone before generation began. `StaleWorkingFolderSweepWorker` sweeps such leftovers with pak_extractor's own `_sweep_stale_scratch` and `DATAFORGE_SCRATCH_PREFIX`, which leaves a folder whose marker is held, so a live run or delete in another window is safe: once a session at startup beside the active cache folder (`_sweep_stale_working_folders_at_startup`, first in `_continue_startup_after_update_gate`, reading the cache folder without creating it, #494, and skipped while an extraction runs, which sweeps there itself), and beside a queued old cache folder that `_cleanup_pending_old_cache` deletes or finds gone, where after a cache folder change no extraction sweeps any more (a cache in use gets none, because the extraction that just ran swept beside it, and a folder the check cannot resolve gets none, out of caution, as its cleanup is skipped and it cannot be compared with the cache in use). `_sweep_working_folders_in_background` starts it on its own thread, never the GUI's, and it first waits for the deletes already running (`after`), so it never works on a folder one of them is finishing. It is kept in `_working_folder_deleters` with them, so a new extraction waits for it the same way and it never runs beside an extraction's own sweep. Closing asks every running delete and sweep to stop (`requestInterruption`) before it waits: a sweep ends at once while it still waits for a delete, and before its next leftover folder while it sweeps (`_sweep_stale_scratch`'s `should_cancel`), finishing the folder in hand and leaving the rest for a later sweep, while a working folder's own delete has no stop and is waited for as before. A crash during the delete of a run that worked in `%TEMP%` (a OneDrive, network, removable or deep cache folder) still leaves that folder for the next DataForge extraction's sweep of `%TEMP%`. Anything that refuses to start work while a job runs, because the work closes the app (an update, Import Settings), must count `_working_folder_deleters` as well as the `_WORKERS_STOPPED_ON_CLOSE` workers: otherwise it can close the app while a delete or sweep runs, and the close then waits for it, hidden, for up to 180 s. `_closing_would_wait()` is that question (`_job_running() or bool(self._working_folder_deleters)`), asked by `_on_update_available` and `_handle_import_settings`, which say why in the status bar with the lock's own text (`extract.locked_while_job_runs`). `_job_running()` itself stays workers only: a generation is meant to run beside a delete, and a new DataForge run waits for one on its own thread, so neither asks the broader question. The two buttons are not locked for a delete (`_refresh_job_lock` follows the jobs, so it would need a refresh when each delete starts and ends), so the check at the relaunch is the whole guard there. `tests/test_extraction_stop.py` (`TestTheWorkerKeepsOnlyASuccessfulRunsFolder`, `TestTheWindowDeletesItInTheBackground`, `TestClosingWaitsForTheDelete`, `TestTheDeletesFinishedSlot`, `TestANewRunWaitsForTheDelete`, `TestLeftoversAreSweptAtStartup`), `tests/test_clear_cache.py` and `tests/test_pending_cache_cleanup.py` (`TestBesideTheBackgroundDelete`, `TestLeftoverWorkingFoldersBesideIt`) lock this. +A DataForge extraction started while such a delete still runs waits for it (#522, the user's choice): `_run_dataforge_extraction` hands `DataForgeExtractWorker` the `done` event of every delete and sweep in `_working_folder_deleters` (`wait_for`), and the run waits for them on its own thread before `extract_dataforge`, so its sweep does not skip that folder and its free-space check does not count it as used. A delete lets go of its `WorkingFolder` before it sets `done`, so after one that failed with something other than an OSError (whose marker was never closed) the marker is closed by then and that sweep takes what is left. The GUI thread never waits, the dialog shows its busy bar with the extraction's label meanwhile (nothing is counted yet), and a Stop or a close ends the wait at once, as a stop. A crash or a kill during a delete leaves the working folder, with the old cache set aside in it, and before #522 it was gone before generation began. `StaleWorkingFolderSweepWorker` sweeps such leftovers with pak_extractor's own `_sweep_stale_scratch` and `DATAFORGE_SCRATCH_PREFIX`, which leaves a folder whose marker is held, so a live run or delete in another window is safe: once a session at startup beside the active cache folder (`_sweep_stale_working_folders_at_startup`, first in `_continue_startup_after_update_gate`, reading the cache folder without creating it, #494, and skipped while an extraction runs, which sweeps there itself), and beside a queued old cache folder that `_cleanup_pending_old_cache` deletes or finds gone, where after a cache folder change no extraction sweeps any more (a cache in use gets none, because the extraction that just ran swept beside it, and a folder the check cannot resolve gets none, out of caution, as its cleanup is skipped and it cannot be compared with the cache in use). `_sweep_working_folders_in_background` starts it on its own thread, never the GUI's, and it first waits for the deletes already running (`after`), so it never works on a folder one of them is finishing. It is kept in `_working_folder_deleters` with them, so a new extraction waits for it the same way and it never runs beside an extraction's own sweep. Closing asks every running delete and sweep to stop (`requestInterruption`) before it waits: a sweep ends at once while it still waits for a delete, and before its next leftover folder while it sweeps (`_sweep_stale_scratch`'s `should_cancel`), finishing the folder in hand and leaving the rest for a later sweep, while a working folder's own delete has no stop and is waited for as before. A crash during the delete of a run that worked in `%TEMP%` (a OneDrive, network, removable or deep cache folder) still leaves that folder for the next DataForge extraction's sweep of `%TEMP%`. Anything that refuses to start work while a job runs, because the work closes the app (an update, Import Settings), must count `_working_folder_deleters` as well as the `_WORKERS_STOPPED_ON_CLOSE` workers: otherwise it can close the app while a delete or sweep runs, and the close then waits for it, hidden, for up to 180 s. `_closing_would_wait()` is that question (`_job_running() or bool(self._working_folder_deleters)`), asked by `_on_update_available` and `_handle_import_settings`, which say why in the status bar with the lock's own text (`extract.locked_while_job_runs`). `_job_running()` itself stays workers only: a generation is meant to run beside a delete, and a new DataForge run waits for one on its own thread, so neither asks the broader question. The two buttons are not locked for a delete (`_refresh_job_lock` follows the jobs, so it would need a refresh when each delete starts and ends), so the check at the relaunch is the whole guard there. `tests/test_extraction_stop.py` (`TestTheWorkerKeepsOnlyASuccessfulRunsFolder`, `TestTheWindowDeletesItInTheBackground`, `TestClosingWaitsForTheDelete`, `TestTheDeletesFinishedSlot`, `TestANewRunWaitsForTheDelete`, `TestLeftoversAreSweptAtStartup`), `tests/test_clear_cache.py` and `tests/test_pending_cache_cleanup.py` (`TestBesideTheBackgroundDelete`, `TestLeftoverWorkingFoldersBesideIt`) lock this. ### A cut-off generation is offered again at the next start (#476) The startup check (`_check_enhancements_freshness`) used to ask only whether each enabled enhancement INI existed. A generation cut off part-way (the window closed, the app crashed, a write failed) left every INI there, each whole but the set a mix of old and new, and the next start stayed quiet. Now `EnhancementsGeneratorWorker` empties the `.dataforge_stamp` of the language it generates just before the generator's `main()` (after the patches and the module load, so a failure there keeps a good stamp) and writes it only once `main()` has returned. The check also asks when every file is there but `AppSettings.enhancements_built_from_current_dataforge()` says the set was not built from the current DataForge build: the last run was cut off or failed, or the set came from older game data. It asks that only when the DataForge cache is fresh (`_out_of_date_build_key`): an outdated cache is `_maybe_prompt_dataforge_refresh`'s question, so this one would stack on it or ask again after it was declined, and a cut-off extraction or a cache never extracted has nothing to generate from. In that case the category dialog offers every enabled category (less any the Enhancements tab has unticked and not saved), ticked with its plain label, under `extract.outdated_categories_body` ("out of date", with no count: the count's noun says "missing" in several languages). The answer holds for the rest of the session (`_enhancements_prompt_answer`, reset by a channel or data-folder change). Skip is never turned into a run by a later call, such as the one after a global.ini extraction, which used to run the pipeline silently, possibly a DataForge extraction the user had just declined. It is recorded before the dialog opens, because the dialog's event loop can deliver another load, and that call must start nothing. Nothing about the answer is kept on disk, so the question comes back at every start until the set is generated, like the missing-files question. The Generate button stays lit while a ticked category's set is not current, and the language switch (`_language_enhancements_fresh`, English included) uses the same test, so the three never disagree. A run that finishes re-reads this from disk (`set_operation_idle`), so one whose final stamp write failed (`set_enhancements_stamp` only logs that, and the run still succeeds) leaves the button lit over the set the next start will ask about. The worker also captures its folder, the build key and the tag fingerprint when its run starts: reading them when it ended let a channel, data-folder or cache-folder change during a hidden run stamp a set the run never wrote. Closing the window during a generation stops it at its next job boundary (see *Closing stops a running extraction* above), which after `main()` began is one of these cut-off runs, so the next start asks. `tests/test_enhancements_stale_startup.py` locks this. -### base.ini freshness follows Data.p4k's size (perf proposal) +### base.ini freshness follows Data.p4k's size (#522) `_check_p4k_freshness` (startup, a channel switch, a data folder change) and the Config tab's `_refresh_p4k_status` (the Extract button's red nudge) compared Data.p4k's modified time with base.ini's own, and the RSI launcher's file verification moves that time on an ordinary game launch (#209), so playing was enough to be asked to extract global.ini again. Both now ask `pak_extractor.base_ini_is_fresh` (see `src/utils/CLAUDE.md`): a launch alone asks nothing and leaves the button alone, and a patch, which changes Data.p4k's size, still asks with the same `extract.p4k_prompt_p4k_newer` text. The language switch and Simple mode's Apply never checked base.ini and still don't; Simple mode meets the prompt only at startup, as Advanced does. `tests/test_base_ini_freshness.py` locks this. diff --git a/src/gui/config_tab.py b/src/gui/config_tab.py index a9912429..5ab5e607 100644 --- a/src/gui/config_tab.py +++ b/src/gui/config_tab.py @@ -1337,7 +1337,7 @@ def _refresh_p4k_status(self): tr("config.p4k_status_found_with_base", date=last_str) ) # The same rule as the startup prompt: Data.p4k's size, not - # the modified time a game launch moves (perf proposal). + # the modified time a game launch moves (#522). needs_update = not base_ini_is_fresh(p4k_path, base_ini) else: self._p4k_status_label.setText(tr("config.p4k_status_found_no_base")) diff --git a/src/gui/main_window.py b/src/gui/main_window.py index c61fd3d7..f044feef 100644 --- a/src/gui/main_window.py +++ b/src/gui/main_window.py @@ -802,7 +802,7 @@ def __init__(self, installer_data_move=None): self._forge_worker: Optional[DataForgeExtractWorker] = None self._forge_progress_dialog: Optional[AnimatedProgressDialog] = None # Threads deleting a finished extraction's working folder, or - # sweeping the ones a crash left, in the background (perf proposal). + # sweeping the ones a crash left, in the background (#522). # Closing waits for them, and so does a new DataForge extraction. self._working_folder_deleters: list[WorkingFolderDeleteWorker] = [] # The startup sweep of leftover working folders runs once a session. @@ -5668,7 +5668,7 @@ def _cleanup_pending_old_cache(self) -> None: A folder that is not in use also has the working folders a crash or a kill left beside it swept, in the background, whether it is still - there or not (perf proposal). + there or not (#522). """ queued = AppSettings.get_pending_cache_cleanup() if not queued: @@ -5699,7 +5699,7 @@ def _cleanup_pending_old_cache(self) -> None: ) return # A working folder a crash or a kill left beside the old cache folder - # (perf proposal: with the old cache set aside in it, it lives on + # (#522: with the old cache set aside in it, it lives on # while enhancements are generated). The extractions sweep only # beside the cache folder in use, so nothing else ever would. self._sweep_working_folders_in_background(old_path.parent) @@ -5908,8 +5908,8 @@ def _check_p4k_freshness(self, if_unfinished=None) -> bool: base_missing = not base_ini.exists() # Data.p4k's size says whether the game changed, not its modified - # time, which the launcher's file check moves on a game launch (perf - # proposal, the rule #209 gave the DataForge cache). + # time, which the launcher's file check moves on a game launch (#522, + # the rule #209 gave the DataForge cache). p4k_changed = (not base_missing) and not base_ini_is_fresh(p4k_path, base_ini) if not base_missing and not p4k_changed: @@ -6708,7 +6708,7 @@ def _run_dataforge_extraction(self): # The run waits on its own thread for any background delete of an # earlier run's working folder, or sweep of leftover ones, still - # going (perf proposal): its sweep would skip that folder and its + # going (#522): its sweep would skip that folder and its # free-space check count it as used. worker = DataForgeExtractWorker( p4k_path, unp4k_exe, unforge_exe, forge_dir, @@ -6925,7 +6925,7 @@ def _on_dataforge_extract_error(self, message: str, worker): def _on_dataforge_extract_finished(self, success: bool, worker): worker.quit() worker.wait() - # The working folder a successful run kept (perf proposal), taken + # The working folder a successful run kept (#522), taken # once the thread is done. working_folder = worker.take_working_folder() # As _on_applied_state_ready does: frees the thread and the lambda @@ -6954,8 +6954,8 @@ def _on_dataforge_extract_finished(self, success: bool, worker): return if working_folder is not None: - # Deleted while enhancements are generated, not before (perf - # proposal). Started first, so the queued cleanup below runs + # Deleted while enhancements are generated, not before (#522). + # Started first, so the queued cleanup below runs # beside it; the two never share a folder. self._delete_working_folder_in_background(working_folder) @@ -7175,7 +7175,7 @@ def _on_p4k_extract_finished(self, success: bool, worker): def _delete_working_folder_in_background(self, working_folder) -> None: """Delete a finished DataForge extraction's working folder on a - thread of its own (perf proposal), which generation no longer waits + thread of its own (#522), which generation no longer waits for: on a re-extract beside the cache the ~31,000 files of the old cache set aside there (and ~31,000 more that an unforge without --exclude-file wrote and the cache does not keep), and all of @@ -7195,7 +7195,7 @@ def _delete_working_folder_in_background(self, working_folder) -> None: def _sweep_working_folders_in_background(self, parent) -> None: """Sweep the DataForge working folders a crash or a kill left in - *parent* on a thread of its own (perf proposal), never the GUI + *parent* on a thread of its own (#522), never the GUI thread: such a folder can hold an old cache of about 31,000 files. The sweep leaves a folder whose marker is still held. It first waits for the background deletes already running, and is kept with them @@ -7210,7 +7210,7 @@ def _sweep_working_folders_in_background(self, parent) -> None: def _sweep_stale_working_folders_at_startup(self) -> None: """Once a session, sweep the DataForge working folders a crash or a - kill left beside the active cache folder (perf proposal). + kill left beside the active cache folder (#522). A successful run's working folder, with the old cache set aside in it, now lives on while enhancements are generated, and a crash @@ -7260,7 +7260,7 @@ def _job_running(self) -> bool: def _closing_would_wait(self) -> bool: """True while a job runs (#483) or a finished DataForge run's working - folder is still being deleted in the background (perf proposal). + folder is still being deleted in the background (#522). Import Settings and an update's install start the new copy before this one closes, and closing waits for such a delete with the window hidden (_stop_workers_for_close), so they wait for it as they do for @@ -7324,7 +7324,7 @@ def _stop_workers_for_close(self, timeout_ms: int = 180_000) -> None: queued. The background deletes of finished extractions' working folders - (perf proposal) are waited for too, within the same deadline, with a + (#522) are waited for too, within the same deadline, with a DataForge run's own folder when it finished before its finished slot ran. Each holds its folder's marker, so past the deadline another window's sweep still leaves it alone until the process ends, and @@ -7367,8 +7367,8 @@ def _stop_workers_for_close(self, timeout_ms: int = 180_000) -> None: continue # keep the reference so Qt doesn't destroy a live thread setattr(self, worker_attr, None) if worker_attr == "_forge_worker": - # Only a DataForge run keeps its working folder (perf - # proposal): one that succeeded just before the close. + # Only a DataForge run keeps its working folder (#522): + # one that succeeded just before the close. working_folder = worker.take_working_folder() if working_folder is not None: self._delete_working_folder_in_background(working_folder) @@ -7380,7 +7380,7 @@ def _stop_workers_for_close(self, timeout_ms: int = 180_000) -> None: self.hide() for deleter in deleting: # Only a sweep of leftovers stops here, before its next folder - # (perf proposal): the folder in hand is finished. A working + # (#522): the folder in hand is finished. A working # folder's own delete has no stop and finishes, or its marker # stays held for the next sweep. deleter.requestInterruption() diff --git a/src/gui/workers.py b/src/gui/workers.py index aff0fa6e..e98b7aae 100644 --- a/src/gui/workers.py +++ b/src/gui/workers.py @@ -13,8 +13,8 @@ - InstallScanWorker — finds every Star Citizen install on the machine (2.4) - P4kExtractWorker — unp4k extraction of global.ini - DataForgeExtractWorker — unp4k + unforge + patch pipeline -- WorkingFolderDeleteWorker — deletes a finished extraction's working folder (perf proposal) -- StaleWorkingFolderSweepWorker — sweeps working folders a crash or a kill left (perf proposal) +- WorkingFolderDeleteWorker — deletes a finished extraction's working folder (#522) +- StaleWorkingFolderSweepWorker — sweeps working folders a crash or a kill left (#522) - SelectAllDelegate — Custom Value cell delegate (auto-select, EM3/EM4 wrap) - OrderSpinBoxDelegate — Sort Order cell delegate (0-99 spin box, #142) - TestPlanSubmitWorker — POSTs a tester's test-plan report to a Discord webhook @@ -875,13 +875,13 @@ def run(self): self.finished.emit(False) -# How often a wait for a background delete looks for a stop (perf proposal). +# How often a wait for a background delete looks for a stop (#522). _BACKGROUND_DELETE_POLL_SECONDS = 0.05 def _wait_for_background_deletes(done_events, should_stop) -> bool: """Wait on this thread until every background delete in *done_events* - is over, each a WorkingFolderDeleteWorker's ``done`` (perf proposal). + is over, each a WorkingFolderDeleteWorker's ``done`` (#522). Returns False as soon as *should_stop()* turns True instead, so a stop or a close is never held up by a delete.""" for done in done_events: @@ -903,7 +903,7 @@ class DataForgeExtractWorker(QThread): which would fire the global ErrorDialogHandler for something the user asked for. - A successful run keeps its working folder (perf proposal): + A successful run keeps its working folder (#522): extract_dataforge hands it over instead of deleting it, and MainWindow takes it with take_working_folder() once this thread is done and has it deleted in the background (WorkingFolderDeleteWorker) while enhancements @@ -912,7 +912,7 @@ class DataForgeExtractWorker(QThread): closing waits for it with the thread. *wait_for* holds the ``done`` events of the background deletes still - going when the run was started (perf proposal). The run waits for them + going when the run was started (#522). The run waits for them before extract_dataforge, so its sweep does not skip an earlier run's working folder still being deleted and its free-space check does not count that folder as used. It waits here, never on the GUI thread, with @@ -945,7 +945,7 @@ def __init__(self, p4k_path, unp4k_exe, unforge_exe, cache_dir, wait_for=(), *, # QThread reports no interruption once the thread has finished, so a # finished slot that waits for the thread first would always see False. self.stopped = False - # The working folder extract_dataforge handed over (perf proposal). + # The working folder extract_dataforge handed over (#522). self._working_folder = None # Set when the user sends the run to the background from the progress # dialog's question (#483): the generation that follows runs without a @@ -965,7 +965,7 @@ def take_working_folder(self): def _finish(self, success: bool) -> None: """Emit finished. A run that did not succeed deletes a working - folder it kept first (perf proposal): a stop after the cache was + folder it kept first (#522): a stop after the cache was written, during the patches or as a failure on the way out, or a failure after extract_dataforge returned. @@ -1084,7 +1084,7 @@ def run(self): class WorkingFolderDeleteWorker(QThread): """Deletes the working folder a successful DataForge extraction kept - (perf proposal), so enhancement generation does not wait for it. + (#522), so enhancement generation does not wait for it. MainWindow starts it on the GUI thread once the extraction's own thread is done, never from inside another worker, and closing waits for it @@ -1128,7 +1128,7 @@ def _delete(self) -> None: class StaleWorkingFolderSweepWorker(WorkingFolderDeleteWorker): """Sweeps the DataForge working folders a crash or a kill left in one - folder (perf proposal): pak_extractor's own sweep, which deletes only a + folder (#522): pak_extractor's own sweep, which deletes only a folder whose marker it can delete, so it leaves alone a folder a live run or delete in this or another window still holds. diff --git a/src/utils/CLAUDE.md b/src/utils/CLAUDE.md index 062c8501..ad1db95b 100644 --- a/src/utils/CLAUDE.md +++ b/src/utils/CLAUDE.md @@ -26,23 +26,23 @@ Utility layer: settings, build-mode flag, P4K extraction, enhancement support mo - `app_updater.py` — *separate* from `updater.py`. Polls `GET /repos/Osiris-DevWorks/smart-citizen/releases/latest` and compares `tag_name` to local `VERSION.TXT`. As of #211 (2.1.2) the check runs on every launch (one unauthenticated call, far under GitHub's 60-req/hr cap) and gates the rest of startup in `MainWindow` until it resolves. Also owns `pick_installer_asset()` (finds the `SmartCitizen-*-Setup.exe` release asset) and `AppUpdateDownloadWorker` (streams the installer to `%TEMP%\SmartCitizen-Update\` with progress; `.part`-then-rename so a partial download is never runnable). The install/relaunch side lives in `MainWindow._launch_installer_and_quit` + `installer.iss`'s `/AUTOUPDATE=1` relaunch, which runs in `CurStepChanged` at `ssPostInstall` right after the registry writes (not as a `[Run]` entry, which Setup would run first). - `pak_extractor.py` — P4K extraction pipeline: `unp4k.exe` (extracts Game2.dcb) → `unforge.exe` (converts to entity XMLs). unforge is handed `DATAFORGE_EXCLUDE_SUBPATHS` (`dataforge_exclude.py`) with `--exclude-file`, a file `_write_unforge_exclude_list` writes into the run's working folder, one record path a line: every record outside the `DATAFORGE_KEEP_SUBPATHS` subtrees on the game build the list was made from (LIVE, 23 Oct 2026: 538 folders and files, 31,424 of 62,623 records), so it writes about half the tree. `_cache_filtered_records()` puts the `DATAFORGE_KEEP_SUBPATHS` subtrees (what the generator reads) into the persistent cache first, then everything else unforge wrote that the exclude list does not name (`_unlisted_records`, logged by name, matched ignoring case as unforge matches it): a record folder a game patch adds reaches the cache with no list to update, where a keep list alone dropped it until someone listed it. An unforge that prints its usage for the flag (one built before odw-fast-unp4k ad7c6a7) is run again with the dcb alone, and the cache still leaves the excluded records out. The room check before the wipe (`_require_room_for_new_cache`) counts the same files. The exclude list keeps the cache to about half of unforge's whole tree. When the working folder is beside the cache each subtree is moved with one folder rename (#484), so nothing of it is left there to delete. In `%TEMP%`, or when a rename is refused (another program holds a file under it past a 0.7 s retry, `_MOVE_RETRY_DELAYS`), the subtree is copied, and a rename that finds the drive full is not retried. The renames use `os.rename`, never `os.replace` or `shutil.move`. A new read path in the generator needs an entry in `DATAFORGE_KEEP_SUBPATHS`, and the `DATAFORGE_EXCLUDE_SUBPATHS` entry that holds it taken out or split; `tests/test_pak_extraction.py::TestDataForgeKeepList` locks the contract, and `TestDataForgeExcludeList` that nothing excluded is, holds or lies in a kept subtree. The cache layout itself (`raw/libs`, `foundry/records`) is named once here as `DATAFORGE_LIBS_SUBPATH` / `DATAFORGE_RECORDS_SUBPATH` and joined by every reader, and so is the generator's lookup pickle folder, `DATAFORGE_LOOKUPS_SUBPATH` (`.lookups`), which `_cached_lookup` writes and `EnhancementsGeneratorWorker` sweeps for leftover temps (`TestCacheLayoutNamedOnce` fails on any quoted `"raw"` or `"foundry"` path segment elsewhere in `src/` or `scripts/`, whether joined with `/`, passed to `Path()` or `joinpath()`, or written as `"raw/libs"`, and on any string starting `".lookups"`, a versioned `".lookups-v2"` included). `extract_dataforge()` and `dataforge_cache_is_fresh()` both wrap `dataforge_cache_dir` via `win_paths.win_long_path()` at entry — see that file. - **Working folder and stopping (#471).** The tools used to work in `tempfile.TemporaryDirectory()` under `%TEMP%`, so a full run's ~2 GB (Game2.dcb plus ~62k XMLs) hit the system drive whatever cache folder the user had picked, and closing the app left unforge running on its own while Python's exit-time cleanup failed under it. Now each DataForge run works in its own `dataforge.extracting-XXXXXXXX` folder beside the cache leaf (`_scratch_location`: same drive, outside the leaf that step 3 wipes, inside the channel folder the uninstaller's wipe covers). It falls back to `%TEMP%\SmartCitizen-DataForge-XXXXXXXX` for a network share, OneDrive, a removable or network drive, or a run folder over 190 characters (unp4k's working folder must stay within what CreateProcess accepts; unforge itself was measured fine with 368-character output paths). The tools are handed plain paths (`win_plain_path`), while Python's own reads of their output stay long-path safe. The global.ini extraction (the English global.ini alone, ~10 MB, see *What the tools are asked for and what they print* below) follows the same rules in its own `globalini.extracting-XXXXXXXX` folder (`scratch_near=` the DataForge cache folder, which `MainWindow._run_p4k_extraction` resolves on the main thread and replaces with `None`, meaning `%TEMP%`, when that drive is offline), so neither extraction writes to `%TEMP%` unless a fallback applies. The global.ini extraction also falls back to `%TEMP%` when the folder beside the cache can't be made (permissions, Controlled Folder Access, a read-only drive), since it only ever needed `%TEMP%` before. DataForge does not: its cache writes go to that same drive and would fail anyway, after a minute of unforge. `_begin_scratch` sweeps both places and opens the run's folder for either extraction. `_make_run_folder` makes it with one `os.mkdir` per random name (the prefix and `_MKDTEMP_RANDOM_CHARS`, 8 hex characters from `secrets.token_hex`), not `tempfile.mkdtemp` (#490). On the Python 3.11 the release builds use, mkdtemp takes a `PermissionError` for a name that may be taken and tries again up to `os.TMP_MAX` (2,147,483,647) times, about 39 hours against a folder that refuses, so the extraction looked stuck and the fallback above never started (Python 3.14.6 stops at 20). Only a taken name is tried again (a `FileExistsError`, or a `PermissionError` for a name that is there, which a folder Windows is still deleting can give), up to `_SCRATCH_NAME_ATTEMPTS` (10) names, then `FileExistsError`. A `PermissionError` making the folder or opening its marker becomes `WorkingFolderRefusedError` in `_begin_scratch` (`_folder_refused_error`: the Windows error at WARNING, then a `tr()` message that names the folder and says Windows denied permission there; Python raises the whole range of Windows errors 19 to 36, a write-protected or failing drive or a file another program holds, as `PermissionError` too, so `windows_refused()` counts only ERROR_ACCESS_DENIED, ERROR_NETWORK_ACCESS_DENIED or no Windows code as a refusal, and any other gets `extract.folder_unusable`, which gives `windows_reason()`, the error's own text in the user's Windows language, with no advice about permissions), unless the global.ini fallback moves the work first, so global.ini reports a refusal only when `%TEMP%` refuses too. Beside the cache the advice is to check the folder's permissions, allow Smart Citizen through Controlled folder access, or choose another DataForge cache folder (`extract.folder_refused_fix_cache`). In `%TEMP%` it leaves out the cache folder, which may not help there (`extract.folder_refused_fix`). Any other OSError (a drive that is gone, a path too long) behaves as before: DataForge raises it, and global.ini moves to `%TEMP%` on it. Step 3's wipe deletes the cache leaf and makes it again in the folder that holds it. Beside the cache, making the working folder in that same folder has already shown it takes a new folder; in `%TEMP%` nothing has, and a folder that refuses new folders let the run delete the old cache and then fail to make the new one with a raw `PermissionError` (#494). So a DataForge run in `%TEMP%` calls `_require_cache_can_be_remade` right after `_begin_scratch`, before any tool runs: it makes one empty folder there with `_make_run_folder` and the DataForge prefix (making the folder that holds the cache folder first when it is missing) and removes it with `os.rmdir`, and a `PermissionError` becomes `_folder_refused_error` for that folder with the cache advice, the old cache untouched. Any other OSError is raised as it is, as above. It runs after the free-space check, so a full drive still gets its own message. A test folder that will not go is logged at WARNING and left for a later DataForge run's sweep, which takes it as an unmarked leftover once it is 10 minutes old. A run holds an `.in_use` marker open, and `_sweep_stale_scratch` at the start of every extraction deletes only leftovers whose marker it can delete (a crash or hard kill releases it) or that have no marker and are over 10 minutes old (a run that died before making one, or a removal that got part-way), so a live run in another window is never touched. The window also runs it with the DataForge prefix, off the GUI thread (`StaleWorkingFolderSweepWorker`, perf proposal), once a session beside the active cache folder at startup and beside a queued old cache folder after `_cleanup_pending_old_cache`, because a working folder now outlives its run while it is deleted in the background, and one a crash left there would otherwise wait for the next extraction beside the same cache folder (see `src/gui/CLAUDE.md`). `_run_tool` replaces `subprocess.run`: it polls `communicate(timeout=...)` so a `should_cancel()` request kills the tool within a quarter of a second, and raises `ExtractionCancelled` (deliberately not a `RuntimeError`) before any exit code is read. `extract_dataforge` also checks for a stop just before the old cache is wiped (the last point that leaves it untouched) and after the stamps (the cache is complete, but the stop is reported so nothing chains into generation), so a stop never leaves a half-filled cache: nothing checks for a stop between the wipe and the stamps, so the renames, their retries and any fallback copy always finish. unforge's `foundry/records` folder is checked before the gate, so output without it, or with none of the kept subtrees in it, keeps the old cache. The working folder goes in a `finally` (`_close_scratch` logs every OSError, never raising it over the real outcome), unless a successful run hands it over (see below). The workers feed `should_cancel` from `QThread.isInterruptionRequested`; `tests/test_extraction_working_folder.py`, `tests/test_extraction_stop.py` and `tests/test_folder_refused_strings.py` lock all of this. + **Working folder and stopping (#471).** The tools used to work in `tempfile.TemporaryDirectory()` under `%TEMP%`, so a full run's ~2 GB (Game2.dcb plus ~62k XMLs) hit the system drive whatever cache folder the user had picked, and closing the app left unforge running on its own while Python's exit-time cleanup failed under it. Now each DataForge run works in its own `dataforge.extracting-XXXXXXXX` folder beside the cache leaf (`_scratch_location`: same drive, outside the leaf that step 3 wipes, inside the channel folder the uninstaller's wipe covers). It falls back to `%TEMP%\SmartCitizen-DataForge-XXXXXXXX` for a network share, OneDrive, a removable or network drive, or a run folder over 190 characters (unp4k's working folder must stay within what CreateProcess accepts; unforge itself was measured fine with 368-character output paths). The tools are handed plain paths (`win_plain_path`), while Python's own reads of their output stay long-path safe. The global.ini extraction (the English global.ini alone, ~10 MB, see *What the tools are asked for and what they print* below) follows the same rules in its own `globalini.extracting-XXXXXXXX` folder (`scratch_near=` the DataForge cache folder, which `MainWindow._run_p4k_extraction` resolves on the main thread and replaces with `None`, meaning `%TEMP%`, when that drive is offline), so neither extraction writes to `%TEMP%` unless a fallback applies. The global.ini extraction also falls back to `%TEMP%` when the folder beside the cache can't be made (permissions, Controlled Folder Access, a read-only drive), since it only ever needed `%TEMP%` before. DataForge does not: its cache writes go to that same drive and would fail anyway, after a minute of unforge. `_begin_scratch` sweeps both places and opens the run's folder for either extraction. `_make_run_folder` makes it with one `os.mkdir` per random name (the prefix and `_MKDTEMP_RANDOM_CHARS`, 8 hex characters from `secrets.token_hex`), not `tempfile.mkdtemp` (#490). On the Python 3.11 the release builds use, mkdtemp takes a `PermissionError` for a name that may be taken and tries again up to `os.TMP_MAX` (2,147,483,647) times, about 39 hours against a folder that refuses, so the extraction looked stuck and the fallback above never started (Python 3.14.6 stops at 20). Only a taken name is tried again (a `FileExistsError`, or a `PermissionError` for a name that is there, which a folder Windows is still deleting can give), up to `_SCRATCH_NAME_ATTEMPTS` (10) names, then `FileExistsError`. A `PermissionError` making the folder or opening its marker becomes `WorkingFolderRefusedError` in `_begin_scratch` (`_folder_refused_error`: the Windows error at WARNING, then a `tr()` message that names the folder and says Windows denied permission there; Python raises the whole range of Windows errors 19 to 36, a write-protected or failing drive or a file another program holds, as `PermissionError` too, so `windows_refused()` counts only ERROR_ACCESS_DENIED, ERROR_NETWORK_ACCESS_DENIED or no Windows code as a refusal, and any other gets `extract.folder_unusable`, which gives `windows_reason()`, the error's own text in the user's Windows language, with no advice about permissions), unless the global.ini fallback moves the work first, so global.ini reports a refusal only when `%TEMP%` refuses too. Beside the cache the advice is to check the folder's permissions, allow Smart Citizen through Controlled folder access, or choose another DataForge cache folder (`extract.folder_refused_fix_cache`). In `%TEMP%` it leaves out the cache folder, which may not help there (`extract.folder_refused_fix`). Any other OSError (a drive that is gone, a path too long) behaves as before: DataForge raises it, and global.ini moves to `%TEMP%` on it. Step 3's wipe deletes the cache leaf and makes it again in the folder that holds it. Beside the cache, making the working folder in that same folder has already shown it takes a new folder; in `%TEMP%` nothing has, and a folder that refuses new folders let the run delete the old cache and then fail to make the new one with a raw `PermissionError` (#494). So a DataForge run in `%TEMP%` calls `_require_cache_can_be_remade` right after `_begin_scratch`, before any tool runs: it makes one empty folder there with `_make_run_folder` and the DataForge prefix (making the folder that holds the cache folder first when it is missing) and removes it with `os.rmdir`, and a `PermissionError` becomes `_folder_refused_error` for that folder with the cache advice, the old cache untouched. Any other OSError is raised as it is, as above. It runs after the free-space check, so a full drive still gets its own message. A test folder that will not go is logged at WARNING and left for a later DataForge run's sweep, which takes it as an unmarked leftover once it is 10 minutes old. A run holds an `.in_use` marker open, and `_sweep_stale_scratch` at the start of every extraction deletes only leftovers whose marker it can delete (a crash or hard kill releases it) or that have no marker and are over 10 minutes old (a run that died before making one, or a removal that got part-way), so a live run in another window is never touched. The window also runs it with the DataForge prefix, off the GUI thread (`StaleWorkingFolderSweepWorker`, #522), once a session beside the active cache folder at startup and beside a queued old cache folder after `_cleanup_pending_old_cache`, because a working folder now outlives its run while it is deleted in the background, and one a crash left there would otherwise wait for the next extraction beside the same cache folder (see `src/gui/CLAUDE.md`). `_run_tool` replaces `subprocess.run`: it polls `communicate(timeout=...)` so a `should_cancel()` request kills the tool within a quarter of a second, and raises `ExtractionCancelled` (deliberately not a `RuntimeError`) before any exit code is read. `extract_dataforge` also checks for a stop just before the old cache is wiped (the last point that leaves it untouched) and after the stamps (the cache is complete, but the stop is reported so nothing chains into generation), so a stop never leaves a half-filled cache: nothing checks for a stop between the wipe and the stamps, so the renames, their retries and any fallback copy always finish. unforge's `foundry/records` folder is checked before the gate, so output without it, or with none of the kept subtrees in it, keeps the old cache. The working folder goes in a `finally` (`_close_scratch` logs every OSError, never raising it over the real outcome), unless a successful run hands it over (see below). The workers feed `should_cancel` from `QThread.isInterruptionRequested`; `tests/test_extraction_working_folder.py`, `tests/test_extraction_stop.py` and `tests/test_folder_refused_strings.py` lock all of this. - **Deleting the working folder after the run (perf proposal).** Both big deletes of a DataForge run (the old cache, about 31,000 files, and what unforge wrote that the cache does not keep, about 31,000 more) used to come before generation could start. Beside the cache, step 3 no longer wipes the old cache: once its stamps are gone, `_set_old_cache_aside` renames it into the run's working folder (`_SET_ASIDE_CACHE_NAME`, one rename on one drive, with #484's retries for a held file), so nothing can read it as fresh there. It is wiped in place, as before, in `%TEMP%` (another drive), when a stamp would not go, when the rename is refused, and on a drive that has room only once it is gone: `_require_room_for_new_cache` returns True when it counted the old cache's room (or the drive is short and that room could not be measured). A copy after a refused move deletes a set-aside cache first (`_cache_filtered_records(before_copy=)`), so it gets the room the #482 checks counted. A caller that passes `defer_cleanup` (the DataForge worker) is handed a `WorkingFolder` for a run that saved and stamped the new cache with no stop pending, and not on such a tight drive, instead of the folder being deleted before `extract_dataforge` returns; every other outcome still deletes it in the `finally`, except that a stop arriving right after the hand-over still raises `ExtractionCancelled` with the folder already handed over, and the caller then owns it (`DataForgeExtractWorker._finish` deletes it). The folder is beside the cache folder, or in `%TEMP%` when the work was there, never inside the cache folder. `_close_scratch`, which `WorkingFolder.delete` uses, deletes everything but the `.in_use` marker while the marker is held, then the marker and the folder, so another window's sweep never touches a folder still being deleted, and a crash part-way leaves the marker file for the next sweep to take at once (on the Python 3.11 the release builds use, rmtree deleted the marker first). An entry that will not go (a file another program holds under `Data`) does not stop the others: each entry is tried, the first error is logged once, and the set-aside old cache still goes, with the marker file left for the next sweep. `WorkingFolder.release` closes the marker and leaves the folder to that sweep. Who deletes it, when, and how closing waits is in `src/gui/CLAUDE.md`. `tests/test_extraction_working_folder.py` (`TestTheOldCacheIsSetAside`, `TestTheWorkingFolderGoesAfterTheRun`) locks this. + **Deleting the working folder after the run (#522).** Both big deletes of a DataForge run (the old cache, about 31,000 files, and what unforge wrote that the cache does not keep, about 31,000 more) used to come before generation could start. Beside the cache, step 3 no longer wipes the old cache: once its stamps are gone, `_set_old_cache_aside` renames it into the run's working folder (`_SET_ASIDE_CACHE_NAME`, one rename on one drive, with #484's retries for a held file), so nothing can read it as fresh there. It is wiped in place, as before, in `%TEMP%` (another drive), when a stamp would not go, when the rename is refused, and on a drive that has room only once it is gone: `_require_room_for_new_cache` returns True when it counted the old cache's room (or the drive is short and that room could not be measured). A copy after a refused move deletes a set-aside cache first (`_cache_filtered_records(before_copy=)`), so it gets the room the #482 checks counted. A caller that passes `defer_cleanup` (the DataForge worker) is handed a `WorkingFolder` for a run that saved and stamped the new cache with no stop pending, and not on such a tight drive, instead of the folder being deleted before `extract_dataforge` returns; every other outcome still deletes it in the `finally`, except that a stop arriving right after the hand-over still raises `ExtractionCancelled` with the folder already handed over, and the caller then owns it (`DataForgeExtractWorker._finish` deletes it). The folder is beside the cache folder, or in `%TEMP%` when the work was there, never inside the cache folder. `_close_scratch`, which `WorkingFolder.delete` uses, deletes everything but the `.in_use` marker while the marker is held, then the marker and the folder, so another window's sweep never touches a folder still being deleted, and a crash part-way leaves the marker file for the next sweep to take at once (on the Python 3.11 the release builds use, rmtree deleted the marker first). An entry that will not go (a file another program holds under `Data`) does not stop the others: each entry is tried, the first error is logged once, and the set-aside old cache still goes, with the marker file left for the next sweep. `WorkingFolder.release` closes the marker and leaves the folder to that sweep. Who deletes it, when, and how closing waits is in `src/gui/CLAUDE.md`. `tests/test_extraction_working_folder.py` (`TestTheOldCacheIsSetAside`, `TestTheWorkingFolderGoesAfterTheRun`) locks this. - **Free space (#482).** Each extraction checks free space in `_begin_scratch`, through its `check_space` callback, after both sweeps (a crashed run's leftover is gone before it is counted) and before any folder is made (a drive with nothing left still gets the clear message, not a raw `os.mkdir` error). The input checks come first, so a missing tool is still reported ahead of a short drive. `_require_free_space` counts each part on the volume it lands on: for DataForge (`_dataforge_needs`) the working folder (Game2.dcb plus unforge's whole tree, ~62k files) where the run works and the new cache (~31k files) at the leaf, less the old cache when the leaf holds a stamped one (`.p4k_mtime` and `raw/libs`, the same probe as the Config tab's), which the run sets aside or wipes, and frees before any copy (`_credit`: at least its bytes and at least a cluster per file, as the new cache is charged); for global.ini (`_global_ini_needs`) its work, one English global.ini (`_GLOBAL_INI_WORK_BYTES` is `_BASE_INI_BYTES`, 11 MiB in one file, measured at 10.1 MB), and base.ini on the data folder's drive. Parts on one volume (`os.stat().st_dev`, which gets subst drives and mounted folders right) are added together, and each part is measured at its nearest existing folder, because the Windows calls fail on one that does not exist yet. `_space_for` adds 15% and one cluster per file (`GetDiskFreeSpaceW` on the volume, 4096 when unknown), so a 128 KiB-cluster exFAT stick asks for far more, and free is the smaller of `shutil.disk_usage` and `GetDiskFreeSpaceExW`'s caller figure (NTFS quotas). Each volume gets one INFO line, `Free space on D: is N bytes, about M needed (4096-byte clusters)`, named by its mount path from `GetVolumePathNameW` (`_drive_label`, the drive in the path's text, only when that call fails). The figures are the `_DATAFORGE_*`, `_GLOBAL_INI_*` and `_BASE_INI_BYTES` constants, measured on LIVE 4.10 with GB read as GiB: a first run on an NTFS drive needs about 4.2 GiB, a re-extract over a complete cache about 3.0 GiB. The figures stay sized for a copy of the new cache even though the work beside the cache moves it (#484): a rename is refused while another program holds any file under the subtree, and that subtree is then copied after the wipe, so only a copy-sized check keeps that fallback from running out of room. The old cache's credit holds although beside the cache it is now set aside rather than wiped, and freed only after the run (perf proposal): a run that needed that room to pass the check before the wipe wipes it in place as before (`_require_room_for_new_cache` returns True), a move takes no new room, and a copy deletes a set-aside cache first (`before_copy`). Every probe is best effort: one that fails, or a part with no existing folder above it (a drive that is gone), is logged at WARNING and skipped, so measuring never refuses a run. `_require_room_for_new_cache` checks again just before the old cache is wiped (the drive can fill while unforge runs), from one walk of what unforge really wrote, and counts the old cache's allocation as room when the drive is short, except files the directory listing marks `OFFLINE`, `RECALL_ON_OPEN` or `RECALL_ON_DATA_ACCESS` (OneDrive online-only files, `_ONLINE_ONLY_ATTRIBUTES`, `_walk_files(local_only=True)`), which free nothing. When it refuses, it adds this run's working folder's allocation to both figures if that folder is on the cache's volume (`st_dev`), so the box matches the drive once the working folder is deleted, and it never names less than a retry's up-front check asks that drive for (the new cache, plus the working folder when it was there). Its INFO line (`DataForge run wrote ... The new cache is ...`) is how the constants get checked against a real run. A copy (in `%TEMP%`, or after a refused rename) passes `copy_function=_copy_file_checked`, which turns a full drive (ENOSPC, or winerror 39, 112 or 1295, of which Python maps only 112 to ENOSPC) into `_DiskFullDuringCopy`, not an OSError, so `copytree` stops at once instead of folding it into `shutil.Error` and copying on, and a rename that finds the drive full raises `_DiskFullDuringCopy` too; a full drive anywhere from the cache's mkdir to its stamps becomes the "cache incomplete" message, both stamps are removed (`_remove_stamps`, after any failure in that stretch and before the wipe; the mtime stamp alone reads as fresh), and the unstamped leaf is offered for a re-extract at startup. A shortage raises `NotEnoughSpaceError`, a `RuntimeError` (never an OSError, so `_begin_scratch`'s `except OSError` cannot read it as a folder that cannot be made) with a `tr()` message that names each short drive with what it has (rounded down) and needs (rounded up), and the advice that fits it: the cache advice for DataForge's cache drive, the temp advice for DataForge's work in `%TEMP%`, and the plain advice for every global.ini part, because a global.ini run can be in `%TEMP%` for reasons the temp advice does not list. It sits in `ANTICIPATED_EXTRACTION_ERRORS` with `P4kLockedError`, `DataForgeTimeoutError` and `WorkingFolderRefusedError` (#490), which the workers report at WARNING (see `src/gui/CLAUDE.md`). The global.ini extraction moves its work to `%TEMP%` only when the work's own drive beside the cache is short, as it does when that folder cannot be made. That shortage and the move are both INFO lines, with nothing at WARNING: `check_space`'s third argument (`movable`) hands the folder beside the cache to `_require_free_space(movable=)`, which then raises the private `_WorkDriveShort` that `_begin_scratch` catches. A short data folder drive alone (base.ini's part, which no move helps) refuses at once, at WARNING, naming only that drive with the plain advice. When the drive beside the cache and `%TEMP%`'s are both short, the refusal names `%TEMP%`'s drive only. DataForge never falls back: its cache needs that drive anyway. `tests/test_extraction_working_folder.py` (`TestFreeSpaceCheck` and the global.ini tests) and `tests/test_free_space_strings.py` lock this. + **Free space (#482).** Each extraction checks free space in `_begin_scratch`, through its `check_space` callback, after both sweeps (a crashed run's leftover is gone before it is counted) and before any folder is made (a drive with nothing left still gets the clear message, not a raw `os.mkdir` error). The input checks come first, so a missing tool is still reported ahead of a short drive. `_require_free_space` counts each part on the volume it lands on: for DataForge (`_dataforge_needs`) the working folder (Game2.dcb plus unforge's whole tree, ~62k files) where the run works and the new cache (~31k files) at the leaf, less the old cache when the leaf holds a stamped one (`.p4k_mtime` and `raw/libs`, the same probe as the Config tab's), which the run sets aside or wipes, and frees before any copy (`_credit`: at least its bytes and at least a cluster per file, as the new cache is charged); for global.ini (`_global_ini_needs`) its work, one English global.ini (`_GLOBAL_INI_WORK_BYTES` is `_BASE_INI_BYTES`, 11 MiB in one file, measured at 10.1 MB), and base.ini on the data folder's drive. Parts on one volume (`os.stat().st_dev`, which gets subst drives and mounted folders right) are added together, and each part is measured at its nearest existing folder, because the Windows calls fail on one that does not exist yet. `_space_for` adds 15% and one cluster per file (`GetDiskFreeSpaceW` on the volume, 4096 when unknown), so a 128 KiB-cluster exFAT stick asks for far more, and free is the smaller of `shutil.disk_usage` and `GetDiskFreeSpaceExW`'s caller figure (NTFS quotas). Each volume gets one INFO line, `Free space on D: is N bytes, about M needed (4096-byte clusters)`, named by its mount path from `GetVolumePathNameW` (`_drive_label`, the drive in the path's text, only when that call fails). The figures are the `_DATAFORGE_*`, `_GLOBAL_INI_*` and `_BASE_INI_BYTES` constants, measured on LIVE 4.10 with GB read as GiB: a first run on an NTFS drive needs about 4.2 GiB, a re-extract over a complete cache about 3.0 GiB. The figures stay sized for a copy of the new cache even though the work beside the cache moves it (#484): a rename is refused while another program holds any file under the subtree, and that subtree is then copied after the wipe, so only a copy-sized check keeps that fallback from running out of room. The old cache's credit holds although beside the cache it is now set aside rather than wiped, and freed only after the run (#522): a run that needed that room to pass the check before the wipe wipes it in place as before (`_require_room_for_new_cache` returns True), a move takes no new room, and a copy deletes a set-aside cache first (`before_copy`). Every probe is best effort: one that fails, or a part with no existing folder above it (a drive that is gone), is logged at WARNING and skipped, so measuring never refuses a run. `_require_room_for_new_cache` checks again just before the old cache is wiped (the drive can fill while unforge runs), from one walk of what unforge really wrote, and counts the old cache's allocation as room when the drive is short, except files the directory listing marks `OFFLINE`, `RECALL_ON_OPEN` or `RECALL_ON_DATA_ACCESS` (OneDrive online-only files, `_ONLINE_ONLY_ATTRIBUTES`, `_walk_files(local_only=True)`), which free nothing. When it refuses, it adds this run's working folder's allocation to both figures if that folder is on the cache's volume (`st_dev`), so the box matches the drive once the working folder is deleted, and it never names less than a retry's up-front check asks that drive for (the new cache, plus the working folder when it was there). Its INFO line (`DataForge run wrote ... The new cache is ...`) is how the constants get checked against a real run. A copy (in `%TEMP%`, or after a refused rename) passes `copy_function=_copy_file_checked`, which turns a full drive (ENOSPC, or winerror 39, 112 or 1295, of which Python maps only 112 to ENOSPC) into `_DiskFullDuringCopy`, not an OSError, so `copytree` stops at once instead of folding it into `shutil.Error` and copying on, and a rename that finds the drive full raises `_DiskFullDuringCopy` too; a full drive anywhere from the cache's mkdir to its stamps becomes the "cache incomplete" message, both stamps are removed (`_remove_stamps`, after any failure in that stretch and before the wipe; the mtime stamp alone reads as fresh), and the unstamped leaf is offered for a re-extract at startup. A shortage raises `NotEnoughSpaceError`, a `RuntimeError` (never an OSError, so `_begin_scratch`'s `except OSError` cannot read it as a folder that cannot be made) with a `tr()` message that names each short drive with what it has (rounded down) and needs (rounded up), and the advice that fits it: the cache advice for DataForge's cache drive, the temp advice for DataForge's work in `%TEMP%`, and the plain advice for every global.ini part, because a global.ini run can be in `%TEMP%` for reasons the temp advice does not list. It sits in `ANTICIPATED_EXTRACTION_ERRORS` with `P4kLockedError`, `DataForgeTimeoutError` and `WorkingFolderRefusedError` (#490), which the workers report at WARNING (see `src/gui/CLAUDE.md`). The global.ini extraction moves its work to `%TEMP%` only when the work's own drive beside the cache is short, as it does when that folder cannot be made. That shortage and the move are both INFO lines, with nothing at WARNING: `check_space`'s third argument (`movable`) hands the folder beside the cache to `_require_free_space(movable=)`, which then raises the private `_WorkDriveShort` that `_begin_scratch` catches. A short data folder drive alone (base.ini's part, which no move helps) refuses at once, at WARNING, naming only that drive with the plain advice. When the drive beside the cache and `%TEMP%`'s are both short, the refusal names `%TEMP%`'s drive only. DataForge never falls back: its cache needs that drive anyway. `tests/test_extraction_working_folder.py` (`TestFreeSpaceCheck` and the global.ini tests) and `tests/test_free_space_strings.py` lock this. **Data.p4k being written (#516).** Both extractions call `_refuse_if_p4k_being_written` right after the input checks, before any folder or tool. It raises `P4kLockedError` (the same "wait for the launcher" message as the locked case, logged at WARNING) when `_p4k_has_writer` finds another handle that can write the archive (a `CreateFileW` read with `FILE_SHARE_READ` only fails with a sharing violation; the game and an idle launcher only read, so they pass) or `_p4k_changed_recently` finds its modified time inside `_P4K_RECENT_WRITE_SECONDS` (120). The launcher does not always lock readers out while it patches, and unp4k then crashed inside SharpZipLib's `ReadEntries`. A modified time up to `_P4K_CLOCK_SLACK_SECONDS` (2) ahead of the clock counts as now, because before Python 3.13 `time.time()` on Windows reads the same 15.6 ms tick Windows stamps a new file with, so a file written a moment ago comes out 0 seconds old or a rounding error less, which failed the just-written tests at random on CI (Python 3.11); a time further ahead (a clock that moved) does not count. `tests/conftest.py`'s `no_recent_p4k_refusal` sets the window to 0 for the session, since every test's Data.p4k is new, and a window of 0 counts nothing, a time ahead of the clock included; `tests/test_p4k_being_written.py` locks this. **A cache folder that is a link (#491).** A user can make the DataForge cache folder itself a junction or a symbolic link, usually to keep the cache on another drive. `shutil.rmtree` refuses a link and deletes nothing, and `robust_rmtree`'s error hook used to call the refusing `os.path.islink` again, which returns instead of raising, so the wipe read as done: every re-extract then met the old cache after unforge and failed with a raw `FileExistsError`, and Clear Cache and the queued old-cache cleanup reported a folder gone that was not. Nothing is deleted through a link now. `linked_folder_target(path)` reads a link by rmtree's own rule (a symbolic link, or a junction, `IO_REPARSE_TAG_MOUNT_POINT`, which is also how a drive mounted in a folder looks) without following it, so a link whose target is gone counts, and names the folder at its end (`os.path.realpath`, without the long-path prefix, or the link's own text from `os.readlink` when realpath raises, as it does for an error it does not expect such as a network that cannot be reached, or "" when neither can be read); any other reparse point, such as a OneDrive folder, is not a link to it. `robust_rmtree` raises `LinkedFolderError` for a link, also one whose target is gone (it asks `os.path.lexists`, not `exists`), at once and never again: an `OSError`, so every caller's `except OSError` reports it. A folder that only holds a link is still deleted, the link with it and its target kept. `extract_dataforge` asks `linked_cache_folder_error(leaf)` before anything runs, which logs the link at WARNING and returns `CacheFolderIsLinkError` (in `ANTICIPATED_EXTRACTION_ERRORS`, so one box), its `extract.cache_folder_is_link` naming the folder and where it leads and saying to choose that folder as the DataForge cache folder in the Config tab, with the old cache and the link kept. `MainWindow._run_dataforge_extraction` asks the same before any worker exists (see `src/gui/CLAUDE.md`). Only the cache folder itself counts: a link higher up the path is an ordinary path to the wipe. `tests/test_linked_cache_folder.py` locks this. - **Deleting a tree (perf proposal).** `robust_rmtree` deletes the working folder (and the old cache set aside in it), the old cache where a re-extract wipes it in place or deletes a set-aside one before a copy, the cache for Clear Cache and the queued old-cache cleanup. One file at a time it took about 220 us a file on a real run (6.9 s for the ~31k files unforge wrote that the cache does not keep, and a re-extract also wipes the ~31k-file old cache). Each attempt now runs `_delete_files_first` first: it walks the tree on one thread and hands the plain files to `_DELETE_THREADS` threads (`min(8, os.cpu_count())`) in batches of `_DELETE_BATCH` (64), each clearing the read-only bit and trying once more when a delete fails, and leaves anything that still will not go. The threads are daemon threads, started one per batch up to `_DELETE_THREADS` and handed at most twice that many batches ahead through a bounded `queue.Queue`, so Python's exit never waits for them. A `ThreadPoolExecutor`'s threads were joined at exit only after every queued batch had run, and the walk queues nearly the whole tree while the first deletes run, so a close past `_stop_workers_for_close`'s 180 s deadline kept the hidden process alive until the tree was gone, or for ever behind one delete that blocked. Once Python begins to exit (`_EXITING`, set by `atexit`) the walk stops and the threads drop the batches still queued: `shutil.rmtree` goes on with the rest on the calling thread, which Python does not wait for either (the delete worker's `QThread`), and what the process does not finish stays with the working folder's marker file for the next sweep, since `_close_scratch` deletes the marker last. Where Python starts no new thread (while it exits, from 3.12) rmtree deletes it all, one file at a time. Any other exception in a thread (an OSError never gets out of one) stops the walk and is raised on the calling thread, its failure list emptied as it is raised: the list and the exception's traceback held each other, which kept everything in those frames alive until the cyclic GC ran, a working folder's marker included, so a sweep straight after skipped that folder. `shutil.rmtree` then runs as before over what is left, the folders bottom-up and any file the threads could not delete, one at a time with the read-only hook, so every error a caller sees is still rmtree's, raised by the hook. Measured on this PC on a synthetic 30,000-file tree shaped like the DataForge records: 3.9 to 5.1 s one at a time, 1.3 to 1.4 s with 8 threads (16 were no faster). The daemon threads are as fast as the pool they replaced: medians of 1.31 s against the pool's 1.37 s and 4.08 s one at a time, over six interleaved rounds. The walk follows rmtree's own rule (`is_link_stat`, which `linked_folder_target` and `shader_cache._is_link` use too): it goes into a real folder only, never a junction or a symbolic link, and deletes only plain files, so a link handed in is still refused by rmtree's hook with `LinkedFolderError` and nothing deleted, and a link inside is still unlinked by rmtree without being followed (#491). The whole-tree retry keeps its backoff (0.2, 0.4, 0.8, 1.6 and 3.0 s between six attempts), but nothing waits after the last attempt: the last failure used to sleep up to 3 s before raising, and the leftover sweep's second attempt slept for nothing. `tests/test_robust_rmtree.py` locks this. + **Deleting a tree (#522).** `robust_rmtree` deletes the working folder (and the old cache set aside in it), the old cache where a re-extract wipes it in place or deletes a set-aside one before a copy, the cache for Clear Cache and the queued old-cache cleanup. One file at a time it took about 220 us a file on a real run (6.9 s for the ~31k files unforge wrote that the cache does not keep, and a re-extract also wipes the ~31k-file old cache). Each attempt now runs `_delete_files_first` first: it walks the tree on one thread and hands the plain files to `_DELETE_THREADS` threads (`min(8, os.cpu_count())`) in batches of `_DELETE_BATCH` (64), each clearing the read-only bit and trying once more when a delete fails, and leaves anything that still will not go. The threads are daemon threads, started one per batch up to `_DELETE_THREADS` and handed at most twice that many batches ahead through a bounded `queue.Queue`, so Python's exit never waits for them. A `ThreadPoolExecutor`'s threads were joined at exit only after every queued batch had run, and the walk queues nearly the whole tree while the first deletes run, so a close past `_stop_workers_for_close`'s 180 s deadline kept the hidden process alive until the tree was gone, or for ever behind one delete that blocked. Once Python begins to exit (`_EXITING`, set by `atexit`) the walk stops and the threads drop the batches still queued: `shutil.rmtree` goes on with the rest on the calling thread, which Python does not wait for either (the delete worker's `QThread`), and what the process does not finish stays with the working folder's marker file for the next sweep, since `_close_scratch` deletes the marker last. Where Python starts no new thread (while it exits, from 3.12) rmtree deletes it all, one file at a time. Any other exception in a thread (an OSError never gets out of one) stops the walk and is raised on the calling thread, its failure list emptied as it is raised: the list and the exception's traceback held each other, which kept everything in those frames alive until the cyclic GC ran, a working folder's marker included, so a sweep straight after skipped that folder. `shutil.rmtree` then runs as before over what is left, the folders bottom-up and any file the threads could not delete, one at a time with the read-only hook, so every error a caller sees is still rmtree's, raised by the hook. Measured on this PC on a synthetic 30,000-file tree shaped like the DataForge records: 3.9 to 5.1 s one at a time, 1.3 to 1.4 s with 8 threads (16 were no faster). The daemon threads are as fast as the pool they replaced: medians of 1.31 s against the pool's 1.37 s and 4.08 s one at a time, over six interleaved rounds. The walk follows rmtree's own rule (`is_link_stat`, which `linked_folder_target` and `shader_cache._is_link` use too): it goes into a real folder only, never a junction or a symbolic link, and deletes only plain files, so a link handed in is still refused by rmtree's hook with `LinkedFolderError` and nothing deleted, and a link inside is still unlinked by rmtree without being followed (#491). The whole-tree retry keeps its backoff (0.2, 0.4, 0.8, 1.6 and 3.0 s between six attempts), but nothing waits after the last attempt: the last failure used to sleep up to 3 s before raising, and the leftover sweep's second attempt slept for nothing. `tests/test_robust_rmtree.py` locks this. **The bar while a tool runs (#507).** unp4k and unforge report nothing while they run, so each extraction reports those phases with a total of 0, which `AnimatedProgressDialog.set_progress` draws as its moving busy bar: `progress.unp4k_launch_short` for global.ini, and `progress.extract_dcb` and `progress.unforge_short` for DataForge. A counted bar used to stand at 0% for the whole unp4k run, and at 33% for the minutes unforge takes, which looked as if the app had hung. The counted phases stay as they were: copying global.ini (1 of 2), caching the entity files (2 of 3) and Done. The dialog shows the busy bar for any count still at 0 too (see `src/gui/CLAUDE.md`), so the generator's first lookups move as well. `tests/test_extraction_working_folder.py` (`TestToolPhasesShowTheBusyBar`) and `tests/test_extraction_stop.py` (`TestProgressDialogBusyBar`) lock this. - **base.ini freshness (perf proposal).** `base_ini_is_fresh(p4k_path, base_ini)` says whether the extracted English base.ini is still current, by #209's rule for the DataForge cache: Data.p4k's byte size, not its modified time, which the RSI launcher's file verification moves on an ordinary game launch. The old rule (Data.p4k newer than base.ini) asked to extract global.ini again (20 to 26 s) at the next start, channel switch or data folder change after nothing more than playing, and lit the Config tab's Extract button. `extract_global_ini` reads Data.p4k's size before unp4k runs, so a Data.p4k that changes during the run reads as changed afterwards (`extract_dataforge` now reads the stat for its `.p4k_mtime` and `.p4k_size` stamps there too: read after unforge, a patch landing during a DataForge run made the old build's cache read as current), and `_write_base_ini_stamp` records it with base.ini's own size in `BASE_INI_STAMP` (`.base_ini_stamp`, beside base.ini). base.ini's size ties the stamp to the base.ini it was written for, because the files in that cache folder come and go on their own: Clear Cache deletes `*.ini` and leaves the stamp, and a data folder change copies a file only where the new folder has none. A stamp for another base.ini, or one that is not two numbers, counts as no stamp, and then the old rule decides, so an upgrade asks for nothing the old rule would not have asked for, and reads a patch as the old rule reads it at that check. When the old rule reads fresh (Data.p4k no newer than base.ini, which normally means untouched since base.ini was written), its size is recorded there and then, and the next launcher touch needs no extraction first. A patch the old rule misses (one that left Data.p4k's modified time no newer than base.ini's) is recorded the same way, so it stays missed until the next patch changes the size, where the old rule would have asked after the next game launch: nothing else tells a launch from such a patch. `extract_global_ini` removes the old stamp once the new base.ini is in place and before it writes the new one (one that will not go is logged at WARNING; removed before the copy, a copy that failed left the old base.ini with no stamp, and the modified times could read it fresh against a patched Data.p4k), and a new stamp that cannot be written is logged at WARNING and fails nothing, so the old rule then decides. Kept past such a failed write, the old stamp spoke for the new base.ini whenever the two were the same size, and after a patch that left global.ini as it was, base.ini read stale straight after every extraction. Both callers use it: `MainWindow._check_p4k_freshness` and `ConfigTab._refresh_p4k_status` (see `src/gui/CLAUDE.md`). As with #209, a patch that left Data.p4k's size exactly as it was would go unnoticed. `tests/test_base_ini_freshness.py` locks this. + **base.ini freshness (#522).** `base_ini_is_fresh(p4k_path, base_ini)` says whether the extracted English base.ini is still current, by #209's rule for the DataForge cache: Data.p4k's byte size, not its modified time, which the RSI launcher's file verification moves on an ordinary game launch. The old rule (Data.p4k newer than base.ini) asked to extract global.ini again (20 to 26 s) at the next start, channel switch or data folder change after nothing more than playing, and lit the Config tab's Extract button. `extract_global_ini` reads Data.p4k's size before unp4k runs, so a Data.p4k that changes during the run reads as changed afterwards (`extract_dataforge` now reads the stat for its `.p4k_mtime` and `.p4k_size` stamps there too: read after unforge, a patch landing during a DataForge run made the old build's cache read as current), and `_write_base_ini_stamp` records it with base.ini's own size in `BASE_INI_STAMP` (`.base_ini_stamp`, beside base.ini). base.ini's size ties the stamp to the base.ini it was written for, because the files in that cache folder come and go on their own: Clear Cache deletes `*.ini` and leaves the stamp, and a data folder change copies a file only where the new folder has none. A stamp for another base.ini, or one that is not two numbers, counts as no stamp, and then the old rule decides, so an upgrade asks for nothing the old rule would not have asked for, and reads a patch as the old rule reads it at that check. When the old rule reads fresh (Data.p4k no newer than base.ini, which normally means untouched since base.ini was written), its size is recorded there and then, and the next launcher touch needs no extraction first. A patch the old rule misses (one that left Data.p4k's modified time no newer than base.ini's) is recorded the same way, so it stays missed until the next patch changes the size, where the old rule would have asked after the next game launch: nothing else tells a launch from such a patch. `extract_global_ini` removes the old stamp once the new base.ini is in place and before it writes the new one (one that will not go is logged at WARNING; removed before the copy, a copy that failed left the old base.ini with no stamp, and the modified times could read it fresh against a patched Data.p4k), and a new stamp that cannot be written is logged at WARNING and fails nothing, so the old rule then decides. Kept past such a failed write, the old stamp spoke for the new base.ini whenever the two were the same size, and after a patch that left global.ini as it was, base.ini read stale straight after every extraction. Both callers use it: `MainWindow._check_p4k_freshness` and `ConfigTab._refresh_p4k_status` (see `src/gui/CLAUDE.md`). As with #209, a patch that left Data.p4k's size exactly as it was would go unnoticed. `tests/test_base_ini_freshness.py` locks this. - **What the tools are asked for and what they print (perf proposal).** unp4k extracts every entry whose name holds its filter, ignoring case, and names entries with `/` (`Data/Localization/english/global.ini`). The global.ini extraction used to hand it `global.ini`, which took all 11 languages' files, and unp4k reads the whole archive's directory once per parallel worker: measured on a 150 GB Data.p4k, 20 to 26 s and 9 GB of peak memory. It now hands it `_GLOBAL_INI_FILTER`, `_GLOBAL_INI_RELATIVE` without its `data/` (`Localization/english/global.ini`): one 10.1 MB file, byte for byte the same English file, in 5.4 s with 1.65 GB peak with the release's unp4k.exe, whose single parallel worker still opens the archive and reads its directory a second time. The bundled unp4k.exe and unforge.exe come from odw-fast-unp4k's release v4.0.4 (main eee6ac4), built by that repo's own release workflow: `unp4k-win-x64-v4.0.4.zip` and `unforge-win-x64-v4.0.4.zip`, with `unforge.cli.exe` renamed to `unforge.exe`. That main holds f209183 (on 69575e1, which takes out unp4k's error telemetry), whose workers all extract through the archive unp4k opened for the listing, so a run that extracts Game2.dcb or this file reads the archive's directory once: about 2 s and 1 GB for global.ini. A rebuilt unp4k.exe without that change keeps the filter's gain but not the rest. It also holds ad7c6a7, which adds `--exclude` and `--exclude-file` to 32afd06, with which unforge reads the records on every core with server GC: about 14 s for Game2.dcb against the release's 40 s, writing the same files, and about 9 s handed the exclude list. The four commits reached odw-fast-unp4k through its PRs #9 to #12 (#526). Both tools are self-contained single-file net10.0 builds for win-x64 (`x64\libzstd.dll` and `x86\libzstd.dll` come with unp4k). unforge was a net8.0 build before, which peaked at about 0.86 GB on Game2.dcb against about 1.3 GB on net10.0. To rebuild by hand, from a checkout of that main: `dotnet publish src\unp4k\unp4k.csproj -c Release -r win-x64 -f net10.0` and `dotnet publish src\unforge.cli\unforge.cli.csproj -c Release -r win-x64 -f net10.0`, with `unforge.cli.exe` renamed to `unforge.exe`. The run only ever reads that file; other languages come from community downloads (`languages/sources.json`). The free-space figures follow (see *Free space* above). unp4k exits 0 after an entry it could not extract, printing `Exception while extracting : ` for each, and unforge prints `Error converting : ` when anything in its conversion fails, from reading the dcb to saving a file (one catch around it all), which can leave a partial tree that passes `_unforge_records` and would be stamped as a complete cache. The bundled unforge (ad7c6a7) then exits 1, but one built before it exits 0, and that is the one the fallback for an unforge without `--exclude-file` runs. `_UNP4K_ENTRY_FAILED` and `_UNFORGE_FILE_FAILED` are copies of literals in odw-fast-unp4k's `src/unp4k/Program.cs` and `src/unforge.cli/Program.cs`, so a rebuilt tool that words either differently must change them too: `test_the_bundled_tools_hold_their_failure_text` checks the bundled `assets/unp4k/unp4k.exe` and `unforge.exe` still hold them as UTF-16LE text, and `test_the_bundled_unforge_takes_the_exclude_list` that `unforge.exe` holds `--exclude-file` and `_UNFORGE_USAGE`. `_check_unp4k`, which both unp4k runs go through, fails a run whose stdout or stderr holds `_UNP4K_ENTRY_FAILED` through `_raise_unp4k_failure`, as a non-zero exit does (a locked Data.p4k is still `P4kLockedError`, anything else a `RuntimeError` logged at ERROR, worded "exited with code 0 but could not extract a file"), and unforge output holding `_UNFORGE_FILE_FAILED` raises the same `RuntimeError` as its non-zero exit, with the output, before the old cache is wiped. So a dcb unforge cannot read fails before the `libs/` check, as `unforge.exe failed (code 1)` with the bundled unforge and `(code 0)` with an older one. Nothing is required of the output on success. `_check_unp4k` also logs unp4k's stdout at INFO, cut at 2000 characters as unforge's is, so its `Found N files to extract.` line shows what the filter matched. `extract_dataforge` no longer runs `gc.collect()` and sleeps 0.1 s after each tool and before caching, or collects at the end: each tool is a separate process `_run_tool` has already reaped, a refused move has its own retries (`_MOVE_RETRY_DELAYS`) and the wipe `robust_rmtree`'s backoff, whose own `gc.collect()` stays. `tests/test_extraction_working_folder.py` (`TestGlobalIniTakesTheEnglishFileAlone` and `TestToolsThatExitZeroAfterAnError`) locks this. + **What the tools are asked for and what they print (#522 and #529).** unp4k extracts every entry whose name holds its filter, ignoring case, and names entries with `/` (`Data/Localization/english/global.ini`). The global.ini extraction used to hand it `global.ini`, which took all 11 languages' files, and unp4k reads the whole archive's directory once per parallel worker: measured on a 150 GB Data.p4k, 20 to 26 s and 9 GB of peak memory. It now hands it `_GLOBAL_INI_FILTER`, `_GLOBAL_INI_RELATIVE` without its `data/` (`Localization/english/global.ini`): one 10.1 MB file, byte for byte the same English file, in 5.4 s with 1.65 GB peak with the release's unp4k.exe, whose single parallel worker still opens the archive and reads its directory a second time. The bundled unp4k.exe and unforge.exe come from odw-fast-unp4k's release v4.0.4 (main eee6ac4), built by that repo's own release workflow: `unp4k-win-x64-v4.0.4.zip` and `unforge-win-x64-v4.0.4.zip`, with `unforge.cli.exe` renamed to `unforge.exe`. That main holds f209183 (on 69575e1, which takes out unp4k's error telemetry), whose workers all extract through the archive unp4k opened for the listing, so a run that extracts Game2.dcb or this file reads the archive's directory once: about 2 s and 1 GB for global.ini. A rebuilt unp4k.exe without that change keeps the filter's gain but not the rest. It also holds ad7c6a7, which adds `--exclude` and `--exclude-file` to 32afd06, with which unforge reads the records on every core with server GC: about 14 s for Game2.dcb against the release's 40 s, writing the same files, and about 9 s handed the exclude list. The four commits reached odw-fast-unp4k through its PRs #9 to #12 (#526). Both tools are self-contained single-file net10.0 builds for win-x64 (`x64\libzstd.dll` and `x86\libzstd.dll` come with unp4k). unforge was a net8.0 build before, which peaked at about 0.86 GB on Game2.dcb against about 1.3 GB on net10.0. To rebuild by hand, from a checkout of that main: `dotnet publish src\unp4k\unp4k.csproj -c Release -r win-x64 -f net10.0` and `dotnet publish src\unforge.cli\unforge.cli.csproj -c Release -r win-x64 -f net10.0`, with `unforge.cli.exe` renamed to `unforge.exe`. The run only ever reads that file; other languages come from community downloads (`languages/sources.json`). The free-space figures follow (see *Free space* above). unp4k exits 0 after an entry it could not extract, printing `Exception while extracting : ` for each, and unforge prints `Error converting : ` when anything in its conversion fails, from reading the dcb to saving a file (one catch around it all), which can leave a partial tree that passes `_unforge_records` and would be stamped as a complete cache. The bundled unforge (ad7c6a7) then exits 1, but one built before it exits 0, and that is the one the fallback for an unforge without `--exclude-file` runs. `_UNP4K_ENTRY_FAILED` and `_UNFORGE_FILE_FAILED` are copies of literals in odw-fast-unp4k's `src/unp4k/Program.cs` and `src/unforge.cli/Program.cs`, so a rebuilt tool that words either differently must change them too: `test_the_bundled_tools_hold_their_failure_text` checks the bundled `assets/unp4k/unp4k.exe` and `unforge.exe` still hold them as UTF-16LE text, and `test_the_bundled_unforge_takes_the_exclude_list` that `unforge.exe` holds `--exclude-file` and `_UNFORGE_USAGE`. `_check_unp4k`, which both unp4k runs go through, fails a run whose stdout or stderr holds `_UNP4K_ENTRY_FAILED` through `_raise_unp4k_failure`, as a non-zero exit does (a locked Data.p4k is still `P4kLockedError`, anything else a `RuntimeError` logged at ERROR, worded "exited with code 0 but could not extract a file"), and unforge output holding `_UNFORGE_FILE_FAILED` raises the same `RuntimeError` as its non-zero exit, with the output, before the old cache is wiped. So a dcb unforge cannot read fails before the `libs/` check, as `unforge.exe failed (code 1)` with the bundled unforge and `(code 0)` with an older one. Nothing is required of the output on success. `_check_unp4k` also logs unp4k's stdout at INFO, cut at 2000 characters as unforge's is, so its `Found N files to extract.` line shows what the filter matched. `extract_dataforge` no longer runs `gc.collect()` and sleeps 0.1 s after each tool and before caching, or collects at the end: each tool is a separate process `_run_tool` has already reaped, a refused move has its own retries (`_MOVE_RETRY_DELAYS`) and the wipe `robust_rmtree`'s backoff, whose own `gc.collect()` stays. `tests/test_extraction_working_folder.py` (`TestGlobalIniTakesTheEnglishFileAlone` and `TestToolsThatExitZeroAfterAnError`) locks this. - `dataforge_exclude.py` — `DATAFORGE_EXCLUDE_SUBPATHS`, the 538 record folders and files under `libs/foundry/records/` that `pak_extractor` hands unforge with `--exclude-file` and the DataForge cache never keeps: every record outside `DATAFORGE_KEEP_SUBPATHS` on LIVE's game build of 23 Oct 2026. An exclude list, not a keep list, so a folder a later patch adds is still written and cached. Nothing here may be, hold or lie in a kept subtree (`tests/test_pak_extraction.py::TestDataForgeExcludeList`). - `win_paths.py` — `win_long_path(path)`: prefixes a path with `\\?\` (or `\\?\UNC\` for a network share) so Windows file APIs accept it past the legacy 260-char `MAX_PATH`, with no machine-wide config needed (unlike the inbox "Enable Win32 long paths" policy, which requires admin rights and isn't guaranteed on for any given user — issue #221). The DataForge XML tree's long CIG-authored filenames routinely cross this limit once nested under a deep portable/tester install path (or a zip double-extracted into an already same-named folder), and `WinError 3` ("cannot find the path specified") fires even though the file exists. No-op on non-Windows and on already-prefixed paths. `win_plain_path(path)` is the inverse (strips the prefix, `\\?\UNC\` back to `\\`), for paths handed to unp4k and unforge (#471). Every DataForge-cache consumer wraps its own root path once at its entry point so every path it derives (via further `/` joins, `rglob`, `ET.parse`, etc.) inherits long-path safety for free — see `pak_extractor.py` (move, copy and cleanup, plus the two entry points above; `_rename_tree` prefixes both sides as well, #484), `dataforge_patcher.py` (patch target reads), and `scripts/generate_enhancements_ini.py`'s `main()` (wraps `forge_dir` once, covering 20+ `ET.parse` sites and 18+ `rglob` walks). `tests/test_win_long_path.py` locks all of the above. - `atomic_io.py`: writes a file in one step. `atomic_writer(path, mode)` (`"w"` or `"wb"`) writes to a `.sc-XXXXXXXX.tmp` file beside the target, flushes, syncs and closes it, then swaps it in with `os.replace`, so a run cut off by a close or a crash leaves the old file or the new one, never part of one. `atomic_write_text()` wraps it. The temp is made by `_create_temp_beside`, not `tempfile.mkstemp`: on Python 3.11, which the release builds use, mkstemp retries a PermissionError up to `os.TMP_MAX` (2^31) times on Windows, so a folder that refuses new files (Controlled Folder Access, a read-only share) hung the write for about two days. Only a name already taken is retried, up to `TEMP_CREATE_ATTEMPTS` (20) times, and any other error raises at once, naming the target rather than the temp, so the error dialog and the log say which file could not be written. Only a PermissionError from the swap is retried, for about 1.3 s (on Windows any other open handle on the target blocks `os.replace`, even one that shares delete). Any failure removes the temp, keeps the old file and raises. It never falls back to writing in place, which is what let readers see partial files. `read_bytes_retrying()` is the reader's side: a reader that opens a file during a swap gets PermissionError for a moment, retried for about 0.15 s. `sweep_stale_temps(folder)` removes temps over 10 minutes old, which only a hard kill between the temp and the swap leaves; `EnhancementsGeneratorWorker` sweeps the INI folder and the lookup pickle folder (`pak_extractor.DATAFORGE_LOOKUPS_SUBPATH`, `.lookups`) at the start of each run, and `dataforge_patcher` sweeps the folder of every record it patches, rewritten or not. Text mode writes the bytes `Path.write_text` wrote (UTF-8, no BOM, CRLF on Windows). Used by the generator's `write_ini` and lookup pickles (through its `src` shim), `mining_catalog`'s `write_crafting_index` and `load_crafting_index`, the stamp setters and getters in `settings.py`, `dataforge_patcher.py` and `ini_io.read_ini_text`. Stdlib only, Qt-free and settings-free, so a test can run it in a child on the base interpreter. `tests/test_atomic_io.py` locks it. diff --git a/src/utils/pak_extractor.py b/src/utils/pak_extractor.py index f7b2bb1d..b03a3d15 100644 --- a/src/utils/pak_extractor.py +++ b/src/utils/pak_extractor.py @@ -33,8 +33,8 @@ P4K_MTIME_STAMP = ".p4k_mtime" P4K_SIZE_STAMP = ".p4k_size" -# base.ini's freshness stamp, beside it in the channel's cache folder (perf -# proposal): the size of the Data.p4k it was extracted from, then base.ini's +# base.ini's freshness stamp, beside it in the channel's cache folder (#522): +# the size of the Data.p4k it was extracted from, then base.ini's # own size. Written by extract_global_ini and read by base_ini_is_fresh, by # the same size rule as the DataForge stamps above. BASE_INI_STAMP = ".base_ini_stamp" @@ -78,8 +78,8 @@ # tree that passes _unforge_records and would be stamped as a complete cache. # The bundled unforge (odw-fast-unp4k ad7c6a7) then exits 1, but one built # before it exits 0, and that is the one the fallback for an unforge without -# --exclude-file runs. Either text fails the run as a non-zero exit does (perf -# proposal), so a dcb unforge cannot read fails before the libs/ check: at +# --exclude-file runs. Either text fails the run as a non-zero exit does (#522), +# so a dcb unforge cannot read fails before the libs/ check: at # its exit code with the bundled unforge, at this text with an older one. # Nothing is required of their output on success. Both texts are # copies of literals in odw-fast-unp4k's src/unp4k/Program.cs and @@ -117,7 +117,7 @@ class DataForgeTimeoutError(RuntimeError): Overwhelmingly this means the game install is inconsistent rather than the machine being slow. Measured on a healthy 4.9.188.23497 install, unforge converts the 330 MB Game2.dcb in about 40 seconds against a - 1800 second budget (the unforge.exe bundled since the perf proposal + 1800 second budget (the unforge.exe bundled since #522 took about 15 s for the 337 MB Game2.dcb of 23 Oct 2026). Issue #370 was a user whose Data.p4k carried a 114 MB ``Game.dcb`` while their build_manifest.id was byte-for-byte @@ -345,7 +345,7 @@ def _check_unp4k(result) -> None: """Log unp4k's stdout at INFO, truncated as unforge's is (its "Found N files to extract." says what the filter matched), then raise through _raise_unp4k_failure when the run failed: a non-zero exit, or an entry - it says it could not extract (_UNP4K_ENTRY_FAILED, perf proposal).""" + it says it could not extract (_UNP4K_ENTRY_FAILED, #522).""" stdout = (result.stdout or "").strip() if stdout: logger.info(f"unp4k stdout ({len(stdout)} bytes, truncated): {stdout[:2000]}") @@ -453,7 +453,7 @@ def linked_cache_folder_error(leaf) -> Optional[CacheFolderIsLinkError]: # robust_rmtree deletes a tree's files with this many threads before -# shutil.rmtree removes the folders (perf proposal). A delete is mostly +# shutil.rmtree removes the folders (#522). A delete is mostly # Windows' own work (the file system, antivirus), done outside Python's lock, # so several overlap well. Measured on this PC on a 30,000-file tree shaped # like the DataForge records: rmtree alone took 3.9 to 5.1 s, with 8 threads @@ -463,7 +463,7 @@ def linked_cache_folder_error(leaf) -> Optional[CacheFolderIsLinkError]: # hand out than to delete, and one per folder would leave a folder with # thousands of files to a single thread. _DELETE_BATCH = 64 -# Set once Python begins to exit (perf proposal). The threaded pass then +# Set once Python begins to exit (#522). The threaded pass then # hands out no more batches, and its threads drop the ones still queued. _EXITING = threading.Event() atexit.register(_EXITING.set) @@ -508,7 +508,7 @@ def _delete_batches(batches: queue.Queue, failed: list) -> None: def _delete_files_first(top: str) -> None: """Delete the plain files under the folder *top* with _DELETE_THREADS - threads (perf proposal), leaving shutil.rmtree, which robust_rmtree runs + threads (#522), leaving shutil.rmtree, which robust_rmtree runs next, the folders and whatever would not go. It walks by rmtree's own rule: it goes into a folder only when that is a @@ -608,13 +608,13 @@ def robust_rmtree(path: Path, attempts: int = 6) -> None: tree on each attempt catches files added or unlocked between tries. Each attempt deletes the files with several threads first - (_delete_files_first, perf proposal: about 3x faster on ~30k files), + (_delete_files_first, #522: about 3x faster on ~30k files), then shutil.rmtree removes the folders bottom-up and anything that would not go, one at a time, with the hook below. Silently succeeds if *path* doesn't exist. Raises the last error if every attempt fails so callers can surface it to the user, without a - wait after the last attempt (perf proposal). + wait after the last attempt (#522). A *path* that is a link (a junction or a symbolic link, also one whose target is gone) raises LinkedFolderError at once, with nothing deleted @@ -661,7 +661,7 @@ def _on_error(func, target, *_): last_err = e if i + 1 == attempts: # Nothing follows the last attempt, so waiting first only - # held up the caller (perf proposal). + # held up the caller (#522). logger.warning(f"rmtree {path} attempt {i + 1}/{attempts} failed ({e}); " "giving up") break @@ -765,7 +765,7 @@ def _on_error(func, target, *_): _DATAFORGE_CACHE_BYTES = math.ceil(1.2 * _GIB) _DATAFORGE_CACHE_FILES = 30_700 _BASE_INI_BYTES = 11 * _MIB # the English global.ini -# unp4k extracts the English global.ini alone (perf proposal): one file, +# unp4k extracts the English global.ini alone (#522): one file, # measured at 10.1 MB, the same file base.ini is a copy of. _GLOBAL_INI_WORK_BYTES = _BASE_INI_BYTES _GLOBAL_INI_WORK_FILES = 1 @@ -824,7 +824,7 @@ def _sweep_stale_scratch(parent: Path, prefix: str, should_cancel=None) -> None: *should_cancel*, when given, is asked before each folder, and a stop leaves the rest for a later sweep: closing stops the window's own sweep - (StaleWorkingFolderSweepWorker, perf proposal) this way.""" + (StaleWorkingFolderSweepWorker, #522) this way.""" try: folders = [p for p in parent.glob(prefix + "*") if p.is_dir()] except OSError: @@ -900,15 +900,15 @@ def _close_scratch(run_dir: Path, marker) -> None: the cache had been stamped). Anything else, such as a MemoryError in a delete thread, still gets out. - Everything but the marker goes while the marker is still held (perf - proposal), so another window's sweep, which deletes only a folder whose + Everything but the marker goes while the marker is still held (#522), + so another window's sweep, which deletes only a folder whose marker it can delete, never works on a folder that is still being deleted, which can now take a while in the background (see WorkingFolder). A crash part-way, or a folder that will not go, leaves the marker file, closed, so the next sweep takes the folder at once rather than waiting out the grace for an unmarked one. An entry that will not go (a file another program holds under Data) does not stop the - others (perf proposal): the set-aside old cache beside it still goes.""" + others (#522): the set-aside old cache beside it still goes.""" failed = None try: entries = list(os.scandir(_win_long_path(run_dir))) @@ -952,8 +952,8 @@ def _close_scratch(run_dir: Path, marker) -> None: class WorkingFolder: """A DataForge run's working folder, handed by extract_dataforge to its - caller to delete later instead of deleted before it returned (perf - proposal). It holds Game2.dcb, the exclude list unforge was handed and + caller to delete later instead of deleted before it returned (#522). + It holds Game2.dcb, the exclude list unforge was handed and the old cache when it was set aside there (_set_old_cache_aside): on a re-extract beside the cache about 31,000 files, nearly all of them the old cache, since unforge writes only what the cache keeps. In %TEMP% the @@ -1264,7 +1264,7 @@ class _SpaceNeed(NamedTuple): def _credit(part: _SpaceNeed, cluster: int) -> int: """Bytes the old cache frees for *part* on a drive with *cluster*-byte clusters. Beside the cache the run now sets the old cache aside rather - than wiping it, and frees it after the run (perf proposal), but the + than wiping it, and frees it after the run (#522), but the credit still holds: on a drive that has room only once the old cache is gone it is wiped in place first, as before (_require_room_for_new_cache returns True), a move takes no new room, and a copy deletes a set-aside @@ -1505,8 +1505,8 @@ def _require_room_for_new_cache(libs_dir: Path, leaf: Path, should_cancel) -> bo Returns True when the run goes on only because the old cache's room was counted (or the drive is short and that room could not be measured): the old cache must then be wiped before the new one is filled, and the - working folder deleted before the run returns, as before the perf - proposal. False when the drive has room for a full copy without it, or + working folder deleted before the run returns, as before #522. + False when the drive has room for a full copy without it, or nothing could be measured.""" # The cache keeps every record unforge wrote less the exclude list # (_cache_filtered_records), so that is what is counted. @@ -1541,7 +1541,7 @@ def _require_room_for_new_cache(libs_dir: Path, leaf: Path, should_cancel) -> bo if free >= need: return False # Short without the old cache's room from here on: the run goes on only - # if the old cache is wiped before the new one is filled (perf proposal). + # if the old cache is wiped before the new one is filled (#522). credit = 0 try: if Path(_win_long_path(leaf)).is_dir(): @@ -1620,7 +1620,7 @@ def _remove_stamps(leaf: Path) -> None: # Path of global.ini inside the p4k archive (unp4k preserves directory structure) _GLOBAL_INI_RELATIVE = Path("data/Localization/english/global.ini") -# The filter unp4k gets for it (perf proposal). unp4k extracts every entry +# The filter unp4k gets for it (#522). unp4k extracts every entry # whose name holds the filter, ignoring case, and names entries with "/" # (Data/Localization/english/global.ini). "global.ini" alone matched every # language's file, 11 of them: 20 to 26 s and 9 GB of memory on a 150 GB @@ -1727,13 +1727,13 @@ def _move_tree(src: Path, dst: Path) -> Optional[OSError]: # The name the old cache takes inside the run's working folder when it is -# set aside rather than wiped (perf proposal). unforge writes only Data\. +# set aside rather than wiped (#522). unforge writes only Data\. _SET_ASIDE_CACHE_NAME = "previous-cache" def _set_old_cache_aside(leaf: Path, run_dir: Path) -> Optional[Path]: """Move the old cache *leaf* into the working folder *run_dir* with one - rename (perf proposal), so it is deleted with that folder after the run + rename (#522), so it is deleted with that folder after the run instead of before the new cache is filled. The working folder is beside the cache, in the same folder, so this is a rename on one drive, as the kept subtrees' moves are (#484), with their retries for a held file. @@ -1866,7 +1866,7 @@ def _cache_filtered_records(src_libs: Path, dst_libs: Path, *, move: bool, drive. Without *move* (the working folder is in %TEMP%) every subtree is copied, as before #484. The free-space checks are sized for a full copy, the most this can need (#482). *before_copy*, when given, is called once - before the first copy (perf proposal: it deletes an old cache that was + before the first copy (#522: it deletes an old cache that was set aside rather than wiped, so a copy gets the room the checks counted). Returns ``(moved, copied, skipped)``, each folder or file moved or @@ -1938,7 +1938,7 @@ def _get_subprocess_kwargs() -> dict: _TOOL_REAP_SECONDS = 30 _UNP4K_GLOBAL_INI_TIMEOUT_SECONDS = 300 _UNP4K_DCB_TIMEOUT_SECONDS = 600 -_UNFORGE_TIMEOUT_SECONDS = 1800 # a healthy run takes ~10 s, ~40 s before the perf proposal (#370) +_UNFORGE_TIMEOUT_SECONDS = 1800 # a healthy run takes ~10 s, ~40 s before #522 and #529 (#370) def _run_tool(args, *, cwd, timeout, should_cancel=None) -> subprocess.CompletedProcess: @@ -2049,7 +2049,7 @@ def extract_global_ini( FileNotFoundError: If unp4k.exe or Data.p4k is missing, or the extracted file is not found after extraction. RuntimeError: If unp4k.exe exits with a non-zero return code, or - says it could not extract the file (perf proposal). + says it could not extract the file (#522). P4kLockedError: Data.p4k is locked, or is being written (another program has it open for writing, or it changed in the last two minutes, #516). The latter is found before anything runs. @@ -2066,7 +2066,7 @@ def extract_global_ini( if not p4k_path.exists(): raise FileNotFoundError(f"Data.p4k not found at: {p4k_path}") _refuse_if_p4k_being_written(p4k_path) - # Read before unp4k opens it (perf proposal): a Data.p4k that changes + # Read before unp4k opens it (#522): a Data.p4k that changes # during the run then reads as changed at the next check, rather than # its new size vouching for a global.ini taken from the old one. p4k_size = p4k_path.stat().st_size @@ -2120,7 +2120,7 @@ def extract_global_ini( shutil.copy2(str(extracted), str(output_path)) logger.info(f"Extracted global.ini → {output_path}") # The old stamp goes once the new base.ini is in place, before the - # new stamp is written (perf proposal). It speaks for the old + # new stamp is written (#522). It speaks for the old # base.ini, and kept past a new stamp that could not be written it # spoke for a new base.ini of the same size too: after a patch that # left global.ini as it was, base.ini read stale straight after every @@ -2132,7 +2132,7 @@ def extract_global_ini( (output_path.parent / BASE_INI_STAMP).unlink(missing_ok=True) except OSError as e: logger.warning(f"Could not remove the old {BASE_INI_STAMP} beside {output_path}: {e}") - # What base_ini_is_fresh keys off (perf proposal): Data.p4k's size, + # What base_ini_is_fresh keys off (#522): Data.p4k's size, # not its modified time, which the launcher moves on a game launch. _write_base_ini_stamp(output_path, p4k_size) finally: @@ -2189,7 +2189,7 @@ def extract_dataforge( check just before the old cache is wiped is the last point a stop leaves that cache untouched; past it the run finishes and stamps the new cache, then reports the stop. - defer_cleanup: Optional callable(WorkingFolder) (perf proposal). A + defer_cleanup: Optional callable(WorkingFolder) (#522). A run that saved and stamped the new cache with no stop requested hands it the working folder (what the cache does not keep, and the old cache when it was set aside there) instead of deleting @@ -2210,7 +2210,7 @@ def extract_dataforge( keeps, or every one of the subtrees it keeps there, all found before the old cache is wiped (#484). RuntimeError: If either subprocess fails: a non-zero exit, or a file - it says it could not extract or convert (perf proposal), found + it says it could not extract or convert (#522), found before the old cache is wiped. P4kLockedError: Data.p4k is locked, or is being written (another program has it open for writing, or it changed in the last two @@ -2263,7 +2263,7 @@ def extract_dataforge( _require_cache_can_be_remade(run_dir, dataforge_cache_dir) # Data.p4k as unp4k is about to read it, for the stamps in step 3 - # (perf proposal). Read after unforge, a game patch that landed + # (#522). Read after unforge, a game patch that landed # during the run made a cache of the old build read as current, as # extract_global_ini's base.ini stamp would have. p4k_stat = p4k_path.stat() @@ -2284,7 +2284,7 @@ def extract_dataforge( should_cancel=should_cancel, ) _check_unp4k(result) - # No pause and no gc.collect() after either tool (perf proposal): + # No pause and no gc.collect() after either tool (#522): # each is a separate process _run_tool has already reaped, and Python # holds none of the files it wrote. A refused move has its own retries # (_MOVE_RETRY_DELAYS), and the wipe has robust_rmtree's backoff. @@ -2358,7 +2358,7 @@ def run_unforge(*extra): f"{_stderr or _stdout or '(no output)'}") # Exit code 0, but a file it could not save fails the run all the # same (_UNFORGE_FILE_FAILED), here, while the old cache is still - # there (perf proposal). + # there (#522). if _UNFORGE_FILE_FAILED in f"{_stdout}\n{_stderr}": raise RuntimeError(f"unforge.exe failed (code {result.returncode}):\n" + "\n".join(filter(None, (_stderr, _stdout)))) @@ -2406,7 +2406,7 @@ def run_unforge(*extra): # The drive may have filled while unforge ran (#482). Refused here, # the old cache is kept. *tight*: it has room only once the old - # cache is gone (perf proposal). + # cache is gone (#522). tight = _require_room_for_new_cache(libs_dir, dataforge_cache_dir, should_cancel) # The last point a stop leaves the previous cache untouched (#471). @@ -2428,7 +2428,7 @@ def run_unforge(*extra): # Data.p4k, and the next free-space check counts it as a complete # old cache. Beside the cache it is then set aside in the working # folder with one rename, and deleted with that folder after the - # run (perf proposal). It is wiped in place, as before, in %TEMP% + # run (#522). It is wiped in place, as before, in %TEMP% # (another drive), on a drive that has room only once it is gone, # or when the rename is refused. The wipe uses a retry loop because # on Windows (particularly under OneDrive) a transient handle from @@ -2444,7 +2444,7 @@ def run_unforge(*extra): def free_the_old_cache(): # A copy needs the room the wipe would have freed (#482), so a - # set-aside old cache goes first (perf proposal). Best effort: a + # set-aside old cache goes first (#522). Best effort: a # copy that then finds the drive full is the "cache incomplete" # message, as it would have been. try: @@ -2498,7 +2498,7 @@ def free_the_old_cache(): raise _cache_incomplete_error(dataforge_cache_dir, e) from e logger.info(f"DataForge cache written to {dataforge_cache_dir}") # The caller deletes the working folder after the run, in the - # background (perf proposal), unless a stop is already pending or + # background (#522), unless a stop is already pending or # the drive had room only once the old cache was gone. if defer_cleanup is not None and not tight and not ( should_cancel is not None and should_cancel()): @@ -2573,7 +2573,7 @@ def dataforge_cache_is_fresh(p4k_path: Path, dataforge_cache_dir: Path) -> bool: def _write_base_ini_stamp(base_ini: Path, p4k_size: int) -> None: """Record that *base_ini* came from a Data.p4k of *p4k_size* bytes - (perf proposal). + (#522). base.ini's own size goes in too, so the stamp speaks only for the base.ini it was written for. The cache folder holds other files that @@ -2594,7 +2594,7 @@ def _write_base_ini_stamp(base_ini: Path, p4k_size: int) -> None: def base_ini_is_fresh(p4k_path: Path, base_ini: Path) -> bool: """Return True if *base_ini*, the global.ini extracted from Data.p4k, is - still current with *p4k_path* (perf proposal). + still current with *p4k_path* (#522). Keyed off Data.p4k's byte size, by #209's rule for the DataForge cache (see dataforge_cache_is_fresh): the RSI launcher's file verification diff --git a/tests/CLAUDE.md b/tests/CLAUDE.md index c7a18134..35f1c854 100644 --- a/tests/CLAUDE.md +++ b/tests/CLAUDE.md @@ -14,16 +14,16 @@ Split by domain: - `test_mission_variant_tuple.py` — 1.4.1 regression: a variant adding an 11th tuple field threw `ValueError: too many values to unpack (expected 10)`. Locks the variant-tuple shape. - `test_mission_desc_annotation_toggle.py` — 1.4.1 Tag-Builder toggle that suppresses `[CLASS-…]` component annotations (and since #492 armor weight tags, see `test_armor_blueprint_bullets.py`) inside mission descriptions while leaving the standalone component INIs annotated. - `test_missile_name_tag.py` — 1.4.1: bomb name tags carry their ordinance letter, not size-only, so the variant column distinguishes `[BOMB-X-…]` from `[BOMB-Y-…]`. -- `test_commodity_journal.py` — 1.4.1 generator guard for the `commodity_journal` unpack crash when the crafting cache directory is missing. The SCO crafting index's material names follow the selected language (`ctx["loc"]`, not `tag_loc`), and its materials come in a fixed order, not the set order that changed with each run's string hashing (perf proposal review): the order they are first met, blueprint by blueprint, each blueprint's materials by name, so the index as a whole is not in name order. The test has eight materials in one blueprint, so a set order passes by chance about once in 40,000 runs. +- `test_commodity_journal.py` — 1.4.1 generator guard for the `commodity_journal` unpack crash when the crafting cache directory is missing. The SCO crafting index's material names follow the selected language (`ctx["loc"]`, not `tag_loc`), and its materials come in a fixed order, not the set order that changed with each run's string hashing (#522 review): the order they are first met, blueprint by blueprint, each blueprint's materials by name, so the index as a whole is not in name order. The test has eight materials in one blueprint, so a set order passes by chance about once in 40,000 runs. - `test_blueprint_pools.py` — multi-source pool merge regression, component-style tag annotation, CIG-prefix strip, pool rank-tier label. - `test_blueprint_filename_fallback.py` — #281 fuel-nozzle alias correction in `_name_from_blueprint_filename`, so all 8 variants resolve to their real names instead of garbled title-cased-slug fallback text in mission blueprint bullets. - `test_blueprint_log_scanner.py` — #222 BP Scan log scanner (#223 moved its button + result UI onto the Blueprint Tracker tab; the scanning internals tested here are unchanged by that move). `parse_events` keys only on the ` Added notification "Received Blueprint: : "` line (embedded quotes survive, UI-echo/continuation lines and malformed timestamps ignored); `scan_files` applies the March-2026 epoch floor + exclusive watermark, de-dups names across files, and advances `latest_timestamp` past the watermark for a monotonic mark; `find_log_files` discovers `logbackups\*.log` + `Game.log` oldest-first with an mtime pre-filter (also honoring the watermark, so an unchanged rotated log isn't re-opened on a later scan); `scan_channel` composes the two for the worker's entry point. Fabricated logs via `tmp_path`, no SC install or Qt. The worker + MainWindow fold-in stay manual-test only. - `test_blueprints_shuttle.py` — Blueprint Tracker's Available/Owned split (`BlueprintTrackerTab._available_blueprints`, a pure staticmethod). - `test_pak_extraction.py` — P4K/DataForge. `TestCacheLayoutNamedOnce` (#471) fails on any quoted `"raw"` or `"foundry"` path segment, or any quoted string starting `.lookups` (a versioned `.lookups-v2` included), in `src/` or `scripts/` outside `pak_extractor`'s layout constants (`DATAFORGE_LOOKUPS_SUBPATH` among them), the generator's missing-`src/` fallback and the legacy build cleaner. The generator's `_cached_lookup` and `EnhancementsGeneratorWorker`'s temp sweep both join `DATAFORGE_LOOKUPS_SUBPATH`. `TestDataForgeKeepList` also fails on entries that clash or nest ignoring case (NTFS ignores it) and on any entry that is not a plain relative forward-slash path (a rename takes its source away). `TestDataForgeExcludeList`: nothing excluded is, holds or lies in a kept subtree (ignoring case), every entry is plain, lower case, named once and not nested in another, and the file unforge reads names each under `libs/foundry/records/`, one a line. `TestCacheFilteredRecords` copies and moves a synthetic tree: counts, sources left or gone, file IDs kept by a move, a file where a subtree goes is never replaced, what the exclude list does not name cached whole (beside a kept subtree, beside a folder above one, at the top), a folder holding an excluded entry read rather than taken whole, and the list matched ignoring case. `TestDataForgeExtraction` drives `extract_dataforge` with a fake `_run_tool` set on `src.utils.pak_extractor`, the module production imports, and the file imports only that copy. A missing `unp4k.exe`, `unforge.exe` or `Data.p4k` fails before any tool runs, and the same run with the file put back goes through the fake, which shows the fake is wired. A failing unp4k, by a non-zero exit (its stderr in the message, and a locked `Data.p4k` still a `P4kLockedError`) or by an exception (raised unchanged), stops the run before unforge, keeps the old cache and leaves no working folder. `TestNoSubprocessRunPatches` fails when a test file that mentions `pak_extractor` has a `patch`, `patch.object`, `mocker.patch`, `monkeypatch.setattr` or `setattr` call whose target, given positionally or as the `target=`, `attribute=` or `name=` keyword, is spelled through the name `subprocess`: the string `subprocess.run`, `utils.pak_extractor.subprocess.run` or `src.utils.pak_extractor.subprocess.run`, or an object such as `subprocess` or `pe.subprocess` with the name `"run"`. Such a patch is never hit, because `_run_tool` uses `Popen`. An aliased import is not seen (`import subprocess as sp`, then `patch.object(sp, "run")`). Synthetic snippets show it sees each of those spellings, the keyword forms included, and passes a `_run_tool` or `Popen` patch, a fake that calls `subprocess.run`, and a string. `TestUnp4kFailureMessage` sets the language on `src.utils.i18n`, the module the extractor translates through, and a test pins that, so its English checks no longer depend on the language an earlier test left. The rest of `extract_dataforge` is in `test_extraction_working_folder.py`. -- `test_extraction_working_folder.py`: #471, Qt-free. Where unp4k and unforge work and how they are stopped. `_run_tool` against real child processes (2 MB on each pipe without a stall, exit codes, a timeout that kills the tool and raises run()'s `TimeoutExpired`, a stop that kills it, a stop before the start that never calls Popen, a stop that wins over a clean exit, a killed tool whose own child still holds the pipes given up on after `_TOOL_REAP_SECONDS`, the Popen kwargs). `_scratch_location` (beside the cache, the 190-character cap exactly, network share, OneDrive, removable drive). `_sweep_stale_scratch` (a dead run goes, a run whose `.in_use` marker is held stays, the grace period for unmarked folders, only its own names, a marker that cannot be opened leaving no folder, a stop asked before each folder leaves the folders after it, perf proposal). The fake unforge's records beside the kept subtrees (`unused/elsewhere` and `_SIBLINGS`) are named in the exclude list by the autouse `_fake_records_are_excluded`, and it writes them whatever it is handed, so the cache must leave them out itself. `extract_dataforge` with a fake `_run_tool` (placement, the held marker, plain paths for the tools, the dcb and the exclude list as unforge's arguments (`--exclude-file`, every entry under its records folder, one a line), both lists read as they are when the run caches (with the keep list patched to two, the other subtrees unforge wrote are cached all the same, apart from reputation's two, which the exclude list is patched to name), a record folder or file a patch adds cached, logged by name and counted by the room check, an unforge that prints its usage for the flag run again with the dcb alone, one that cannot read the list failing the run with no second try, the %TEMP% fallback and its leftovers swept, a folder beside the cache that cannot be made failing the run before any tool runs (DataForge has no fallback there), a failure and a stop before the wipe (also during the check's walk of the old cache) keep the old cache stamped and fresh, an unforge that writes no `libs/` fails with what it printed, or with the antivirus advice when it printed nothing, the old cache stamped and fresh, a stop while the cache is filled (during the renames or a fallback copy) finishes and stamps it, then reports the stop, a manifest an older version left in the old cache goes with it (#495), the stamps record Data.p4k as unp4k read it, so a patch that lands during the run, while unp4k or unforge runs (`_FakeTools`' `during_unp4k` and `during_unforge`), leaves the cache reading as out of date (perf proposal), no tool runs without a working folder, a cleanup failure never replaces the outcome, a deep cache still moves and still copies a keep file past 260 characters), the move (#484: beside the cache every kept subtree is moved (file IDs kept, `copy2` never called), unkept siblings stay behind, a refused subtree (17, 5, 32, or a real held file) is retried when held and then copied while the rest move, a full drive on a rename says the cache is incomplete, `%TEMP%` work is copied, a moved and a copied cache are the same, and unforge output without `foundry/records`, with none of the kept subtrees in it or with `records` a file each keep the old cache, while a wipe cut short and a failed stamp each leave no stamp, and a kill between the two stamp removals leaves no `.p4k_mtime` (it goes first, and alone it would read as fresh); each retry waits its delay, the shipped schedule is real but short, the fill and cleanup times are measured, and a run whose old cache has no stamps logs nothing at WARNING), then end to end with Python scripts standing in for unp4k and unforge (full run (which moves what the child process wrote, file IDs kept), a stop mid-unforge and a timeout, each leaving no live process and no folder), and `extract_global_ini` the same way (beside the DataForge cache, %TEMP% for OneDrive, no cache folder or a refused folder beside the cache, its own leftovers swept and a DataForge run's never touched). Children run on `sys._base_executable`, because a venv's `python.exe` is a launcher whose child a kill would orphan, and publish their pid atomically so the tests wait on the child rather than a fixed delay (no start-up-time flakes on a slow CI runner). Each test gets a private `%TEMP%` and no OneDrive environment (every variable in `src.utils.onedrive._ONEDRIVE_ENV_VARS` removed). #490, making the folder: `TestMakeRunFolder` (the prefix and 8 random hex characters, a refusal failing on the first try, a taken name and a refused name that is there each getting a new name, and a name that stays taken giving up with `FileExistsError` after `_SCRATCH_NAME_ATTEMPTS`), `TestRefusedWorkingFolder` (DataForge refused beside the cache: one `os.mkdir`, the message naming the folder with the cache advice, the Windows error at WARNING and nothing at ERROR, no tool run, the old cache kept; DataForge working in `%TEMP%` gets the advice without the cache folder; a refused marker is a refusal too, and so is a refused parent of the cache folder (rare, since the app makes the cache folder first); global.ini moves to `%TEMP%` on one refusal with only an INFO line, and is refused naming `%TEMP%` only when `%TEMP%` refuses too or there is no cache folder; a drive that is gone and a bad path keep their old handling for both extractions; a write-protected drive, WinError 19, which Python raises as a `PermissionError` although nothing was refused, gets Windows' own reason (`extract.folder_unusable`) for DataForge on the first try with the old cache kept, and moves global.ini to `%TEMP%` first, naming `%TEMP%` only when it fails there too; `windows_refused` counts only ERROR_ACCESS_DENIED, ERROR_NETWORK_ACCESS_DENIED or no Windows code as a refusal, over a table of the errors Python raises as `PermissionError`, and `windows_reason` gives the error's own text) and `TestRealRefusal` (the same against folders whose access control list denies Everyone new subfolders, `icacls /deny *S-1-1-0:(AD)`, removed in a `finally` by `new_folders_denied` from `tests/unusable_folder.py`, which skips only when icacls cannot set the deny; DataForge runs once the deny is gone). #494: a DataForge run working in `%TEMP%` makes one empty `dataforge.extracting-` test folder in the folder that holds the cache folder, and removes it, before any tool runs, because the wipe makes the cache folder again there. In `TestRefusedWorkingFolder` a refusal of it (faked) is the same refusal, naming that folder with the cache advice, at WARNING with nothing at ERROR, no tool run and the old cache stamped and fresh; a folder that takes new folders gets exactly one test folder, gone before the tools run; with no cache folder yet, no folder to hold it and no channel folder above that, the check makes those folders first (all the missing ones, not only the last), so the run goes through and stamps the new cache, and Windows refusing one of them (faked) is the same refusal, naming the folder that holds the cache folder with the cache advice, before any tool runs; beside the cache no test folder is made; a test folder that will not go is logged at WARNING and swept by a later run once it is past the grace period; a drive that is gone or a bad path is raised as it is, before any tool runs; and a short drive is still reported first, with no folder made. `TestRealRefusal` has the refusal for real: the old cache stays stamped and fresh, no test folder is left, and the run goes through once the deny is gone (before the fix the run wiped the old cache and then failed to make it again with a raw `PermissionError`). Every test that refuses a folder goes through `_counting_mkdir`, which counts the run folders `os.mkdir` is asked for and stops a loop after 100 with a non-OSError, because the old `tempfile.mkdtemp` on Python 3.11 (CI) would retry a refusal 2**31 - 1 times instead of failing. The mkdtemp-patching test moved to that seam. #507: `TestToolPhasesShowTheBusyBar` runs both extractions with fake tools and records every progress call, and each fake tool notes the last call made before it started, which is what the bar shows while it runs: a total of 0 (the busy bar) with that phase's label for unp4k and unforge. The DataForge run reports exactly those two, caching 2 of 3 and Done 3 of 3, and global.ini reports exactly the busy bar, 1 of 2 and 2 of 2. The fake tools print the failure constants' own text, so `test_the_bundled_tools_hold_their_failure_text` reads the bundled `unp4k.exe` for `_UNP4K_ENTRY_FAILED` and `unforge.exe` for `_UNFORGE_FILE_FAILED` without running them, as the UTF-16LE text .NET keeps string literals in: a tool rebuilt with either worded otherwise fails it. `test_the_bundled_unforge_takes_the_exclude_list` reads `unforge.exe` the same way for `--exclude-file` and `_UNFORGE_USAGE`. Perf proposal: the global.ini fake, like unp4k, writes each of the game's 11 languages' files whose entry name (`Data/Localization//global.ini`) holds its filter, ignoring case, and `_FakeTools` takes what each tool prints. `TestGlobalIniTakesTheEnglishFileAlone`: the run hands unp4k `Localization/english/global.ini` and only the English file is written, the filter is `_GLOBAL_INI_RELATIVE` without its `data/`, and room beside the cache for one English global.ini (at 128 KiB clusters, where each counted file costs a cluster) keeps the work there while a byte less moves it to `%TEMP%`. `TestToolsThatExitZeroAfterAnError`: an unp4k `Exception while extracting` line on stdout or stderr, exit code 0, fails the DataForge run at ERROR ("exited with code 0 but could not extract a file") before unforge runs, with the old cache stamped and fresh and the working folder gone; one about a locked Data.p4k gets `P4kLockedError` with nothing at ERROR, and the test checks unp4k ran, since #516's up-front refusal raises the same error before any tool (defence in depth: the bundled unp4k opens Data.p4k outside its per-entry catch, so a lock ends it with a non-zero exit); the same line for global.ini keeps the old base.ini although the file is in the working folder; an unforge `Error converting` line on either stream fails the run (`unforge.exe failed (code 0)`, with the line) before anything is moved, the old cache stamped and fresh; unp4k's stdout is logged once at INFO, cut at 2000 characters, for both extractions; and with nothing refused nothing sleeps and only `robust_rmtree` calls `gc.collect()` (each caller read from its frame). `test_the_figures_cover_the_measured_run` pins the global.ini work at no less than the English file (10.1 MB measured, 11,128,009 bytes before) in at least one file. Perf proposal: `TestTheOldCacheIsSetAside` (beside the cache one rename sets the old cache aside in the working folder, file IDs kept, without its stamps, which are gone before the rename, and nothing wipes the cache folder in place; it is wiped in place instead when a stamp will not go, when the rename is refused (17 at once, 5 and 32 after the retries, and a real held file), with the work in `%TEMP%` (no rename at all, the `%TEMP%` folder still handed over), and on a drive with room only once the old cache is gone (by the gate's figures, or short with the old cache unmeasurable), where nothing is handed over either, while exactly a full copy's room sets it aside; a copy after a refused move deletes the set-aside cache once, before the first file is copied, and logs a delete that fails while the copy goes on; a cache folder that is a junction is still refused before anything is renamed or handed over, #491) and `TestTheWorkingFolderGoesAfterTheRun` (with `defer_cleanup` the folder outlives the run with its marker held, so a sweep leaves it, then `delete()` removes it all, once; on its own thread the marker stays held while the rest goes, a sweep meanwhile changes nothing, and the order is `Data`, the set-aside cache, then the folder with its marker; a crash part-way (a `BaseException` on the second file deleted) leaves the marker file, so the next run's sweep takes the folder at once; an entry that will not go (`Data`) is logged once while the set-aside cache is still tried and goes, and the marker file stays for the next sweep; every delete stays inside the working folder while a cache file is held open as the generator holds one; `release()` leaves the folder to the next sweep; a stop pending at the hand-over, a failed unforge, a full drive on a stamp and a stop before the wipe hand nothing over and delete the folder before the run ends, while a stop right after the hand-over leaves the folder to the taker). -- `test_extraction_stop.py`: #471, the Qt side. `AnimatedProgressDialog.set_close_guard` (Esc and a spontaneous close ask once, a programmatic `close()` never does), its `set_progress` with a total of 0 (#507, `TestProgressDialogBusyBar`: after a counted phase, a 0 to 0 range with no percentage and the counted bar's style sheet cleared, since that style stops the animation, and a counted phase after it counted again; nothing counted yet, 0 of N, the same until the first count, also after a counted phase), `DataForgeExtractWorker` and `P4kExtractWorker` on real threads (a stop ends with `finished(False)`, no `error` and nothing at ERROR, also when it lands after the cache was written or as a failure on the way out, while a real failure still reports), and `MainWindow` on stub selves: `_stop_workers_for_close` (cancel, not hide or close, release within the bound, keep a busy thread past it; enhancement generation is stopped the same way, asked with the extractions, its dialog detached and cancelled, waited for under the same one deadline, kept past it with its own warning, and on a real thread the close waits for the job in hand), `closeEvent`'s order (commit, ask the tools to stop, loader wait, window state, wait, accept) and that Cancel stops nothing, the identity-bound slots (an error while closing opens no dialog, an ordinary one still does), the `_close_committed` guards and no second global.ini extraction while one runs, the Stop / Run in Background / Cancel prompt (Cancel is the default and keeps the dialog and the run, Stop dismisses the dialog for good and keeps the Simple page busy until the stop has finished, a Stop after the run already finished still ends a Simple-mode run, and the generation that follows releases the page; Run in Background, #483, dismisses the dialog for good without stopping the run and marks it so the generation that follows runs without a dialog too, after a run that finished meanwhile detaches generation's dialog instead (its other progress connection and the status bar message kept), and in both of those a Simple run still applies at the end; after one that failed meanwhile it just closes), the global.ini dialog's stop (Esc or the X stops a running extraction and cancels the dialog, otherwise it just closes, then end to end on a real dialog and thread for both), its "stopped" status, and a stopped or failed extraction that startup, a channel switch or a data-folder change was waiting on carrying on as if it had been declined (each caller's `if_unfinished`, and the declined paths moved into methods unchanged), after a failure's error box has closed rather than on top of it, `TestDismissedDialogStaysHidden` (a control showing a hidden dialog come back under steady progress, and the detach that keeps it hidden; a dialog under 4 s old coming back after a hide but not a cancel, and an Esc stop on one that stays dismissed while the stop takes longer, both with Qt's show timer started by `setValue` at the bar's minimum, since `set_progress` sets no value for a count of 0 any more), the wiring (the DataForge worker built with the active channel and cache base, which the finished slot's settings check compares, #483), the hand-off to generation swapping in generation's own question (and a generation started on its own getting it too), `TestGenerationStopPrompt` (#483: Esc or X on the generation dialog asks Stop, Run in Background or Cancel, with generation's wording and Cancel the default; a generation already over closes without asking; Cancel keeps the dialog and the run; Stop interrupts it, dismisses the dialog for good and lets a Simple run apply nothing; Run in Background dismisses the dialog and keeps the Simple run; one that finished meanwhile just closes; a DataForge Stop after generation took its dialog over stops that generation; the finished slot says stopped, not failed, for a stopped worker), and the startup refresh prompt for a cache with content but no freshness stamp. `TestRefusedWorkingFolderReachesTheUser` (#490) runs the real `extract_dataforge` and `extract_global_ini` with every working folder refused (`_open_scratch` raising PermissionError) through the worker on a real thread into the real error slot with a fake `QMessageBox`: one box, naming the folder beside the cache for DataForge and `%TEMP%` for global.ini, WARNING records at each step and nothing at ERROR. `TestUnusableCacheFolderAtStart` (#494): `_run_dataforge_extraction` creates the cache folder because it is about to write the cache, so a cache folder it cannot make (Windows refuses it, its drive is gone or not ready, or Windows rejects it for another reason) is reported there, before any worker or progress dialog is built: one box titled `extract.dataforge_extraction_error_title` naming the cache folder the Config tab shows, `extract.cache_folder_refused` with the cache advice for a `PermissionError`, `extract.cache_folder_unreachable` for a `FileNotFoundError` and for a drive that is not ready (WinError 21, which Python raises as a `PermissionError`, faked), and `extract.cache_folder_unusable` with Windows' own reason (its `strerror`, or the whole error when it has none) for any other OSError (a bad name, a file in the way, a full drive and an error with no reason, each faked, with the English text asserted in full), the Windows error once at WARNING and nothing at ERROR, the Simple run ended, Generate lit again, the failed status, and False returned (True once a worker started, False while closing). Each with a faked error and for real (a folder that denies new folders, a drive that is not there), and end to end from Simple mode's Apply, which leaves the page ready with nothing applied. The startup refresh prompt reads the cache through `get_dataforge_cache_leaf`. `TestLinkedCacheFolderAtStart` (#491): a cache folder that is a junction, and one whose target is gone, is refused there too, before the creating getter, any worker or progress dialog, with one `extract.cache_folder_is_link` box, the link once at WARNING and the same Simple run, Generate and status bar handling, while a junction higher up (the default location's channel folder) still starts the run. `TestLinkedCacheFolderReachesTheUser` runs the real `extract_dataforge` with its cache folder a junction through the worker on a real thread into the real error slot: one box with the link text, the `CacheFolderIsLinkError` at WARNING, nothing at ERROR, no tool, no patches and no working folder. `TestDataForgeQuestionAfterASuccess` (#478) runs the real success slot with the real `_maybe_prompt_dataforge_refresh` on a stub window, against a cache folder in `tmp_path` and a fake `QMessageBox`: a success with a follow-up asks about a cache stamped for an older Data.p4k before the reload, and a Yes starts the DataForge extraction with the reload still following, while the Config tab's Extract button (no follow-up), a fresh cache, one never extracted, a DataForge extraction or a generation already running, and a close in progress ask nothing. `TestGenerationSlots` holds enhancement generation's slots to the same rules: bound to their worker (the wiring passes it), a worker that is no longer the current one only reaped, a finish after the close committed applying and reloading nothing (Simple mode too) and only ending the Simple run, and an error while closing or from a released worker logged at WARNING with no dialog, while an ordinary one still logs at ERROR. `test_no_freshness_prompts` also checks that nothing is read for the out-of-date check once closing. Perf proposal: `TestTheWorkerKeepsOnlyASuccessfulRunsFolder` (on a real thread a success keeps the handed-over folder for `take_working_folder()`, once, and a stop after the cache was written, a stop during the patches, a failure after a stop, `ExtractionCancelled` after the hand-over and failing patches each delete it before `finished(False)`, and on each of those five a delete that raises something other than an OSError, a RuntimeError or a MemoryError (whose text is empty), is logged once at WARNING naming its type, with `finished(False)` still emitted once and nothing reaching `sys.excepthook`: the test requests `slot_errors`, because without a hook an exception leaving `run()` ends the whole pytest run with no test named), `TestTheWindowDeletesItInTheBackground` (the finished slot takes it after `wait()` and starts its delete before the queued cleanup and generation; one released by the close or finishing during it is let go with `release()`; end to end on real threads the delete starts once the extraction's thread is done, runs on a thread that is neither the GUI's nor the extraction's, parses no XML in a folder full of it (lxml and ElementTree's parsers patched to record), and the window drops the thread once it is done) and `TestClosingWaitsForTheDelete` (a running delete is waited for, hidden; one busy past the deadline is kept with a WARNING; the deadline is shared with the extraction workers; a DataForge run released by the close has its folder deleted and waited for too; the global.ini worker is never asked for a folder; a sweep of leftovers still waiting on a delete that never ends is stopped at once rather than waited for, and the sweep worker hands its own stop to `_sweep_stale_scratch`). `TestTheDeletesFinishedSlot`: the delete's finished slot calls `quit()` before `wait()`, as every other finished slot does, and a delete that raises on a real thread is logged once at WARNING (never ERROR, pinned with its level) and still sets `done`, so an extraction waiting on it starts, with nothing reaching `sys.excepthook` (`slot_errors`). After a delete thread's MemoryError, with the cyclic GC off and the worker still held, the worker has let go of the `WorkingFolder` by the time `done` is set, so its marker is closed and a sweep straight after takes the folder. `TestANewRunWaitsForTheDelete` (perf proposal, the user's choice): on a real thread a DataForge run handed a delete still going calls `extract_dataforge` (where its sweep and free-space check are) only once that delete is over, and a Stop or a close during the wait ends the run within a second as a stop (`finished(False)`, `stopped`, no `error`, nothing at ERROR, no patches); `_run_dataforge_extraction` hands the run the `done` event of every delete and sweep in `_working_folder_deleters`; and from the window on real threads, with a real working folder whose delete is held up, the window returns at once, the dialog shows its busy bar with the extraction's label while the run waits, and `extract_dataforge` runs on the run's own thread once the folder is gone. `TestLeftoversAreSweptAtStartup` (perf proposal): once a session the startup sweep beside the active cache folder removes a crashed run's working folder, with an old cache set aside in it, and leaves one whose marker is held and the cache itself, on a thread that is not the GUI's, reading the cache folder without creating it; there is none while an extraction runs or the window closes; `_continue_startup_after_update_gate` sweeps before `_start_startup_sync`; and a sweep waits for the deletes already running. +- `test_extraction_working_folder.py`: #471, Qt-free. Where unp4k and unforge work and how they are stopped. `_run_tool` against real child processes (2 MB on each pipe without a stall, exit codes, a timeout that kills the tool and raises run()'s `TimeoutExpired`, a stop that kills it, a stop before the start that never calls Popen, a stop that wins over a clean exit, a killed tool whose own child still holds the pipes given up on after `_TOOL_REAP_SECONDS`, the Popen kwargs). `_scratch_location` (beside the cache, the 190-character cap exactly, network share, OneDrive, removable drive). `_sweep_stale_scratch` (a dead run goes, a run whose `.in_use` marker is held stays, the grace period for unmarked folders, only its own names, a marker that cannot be opened leaving no folder, a stop asked before each folder leaves the folders after it, #522). The fake unforge's records beside the kept subtrees (`unused/elsewhere` and `_SIBLINGS`) are named in the exclude list by the autouse `_fake_records_are_excluded`, and it writes them whatever it is handed, so the cache must leave them out itself. `extract_dataforge` with a fake `_run_tool` (placement, the held marker, plain paths for the tools, the dcb and the exclude list as unforge's arguments (`--exclude-file`, every entry under its records folder, one a line), both lists read as they are when the run caches (with the keep list patched to two, the other subtrees unforge wrote are cached all the same, apart from reputation's two, which the exclude list is patched to name), a record folder or file a patch adds cached, logged by name and counted by the room check, an unforge that prints its usage for the flag run again with the dcb alone, one that cannot read the list failing the run with no second try, the %TEMP% fallback and its leftovers swept, a folder beside the cache that cannot be made failing the run before any tool runs (DataForge has no fallback there), a failure and a stop before the wipe (also during the check's walk of the old cache) keep the old cache stamped and fresh, an unforge that writes no `libs/` fails with what it printed, or with the antivirus advice when it printed nothing, the old cache stamped and fresh, a stop while the cache is filled (during the renames or a fallback copy) finishes and stamps it, then reports the stop, a manifest an older version left in the old cache goes with it (#495), the stamps record Data.p4k as unp4k read it, so a patch that lands during the run, while unp4k or unforge runs (`_FakeTools`' `during_unp4k` and `during_unforge`), leaves the cache reading as out of date (#522), no tool runs without a working folder, a cleanup failure never replaces the outcome, a deep cache still moves and still copies a keep file past 260 characters), the move (#484: beside the cache every kept subtree is moved (file IDs kept, `copy2` never called), unkept siblings stay behind, a refused subtree (17, 5, 32, or a real held file) is retried when held and then copied while the rest move, a full drive on a rename says the cache is incomplete, `%TEMP%` work is copied, a moved and a copied cache are the same, and unforge output without `foundry/records`, with none of the kept subtrees in it or with `records` a file each keep the old cache, while a wipe cut short and a failed stamp each leave no stamp, and a kill between the two stamp removals leaves no `.p4k_mtime` (it goes first, and alone it would read as fresh); each retry waits its delay, the shipped schedule is real but short, the fill and cleanup times are measured, and a run whose old cache has no stamps logs nothing at WARNING), then end to end with Python scripts standing in for unp4k and unforge (full run (which moves what the child process wrote, file IDs kept), a stop mid-unforge and a timeout, each leaving no live process and no folder), and `extract_global_ini` the same way (beside the DataForge cache, %TEMP% for OneDrive, no cache folder or a refused folder beside the cache, its own leftovers swept and a DataForge run's never touched). Children run on `sys._base_executable`, because a venv's `python.exe` is a launcher whose child a kill would orphan, and publish their pid atomically so the tests wait on the child rather than a fixed delay (no start-up-time flakes on a slow CI runner). Each test gets a private `%TEMP%` and no OneDrive environment (every variable in `src.utils.onedrive._ONEDRIVE_ENV_VARS` removed). #490, making the folder: `TestMakeRunFolder` (the prefix and 8 random hex characters, a refusal failing on the first try, a taken name and a refused name that is there each getting a new name, and a name that stays taken giving up with `FileExistsError` after `_SCRATCH_NAME_ATTEMPTS`), `TestRefusedWorkingFolder` (DataForge refused beside the cache: one `os.mkdir`, the message naming the folder with the cache advice, the Windows error at WARNING and nothing at ERROR, no tool run, the old cache kept; DataForge working in `%TEMP%` gets the advice without the cache folder; a refused marker is a refusal too, and so is a refused parent of the cache folder (rare, since the app makes the cache folder first); global.ini moves to `%TEMP%` on one refusal with only an INFO line, and is refused naming `%TEMP%` only when `%TEMP%` refuses too or there is no cache folder; a drive that is gone and a bad path keep their old handling for both extractions; a write-protected drive, WinError 19, which Python raises as a `PermissionError` although nothing was refused, gets Windows' own reason (`extract.folder_unusable`) for DataForge on the first try with the old cache kept, and moves global.ini to `%TEMP%` first, naming `%TEMP%` only when it fails there too; `windows_refused` counts only ERROR_ACCESS_DENIED, ERROR_NETWORK_ACCESS_DENIED or no Windows code as a refusal, over a table of the errors Python raises as `PermissionError`, and `windows_reason` gives the error's own text) and `TestRealRefusal` (the same against folders whose access control list denies Everyone new subfolders, `icacls /deny *S-1-1-0:(AD)`, removed in a `finally` by `new_folders_denied` from `tests/unusable_folder.py`, which skips only when icacls cannot set the deny; DataForge runs once the deny is gone). #494: a DataForge run working in `%TEMP%` makes one empty `dataforge.extracting-` test folder in the folder that holds the cache folder, and removes it, before any tool runs, because the wipe makes the cache folder again there. In `TestRefusedWorkingFolder` a refusal of it (faked) is the same refusal, naming that folder with the cache advice, at WARNING with nothing at ERROR, no tool run and the old cache stamped and fresh; a folder that takes new folders gets exactly one test folder, gone before the tools run; with no cache folder yet, no folder to hold it and no channel folder above that, the check makes those folders first (all the missing ones, not only the last), so the run goes through and stamps the new cache, and Windows refusing one of them (faked) is the same refusal, naming the folder that holds the cache folder with the cache advice, before any tool runs; beside the cache no test folder is made; a test folder that will not go is logged at WARNING and swept by a later run once it is past the grace period; a drive that is gone or a bad path is raised as it is, before any tool runs; and a short drive is still reported first, with no folder made. `TestRealRefusal` has the refusal for real: the old cache stays stamped and fresh, no test folder is left, and the run goes through once the deny is gone (before the fix the run wiped the old cache and then failed to make it again with a raw `PermissionError`). Every test that refuses a folder goes through `_counting_mkdir`, which counts the run folders `os.mkdir` is asked for and stops a loop after 100 with a non-OSError, because the old `tempfile.mkdtemp` on Python 3.11 (CI) would retry a refusal 2**31 - 1 times instead of failing. The mkdtemp-patching test moved to that seam. #507: `TestToolPhasesShowTheBusyBar` runs both extractions with fake tools and records every progress call, and each fake tool notes the last call made before it started, which is what the bar shows while it runs: a total of 0 (the busy bar) with that phase's label for unp4k and unforge. The DataForge run reports exactly those two, caching 2 of 3 and Done 3 of 3, and global.ini reports exactly the busy bar, 1 of 2 and 2 of 2. The fake tools print the failure constants' own text, so `test_the_bundled_tools_hold_their_failure_text` reads the bundled `unp4k.exe` for `_UNP4K_ENTRY_FAILED` and `unforge.exe` for `_UNFORGE_FILE_FAILED` without running them, as the UTF-16LE text .NET keeps string literals in: a tool rebuilt with either worded otherwise fails it. `test_the_bundled_unforge_takes_the_exclude_list` reads `unforge.exe` the same way for `--exclude-file` and `_UNFORGE_USAGE`. #522: the global.ini fake, like unp4k, writes each of the game's 11 languages' files whose entry name (`Data/Localization//global.ini`) holds its filter, ignoring case, and `_FakeTools` takes what each tool prints. `TestGlobalIniTakesTheEnglishFileAlone`: the run hands unp4k `Localization/english/global.ini` and only the English file is written, the filter is `_GLOBAL_INI_RELATIVE` without its `data/`, and room beside the cache for one English global.ini (at 128 KiB clusters, where each counted file costs a cluster) keeps the work there while a byte less moves it to `%TEMP%`. `TestToolsThatExitZeroAfterAnError`: an unp4k `Exception while extracting` line on stdout or stderr, exit code 0, fails the DataForge run at ERROR ("exited with code 0 but could not extract a file") before unforge runs, with the old cache stamped and fresh and the working folder gone; one about a locked Data.p4k gets `P4kLockedError` with nothing at ERROR, and the test checks unp4k ran, since #516's up-front refusal raises the same error before any tool (defence in depth: the bundled unp4k opens Data.p4k outside its per-entry catch, so a lock ends it with a non-zero exit); the same line for global.ini keeps the old base.ini although the file is in the working folder; an unforge `Error converting` line on either stream fails the run (`unforge.exe failed (code 0)`, with the line) before anything is moved, the old cache stamped and fresh; unp4k's stdout is logged once at INFO, cut at 2000 characters, for both extractions; and with nothing refused nothing sleeps and only `robust_rmtree` calls `gc.collect()` (each caller read from its frame). `test_the_figures_cover_the_measured_run` pins the global.ini work at no less than the English file (10.1 MB measured, 11,128,009 bytes before) in at least one file. #522: `TestTheOldCacheIsSetAside` (beside the cache one rename sets the old cache aside in the working folder, file IDs kept, without its stamps, which are gone before the rename, and nothing wipes the cache folder in place; it is wiped in place instead when a stamp will not go, when the rename is refused (17 at once, 5 and 32 after the retries, and a real held file), with the work in `%TEMP%` (no rename at all, the `%TEMP%` folder still handed over), and on a drive with room only once the old cache is gone (by the gate's figures, or short with the old cache unmeasurable), where nothing is handed over either, while exactly a full copy's room sets it aside; a copy after a refused move deletes the set-aside cache once, before the first file is copied, and logs a delete that fails while the copy goes on; a cache folder that is a junction is still refused before anything is renamed or handed over, #491) and `TestTheWorkingFolderGoesAfterTheRun` (with `defer_cleanup` the folder outlives the run with its marker held, so a sweep leaves it, then `delete()` removes it all, once; on its own thread the marker stays held while the rest goes, a sweep meanwhile changes nothing, and the order is `Data`, the set-aside cache, then the folder with its marker; a crash part-way (a `BaseException` on the second file deleted) leaves the marker file, so the next run's sweep takes the folder at once; an entry that will not go (`Data`) is logged once while the set-aside cache is still tried and goes, and the marker file stays for the next sweep; every delete stays inside the working folder while a cache file is held open as the generator holds one; `release()` leaves the folder to the next sweep; a stop pending at the hand-over, a failed unforge, a full drive on a stamp and a stop before the wipe hand nothing over and delete the folder before the run ends, while a stop right after the hand-over leaves the folder to the taker). +- `test_extraction_stop.py`: #471, the Qt side. `AnimatedProgressDialog.set_close_guard` (Esc and a spontaneous close ask once, a programmatic `close()` never does), its `set_progress` with a total of 0 (#507, `TestProgressDialogBusyBar`: after a counted phase, a 0 to 0 range with no percentage and the counted bar's style sheet cleared, since that style stops the animation, and a counted phase after it counted again; nothing counted yet, 0 of N, the same until the first count, also after a counted phase), `DataForgeExtractWorker` and `P4kExtractWorker` on real threads (a stop ends with `finished(False)`, no `error` and nothing at ERROR, also when it lands after the cache was written or as a failure on the way out, while a real failure still reports), and `MainWindow` on stub selves: `_stop_workers_for_close` (cancel, not hide or close, release within the bound, keep a busy thread past it; enhancement generation is stopped the same way, asked with the extractions, its dialog detached and cancelled, waited for under the same one deadline, kept past it with its own warning, and on a real thread the close waits for the job in hand), `closeEvent`'s order (commit, ask the tools to stop, loader wait, window state, wait, accept) and that Cancel stops nothing, the identity-bound slots (an error while closing opens no dialog, an ordinary one still does), the `_close_committed` guards and no second global.ini extraction while one runs, the Stop / Run in Background / Cancel prompt (Cancel is the default and keeps the dialog and the run, Stop dismisses the dialog for good and keeps the Simple page busy until the stop has finished, a Stop after the run already finished still ends a Simple-mode run, and the generation that follows releases the page; Run in Background, #483, dismisses the dialog for good without stopping the run and marks it so the generation that follows runs without a dialog too, after a run that finished meanwhile detaches generation's dialog instead (its other progress connection and the status bar message kept), and in both of those a Simple run still applies at the end; after one that failed meanwhile it just closes), the global.ini dialog's stop (Esc or the X stops a running extraction and cancels the dialog, otherwise it just closes, then end to end on a real dialog and thread for both), its "stopped" status, and a stopped or failed extraction that startup, a channel switch or a data-folder change was waiting on carrying on as if it had been declined (each caller's `if_unfinished`, and the declined paths moved into methods unchanged), after a failure's error box has closed rather than on top of it, `TestDismissedDialogStaysHidden` (a control showing a hidden dialog come back under steady progress, and the detach that keeps it hidden; a dialog under 4 s old coming back after a hide but not a cancel, and an Esc stop on one that stays dismissed while the stop takes longer, both with Qt's show timer started by `setValue` at the bar's minimum, since `set_progress` sets no value for a count of 0 any more), the wiring (the DataForge worker built with the active channel and cache base, which the finished slot's settings check compares, #483), the hand-off to generation swapping in generation's own question (and a generation started on its own getting it too), `TestGenerationStopPrompt` (#483: Esc or X on the generation dialog asks Stop, Run in Background or Cancel, with generation's wording and Cancel the default; a generation already over closes without asking; Cancel keeps the dialog and the run; Stop interrupts it, dismisses the dialog for good and lets a Simple run apply nothing; Run in Background dismisses the dialog and keeps the Simple run; one that finished meanwhile just closes; a DataForge Stop after generation took its dialog over stops that generation; the finished slot says stopped, not failed, for a stopped worker), and the startup refresh prompt for a cache with content but no freshness stamp. `TestRefusedWorkingFolderReachesTheUser` (#490) runs the real `extract_dataforge` and `extract_global_ini` with every working folder refused (`_open_scratch` raising PermissionError) through the worker on a real thread into the real error slot with a fake `QMessageBox`: one box, naming the folder beside the cache for DataForge and `%TEMP%` for global.ini, WARNING records at each step and nothing at ERROR. `TestUnusableCacheFolderAtStart` (#494): `_run_dataforge_extraction` creates the cache folder because it is about to write the cache, so a cache folder it cannot make (Windows refuses it, its drive is gone or not ready, or Windows rejects it for another reason) is reported there, before any worker or progress dialog is built: one box titled `extract.dataforge_extraction_error_title` naming the cache folder the Config tab shows, `extract.cache_folder_refused` with the cache advice for a `PermissionError`, `extract.cache_folder_unreachable` for a `FileNotFoundError` and for a drive that is not ready (WinError 21, which Python raises as a `PermissionError`, faked), and `extract.cache_folder_unusable` with Windows' own reason (its `strerror`, or the whole error when it has none) for any other OSError (a bad name, a file in the way, a full drive and an error with no reason, each faked, with the English text asserted in full), the Windows error once at WARNING and nothing at ERROR, the Simple run ended, Generate lit again, the failed status, and False returned (True once a worker started, False while closing). Each with a faked error and for real (a folder that denies new folders, a drive that is not there), and end to end from Simple mode's Apply, which leaves the page ready with nothing applied. The startup refresh prompt reads the cache through `get_dataforge_cache_leaf`. `TestLinkedCacheFolderAtStart` (#491): a cache folder that is a junction, and one whose target is gone, is refused there too, before the creating getter, any worker or progress dialog, with one `extract.cache_folder_is_link` box, the link once at WARNING and the same Simple run, Generate and status bar handling, while a junction higher up (the default location's channel folder) still starts the run. `TestLinkedCacheFolderReachesTheUser` runs the real `extract_dataforge` with its cache folder a junction through the worker on a real thread into the real error slot: one box with the link text, the `CacheFolderIsLinkError` at WARNING, nothing at ERROR, no tool, no patches and no working folder. `TestDataForgeQuestionAfterASuccess` (#478) runs the real success slot with the real `_maybe_prompt_dataforge_refresh` on a stub window, against a cache folder in `tmp_path` and a fake `QMessageBox`: a success with a follow-up asks about a cache stamped for an older Data.p4k before the reload, and a Yes starts the DataForge extraction with the reload still following, while the Config tab's Extract button (no follow-up), a fresh cache, one never extracted, a DataForge extraction or a generation already running, and a close in progress ask nothing. `TestGenerationSlots` holds enhancement generation's slots to the same rules: bound to their worker (the wiring passes it), a worker that is no longer the current one only reaped, a finish after the close committed applying and reloading nothing (Simple mode too) and only ending the Simple run, and an error while closing or from a released worker logged at WARNING with no dialog, while an ordinary one still logs at ERROR. `test_no_freshness_prompts` also checks that nothing is read for the out-of-date check once closing. #522: `TestTheWorkerKeepsOnlyASuccessfulRunsFolder` (on a real thread a success keeps the handed-over folder for `take_working_folder()`, once, and a stop after the cache was written, a stop during the patches, a failure after a stop, `ExtractionCancelled` after the hand-over and failing patches each delete it before `finished(False)`, and on each of those five a delete that raises something other than an OSError, a RuntimeError or a MemoryError (whose text is empty), is logged once at WARNING naming its type, with `finished(False)` still emitted once and nothing reaching `sys.excepthook`: the test requests `slot_errors`, because without a hook an exception leaving `run()` ends the whole pytest run with no test named), `TestTheWindowDeletesItInTheBackground` (the finished slot takes it after `wait()` and starts its delete before the queued cleanup and generation; one released by the close or finishing during it is let go with `release()`; end to end on real threads the delete starts once the extraction's thread is done, runs on a thread that is neither the GUI's nor the extraction's, parses no XML in a folder full of it (lxml and ElementTree's parsers patched to record), and the window drops the thread once it is done) and `TestClosingWaitsForTheDelete` (a running delete is waited for, hidden; one busy past the deadline is kept with a WARNING; the deadline is shared with the extraction workers; a DataForge run released by the close has its folder deleted and waited for too; the global.ini worker is never asked for a folder; a sweep of leftovers still waiting on a delete that never ends is stopped at once rather than waited for, and the sweep worker hands its own stop to `_sweep_stale_scratch`). `TestTheDeletesFinishedSlot`: the delete's finished slot calls `quit()` before `wait()`, as every other finished slot does, and a delete that raises on a real thread is logged once at WARNING (never ERROR, pinned with its level) and still sets `done`, so an extraction waiting on it starts, with nothing reaching `sys.excepthook` (`slot_errors`). After a delete thread's MemoryError, with the cyclic GC off and the worker still held, the worker has let go of the `WorkingFolder` by the time `done` is set, so its marker is closed and a sweep straight after takes the folder. `TestANewRunWaitsForTheDelete` (#522, the user's choice): on a real thread a DataForge run handed a delete still going calls `extract_dataforge` (where its sweep and free-space check are) only once that delete is over, and a Stop or a close during the wait ends the run within a second as a stop (`finished(False)`, `stopped`, no `error`, nothing at ERROR, no patches); `_run_dataforge_extraction` hands the run the `done` event of every delete and sweep in `_working_folder_deleters`; and from the window on real threads, with a real working folder whose delete is held up, the window returns at once, the dialog shows its busy bar with the extraction's label while the run waits, and `extract_dataforge` runs on the run's own thread once the folder is gone. `TestLeftoversAreSweptAtStartup` (#522): once a session the startup sweep beside the active cache folder removes a crashed run's working folder, with an old cache set aside in it, and leaves one whose marker is held and the cache itself, on a thread that is not the GUI's, reading the cache folder without creating it; there is none while an extraction runs or the window closes; `_continue_startup_after_update_gate` sweeps before `_start_startup_sync`; and a sweep waits for the deletes already running. - `test_p4k_being_written.py`: #516, Qt-free. Both extractions refuse a Data.p4k another program holds open for writing, or one modified in the last 120 seconds, with `P4kLockedError` before any tool or working folder; a reader is not a writer. A modified time a moment ahead of the clock counts (a rounding error, one Windows clock tick, a second, and just under `_P4K_CLOCK_SLACK_SECONDS`, given through `now=`: before Python 3.13 a file written just before the clock was read could come out a rounding error under 0 seconds old, which made the just-written tests fail at random on CI), while one just past the slack, an hour ahead and a missing file do not, and a window of 0 counts nothing, a time ahead of the clock included. The handle tests run on Windows only. The session-wide `no_recent_p4k_refusal` fixture in `conftest.py` sets the window to 0 for every other test; this file puts the real one back. -- `test_base_ini_freshness.py`: perf proposal, base.ini freshness follows Data.p4k's size, not its modified time, by #209's rule for the DataForge cache. `base_ini_is_fresh` on real files with modified times set by hand: a game launch alone (a newer modified time, the same size) keeps base.ini fresh, a patch (a new size) makes it stale whether Data.p4k's modified time is older or newer than base.ini's, a stamp written for another base.ini (base.ini's recorded size differs) or one that is not two numbers (empty, one, three, a word, bytes that are not text) counts for nothing and the modified times decide, both ways: with Data.p4k newer it reads stale and the stamp is left as it was, and with base.ini newer it reads fresh and the stamp is replaced with its own. A missing file reads stale. The upgrade: a base.ini with no stamp that is newer than Data.p4k reads fresh and gets its stamp, so the next launch reads fresh while a patch still reads stale, and one older than Data.p4k still reads stale with no stamp made up for it. `extract_global_ini` with a fake unp4k (a private `%TEMP%`) records Data.p4k's size and base.ini's, so a launch after it needs no other extraction; the size is read before unp4k runs, so a Data.p4k that grows during the run reads stale afterwards; a stamp that can be neither removed nor written (a folder in its place) logs a WARNING for each with nothing at ERROR while the extraction still succeeds; and an earlier stamp for a base.ini of the same size goes once the new base.ini is in place, so after a patch a new stamp that cannot be written (a full drive, faked) leaves none, and base.ini then reads fresh by the modified times and gets its stamp (kept, the old stamp would read it as stale), while a copy of base.ini that fails (a held file, faked) keeps the old stamp, so the old base.ini still reads stale against the patched Data.p4k. The callers on stub selves with `AppSettings` and `QMessageBox` faked: startup, a channel switch and a data folder change ask nothing after a launch and carry on with their reload, and after a patch (with Data.p4k's modified time older than base.ini's, which the old rule missed) ask with `extract.p4k_prompt_p4k_newer` and leave the reload to the extraction; startup stamps a base.ini that has none, so a channel switch after the next launch asks nothing; and the Config tab's Extract button is not red after a launch and red after a patch. +- `test_base_ini_freshness.py`: #522, base.ini freshness follows Data.p4k's size, not its modified time, by #209's rule for the DataForge cache. `base_ini_is_fresh` on real files with modified times set by hand: a game launch alone (a newer modified time, the same size) keeps base.ini fresh, a patch (a new size) makes it stale whether Data.p4k's modified time is older or newer than base.ini's, a stamp written for another base.ini (base.ini's recorded size differs) or one that is not two numbers (empty, one, three, a word, bytes that are not text) counts for nothing and the modified times decide, both ways: with Data.p4k newer it reads stale and the stamp is left as it was, and with base.ini newer it reads fresh and the stamp is replaced with its own. A missing file reads stale. The upgrade: a base.ini with no stamp that is newer than Data.p4k reads fresh and gets its stamp, so the next launch reads fresh while a patch still reads stale, and one older than Data.p4k still reads stale with no stamp made up for it. `extract_global_ini` with a fake unp4k (a private `%TEMP%`) records Data.p4k's size and base.ini's, so a launch after it needs no other extraction; the size is read before unp4k runs, so a Data.p4k that grows during the run reads stale afterwards; a stamp that can be neither removed nor written (a folder in its place) logs a WARNING for each with nothing at ERROR while the extraction still succeeds; and an earlier stamp for a base.ini of the same size goes once the new base.ini is in place, so after a patch a new stamp that cannot be written (a full drive, faked) leaves none, and base.ini then reads fresh by the modified times and gets its stamp (kept, the old stamp would read it as stale), while a copy of base.ini that fails (a held file, faked) keeps the old stamp, so the old base.ini still reads stale against the patched Data.p4k. The callers on stub selves with `AppSettings` and `QMessageBox` faked: startup, a channel switch and a data folder change ask nothing after a launch and carry on with their reload, and after a patch (with Data.p4k's modified time older than base.ini's, which the old rule missed) ask with `extract.p4k_prompt_p4k_newer` and leave the reload to the extraction; startup stamps a base.ini that has none, so a channel switch after the next launch asks nothing; and the Config tab's Extract button is not red after a launch and red after a patch. - `test_folder_refused_strings.py`: #490, the refused working folder message in every language. The `extract.folder_refused*` keys are recorded from real refusals of `extract_dataforge` (beside the cache, and in `%TEMP%` for a OneDrive cache) and `extract_global_ini` (both places refused), which pins which advice each gets (the cache advice beside the cache only), and `extract.folder_unusable` from a write-protected drive beside the cache, which gets Windows' own reason and no advice. Every language has a value for each, keeps its placeholders, formats with the recorded arguments, and puts the folder, and Windows' reason, on a line of its own. English has its text in `ht` and every other language an AI text in `at` with an empty `ht`, as `languages/TRANSLATIONS.md` asks. - `test_run_in_background.py`: #483, Run in Background and the job lock. On a real `MainWindow` (`tests/gui_window.py`, built once for the module): each of the three jobs, set as a worker, locks the Config tab's game, data and cache folders, channel, language, Map Language File, Extract, Import Settings, install scan and Check for Updates with the reason in its label, Apply Category Changes, Save Tag Changes and the Simple page's button with the reason as their tooltips, and all come back when it ends; the locked list itself is pinned; a Simple run alone keeps its own tooltip, and ending one while a job still runs keeps the page busy. An update check or an install scan keeps its own button off through a lock and an unlock, and a check that ends during a job leaves it locked. The game folder is not saved while locked, an install the scan found picked during a job gets the lock reason in its box with the field and the dialog left as they were, and the data folder is not changed while locked through `change_data_dir_to` (the window's data folder slot and the move prompt swapped for recorders, so a change that got through fails instead of reloading the shared window). Apply Category Changes stays locked with a change pending and comes back after, and a locked apply changes nothing; Save Tag Changes the same with tags unsaved, and a locked save saves nothing and asks for no regeneration; Generate stays off through a change made while locked, with the reason or the running job's own message as its tooltip, and comes back after. A lit Save Tag Changes, Generate Enhancements or Config Extract that a job disables is drawn in a dimmer red, and Save Tag Changes and Extract are full red again once unlocked. The Simple page's button, red and green, on the default theme (pinned, since its black label is what the ink reading needs), stands out from the window background less than 0.7 times as much while busy, both its fill (the mean distance: measured 0.44, and 0.92 with the fill undimmed, since the style shades a disabled gradient a little on its own) and its label ink (the largest distance: measured 0.5, and 1.0 with the text undimmed), and as much again after. Every language has the seven `extract.*` texts and `status_bar.enhancement_generation_stopped`, English in `ht` and the rest in `at`, and both questions name the background choice. On the real methods with a `MagicMock` self: the finished slot goes on as before when the channel and cache base held (compared as Windows does, case and spelling aside) and skips the old-cache cleanup and generation, saying why, when either changed; one job at a time (no DataForge extraction beside a generation or a global.ini extraction, no global.ini extraction beside the other two, no generation or pipeline beside an extraction, and a refused pipeline says why; the startup enhancements question waits for the job); the post-import Apply Now is put off while a job runs, Import Settings' flag kept, and an Apply Now answered after a job started under the open question puts it off again with the flag back; the OneDrive warning is put off the same way, before it is shown or after a Move a job started under, without keeping its "don't warn again", and with no job the Move goes ahead; a put-off question is put off once, asked when nothing is going (not during a load, a job or a Simple run, nor while closing), looked at again after `_ASK_AFTER_MODAL_RETRY_MS`, never at once, while a modal box is open and asked once it has gone (a close answered in that prompt asks nothing), queued by a load only when one waits, kept when it puts itself off again during a flush (a Move that starts an extraction, then the post-import question), and on the real methods asked after the job; a language switch during an extraction loads the prose (the cache read through `get_dataforge_cache_leaf`, the getter the method uses since #494); Simple mode asks nothing during any of the three jobs; Import Settings and an update's install wait for the job, and for a background delete of a working folder (`_closing_would_wait`: a delete is not a job, so `_job_running` stays False for it, nothing running waits for nothing, and an AST check pins that both relaunches ask `_closing_would_wait` and not `_job_running`), and an update with none running still downloads; a background generation runs with no dialog, its progress to the status bar only, closes a leftover extraction dialog and is stopped on close without one; and the lock is refreshed as each job starts (its worker set) and ends (cleared), and before a Simple run's pipeline. - `test_atomic_io.py`: `src/utils/atomic_io.py`, Qt-free. A generation's INIs, its stamps, the lookup pickles and the DataForge patcher's XMLs are written to a temp file beside the target and swapped in with `os.replace`, so a run cut off by a close or a crash leaves each file whole, old or new. Pins the text bytes (what `Path.write_text` wrote: UTF-8, no BOM, CRLF on Windows), the temp's place and name (beside the target, `.sc-*.tmp`, 16 characters, never `*.ini`), and how the temp is made. A folder that refuses new files fails on the first `os.open`, naming the target rather than the temp. This is tested with `os.open` faked, and on Windows with a real folder whose ACL denies Everyone (`*S-1-1-0`) adding a file, set and removed with System32's `icacls`. A missing folder fails the same way. A name already taken is retried with a new one, and the other write's temp is left alone. Endless clashes give up after `TEMP_CREATE_ATTEMPTS`, naming the target. A 100-call spin guard turns a runaway loop (`tempfile.mkstemp` on Python 3.11 retries a refusal up to 2**31 times) into a failure instead of a hang. Also that nothing else is left, that the old file stays and the temp goes when the swap fails, the sync fails or the block is interrupted (KeyboardInterrupt too), that an unknown encoding raises LookupError before any temp is made, that a failed `os.fdopen` that closed the descriptor itself raises its own error rather than the second close's EBADF, that only a PermissionError from the swap is retried and only within its budget, with the real sleeps recorded (`time.sleep` patched): 0.01 s doubling to the 0.25 s cap, nine sleeps, 1.0 to 1.6 s in all. A real reader holding the file for 0.3 s does not fail the write and one that never lets go fails it (Windows only). The reader's short retry (`read_bytes_retrying`) is pinned with its three 0.05 s sleeps recorded the same way. `sweep_stale_temps` removes only `.sc-*.tmp` files over 10 minutes old, only in that one folder. A real child on `sys._base_executable` is killed just before its swap: the target keeps its old bytes, the next write succeeds, and the sweep removes the leftover once it is old enough. @@ -45,20 +45,20 @@ Split by domain: - `unusable_folder.py` (a helper module, not a test file, like `junction.py`): a folder Windows refuses or cannot reach, for real. `new_folders_denied(*folders)` denies Everyone the right to add a subfolder (`icacls /deny *S-1-1-0:(AD)`, the SID because the account name is translated on non-English Windows, icacls from System32 with its output read in the OEM code page) for a `with` block and removes the deny in a `finally`. It skips the test only when icacls cannot set the deny, so it runs under CI, and a folder that is not there is an error, not a skip. `missing_drive_folder(name)` names a folder at the top of a drive letter nothing is assigned to (`GetLogicalDrives`, from Z: down), or skips when every letter is taken. #490's `test_extraction_working_folder.py` had the deny to itself; #494's cache folder tests use it too: `test_cache_dir.py`, `test_extraction_stop.py`, `test_enhancements_tab_refused_cache.py`, `test_startup_refused_cache.py`, `test_config_tab_cache_folder.py` and `test_cache_folder_strings.py` both helpers, `test_refused_cache_window_paths.py` and `test_generator_refused_cache.py` the deny only. Import it as `from tests.unusable_folder import new_folders_denied`. - `test_unusable_folder_helper.py`: `new_folders_denied` on real folders under `tmp_path`: a new subfolder is refused (WinError 5) only while the deny is held, also in a folder named with letters outside the OEM code page, files and the folders already there are left alone, several folders are each denied and all released, and the deny goes when the block raises. With icacls faked or made to fail: a deny icacls cannot set skips the test and removes the deny already set on an earlier folder, a deny that cannot be removed fails the test, and a folder that is not there is an error with icacls never run. `missing_drive_folder` names a folder whose drive is not there (WinError 3 on `mkdir`), passes over a letter in use, and skips when none is left. - `test_junction_helper.py`: `make_junction` on real folders under `tmp_path`, each case with and without `CI`: a junction (its reparse tag checked, not a symlink) that leads to its target, made when missing, a real refusal (the link's name already a folder, matched by `WinError 183` so it reads the same in any Windows language) and a link that leads to no folder (`CreateJunction` faked) each skip, or fail under CI. The outcomes are read through a helper that catches the skip too, as in `test_inno_setup_helper.py`. -- `event_loop.py` (a helper module, not a test file, like `junction.py`): `pump_until(predicate, seconds=10.0)` runs the session app's event loop (`QCoreApplication.processEvents`, then a 10 ms sleep) until the predicate holds and returns whether it did, asked once more at the deadline. A test on real threads needs it to have a queued finished slot or signal delivered on the GUI thread. `test_applied_state_worker.py`, `test_extraction_stop.py`, `test_live_blueprint_log_worker.py` and `test_pending_cache_cleanup.py` each had a copy of this loop (perf proposal review: the third copy is the signal to extract) and use it now. Import it as `from tests.event_loop import pump_until`, from a test that requests `qapp`. +- `event_loop.py` (a helper module, not a test file, like `junction.py`): `pump_until(predicate, seconds=10.0)` runs the session app's event loop (`QCoreApplication.processEvents`, then a 10 ms sleep) until the predicate holds and returns whether it did, asked once more at the deadline. A test on real threads needs it to have a queued finished slot or signal delivered on the GUI thread. `test_applied_state_worker.py`, `test_extraction_stop.py`, `test_live_blueprint_log_worker.py` and `test_pending_cache_cleanup.py` each had a copy of this loop (#522 review: the third copy is the signal to extract) and use it now. Import it as `from tests.event_loop import pump_until`, from a test that requests `qapp`. - `test_event_loop_helper.py`: `pump_until` with the session app: a signal from a real `QThread` reaches the GUI thread only once the loop runs, it returns as soon as the condition holds (the third ask) and gives up at the deadline, and with a fake clock it asks once more at the deadline, where a condition true only from then on still counts. - `test_channel_layout.py` — per-channel directory migration. - `test_cache_dir.py` — 1.4.1 split of the DataForge cache override from the user-data override: `CACHE_DIR` is its own registry key, defaults to `%LOCALAPPDATA%\Smart Citizen\\cache\dataforge\` when unset, and never falls back to the user-data override. Migrator coverage locks deferred cleanup (no startup copy/delete) for overrides, same-path no-ops, unavailable destinations, and staged no-override moves that preserve the source after interrupted cross-volume copies. #494: `TestCachePathDoesNotCreate` locks `get_dataforge_cache_leaf` as the same folder `get_dataforge_cache_dir` returns (registry default, override, portable) without creating it, and, for real, a cache folder that denies new folders or sits on a drive that is not there: the leaf is named, `get_dataforge_build_key` reads `unknown`, and only `get_dataforge_cache_dir` raises (`PermissionError`, `FileNotFoundError`), while a cache already there under folders that all deny new ones works for both. `get_default_dataforge_cache_base` (the folder the Config tab names when a reset to the default fails) is the default whatever the override, in both build modes, and makes nothing. `TestOnlyWritersCreateTheCache` is the drift guard: an AST scan of `src/` and `scripts/` (`_SCANNED_FOLDERS`) counts every use of `get_dataforge_cache_dir` (a call, a bare reference, its name as a string) by the function it sits in and fails unless they are exactly `_CACHE_WRITERS`, once each: the migrator, `_run_dataforge_extraction`, `_run_p4k_extraction` and `ConfigTab._maybe_apply_cache_change` (for the new folder). A synthetic source shows each kind of use is counted, and the six modules that name the cache folder today are checked to be among the files scanned, so a scan narrowed to `src/` fails instead of letting the generator script (the one outside it) make the cache folder again unseen. -- `test_pending_cache_cleanup.py`: #486. The old cache folder the Config tab queues with "Re-extract and delete old" is deleted after the next successful extraction (`MainWindow._cleanup_pending_old_cache`), but never when it is a cache in use. A queued folder that is a cache folder under the current cache location, sits inside one or holds one is kept, for any channel's folder, not only the active one's (`AppSettings.get_dataforge_cache_leaf`). Two folders overlap when their normalized text does (`_normalized_folder`) or their resolved paths do (`_resolved_folder`, a non-strict `os.path.realpath`). Each spelling (as stored, in another case, with a trailing separator, with forward slashes, with the long-path prefix) is checked twice: end to end through the cleanup, and by `_normalized_folder` and `_cache_folder_overlapping` on the raw text, because the cleanup wraps the text in `Path` first and the resolved half reads every spelling as the cache folder too, so only the `_normalized_folder` comparison fails when the text half breaks (`_cache_folder_overlapping` still matches through the resolved half). The resolved half is checked by itself with a fake `realpath` (an alias of the cache folder overlaps it, and so does an alias of a folder inside it or holding it, while a sibling reached through the alias or another folder does not, and nothing is created), and end to end with a junction (`_winapi.CreateJunction`) and an 8.3 short name (`GetShortPathNameW`), each in the queued text and in `%LOCALAPPDATA%`, the default location the settings never resolve, while an override is always resolved. A folder `realpath` cannot resolve is kept and the skip is logged at WARNING. The queue is still cleared and an in-use skip is logged at INFO. A really old folder elsewhere is still deleted, and so is a sibling whose name only starts like the cache folder's, or that the cache folder's name starts like. An absent queued folder only clears the queue. The check creates nothing on disk (`get_dataforge_cache_dir` would make the active channel's folder). The issue's scenario runs end to end on stub selves with a fake prompt: the real `ConfigTab._maybe_apply_cache_change` queues A, switches to B, then back to A with no prompt, and the real `_on_dataforge_extract_finished` keeps A and clears the queue, also when the channel changes before that successful extraction. Windows only, skipped elsewhere: the other-case, forward-slash and long-path-prefix spellings, the junction cases (made with `tests/junction.py`, so they fail rather than skip under CI when no junction can be made) and the 8.3 cases (which also skip where the volume makes no short names). Settings live in a `JsonSettings` file under `tmp_path` (no registry), and no widget is built, so it needs no `qapp` except for the tests on real threads, which pump queued finished slots with `tests/event_loop.py`'s `pump_until`. `TestBesideTheBackgroundDelete` (perf proposal): the real finished slot starts the real background delete of a working folder (with a set-aside old cache in it) and then runs the real cleanup, on a real thread with its queued finished slot pumped, and each keeps to its own folder: a queued old cache elsewhere is deleted, a queued folder holding both the cache in use and the working folder is kept (#486), and the working folder is gone either way with the cache in use intact. `TestLeftoverWorkingFoldersBesideIt` (perf proposal): a queued folder that is deleted or already gone has the folder beside it swept in the background, while a cache in use or a folder the check cannot resolve has none, and end to end on real threads, after a cache folder change, a crashed run's working folder beside the old cache folder is swept on a thread that is not the GUI's, once this run's own delete is over, while one whose marker is held stays and the cache in use is intact. +- `test_pending_cache_cleanup.py`: #486. The old cache folder the Config tab queues with "Re-extract and delete old" is deleted after the next successful extraction (`MainWindow._cleanup_pending_old_cache`), but never when it is a cache in use. A queued folder that is a cache folder under the current cache location, sits inside one or holds one is kept, for any channel's folder, not only the active one's (`AppSettings.get_dataforge_cache_leaf`). Two folders overlap when their normalized text does (`_normalized_folder`) or their resolved paths do (`_resolved_folder`, a non-strict `os.path.realpath`). Each spelling (as stored, in another case, with a trailing separator, with forward slashes, with the long-path prefix) is checked twice: end to end through the cleanup, and by `_normalized_folder` and `_cache_folder_overlapping` on the raw text, because the cleanup wraps the text in `Path` first and the resolved half reads every spelling as the cache folder too, so only the `_normalized_folder` comparison fails when the text half breaks (`_cache_folder_overlapping` still matches through the resolved half). The resolved half is checked by itself with a fake `realpath` (an alias of the cache folder overlaps it, and so does an alias of a folder inside it or holding it, while a sibling reached through the alias or another folder does not, and nothing is created), and end to end with a junction (`_winapi.CreateJunction`) and an 8.3 short name (`GetShortPathNameW`), each in the queued text and in `%LOCALAPPDATA%`, the default location the settings never resolve, while an override is always resolved. A folder `realpath` cannot resolve is kept and the skip is logged at WARNING. The queue is still cleared and an in-use skip is logged at INFO. A really old folder elsewhere is still deleted, and so is a sibling whose name only starts like the cache folder's, or that the cache folder's name starts like. An absent queued folder only clears the queue. The check creates nothing on disk (`get_dataforge_cache_dir` would make the active channel's folder). The issue's scenario runs end to end on stub selves with a fake prompt: the real `ConfigTab._maybe_apply_cache_change` queues A, switches to B, then back to A with no prompt, and the real `_on_dataforge_extract_finished` keeps A and clears the queue, also when the channel changes before that successful extraction. Windows only, skipped elsewhere: the other-case, forward-slash and long-path-prefix spellings, the junction cases (made with `tests/junction.py`, so they fail rather than skip under CI when no junction can be made) and the 8.3 cases (which also skip where the volume makes no short names). Settings live in a `JsonSettings` file under `tmp_path` (no registry), and no widget is built, so it needs no `qapp` except for the tests on real threads, which pump queued finished slots with `tests/event_loop.py`'s `pump_until`. `TestBesideTheBackgroundDelete` (#522): the real finished slot starts the real background delete of a working folder (with a set-aside old cache in it) and then runs the real cleanup, on a real thread with its queued finished slot pumped, and each keeps to its own folder: a queued old cache elsewhere is deleted, a queued folder holding both the cache in use and the working folder is kept (#486), and the working folder is gone either way with the cache in use intact. `TestLeftoverWorkingFoldersBesideIt` (#522): a queued folder that is deleted or already gone has the folder beside it swept in the background, while a cache in use or a folder the check cannot resolve has none, and end to end on real threads, after a cache folder change, a crashed run's working folder beside the old cache folder is swept on a thread that is not the GUI's, once this run's own delete is over, while one whose marker is held stays and the cache in use is intact. - `test_enhancements_tab_refused_cache.py`: #494, the Enhancements tab offscreen with a DataForge cache folder that cannot be made, for real (`tests/unusable_folder.py`): an override that denies new folders, the default `%LOCALAPPDATA%\Smart Citizen` denied the same way, and an override on a drive that is not there. The tab builds (it is built with the main window, so this is what stopped Smart Citizen opening), reads not extracted with Generate lit, and makes nothing. A German `retranslate_ui` translates the whole tab (the Tag Builder title comes after the forge status), an Apply click asks for one reload with nothing reaching `sys.excepthook` (recorded by conftest's `slot_errors`), and a stamped cache under folders that all deny new ones reads up to date. Settings in `tmp_profile`. - `test_refused_cache_window_paths.py`: #494, the main window partway through a session, on a stub self with the real Enhancements tab: built on LIVE, whose cache is there, then PTU active with the cache folder above both denying new folders. A channel switch reaches `_check_p4k_freshness` and the reload and resets the prompted flag, the reload after it (`_reload_after_channel_switch`, with PTU's Data.p4k and both tools there) reads PTU's cache folder as never extracted in `_maybe_prompt_dataforge_refresh`, so there is no prompt and no extraction, and the reload follows, a language switch (with the window's cascade to the tab) reaches `_apply_language_base_source`, a data folder change reaches the freshness check and the reload, a language reload with and without a stamp loads the prose, the first load's category dialog runs once and its Generate refreshes the tab once, then starts the pipeline and sets `_initial_load_done` (so a change that starts the pipeline without the dialog, or drops the refresh, fails), the Enhancements Apply asks for one reload, and `_on_dataforge_extract_finished` (success or failure) and a failed generation's finished slot end without raising. PTU's cache folder is never made. - `test_startup_refused_cache.py`: #494, the real `src.main.main()` in a child process (`sys.executable -c`, importing `_start_smart_citizen` from the test file), as a launch does: a cache folder override that denies new folders, one on a drive that is not there, a saved PTU channel whose cache folder cannot be made, and the portable build's default `data\cache` denying new folders. Each must show the window (the child prints the cache status it shows, not extracted), exit 0 and write no `crash_*.log`; each exited 1 with a crash dump before the fix. The child keeps to `tmp_path`: settings in a `JsonSettings` file (the parent writes them through `AppSettings`), `%APPDATA%`, `%LOCALAPPDATA%` and `%USERPROFILE%` pointed there, the registry rename, the old installer's key and the Documents lookup replaced (any other `winreg` or `QSettings` use in settings fails the case), install detection stubbed as conftest's `no_real_install_detection_inputs` does, and sockets refused. `QApplication.exec` returns at once, so no startup task runs, the crash dialog prints instead of opening, and the child leaves with `os._exit` after `main()`'s `sys.exit`. - `test_config_tab_cache_folder.py`: #494, the real Config tab offscreen (`TestCacheFolderChangeIsAtomic`), settings in `tmp_profile`, a fake `QMessageBox` whose migration prompt clicks a chosen button, and real refusals from `tests/unusable_folder.py`. A folder that denies new folders, typed or picked with Browse (the picker faked, answering with forward slashes), and a new folder inside one, are not saved even with a populated old cache that would prompt: the previous folder stays in the settings file and the field, one box (`config.invalid_cache_folder_title`, `config.cache_folder_refused` with #490's `extract.folder_refused_fix`, English asserted in full) names the folder, the Windows error is logged once at WARNING naming it and nothing at ERROR, and there is no prompt, no `cache_dir_changed`, no cleanup queued and nothing made in the folder. A folder on a drive that is not there gets `config.cache_folder_unreachable`, and so does a `FileNotFoundError` making the new cache folder (faked), which is undone too, and a drive that is not ready (faked WinError 21, which Python raises as a `PermissionError`). A typed name Windows does not take (`SC|Cache`, WinError 123) and a path that runs through a file (WinError 183), both for real, get `config.cache_folder_unusable` with Windows' own reason, read from the same failing `mkdir` in the test so it holds in any Windows language, with the English text asserted in full. So does a typed folder whose cache folder cannot be made once it is saved, because a file named LIVE sits where the channel's folder goes (WinError 183, for real), which is undone, and Reset and a cleared field with the default folder in that state, which keep the saved folder and send nothing to `sys.excepthook`. A refused folder typed as `%LOCALAPPDATA%\refused` is named in the box and at the start of the WARNING line as the folder it stands for, not as typed (it fails when `_save_cache_dir` names the typed text), and a typed junction to a refused folder (made with `tests/junction.py`) as the folder it leads to, not the junction (it fails when the typed path is only expanded, not resolved). Reset and a cleared field with a default folder that denies new folders keep the saved folder, name the default (`%LOCALAPPDATA%\Smart Citizen`, or `data\cache` beside the portable exe) and send nothing to `sys.excepthook` (conftest's `slot_errors`). A saved folder that denies new folders or sits on a missing drive can be left with Reset or a good typed folder: no box, no prompt, saved, and one `cache_dir_changed` with the new cache folder made. Cancel still undoes a change, Re-extract and delete old still saves it, queues the old cache and signals once, the same folder again changes nothing, and a file gets `config.cache_folder_is_file`. The saved folder typed again while it denies new folders and holds no cache folder yet is not a quiet no-op, because the same-folder return comes after the cache folder is made: one refused box names it, with no prompt and no signal, and the saved folder stays. The field saves only a folder the user typed (its modified flag), so the typing helpers here and in `test_cache_folder_strings.py` set that flag, as typing does. `TestOnlyATypedFolderIsSaved` shows the tab as the active window and clicks and types with QTest, so the focus moves as a user's does. With a saved folder that denies new folders or sits on a missing drive, a click into the field and out again (to the data folder field, or to Browse with the picker cancelled) and Enter with nothing typed open no box and log nothing at WARNING, though Qt sends `editingFinished` on each (asserted). A folder typed key by key (another one on a missing drive, or the saved folder again) and Enter, then a click away, open exactly one box: the click away used to try the saved folder again and open a second. Every case failed before the fix except Cancel, Re-extract and delete old and the same folder again with nothing refused, which lock behaviour the fix keeps. - `test_cache_folder_strings.py`: #494, the messages for a cache folder that cannot be made in every language, mirroring `test_folder_refused_strings.py`. `settings.cache_folder_failure` is checked against Windows' codes built the way Python builds them: 2, 3, 15, 53 and 67 (a `FileNotFoundError`) and 21 (a drive that is not ready, a `PermissionError`) could not be reached, 5 and 65 were refused (as #490's `pak_extractor.windows_refused` reads them), and 19, 23 and 32 (the rest of the range Python raises as `PermissionError`: a write-protected or failing drive, a file another program holds) and 112, 123, 161, 183 and 206 could not be used (161, a path Windows cannot parse, and 206, a path too long for it, are a `FileNotFoundError` too); `pak_extractor.windows_reason` gives `strerror`, or the whole error when there is none. Each of those 16 codes is also raised by the creating getter for both places that report it, the Config tab given a typed folder and `MainWindow._run_dataforge_extraction`, which must show `config.cache_folder_` and `extract.cache_folder_` for the kind `cache_folder_failure` gives, so neither can read one of those codes its own way (a code outside the table is not checked). Every `tr()` call under `config.cache_folder_`, `extract.cache_folder_` and `extract.folder_refused` is recorded (by prefix, so a key the code adds there fails the exact expectations too), from real cases: the Config tab given a folder that denies new folders (`config.cache_folder_refused` then `extract.folder_refused_fix`), one on a missing drive (`config.cache_folder_unreachable`), a name Windows does not take (`config.cache_folder_unusable` with Windows' reason) and a file (`config.cache_folder_is_file`), and `MainWindow._run_dataforge_extraction` on a stub window with its cache folder denying new folders (`extract.cache_folder_refused` then `extract.folder_refused_fix_cache`), on a missing drive (`extract.cache_folder_unreachable`) and with a file where it should be (`extract.cache_folder_unusable` with Windows' reason), which pins which advice each gets. Windows' reason for each is read from the same failing `mkdir` in the test, so it holds in any Windows language. Every language has a value for each, keeps its placeholders, formats with the recorded arguments and puts the folder, and the reason where there is one, each on a line of its own. English has its text in `ht` and every other language an AI text in `at` with an empty `ht`. - `test_generator_refused_cache.py`: #494, the enhancement generator names the DataForge cache folder through `get_dataforge_cache_leaf` and makes nothing. `EnhancementsGeneratorWorker` runs on its own thread (`DirectConnection` slots), built with the settings the app hands it, once with a cache folder override that denies new folders (`tests/unusable_folder.py`) and once with one not made yet. The generator script it loads is swapped for a stand-in (`importlib.util.spec_from_file_location` answers with it for the worker's module name, and the module is taken out of `sys.modules` afterwards) whose `main()` records the folder it is handed. The real patches step and the stand-in each get the cache folder, the run finishes with no `error` and nothing at ERROR, and nothing is made in the folder above it (the creating getter raised for the refused folder and made the other one). Every launcher checks the cache is there first, so a real run never starts without one: the test takes it away to show the run only names it. The generator script itself, loaded inside the refusal under a name of its own, names the refused folder in `DEFAULT_FORGE_DIR` and `_get_default_forge_dir()` and makes nothing (the creating getter raised, and the script fell back to `%LOCALAPPDATA%`, not the folder the settings name). Settings in `tmp_profile`. -- `test_robust_rmtree.py`: the perf proposal's faster `robust_rmtree`. The first two threads that delete a file each wait at a barrier until the other is deleting too, so a one-thread delete fails it (after the barrier's 10 s). Read-only files and a file past 260 characters are gone before `shutil.rmtree` runs (it is wrapped to list what it finds, and finds only folders), and the threaded pass is handed the long-path form (#221), checked directly because on a PC with long paths turned on a plain path reaches the deep file as well. With no new thread to be had (as while Python shuts down, from 3.12) rmtree deletes every file and nothing is raised, on a tree with more batches than the bounded queue holds, run on a thread of its own so a walker left waiting on a full queue fails the test after 20 s instead of hanging the run. The threads are daemon threads: with both threads holding a batch and the queue full, once Python begins to exit (`_EXITING`, faked) each finishes the batch in hand and drops every queued one, the walk hands out no more, none is left running when the pass returns, and rmtree deletes what is left (the batches put are counted through a recording queue). A child shows atexit sets `_EXITING`. A child on `sys._base_executable` that deletes a working folder on a daemon thread, its first file delete blocked for a minute, exits within 10 s of main returning (a `ThreadPoolExecutor`'s threads, joined at exit after every queued batch, held it for the minute), with the marker file left, so the next sweep takes the folder at once. A file held open without delete sharing fails the first attempt and goes in the second, once its holder lets go during the one 0.2 s wait. One that stays held raises Windows' own `PermissionError` (WinError 32, naming that file) with every other file gone, after one, two (the leftover sweep's) or six attempts, with a wait between two attempts and none after the last, and a WARNING per attempt, the last ending "giving up". #491's link cases run the same code in `test_linked_cache_folder.py`. A delete thread's exception that is not an OSError (a MemoryError) leaves no reference cycle behind: with the cyclic GC off, a working folder's marker is closed as soon as the exception and the `WorkingFolder` holding the marker are dropped, and the next sweep takes the folder at once. Waits are recorded, not slept. -- `test_linked_cache_folder.py`: #491, a DataForge cache folder that is a link is never deleted through. `linked_folder_target` names the folder a junction leads to (also handed the long-path prefix, and given back without it), a symbolic link (for real where the account may make one, and from a faked `lstat` mode everywhere), a junction whose target is gone, a junction to a junction (the last folder) and a target realpath cannot read (faked WinError 1231: the link's own text, then "" when `readlink` fails too), and nothing for a folder, a file, a missing path or a folder reached through a link. `robust_rmtree` handed a junction, a symbolic link or a junction to nothing, plain or with the long-path prefix, raises `LinkedFolderError` (an `OSError`) naming both folders, at once with no retry (no sleep), with nothing deleted and the link kept; a folder that only holds a junction or a directory symbolic link is still deleted with its target kept, and `_delete_files_first` handed a listing with an entry that reads as Windows lists a directory symbolic link (a link by its own lstat, the folder it leads to when followed) leaves that folder's files alone, which needs no privilege (perf proposal). `extract_dataforge` with its cache folder each of those links is refused before anything runs: `CacheFolderIsLinkError` (anticipated) with the English text asserted in full, the folder and target kept on it, one WARNING and nothing else at WARNING or above, the old cache unchanged, the link kept, and no working folder beside the cache or in a private `%TEMP%`; a junction higher up the path is not refused and the run reaches its first tool. Clear Cache (a stub self, a fake `QMessageBox` answering Yes) with the DataForge cache folder in use a junction lists it under Could not delete with the link named, deleting only the INIs, and the queued old-cache cleanup with a queued junction to an unrelated folder logs one WARNING naming the link instead of "Removed old DataForge cache", clears the queue and deletes nothing. `extract.cache_folder_is_link` is recorded from a real refusal and checked in every language: a value, both placeholders, the folder and where it leads each on a line of their own, and English in `ht`, the rest in `at`. Junctions from `tests/junction.py`; symbolic links skip where Windows refuses them (WinError 1314 without Developer Mode) and fail under CI, as junctions do. Settings in `tmp_profile` where the real paths are read. -- `test_clear_cache.py`: #479, More > Clear Cache clears the DataForge cache in use. The real `MainWindow.clear_cache` on a stub window with real settings (`tmp_profile`), real files and a fake `QMessageBox` that answers each question in turn and fails on one it has no answer for. The cache folder `get_dataforge_cache_leaf()` names is offered and cleared in the registry default, with a cache folder override (the default's cache left alone) and in the portable build (`_portable_data_dir` pinned to `tmp_path`), with the question naming the folder, the summary counting the files and the cache, and the Config tab and the Enhancements tab refreshed; it used to look in the data folder's `cache\dataforge` and never asked. Records without the stamp are offered, a cache folder with no cache in it is not, and the old leftover in the data folder is left to the queued cleanup. A DataForge cache with no files skips the file question, and a No to the DataForge question then changes nothing (no summary, the table and the tabs left alone). With files too, a No to the DataForge question clears the files and keeps the cache, a No to the file question clears nothing, the file question lists the files, and nothing to clear gives the cache-empty box. A cache folder override on a drive that is not there (`tests/unusable_folder.py`) clears the rest and names the folder in the summary, or in the cache-empty box, with one WARNING. While an extraction or a generation runs (each of the three workers) nothing is cleared or asked and the status bar says why, but a running background delete of a working folder (perf proposal) holds nothing up: the cache in use is asked about and cleared, and the working folder, with the old cache set aside in it, is left to its delete. A cached file or a DataForge cache that cannot be deleted (faked) is listed in the summary and logged once at WARNING with nothing at ERROR (a file used to be logged at ERROR, the cache twice), and the other file is still deleted. Every language has the ten texts with their fields, the folder on a line of its own, English in `ht` and the rest in `at`, and an AST scan of `clear_cache` finds no string with a letter in it outside the log lines, the `tr()` keys and the file patterns. +- `test_robust_rmtree.py`: #522's faster `robust_rmtree`. The first two threads that delete a file each wait at a barrier until the other is deleting too, so a one-thread delete fails it (after the barrier's 10 s). Read-only files and a file past 260 characters are gone before `shutil.rmtree` runs (it is wrapped to list what it finds, and finds only folders), and the threaded pass is handed the long-path form (#221), checked directly because on a PC with long paths turned on a plain path reaches the deep file as well. With no new thread to be had (as while Python shuts down, from 3.12) rmtree deletes every file and nothing is raised, on a tree with more batches than the bounded queue holds, run on a thread of its own so a walker left waiting on a full queue fails the test after 20 s instead of hanging the run. The threads are daemon threads: with both threads holding a batch and the queue full, once Python begins to exit (`_EXITING`, faked) each finishes the batch in hand and drops every queued one, the walk hands out no more, none is left running when the pass returns, and rmtree deletes what is left (the batches put are counted through a recording queue). A child shows atexit sets `_EXITING`. A child on `sys._base_executable` that deletes a working folder on a daemon thread, its first file delete blocked for a minute, exits within 10 s of main returning (a `ThreadPoolExecutor`'s threads, joined at exit after every queued batch, held it for the minute), with the marker file left, so the next sweep takes the folder at once. A file held open without delete sharing fails the first attempt and goes in the second, once its holder lets go during the one 0.2 s wait. One that stays held raises Windows' own `PermissionError` (WinError 32, naming that file) with every other file gone, after one, two (the leftover sweep's) or six attempts, with a wait between two attempts and none after the last, and a WARNING per attempt, the last ending "giving up". #491's link cases run the same code in `test_linked_cache_folder.py`. A delete thread's exception that is not an OSError (a MemoryError) leaves no reference cycle behind: with the cyclic GC off, a working folder's marker is closed as soon as the exception and the `WorkingFolder` holding the marker are dropped, and the next sweep takes the folder at once. Waits are recorded, not slept. +- `test_linked_cache_folder.py`: #491, a DataForge cache folder that is a link is never deleted through. `linked_folder_target` names the folder a junction leads to (also handed the long-path prefix, and given back without it), a symbolic link (for real where the account may make one, and from a faked `lstat` mode everywhere), a junction whose target is gone, a junction to a junction (the last folder) and a target realpath cannot read (faked WinError 1231: the link's own text, then "" when `readlink` fails too), and nothing for a folder, a file, a missing path or a folder reached through a link. `robust_rmtree` handed a junction, a symbolic link or a junction to nothing, plain or with the long-path prefix, raises `LinkedFolderError` (an `OSError`) naming both folders, at once with no retry (no sleep), with nothing deleted and the link kept; a folder that only holds a junction or a directory symbolic link is still deleted with its target kept, and `_delete_files_first` handed a listing with an entry that reads as Windows lists a directory symbolic link (a link by its own lstat, the folder it leads to when followed) leaves that folder's files alone, which needs no privilege (#522). `extract_dataforge` with its cache folder each of those links is refused before anything runs: `CacheFolderIsLinkError` (anticipated) with the English text asserted in full, the folder and target kept on it, one WARNING and nothing else at WARNING or above, the old cache unchanged, the link kept, and no working folder beside the cache or in a private `%TEMP%`; a junction higher up the path is not refused and the run reaches its first tool. Clear Cache (a stub self, a fake `QMessageBox` answering Yes) with the DataForge cache folder in use a junction lists it under Could not delete with the link named, deleting only the INIs, and the queued old-cache cleanup with a queued junction to an unrelated folder logs one WARNING naming the link instead of "Removed old DataForge cache", clears the queue and deletes nothing. `extract.cache_folder_is_link` is recorded from a real refusal and checked in every language: a value, both placeholders, the folder and where it leads each on a line of their own, and English in `ht`, the rest in `at`. Junctions from `tests/junction.py`; symbolic links skip where Windows refuses them (WinError 1314 without Developer Mode) and fail under CI, as junctions do. Settings in `tmp_profile` where the real paths are read. +- `test_clear_cache.py`: #479, More > Clear Cache clears the DataForge cache in use. The real `MainWindow.clear_cache` on a stub window with real settings (`tmp_profile`), real files and a fake `QMessageBox` that answers each question in turn and fails on one it has no answer for. The cache folder `get_dataforge_cache_leaf()` names is offered and cleared in the registry default, with a cache folder override (the default's cache left alone) and in the portable build (`_portable_data_dir` pinned to `tmp_path`), with the question naming the folder, the summary counting the files and the cache, and the Config tab and the Enhancements tab refreshed; it used to look in the data folder's `cache\dataforge` and never asked. Records without the stamp are offered, a cache folder with no cache in it is not, and the old leftover in the data folder is left to the queued cleanup. A DataForge cache with no files skips the file question, and a No to the DataForge question then changes nothing (no summary, the table and the tabs left alone). With files too, a No to the DataForge question clears the files and keeps the cache, a No to the file question clears nothing, the file question lists the files, and nothing to clear gives the cache-empty box. A cache folder override on a drive that is not there (`tests/unusable_folder.py`) clears the rest and names the folder in the summary, or in the cache-empty box, with one WARNING. While an extraction or a generation runs (each of the three workers) nothing is cleared or asked and the status bar says why, but a running background delete of a working folder (#522) holds nothing up: the cache in use is asked about and cleared, and the working folder, with the old cache set aside in it, is left to its delete. A cached file or a DataForge cache that cannot be deleted (faked) is listed in the summary and logged once at WARNING with nothing at ERROR (a file used to be logged at ERROR, the cache twice), and the other file is still deleted. Every language has the ten texts with their fields, the folder on a line of its own, English in `ht` and the rest in `at`, and an AST scan of `clear_cache` finds no string with a letter in it outside the log lines, the `tr()` keys and the file patterns. - `test_retired_url_sources_migration.py` — 1.0 cleanup of contracts/components/ships/commodities/gear sources retired in 0.7.0. Covers fresh-install defaults, upgrade-time prune, URL-vs-local guard, idempotence. - `test_applied_file_validator.py` — post-apply `global.ini` vs stock `base.ini`. - `test_entry_filter.py` — column-filter logic plus the `NUM_COLUMNS` getter-tuple drift guard. @@ -106,7 +106,7 @@ Split by domain: - `test_blueprint_list_type_tag.py`: #101 regression. The component Type element renders on component entries inside mission blueprint lists, not just standalone component names. - `test_armor_weight_tags.py`: #492 FPS armor weight-class tags (the Tag Builder's Armor category). `_armor_weight`: the stock "Item Type:" line wins when it names a class (only it knows Super Heavy and Utility), then an armor slot's AttachDef SubType (racing helmets, the "LightArmor" backpacks), then Clothing for any `Char_Clothing_*` or `Char_Accessory_*` slot; undersuits, flight suits, flight helmets with the plain "Helmet" SubType and ammo carriers get nothing, including the racing helmets with that plain SubType (most of them), while a flight or racing helmet in a Light slot (the real OMNI-AFS and Interceptor ones) gets Light. `build_wearable_index` and `_run_gen_armor` on fabricated entity XML: off by default with no scan, only `item_Name*` keys (a clothing record's shared "Body" item-port string stays), base.ini's own key casing for names and descriptions (the description test uses a class only the description has, so a SubType fallback cannot pass it), `_ARMOR_WEIGHT_PRECEDENCE` settling a shared name the same way in either file order, the English description for a translated base.ini, `_short` keys, and the user's style, mapping, enclosing and placement. The real `main()` writes the tags into `fps_weapons_desc_enhancements.ini` (left out with the tag off, nothing without Gear), and `TestSettingsWiring` pins that #492 adds no enhancement file, which would make the startup freshness check prompt every Gear user after upgrading. Also the Save Tag Changes fingerprint leaving the Armor config out while its tag is off (`test_tag_config_staleness_startup.py` adds that a stamp written before the Armor tab keeps the button grey, a saved, switched-on Armor tag lights it, and an unsaved Armor edit made with the tag off keeps it lit through a refresh), the tag config defaults (the issue's L/LT, M/MD, H/HV, S/SH, U/UT, C/CL codes), the Blueprint Tracker's enclosing set (armor's brackets count only while its tag is on, and its None style never, while another category's None still does; the Armor tab offers no None brackets and a stored one loads as Square) and the Armor page's standard controls, preview, tooltip and reset. - `test_armor_blueprint_bullets.py`: #492 armor weight tags on mission POTENTIAL BLUEPRINTS bullets. `armor_bullet_names` gives a bullet exactly its item name's tag (checked against `_run_gen_armor` in several styles), nothing when the tag is off, a hand-edited None bracket read as Square, the user's style and mapping, a bare bullet when one display name has two different tags (with the classes swapped between the files, since the read order is by file name), CIG's size prefix stripped and a non-breaking space kept. `_build_blueprint_body_parts` tags bullets only as they are written: no `bullet_names` leaves the output unchanged, colliding "X Set" headers, the Yormandi Eyes and Irradiated Valakkar Pearls override labels and the drift check all keep reading bare names, and bullet order and section order follow the bare names (several same-system sets come out in the same order tagged or not). `_run_gen_missions` end to end on a fabricated contract/pool/blueprint/wearable tree: tagged when on (Square for a hand-edited None), bare with the tag off or the annotate toggle off, no wearables scan with the tag off, English tags on a translated run, and three generations on one warm forge dir (off, Square, Round append) change the bullet while the stored `blueprint_pools` key stays the same and the pickle holds no armor tag. The real `main()` with Gear and missions loads the wearables once (spy on `build_wearable_index`), not at all with the tag off, and tags the bullets with only the missions enhancements selected (no Gear), as component tags do. `[Owned]` lands after a tagged bullet in every bracketed style and placement (idempotent, a longer name left alone), and a tagged bullet joins its bare Armor row in `build_blueprint_metadata`. -- `test_bp_fallback_tags_language.py`: fuel nozzle `[FN]` tags in mission blueprint lists on a translated run (found in the #492 review). The tag reaches the bullet only through `name_fallback_tags` (the nozzles' entities are not UUID-linked to their blueprints, #281), which must be built from the English loc (`ctx["tag_loc"]`): a French run used to find no "Item Type: Fuel Nozzle" line, pickle untagged pools, and leave a later English run on the same cache untagged too. Covers an English run, a French run, French then English on one forge dir, and that a v14 `blueprint_pools` pickle from the old code is rebuilt (the v15 bump). Also the nozzle's own item name (`enhancements_bare_type_tags` reads the English "Item Type:" line too, so a French run tags it), and a translated run whose English base.ini is missing: the real `main()` warns, the pickle built from that run's loc (`standings`) carries an `|en_loc=fallback:` mark, so the next English run rebuilds it, while `scitem_lookups` and `blueprint_pools` hold XML facts only (perf proposal) and keep one unmarked key for both runs, their names resolved through each run's loc, so the next English run still shows English names on a UUID-linked item's bullet and a second fallback language does not reuse the first one's names. Fabricated contract/pool/blueprint tree with no entity link, as for the real nozzles. +- `test_bp_fallback_tags_language.py`: fuel nozzle `[FN]` tags in mission blueprint lists on a translated run (found in the #492 review). The tag reaches the bullet only through `name_fallback_tags` (the nozzles' entities are not UUID-linked to their blueprints, #281), which must be built from the English loc (`ctx["tag_loc"]`): a French run used to find no "Item Type: Fuel Nozzle" line, pickle untagged pools, and leave a later English run on the same cache untagged too. Covers an English run, a French run, French then English on one forge dir, and that a v14 `blueprint_pools` pickle from the old code is rebuilt (the v15 bump). Also the nozzle's own item name (`enhancements_bare_type_tags` reads the English "Item Type:" line too, so a French run tags it), and a translated run whose English base.ini is missing: the real `main()` warns, the pickle built from that run's loc (`standings`) carries an `|en_loc=fallback:` mark, so the next English run rebuilds it, while `scitem_lookups` and `blueprint_pools` hold XML facts only (#522) and keep one unmarked key for both runs, their names resolved through each run's loc, so the next English run still shows English names on a UUID-linked item's bullet and a second fallback language does not reuse the first one's names. Fabricated contract/pool/blueprint tree with no entity link, as for the real nozzles. - `test_commodity_tagging.py`: #97 commodity tagging. Crafting (CF) and Collection flags share one `[...]` wrapper; single-flag items drop the empty flag; crafting-only output matches pre-1.5.0. - `test_favorite_prefix_whitespace.py`: #100 regression. A single-space favourite prefix survives the user.ini round-trip (`parse_ini_file` gained a `strip_values` flag, False for user.ini). - `test_mission_header_em_tag.py`: #99. Mission-header emphasis options are exactly EM3/EM4 (EM1/EM2 never render in-game); stored legacy values coerce back to the default. @@ -116,7 +116,7 @@ Split by domain: - `test_installer_sc_path_checks.py`: #422, the installer's side of the same guard. `installer.iss`'s `IsValidSCPath` accepts a root with channel folders or an existing channel folder (with or without a trailing backslash) and rejects stale values and channel folders that no longer exist; `IsValidSCRoot` accepts only a root, and is what the `sc_install_root` read uses (a channel path there would pre-fill `...\LIVE\LIVE`). The behaviour tests cut those functions, and the `InitializeWizard` pre-fill block that uses them (registry reads faked inside the probe, hard-coded Program Files defaults pointed into the tree, so nothing touches the registry or a real install), verbatim out of `installer.iss`, compile them into a throwaway setup with ISCC and run it silently against a `tmp_path` tree. The pre-fill test covers the two fixes outside the checks: the root joined with `AddBackslash`, and a trailing backslash dropped before the PTU/EPTU/HOTFIX/TECH-PREVIEW to LIVE swap. They skip where Inno Setup isn't installed (but fail under CI, see `inno_setup.py`) or an Application Control policy (Windows Smart App Control, for example) blocks the unsigned probe. GitHub's Windows runner image ships Inno Setup 6, so they run in CI. The text check on the `sc_install_root` line runs everywhere. #438 adds a third probe that cuts the Star Citizen directory part of `WriteInstallerChoicesToRegistry` (registry writes faked) and checks that `sc_install_root` is the page value when it is a root and its parent when it is a channel folder, and nothing when the page is empty. A text check pins the `forward;` declaration of `IsValidSCRoot` ahead of that write, which Pascal Script needs. `_pascal_function` skips `forward;` declarations, so they are not mistaken for the function. The third probe also cuts the `SCRegNode` and `SCLegacyRegNode` constant lines (#418) out of `installer.iss`, because the cut block names its registry nodes with them, and fails with the constant's name when one is renamed. A text check (`_SC_REG_NODES`) fails, without Inno Setup, when the block uses a registry node constant the probe does not cut. - `test_installer_wipe_safety.py`: the #419 review follow-ups, the installer's folder deletes. The opt-in wipe (#357) deleted `\cache` whole, and `CleanCachedData` / `CleanPerChannelCaches` (every install, upgrade, auto-update and plain uninstall) deleted inside the data folder with no check, so a data folder of `D:\` lost `D:\cache` on the next update. `LooksLikeScCache` now says whether a folder is a Smart Citizen cache (a `base.ini`, a `dataforge` folder, or a `\cache` folder, the channels read from `ScChannelName`), the wipe's `\cache` and the flat `cache` that `CleanCachedData` clears go only when it says yes, and both cleaners ask `UnsafeDeleteRootReason` first (silent, the reason goes to the log). The behaviour test cuts the real routines, with the real refusal check, verbatim out of `installer.iss` (registry reads, `{app}` and the log faked), compiles them into one throwaway setup with ISCC and runs it silently over a table of folder trees under `tmp_path`, reading back which files are left, the wipe's summary and the log: the predicate (each layout, empty, missing, look-alike names), the cleaners (an ordinary and a registry data folder clean, another program's `cache` folder stays, and Documents, a folder with game files, the registered Star Citizen install or a folder holding it, and the Smart Citizen install folder are all refused), `CleanPerChannelCaches` on its own, the wipe (an unrelated `cache` folder stays and keeps its data folder, a folder emptied by the wipe still goes), and `UnsafeDeleteRootReason` on its own for the folders no tree under `tmp_path` can be (a drive root, a drive, the top of a network share, Documents through an admin share such as `\\localhost\C$\Users\...\Documents`, a device path, all read as text only, and only the cases whose check reads a folder as text and deletes nothing, these reason cases and the #454 `install-reason` cases, may name a path outside the tree) and for an ordinary folder. It skips where Inno Setup isn't installed (but fails under CI, see `inno_setup.py`) or an Application Control policy blocks the unsigned probe. Text checks run everywhere. They are structural tripwires on the comment-stripped code, and the compiled test is the semantic check: `UnsafeDeleteRootReason` is forward-declared ahead of the cleaners (Pascal Script needs it), the predicate reads channels from `ScChannelName`, starts from `Result := False` and keeps its three marks, each cleaner opens with `Reason := UnsafeDeleteRootReason()`, `if Reason <> ''` and an `Exit` ahead of its one `DelTree`, the flat cache and the wipe's `\cache` are deleted only inside `if LooksLikeScCache()`, every `DelTree` / `RemoveDir` / `DeleteFile` call (any case, space before the bracket or not) sits in a routine on a short list with the reason it is safe, only listed routines start a program and no string or `[Run]` entry names a shell or delete command, and `[InstallDelete]` is pinned with no `[UninstallDelete]`, so a new delete has to be reviewed and added there. #454 narrowed `[InstallDelete]`, which used to empty whatever install folder was chosen. `{app}\*` now runs only behind `Check: InstallDirMayBeEmptied` (`WholeInstallDirReason`), when nothing in the folder is anything but Smart Citizen's own items or the folder is missing or empty. `ScanInstallDir` judges the top of the folder by long name: a program file a release installed (`IsOwnProgramFile`: `SmartCitizen.exe`, `SCLocalizationEditor.exe`, and `SmartCitizen-v` or `SCLocalizationEditor-v` with a version of three one-digit parts, checked digit by digit in `IsVersionedProgramName`, never an old `-Setup.exe` download or a `SmartCitizen-Portable-v` copy (an old standalone `SCLocalizationEditor-v.exe` or `SmartCitizen-v.exe` has an installed program's name, so it counts)), `_internal`, and `unins000.exe` / `.dat` / `.msg`, which count only beside a program file or `_internal` (on their own they can be another program's leftovers). A folder's name earns it nothing: one called `Smart Citizen` that holds other files is treated like any other. The other entries delete only those names, in any folder: `SmartCitizen.exe`, `_internal`, `SmartCitizen-v?.?.?.exe`, `SCLocalizationEditor.exe` and `SCLocalizationEditor-v?.?.?.exe` (every 0.1.1 to 1.x release had a version of three one-digit parts). Text checks: each program entry is an exact name or `-v?.?.?.exe` with more than 8 fixed characters in front of the wildcard (Windows matches 8.3 names too), so a widened wildcard fails, only `{app}\*` has a Check, `IsOwnProgramFile` names exactly the entries' names and nothing more, `IsVersionedProgramName` keeps its length and digit checks, and the exe `[Icons]` and `[Run]` start is one of the entries. The install-folder checks take their folder as a parameter. The delete Check must be asked afresh every time, because `MigrateUserDocsFolder` can fill a missing install folder between the Preparing page and the delete. A text check pins only `InstallDirMayBeEmptied` (its three statements, one `Result :=`), and the probe's `install-refill` case is the semantic check, which also sees a cache in the routines it calls: it asks on a missing folder, puts `LIVE\user.ini` in it, then runs the deletes, and the file must stay while the second ask logs the narrow answer. `ScanInstallDir` lists the folder with a single `FindFirst`, through `LongPath`, and names the three `unins000` files. `AppFolderName` is the last part of `DefaultDirName` and what `NewInstallDir` adds. `NextButtonClick` hands `wpSelectDir` to `GiveInstallItsOwnFolder` and goes straight on to its data-folder exit. That routine reads `WizardDirValue()`, asks `NewInstallDir` with the previous install's folder, returns True with no box when nothing changed and nothing stopped it for length (`TooLong`). On a silent run it never shows a box, and returns False only for `TooLong`. `NewInstallDir` stops only at a folder that is only Smart Citizen's or is the previous install's, or, after those two checks, before a path longer than `MaxInstallDirLength` (240, Setup's own limit, which it checks on the page and again when the install starts), and then sets its `TooLong` output. A Smart Citizen program in a shared folder does not stop it, since a loose old portable copy has the same name as an installed program. `GiveInstallItsOwnFolder` keeps Setup out of such a folder, because the narrow `_internal` entry could take another program's. On an interactive run `TooLong` becomes a `SuppressibleMsgBox` that says the folder holds other files and is too long for a Smart Citizen folder inside it, asks for a shorter or empty folder, and returns False with the page left as typed. A silent run logs why and returns False, so Setup stops without installing. The in-app updater never gets there, because the previous install's folder is kept before the length is looked at. A text check pins that branch (`if TooLong then`, the log on a silent run and the box otherwise, both naming the folder, `Result := False; Exit;`, ahead of the early exit), so removing it, returning True there or showing the box on a silent run fails. The probe's `install` cases run `NewInstallDir` on the chosen folder, then the real `[InstallDelete]` entries (parsed from `installer.iss`, each behind its real Check, with the `DelTree` call Setup makes) in the folder it picked, as Setup does, all under `tmp_path`. Every layout since 0.1.0, and a missing, empty or leftovers-only folder, is emptied. Shared folders, old installs among other programs, folders named `Smart Citizen` or `SC Localization Editor` with other files in them, the data folder used as the install folder, a portable copy beside the install, links (junctions made with `_winapi.CreateJunction`), another program's leftover uninstaller and look-alike names (old `-Setup.exe` and `-installer.exe` downloads, `SmartCitizen-Vault.exe`, and names of exactly a versioned program name's length that only `IsVersionedProgramName`'s digit, dot, prefix or `.exe` test turns away, such as `SmartCitizen-Viewer.exe`, `SmartCitizen-v1-4-2.exe`, `OtherProgram-v1.2.3.exe` and, on its own, `SmartCitizen-v1.4.x.exe`) keep everything that is not Smart Citizen's. No case uses an install folder that is itself a link: a Setup built with Inno Setup 6.7.0 or newer turns on Windows' RedirectionGuard where the system has it (Windows 11, Windows 10 22H2), which stops it following a junction a normal user made, and with an older compiler or on older Windows the folder reads like any other, so the result depends on both, and neither way deletes anything Smart Citizen did not install. The guard adds `\Smart Citizen`, up to three times, to a folder that holds files Smart Citizen did not install, also when a loose old program sits among another app's files. The `install-reason` cases ask `WholeInstallDirReason` and `NewInstallDir` about folders no tree can be, as text with no deletes (only they and the reason cases may name a path outside the tree): a relative folder is never Smart Citizen's own, the guard stops at 240 characters and says so in `TooLong`, and the previous install's folder is kept however long, without `TooLong`. - `test_string_table_model_bounds.py`: #110 regression. `entry_for_row` tolerates out-of-range rows after a failed/empty load; uses `__new__` to stay Qt-free. -- `test_mission_route_title.py`: #166 hauling/delivery route-in-title, reworked for the #200 hotfix. The pure `_derive_route_fragment` / `_route_token_role` / `_title_route_token` / `_expand_nested_route_vars` helpers in `generate_enhancements_ini.py` driven with synthetic bodies: |Address is the default token modifier (|name fails to resolve on some instances), multi-endpoint sides render comma lists (`A > B, C`), bare `*Token` vars expand one level against the loc table keeping only endpoints every variant registers, per-body intersection drops vars pooled bodies disagree on (var-less bodies abstain), Pickup/Dropoff copied verbatim. Perf proposal: `test_body_order_does_not_change_the_route` gives the same route for every order of the bodies, which the callers build from a set (they are sorted, and the first sorted body sets the order of a comma list), and an endpoint some agreeing bodies write bare and others with `|Address` takes the `|Address` form, whichever body sorts first (`_agreed_endpoint_tokens`, in both orders and through `_derive_route_fragment`), while one written a single way (bare, `|Address` or `|name`) keeps that form, and a mix with `|name` (with the bare form or with `|Address`, in either order) keeps the first sorted body's form, as before the perf proposal. Deliberately does NOT use the stale `kraken_4.7.ini` fixture. +- `test_mission_route_title.py`: #166 hauling/delivery route-in-title, reworked for the #200 hotfix. The pure `_derive_route_fragment` / `_route_token_role` / `_title_route_token` / `_expand_nested_route_vars` helpers in `generate_enhancements_ini.py` driven with synthetic bodies: |Address is the default token modifier (|name fails to resolve on some instances), multi-endpoint sides render comma lists (`A > B, C`), bare `*Token` vars expand one level against the loc table keeping only endpoints every variant registers, per-body intersection drops vars pooled bodies disagree on (var-less bodies abstain), Pickup/Dropoff copied verbatim. #522: `test_body_order_does_not_change_the_route` gives the same route for every order of the bodies, which the callers build from a set (they are sorted, and the first sorted body sets the order of a comma list), and an endpoint some agreeing bodies write bare and others with `|Address` takes the `|Address` form, whichever body sorts first (`_agreed_endpoint_tokens`, in both orders and through `_derive_route_fragment`), while one written a single way (bare, `|Address` or `|name`) keeps that form, and a mix with `|name` (with the bare form or with `|Address`, in either order) keeps the first sorted body's form, as before #522. Deliberately does NOT use the stale `kraken_4.7.ini` fixture. - `test_font_size.py`: the UI font size setting. `AppSettings` contract (default 'medium', round-trip, unknown values coerce to 'medium'), every size has a factor with Medium at 1.0, and `apply_font_size_env` on a plain dict: Medium sets nothing, Small/Large set `QT_SCALE_FACTOR`, a user's own factor is left alone, and one we set (inherited across a restart) can change or go back to Medium. - `test_ui_mode.py`: #180 Simple/Advanced view. AppSettings `ui_mode` contract (default 'simple', round-trip, unknown coerces), `SimpleModeWidget` signal/`set_busy` behavior, and `MainWindow._apply_ui_mode` page-swap driven on a lightweight stub `self` (no full window — no pytest-qt). Imports `AppSettings` via `src.utils.settings` so the monkeypatch hits the same class object `main_window` uses. Also covers the #397 fix: `SimpleModeWidget.set_apply_dirty` mirrors the Advanced-mode Apply button's red/green convention (previously hardcoded green regardless of state) — starts dirty by construction, color/tooltip always track `set_apply_dirty` (the tooltip half was a follow-up: it kept showing the same static string in both states even after color/enabled were fixed), and a run in progress (`set_busy(True)`) always wins for enabled state even if dirty flips mid-run, without disturbing what color or tooltip it reports. A #398 follow-up test runs the real `MainWindow._set_apply_btn_dirty` on a stub with a real `SimpleModeWidget`: every state reaches the Simple button (green never disables it), the state is recorded in `_apply_dirty` (theme, language and mode switches re-apply it from there), and the first toolbar call, made before the page exists, is skipped safely. - `test_scitem_lookup_bare_type_and_mining_laser.py`: #266 bare-type (fuel nozzle) and mining-laser bullet tags, plus the #345 size-line fix. `_SIZE_LINE_RE` is the one pattern every desc-size read shares; `TestSizeLineRegex` covers both CIG size forms (`Size: 1` and the S-prefixed `Size: S0` used by mining heads, bomb racks, scopes) and carries a drift guard that greps the generator for a forked bare-digit copy, which is how the bug happened. @@ -145,9 +145,9 @@ Split by domain: - `test_drop_none_entries.py`: #389 crash mitigation. `_drop_none_entries` (module-level in `main_window.py`, not a `MainWindow` method) strips stray `None` items out of a freshly loaded/merged entries list before anything downstream (`_restore_pending_user_edits`'s `e.key` read, `update_category_combo`'s `e.category` read, the table model) can dereference one and crash. Covers empty/no-`None`/single/multiple/all-`None` inputs, order preservation, and that a warning logs only when something was actually dropped. Also drives the real `_on_loading_finished` on a stub `self`: the worker's pre-computed `sort_keys` (one per original entry) reaches the model untouched when nothing was dropped, and the model's keys stay aligned with its entries when something was (the slot passes `None` so the model recomputes them). - `test_single_job_skips_threadpool.py`: #389 mitigation. `scripts/generate_enhancements_ini.py`'s `main()` must not construct a `ThreadPoolExecutor` whenever the pool would have exactly one worker, `min(max_workers, len(jobs)) == 1`, at both the lookup and the generator pool (a pool of one buys no parallelism, and is the exact shape the reported native heap-corruption crash, 0xC0000374, traced back to: a background thread nested under another background thread doing lxml-based XML parsing). `EnhancementsGeneratorWorker` always calls `main(..., max_workers=1)`, so the several-categories-at-`max_workers=1` test is the real GUI path. Runs the real `main()` with `ThreadPoolExecutor` patched to prove it's never constructed, and confirms the inline paths still run every job and write their output. - `test_generator_needs_dataforge_cache.py`: `scripts/generate_enhancements_ini.py`'s `main()` raises `FileNotFoundError`, naming the folder and asking for Extract DataForge first, when the DataForge cache folder is missing or has no records folder (`DATAFORGE_RECORDS_SUBPATH`), before it builds any lookup (the XML path index is the first) or writes any INI, both recorded on the module. The same run with the records folder made goes through and the recorders see it, so only the missing folder stopped it. -- `test_generator_single_pass.py`: perf proposal, the generator reading the DataForge XMLs as few times as it can (each pu_missions and contractgen XML once) and pickling XML facts only (see `scripts/CLAUDE.md`). Frozen copies of the pre-split scitem walk, component tagger and mining-laser tagger are the oracle: all four scitem lookups match them entry for entry and in insertion order under three components configs (the default, a custom one, and one whose typeless tags the lookup drops), on a tree that reaches every branch (size from the class name, the item_Name key or the AttachDef, grade and class from the AttachDef, type from ItemComponentParams, a sentinel description before the real one, a fuel nozzle, a mining laser, an FPS weapon left out, a null magazine record, an unresolved name, a broken file), and `_component_name_tag` and `_component_name_tag_from_facts` match the old tagger for every file, description and config. A components change renders from one cached scitem walk, and that pickle and the blueprint pool one load with an unpickler that refuses every class. The magazine fast path is taken only for ASCII-compatible bytes without `__polymorphicType` (UTF-16, UTF-32 and EBCDIC are not), runs no `_poly_type` then, and a file holding the attribute keeps the loop (an element tagged as a magazine whose polymorphic type says otherwise). Blueprint pool names and tags re-render per config from one cached walk; the ship lookups hold values that render the old block text, an armour Health of `inf` still fails only its ship, and a controller or armour XML changed under the same DataForge build (as a patch changes it) shows on the next `main()` run, with no pickle written for either. The commodity loc prefilter matches a scan of every key, in loc order, and is built once per `scan_crafting_blueprints`, which parses each blueprint once and reads only the carryables whose filename names a commodity, and the SCO crafting index (#474) takes each material's names from those same keys, as a scan of every key gives them. `_run_gen_missions` parses every pu_missions, contractgen and template XML once (counted through `_parse_xml_file` and `ET.parse`), drops hostiles only from a shared description whose XMLs disagree (the last one read has hostiles), tags the Battaglia title and logs the calculated-reward title from the shared parse with one template lookup, still ends the run on a Battaglia failure, with the Rep tag off too, raises the first error the Battaglia or calculated-reward scan met (a held Battaglia error also beats a later listing failure) and asks a failed scan about no other file, and finishes the walk for the other scans when `scan_contract_generators` stops early. As before the single pass, it ends the run on a contractgen file listing that fails part-way (after that scan has logged it) or outright (before that scan, with nothing logged), and raises a calculated-reward failure only where that scan ran, once a contractgen title has zero XP (with the rep tag off the run finishes). The titles written from the pu_missions pass are checked end to end: a cargo haul and a delivery haul under a [BP] contractgen title keep their bodies, and the cargo haul turns the title into [BP?] (#102, #31), a cargo haul whose description is a sentinel leaves another [BP] title as it is, a combat body there that contractgen never writes is dropped as an orphan, a pu-only delivery title takes its route from its own description and then its XP (#166, #200), a pu-only title with no XP is left alone, and the "titles skipped" count includes a title whose description is a sentinel. `test_battaglia_mineable_rs.py`, `test_calculated_reward_titles.py` and the UUID test in `test_commodity_journal.py` read the RS tags, the calculated-reward titles and the blueprint material UUIDs off the calls a generation makes (`_run_gen_missions`, `scan_crafting_blueprints`), since the separate scans they called (`_build_battaglia_mineable_rs_tags`, `_titles_with_calculated_reward`, `_resolve_resource_uuids`) were removed. `_build_xml_path_index` matches a per-file walk in order with one folder key per folder, and `_parse_xml_file` gives the tree `ET.parse` gives (a byte-order mark, UTF-16, Windows-1252) and the same errors. Every test was shown to fail under at least one scripted break of the change it covers. +- `test_generator_single_pass.py`: #522, the generator reading the DataForge XMLs as few times as it can (each pu_missions and contractgen XML once) and pickling XML facts only (see `scripts/CLAUDE.md`). Frozen copies of the pre-split scitem walk, component tagger and mining-laser tagger are the oracle: all four scitem lookups match them entry for entry and in insertion order under three components configs (the default, a custom one, and one whose typeless tags the lookup drops), on a tree that reaches every branch (size from the class name, the item_Name key or the AttachDef, grade and class from the AttachDef, type from ItemComponentParams, a sentinel description before the real one, a fuel nozzle, a mining laser, an FPS weapon left out, a null magazine record, an unresolved name, a broken file), and `_component_name_tag` and `_component_name_tag_from_facts` match the old tagger for every file, description and config. A components change renders from one cached scitem walk, and that pickle and the blueprint pool one load with an unpickler that refuses every class. The magazine fast path is taken only for ASCII-compatible bytes without `__polymorphicType` (UTF-16, UTF-32 and EBCDIC are not), runs no `_poly_type` then, and a file holding the attribute keeps the loop (an element tagged as a magazine whose polymorphic type says otherwise). Blueprint pool names and tags re-render per config from one cached walk; the ship lookups hold values that render the old block text, an armour Health of `inf` still fails only its ship, and a controller or armour XML changed under the same DataForge build (as a patch changes it) shows on the next `main()` run, with no pickle written for either. The commodity loc prefilter matches a scan of every key, in loc order, and is built once per `scan_crafting_blueprints`, which parses each blueprint once and reads only the carryables whose filename names a commodity, and the SCO crafting index (#474) takes each material's names from those same keys, as a scan of every key gives them. `_run_gen_missions` parses every pu_missions, contractgen and template XML once (counted through `_parse_xml_file` and `ET.parse`), drops hostiles only from a shared description whose XMLs disagree (the last one read has hostiles), tags the Battaglia title and logs the calculated-reward title from the shared parse with one template lookup, still ends the run on a Battaglia failure, with the Rep tag off too, raises the first error the Battaglia or calculated-reward scan met (a held Battaglia error also beats a later listing failure) and asks a failed scan about no other file, and finishes the walk for the other scans when `scan_contract_generators` stops early. As before the single pass, it ends the run on a contractgen file listing that fails part-way (after that scan has logged it) or outright (before that scan, with nothing logged), and raises a calculated-reward failure only where that scan ran, once a contractgen title has zero XP (with the rep tag off the run finishes). The titles written from the pu_missions pass are checked end to end: a cargo haul and a delivery haul under a [BP] contractgen title keep their bodies, and the cargo haul turns the title into [BP?] (#102, #31), a cargo haul whose description is a sentinel leaves another [BP] title as it is, a combat body there that contractgen never writes is dropped as an orphan, a pu-only delivery title takes its route from its own description and then its XP (#166, #200), a pu-only title with no XP is left alone, and the "titles skipped" count includes a title whose description is a sentinel. `test_battaglia_mineable_rs.py`, `test_calculated_reward_titles.py` and the UUID test in `test_commodity_journal.py` read the RS tags, the calculated-reward titles and the blueprint material UUIDs off the calls a generation makes (`_run_gen_missions`, `scan_crafting_blueprints`), since the separate scans they called (`_build_battaglia_mineable_rs_tags`, `_titles_with_calculated_reward`, `_resolve_resource_uuids`) were removed. `_build_xml_path_index` matches a per-file walk in order with one folder key per folder, and `_parse_xml_file` gives the tree `ET.parse` gives (a byte-order mark, UTF-16, Windows-1252) and the same errors. Every test was shown to fail under at least one scripted break of the change it covers. -Apart from `AppliedStateWorker`, the two extraction workers, `WorkingFolderDeleteWorker` and `StaleWorkingFolderSweepWorker` (`test_extraction_stop.py` and `test_pending_cache_cleanup.py`, #471 and the perf proposal) and `EnhancementsGeneratorWorker` (`test_enhancements_stale_startup.py`, with a fake generator module: its stamps with `run()` called synchronously, its stop on a real thread; `test_generator_refused_cache.py`, #494, its cache folder handling with a stand-in generator) and `LiveBlueprintLogWorker` (`test_live_blueprint_log_worker.py`, SCO), the QThread workers in `src/gui/workers.py` have no automated tests: covering them the same way, on a real thread with `DirectConnection` slots, needs no `pytest-qt`. Manual smoke testing is the only path for the rest. +Apart from `AppliedStateWorker`, the two extraction workers, `WorkingFolderDeleteWorker` and `StaleWorkingFolderSweepWorker` (`test_extraction_stop.py` and `test_pending_cache_cleanup.py`, #471 and #522) and `EnhancementsGeneratorWorker` (`test_enhancements_stale_startup.py`, with a fake generator module: its stamps with `run()` called synchronously, its stop on a real thread; `test_generator_refused_cache.py`, #494, its cache folder handling with a stand-in generator) and `LiveBlueprintLogWorker` (`test_live_blueprint_log_worker.py`, SCO), the QThread workers in `src/gui/workers.py` have no automated tests: covering them the same way, on a real thread with `DirectConnection` slots, needs no `pytest-qt`. Manual smoke testing is the only path for the rest. ## Pytest config diff --git a/tests/event_loop.py b/tests/event_loop.py index 7c1cdd3c..e21ff933 100644 --- a/tests/event_loop.py +++ b/tests/event_loop.py @@ -5,8 +5,8 @@ connection, which only the event loop delivers, so a test that waits for such a slot runs the loop meanwhile. test_applied_state_worker.py, test_extraction_stop.py, test_live_blueprint_log_worker.py and -test_pending_cache_cleanup.py each had a copy of this loop (perf proposal -review: the third copy is the signal to extract). +test_pending_cache_cleanup.py each had a copy of this loop (#522 review: +the third copy is the signal to extract). Import it as ``from tests.event_loop import pump_until``. It runs the session ``qapp``'s loop (tests/conftest.py), so a test that pumps requests ``qapp``. """ diff --git a/tests/test_base_ini_freshness.py b/tests/test_base_ini_freshness.py index 70866624..d3d0ecce 100644 --- a/tests/test_base_ini_freshness.py +++ b/tests/test_base_ini_freshness.py @@ -1,5 +1,4 @@ -"""base.ini freshness follows Data.p4k's size, not its modified time (perf -proposal). +"""base.ini freshness follows Data.p4k's size, not its modified time (#522). The RSI launcher's file verification moves Data.p4k's modified time on an ordinary game launch without changing a byte (#209, which moved the diff --git a/tests/test_battaglia_mineable_rs.py b/tests/test_battaglia_mineable_rs.py index 44158204..f54ad37b 100644 --- a/tests/test_battaglia_mineable_rs.py +++ b/tests/test_battaglia_mineable_rs.py @@ -96,7 +96,7 @@ def _generation_rs_tags(gen_module, contractgen_dir: Path): *contractgen_dir*: _run_gen_missions over the records tree it sits in, with both dicts read off its calls to _battaglia_mineable_rs_in. The generation runs that on the contractgen parse it shares with - scan_contract_generators (perf proposal), so there is no separate scan + scan_contract_generators (#522), so there is no separate scan to call.""" records = contractgen_dir.parent.parent loc: dict[str, str] = {} diff --git a/tests/test_bp_fallback_tags_language.py b/tests/test_bp_fallback_tags_language.py index 4fa6fe8f..43526756 100644 --- a/tests/test_bp_fallback_tags_language.py +++ b/tests/test_bp_fallback_tags_language.py @@ -177,8 +177,8 @@ class TestMissingEnglishBaseIni: loc. Every pickle built from that loc (standings) is keyed apart, per language, so neither the next English run nor another language's fallback run reuses it. scitem_lookups and blueprint_pools hold XML - facts only and resolve their names through the loc on every run (perf - proposal), so one pickle serves every language and needs no mark.""" + facts only and resolve their names through the loc on every run (#522), + so one pickle serves every language and needs no mark.""" _LINK_UUID = "77777777-7777-7777-7777-777777777777" _LINK_BP_UUID = "88888888-8888-8888-8888-888888888888" diff --git a/tests/test_calculated_reward_titles.py b/tests/test_calculated_reward_titles.py index e87d59b0..7eb931b3 100644 --- a/tests/test_calculated_reward_titles.py +++ b/tests/test_calculated_reward_titles.py @@ -97,7 +97,7 @@ def _generation_calculated_reward_titles(gen_module, contractgen_dir, xml_path_i *contractgen_dir*: _run_gen_missions over the records tree it sits in, with the set read off its calls to _calculated_reward_titles_in. The generation runs that on the contractgen parse it shares with - scan_contract_generators (perf proposal), so there is no separate scan + scan_contract_generators (#522), so there is no separate scan to call.""" records = contractgen_dir.parent.parent loc: dict[str, str] = {} diff --git a/tests/test_clear_cache.py b/tests/test_clear_cache.py index 6aede12c..a8a16a97 100644 --- a/tests/test_clear_cache.py +++ b/tests/test_clear_cache.py @@ -297,7 +297,7 @@ class TestWhileAWorkingFolderIsDeleted: def test_the_cache_in_use_clears_and_the_working_folder_is_left_to_its_delete( self, clear ): - """perf proposal: a finished extraction's working folder, holding the + """#522: a finished extraction's working folder, holding the old cache it set aside, can still be deleted in the background after generation has ended. Clear Cache does not wait for that: the delete never touches the cache folder, and Clear Cache never touches the diff --git a/tests/test_commodity_journal.py b/tests/test_commodity_journal.py index 2ca6e305..150f5853 100644 --- a/tests/test_commodity_journal.py +++ b/tests/test_commodity_journal.py @@ -84,7 +84,7 @@ def test_blueprint_material_uids_collect_item_entityclass_uuids(self, gen_module encoding="utf-8", ) # The UUIDs scan_crafting_blueprints' blueprint pass hands to the - # carryable resolver (perf proposal: it collects them as it reads + # carryable resolver (#522: it collects them as it reads # each blueprint, so there is no separate UUID scan to call). with mock.patch.object(gen_module, "_build_uuid_to_commodity", return_value={}) as resolve: @@ -219,7 +219,7 @@ def test_fills_the_sco_crafting_index(self, gen_module, tmp_path): def test_the_crafting_index_order_does_not_change_between_runs(self, gen_module, tmp_path): """The materials of one blueprint were read from a set, so their order in crafting_index.json, the order they were first met in, - changed with every run's string hashing (found in the perf proposal's + changed with every run's string hashing (found in #522's review). Now each blueprint's materials are met in name order, so the index keeps them in the order they are first met, blueprint by blueprint, each blueprint's by name. All eight here are in one diff --git a/tests/test_event_loop_helper.py b/tests/test_event_loop_helper.py index f704ac34..f163f010 100644 --- a/tests/test_event_loop_helper.py +++ b/tests/test_event_loop_helper.py @@ -1,5 +1,5 @@ """tests/event_loop.py's ``pump_until``, the one loop the tests run the Qt -event loop with while they wait for a queued slot (perf proposal review: it +event loop with while they wait for a queued slot (#522 review: it replaced a copy each in test_applied_state_worker.py, test_extraction_stop.py, test_live_blueprint_log_worker.py and test_pending_cache_cleanup.py). diff --git a/tests/test_extraction_stop.py b/tests/test_extraction_stop.py index 687bba4c..482f904d 100644 --- a/tests/test_extraction_stop.py +++ b/tests/test_extraction_stop.py @@ -24,7 +24,7 @@ with its own warning. Its worker's stop is covered in test_enhancements_stale_startup.py, and the generator's stop points in test_generation_writes.py. -* The perf proposal: a successful DataForge run's working folder is kept +* #522: a successful DataForge run's working folder is kept by the worker (any other outcome deletes it before finished), deleted by the window on a WorkingFolderDeleteWorker of its own once the extraction thread is done, and waited for by closing within the same deadline. @@ -552,7 +552,7 @@ class _Worker: def __init__(self, *args, running=True, finishes=True, **kwargs): self.args = args - self.kwargs = kwargs # the DataForge worker's wait_for (perf proposal) + self.kwargs = kwargs # the DataForge worker's wait_for (#522) self.running = running self.finishes = finishes self.interrupted = False @@ -562,7 +562,7 @@ def __init__(self, *args, running=True, finishes=True, **kwargs): self.calls = [] self.progress, self.progress_pct = _Signal(), _Signal() self.error, self.finished = _Signal(), _Signal() - # What a successful DataForge run kept (perf proposal). + # What a successful DataForge run kept (#522). self.working_folder = None def take_working_folder(self): @@ -689,7 +689,7 @@ def run(self): self.msleep(10) def take_working_folder(self): - return None # a stopped run keeps none (perf proposal) + return None # a stopped run keeps none (#522) thread = Spinning() thread.start() @@ -1058,7 +1058,7 @@ def make_worker(*args, **kwargs): assert worker.progress_pct.slots == [dialog.set_progress] -# ── the working folder deleted after the run (perf proposal) ──────────────── +# ── the working folder deleted after the run (#522) ───────────────────────── class _Folder: """Stands in for pak_extractor.WorkingFolder: records delete() and @@ -1360,7 +1360,7 @@ def test_a_running_delete_is_waited_for_hidden(self, qapp): me.hide.assert_called_once_with() def test_a_sweep_is_stopped_not_waited_for(self, qapp, tmp_path): - """A sweep of leftovers stops when closing asks (perf proposal): one + """A sweep of leftovers stops when closing asks (#522): one still waiting on a delete that will not end in time ends at once, so closing does not sit out its deadline for a housekeeping job.""" never = threading.Event() # the done event of a delete that never ends @@ -1510,7 +1510,7 @@ def test_a_failed_delete_lets_go_of_the_marker_before_done( extraction waiting on ``done`` sweeps beside it at once. The worker lets go of the WorkingFolder before it sets ``done``, so with the cyclic GC off and the worker itself still held, the marker is closed - by then and that sweep takes the folder (perf proposal review).""" + by then and that sweep takes the folder (#522 review).""" real_unlink = pe._unlink_files def boom(_paths): @@ -1556,7 +1556,7 @@ def slow_delete(): class TestANewRunWaitsForTheDelete: - """The user's choice (perf proposal review): a DataForge extraction + """The user's choice (#522 review): a DataForge extraction started while an earlier run's working folder is still being deleted in the background waits for that delete before extract_dataforge, whose sweep would skip the folder and whose free-space check would count it @@ -1722,7 +1722,7 @@ def record(parent, prefix, should_cancel=None): class TestLeftoversAreSweptAtStartup: """A crash or a kill during a successful run's background delete leaves - its working folder, with the old cache set aside in it (perf proposal), + its working folder, with the old cache set aside in it (#522), which only the next DataForge extraction beside the same cache folder used to sweep. Once a session at startup the window sweeps beside the active cache folder, on a thread of its own, with pak_extractor's own diff --git a/tests/test_extraction_working_folder.py b/tests/test_extraction_working_folder.py index 781545a9..c0dc4da1 100644 --- a/tests/test_extraction_working_folder.py +++ b/tests/test_extraction_working_folder.py @@ -38,12 +38,12 @@ * What the bar shows while a tool runs (#507): a total of 0, the busy bar, for unp4k and unforge in both extractions, and the counted phases as they were. -* What the tools are asked for and what they print (perf proposal): the +* What the tools are asked for and what they print (#522): the global.ini run takes the English file alone and is sized for it, and a tool that exits 0 after an error it caught fails the run before the old cache is wiped, with no pauses between the steps. -* The perf proposal: beside the cache the old cache is set aside in the +* #522: beside the cache the old cache is set aside in the working folder with one rename instead of wiped (wiped in place in %TEMP%, on a drive that has room only once it is gone, or when the rename is refused), a copy after a refused move deletes it first, and a run @@ -97,7 +97,7 @@ # 2**31 - 1 times on Windows, #490) is stopped here with an error the test # reports, instead of hanging the run for a day and a half. _SPIN_GUARD_CALLS = 100 -# The unp4k.exe and unforge.exe Smart Citizen ships (perf proposal). +# The unp4k.exe and unforge.exe Smart Citizen ships (#522 and #529). _BUNDLED_TOOLS = Path(__file__).resolve().parents[1] / "assets" / "unp4k" @@ -247,8 +247,8 @@ def rename(src, dst): def _refuse_set_aside(monkeypatch, leaf: Path, winerror: int = 17) -> list[str]: - """pe._rename_tree refuses to move the cache folder *leaf* itself (the - perf proposal's set-aside rename) as Windows refuses it (17 another + """pe._rename_tree refuses to move the cache folder *leaf* itself + (#522's set-aside rename) as Windows refuses it (17 another drive, 5 or 32 a held file, with no retry delays), and passes every other rename through. Returns one entry per refusal.""" real, refused = pe._rename_tree, [] @@ -561,7 +561,7 @@ def refuse(*args, **kwargs): assert _scratch_folders(root, pe.DATAFORGE_SCRATCH_PREFIX) == [] def test_a_stop_leaves_the_folders_after_it(self, root): - """Asked before each folder (perf proposal: closing stops the + """Asked before each folder (#522: closing stops the window's sweep), a stop after the first leaves the rest for a later sweep, and none is touched once it is asked.""" names = [pe.DATAFORGE_SCRATCH_PREFIX + f"dead000{n}" for n in range(3)] @@ -627,7 +627,7 @@ class _FakeTools: multiple of 100, NTFS's resolution), and ``records=False`` writes a libs/ folder without foundry/records. *unp4k_out* and *unforge_out* are what each tool prints, as ``{"stdout": ..., "stderr": ...}``, and - nothing by default (perf proposal), and ``unforge_writes=False`` makes + nothing by default (#522), and ``unforge_writes=False`` makes unforge write nothing at all. *during_unp4k* and *during_unforge* are called with the working folder once that tool has written its output.""" @@ -758,7 +758,7 @@ def test_the_stamps_record_data_p4k_as_unp4k_read_it(self, setup, tool): """A game patch that lands while the run goes on, while unp4k reads Game2.dcb or while unforge converts it, leaves a cache of the build unp4k read. The stamps were read after unforge, so the patched - Data.p4k's size made that cache read as current (perf proposal); + Data.p4k's size made that cache read as current (#522); read before unp4k runs, the next launch asks again.""" leaf, run = setup p4k = leaf.parents[3] / "tools" / "Data.p4k" @@ -905,7 +905,7 @@ def write_the_shipped_list(cwd): ], ids=["unp4k", "unforge"]) def test_the_bundled_tools_hold_their_failure_text(self, exe, text): """The fakes print the constants' own text, so this is what ties - them to the tools Smart Citizen ships (perf proposal): each is a + them to the tools Smart Citizen ships (#522): each is a copy of a literal in odw-fast-unp4k's Program.cs, and a tool rebuilt with it worded otherwise would pass every other test while the check never fired again, so a run that lost a file would be used, @@ -1108,7 +1108,7 @@ def test_a_wipe_cut_short_leaves_no_stamp(self, setup, monkeypatch): wipe that gave up part-way could leave both stamps on a partial cache, which reads as fresh while they match this Data.p4k (#484, gap B). The stamps go before the wipe. The old cache is wiped in - place here because its set-aside rename is refused (perf proposal).""" + place here because its set-aside rename is refused (#522).""" leaf, run = setup _refuse_set_aside(monkeypatch, leaf) p4k = leaf.parents[3] / "tools" / "Data.p4k" @@ -1593,7 +1593,7 @@ def test_a_moved_cache_and_a_copied_cache_are_the_same(self, setup, monkeypatch, assert _tree(self._cache(leaf)) == moved_tree -# ── deleting the working folder after the run (perf proposal) ─────────────── +# ── deleting the working folder after the run (#522) ──────────────────────── _STAMPS = (pe.P4K_MTIME_STAMP, pe.P4K_SIZE_STAMP) @@ -1625,7 +1625,7 @@ def _kept_missing(leaf: Path) -> list[str]: class TestTheOldCacheIsSetAside: """Beside the cache the old cache is no longer wiped before the new one is filled: its stamps go, then one rename puts it in the working folder, - which is deleted after the run (perf proposal). It is wiped in place, as + which is deleted after the run (#522). It is wiped in place, as before, wherever that rename cannot help or is refused.""" def test_one_rename_sets_it_aside_and_nothing_wipes_it(self, setup, monkeypatch, caplog): @@ -1904,7 +1904,7 @@ def test_a_cache_folder_that_is_a_link_is_still_refused_before_anything_runs( class TestTheWorkingFolderGoesAfterTheRun: """A caller that passes defer_cleanup is handed the working folder of a run that saved and stamped the new cache, and deletes it after the run, - in the background (perf proposal). Every other outcome still deletes it + in the background (#522). Every other outcome still deletes it before the run ends (#471).""" def test_it_outlives_the_run_with_its_marker_held_until_deleted(self, setup, caplog): @@ -2001,7 +2001,7 @@ def test_a_folder_that_will_not_go_keeps_its_marker_for_the_next_sweep( ): """Logged once, never raised, with the marker file left closed, so the next sweep takes the folder at once. The entries after the one - that would not go are still tried (perf proposal): a file held under + that would not go are still tried (#522): a file held under Data used to keep the whole set-aside old cache, about 31,000 files, until the next DataForge extraction.""" leaf, run = setup @@ -2263,7 +2263,7 @@ class _FakeGlobalIniTool: writes into its working folder every language's global.ini whose entry name (``Data/Localization//global.ini``) holds its filter, ignoring case, and records the filters it was handed and the languages - it wrote (perf proposal). *stdout* and *stderr* are what it prints.""" + it wrote (#522). *stdout* and *stderr* are what it prints.""" def __init__(self, *, stdout: str = "", stderr: str = ""): self.cwds = [] @@ -2563,7 +2563,7 @@ def test_a_cache_drive_that_is_gone_is_skipped_and_the_run_moves_to_temp( class TestGlobalIniTakesTheEnglishFileAlone: - """(perf proposal) unp4k takes every entry whose name holds its filter, + """(#522) unp4k takes every entry whose name holds its filter, ignoring case, so the filter "global.ini" took all 11 languages' files (20 to 26 s and 9 GB of memory on a 150 GB Data.p4k), while the run only ever reads the English one. The filter now names that file, and the @@ -2601,7 +2601,7 @@ def test_the_work_is_asked_for_the_english_file_alone(self, env, systemp, monkey assert fake.written == ["english", "english"] -# ── tools that exit 0 after an error (perf proposal) ───────────────────────── +# ── tools that exit 0 after an error (#522) ────────────────────────────────── # What the tools print for a file they could not extract or convert. unp4k # catches the error, prints it and goes on to exit 0, and so does an unforge @@ -2818,7 +2818,7 @@ def watched(args, **kwargs): "unforge": (0, 0, pe.tr("progress.unforge_short")), } # Caching stays 2 of 3, and the run then ends at 3 of 3: no diff - # snapshot counts the cache's files any more (perf proposal, #495). + # snapshot counts the cache's files any more (#522, #495). assert calls == [ (0, 0, pe.tr("progress.extract_dcb")), (0, 0, pe.tr("progress.unforge_short")), @@ -3134,7 +3134,7 @@ def test_a_stamp_without_raw_libs_makes_no_room(self, setup, monkeypatch): def test_the_figures_cover_the_measured_run(self): """The LIVE 4.10 run the constants come from (#482), with GB read as GiB and MB as MiB, the larger readings. The global.ini run extracts - the English global.ini alone (perf proposal), measured at 10.1 MB, + the English global.ini alone (#522), measured at 10.1 MB, and at 11,128,009 bytes before that.""" gib, mib = 1 << 30, 1 << 20 assert pe._DATAFORGE_WORK_BYTES >= 317 * mib + 1.84 * gib # Game2.dcb + unforge's tree diff --git a/tests/test_generator_single_pass.py b/tests/test_generator_single_pass.py index 5eb020e0..7de1e7db 100644 --- a/tests/test_generator_single_pass.py +++ b/tests/test_generator_single_pass.py @@ -1,4 +1,4 @@ -"""Perf proposal: the enhancement generator reads the DataForge XMLs as few +"""#522: the enhancement generator reads the DataForge XMLs as few times as it can (each pu_missions and contractgen XML once, where it used to read them up to five times), and its lookup pickles hold XML facts rather than text rendered from the loc file and the Tag Builder diff --git a/tests/test_linked_cache_folder.py b/tests/test_linked_cache_folder.py index e35ef81d..49c4b5cb 100644 --- a/tests/test_linked_cache_folder.py +++ b/tests/test_linked_cache_folder.py @@ -219,7 +219,7 @@ def test_a_folder_holding_a_link_is_deleted_and_the_target_kept(self, tmp_path, a link, as before, and what it leads to stays. A directory symbolic link too: Windows' DirEntry follows one when its stat is asked to follow, but never a junction, so the junction alone could not show - that the threaded pass's stat does not follow (perf proposal). + that the threaded pass's stat does not follow (#522). Where Windows refuses this account a symbolic link it skips, and under CI it fails, as the other symbolic link tests do.""" target = tmp_path / "elsewhere" diff --git a/tests/test_mission_route_title.py b/tests/test_mission_route_title.py index 7955294c..5f1bbe2d 100644 --- a/tests/test_mission_route_title.py +++ b/tests/test_mission_route_title.py @@ -9,7 +9,7 @@ come from the mission_titles config, and courier titles are eligible. The 2.1.1 hotfix (#200) reworked the shapes: |Address is the default modifier, endpoint sides render comma lists, bare ``*Token`` vars expand one level against the loc -table, and per-body intersection guards shared titles. The perf proposal +table, and per-body intersection guards shared titles. #522 sorted the bodies, so their order no longer changes a route, and an endpoint some bodies write bare and others with |Address takes |Address. The stale `kraken_4.7.ini` fixture is deliberately not used as ground truth. @@ -153,7 +153,7 @@ def test_body_order_does_not_change_the_route(gen_module): list from a set, so its order changed with every run's string hashing. Pooled bodies that list their drop-offs in another order, or spell an endpoint differently, gave three courier titles a different route from - one generation to the next (perf proposal). The bodies are sorted now + one generation to the next (#522). The bodies are sorted now ("Hello" before "Hi"), and an endpoint written both bare and with |Address takes |Address.""" bodies = [ @@ -174,7 +174,7 @@ def test_an_endpoint_written_both_ways_takes_its_address_form(gen_module, first) """Pooled bodies that write Pickup1 and Dropoff1 bare in one and with |Address in another give the |Address form, whichever body sorts first, so a title follows the tokens rather than which greeting sorts first - (perf proposal).""" + (#522).""" bare = "collect from ~mission(Pickup1), deliver to ~mission(Dropoff1)" address = "collect from ~mission(Pickup1|Address), deliver to ~mission(Dropoff1|Address)" greetings = ("A. ", "B. ") if first == "bare" else ("B. ", "A. ") @@ -204,7 +204,7 @@ def test_the_agreed_tokens_take_the_address_form(gen_module, order): def test_a_mix_with_the_name_form_keeps_the_first_bodys_form(gen_module, first, second): """The |Address preference is only for a var written both bare and with |Address: a mix with |name keeps the first sorted body's form, as before - the perf proposal (and never makes up a token no body writes).""" + #522 (and never makes up a token no body writes).""" bodies = [f"A. collect from ~mission(Pickup1{first}), deliver to ~mission(Dropoff1{first})", f"B. collect from ~mission(Pickup1{second}), deliver to ~mission(Dropoff1{second})"] assert gen_module._derive_route_fragment(bodies) == ( diff --git a/tests/test_pending_cache_cleanup.py b/tests/test_pending_cache_cleanup.py index 2681b4a2..2d100f8c 100644 --- a/tests/test_pending_cache_cleanup.py +++ b/tests/test_pending_cache_cleanup.py @@ -20,14 +20,14 @@ be made) and the 8.3 short name cases (which also skip where the volume makes no short names). -The perf proposal: a queued folder that is not in use also has the working +#522: a queued folder that is not in use also has the working folders a crash left beside it swept, in the background, since after a cache folder change no extraction sweeps there. MainWindow and ConfigTab run on stub selves, with a JsonSettings backend in ``tmp_path`` so nothing touches the registry. No widget is built, so the session ``qapp`` is only needed by the tests that run the background delete -of a working folder beside the cleanup and that sweep (perf proposal), for +of a working folder beside the cleanup and that sweep (#522), for their queued finished slots. """ from __future__ import annotations @@ -561,7 +561,7 @@ def test_a_channel_switch_before_the_successful_extraction_keeps_it_too( assert AppSettings.get_pending_cache_cleanup() == "" -# ── Beside a working folder's background delete (perf proposal) ───────────── +# ── Beside a working folder's background delete (#522) ────────────────────── def _finished_run_window(run_dir: Path, marker) -> tuple[MagicMock, MagicMock]: """A stub window, and the worker of a successful run that kept *run_dir*, @@ -624,7 +624,7 @@ def test_each_delete_keeps_to_its_own_folder(self, channel, tmp_path, qapp, queu class TestLeftoverWorkingFoldersBesideIt: """A crash or a kill during a successful run's background delete leaves its working folder beside the cache folder, with the old cache set aside - in it (perf proposal). After a cache folder change no extraction sweeps + in it (#522). After a cache folder change no extraction sweeps beside the old cache folder any more, so the cleanup sweeps there, in the background: beside a queued folder it deletes or finds gone, never beside a cache in use, where the extraction that just ran swept itself, or a diff --git a/tests/test_refused_cache_window_paths.py b/tests/test_refused_cache_window_paths.py index 94f27088..a205cf6e 100644 --- a/tests/test_refused_cache_window_paths.py +++ b/tests/test_refused_cache_window_paths.py @@ -253,7 +253,7 @@ def wait(self, *args): return True def take_working_folder(self): - return None # no working folder kept (perf proposal) + return None # no working folder kept (#522) def deleteLater(self): self.calls.append("deleteLater") diff --git a/tests/test_robust_rmtree.py b/tests/test_robust_rmtree.py index f8ee4e68..8296ba81 100644 --- a/tests/test_robust_rmtree.py +++ b/tests/test_robust_rmtree.py @@ -1,4 +1,4 @@ -"""robust_rmtree, faster (perf proposal). +"""robust_rmtree, faster (#522). Deleting the extraction's working folder took about 220 us a file one at a time (6.9 s for ~31k files), and a re-extract deletes the old cache's ~31k @@ -427,7 +427,7 @@ def test_a_delete_threads_failure_keeps_no_marker_open(tmp_path, monkeypatch): folder's marker open until the cyclic GC ran, so a sweep straight after skipped the folder. With the cyclic GC off, the marker is closed as soon as the exception and the WorkingFolder holding the marker are dropped, - and the next sweep takes the folder (perf proposal review). The + and the next sweep takes the folder (#522 review). The background delete worker drops its WorkingFolder before it sets done (test_extraction_stop.py's TestTheDeletesFinishedSlot).""" real_unlink = pe._unlink_files From 97a1c45db084e8dfb4e04dfc59a300355c673de5 Mon Sep 17 00:00:00 2001 From: Stealrull <93751735+Stealrull@users.noreply.github.com> Date: Fri, 6 Nov 2026 18:24:48 +1000 Subject: [PATCH 04/32] Name the code in two test comments instead of line numbers (#549) Two test comments pointed at file and line numbers that have moved: generate_enhancements_ini.py:2817 is a blank line now, and blueprint_tracker_tab.py:325,349 are no longer the two lists. They now name ContractVariant, scan_contract_generators and the two list attributes. The first comment also described destructures _run_gen_missions no longer has. Comments only: both files have the same AST. Fixes #549 Co-Authored-By: Claude Opus 5.5 --- tests/test_blueprint_tracker_hidden_selection.py | 2 +- tests/test_mission_variant_tuple.py | 9 ++++----- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/tests/test_blueprint_tracker_hidden_selection.py b/tests/test_blueprint_tracker_hidden_selection.py index 62c0a0e1..bbcc4cba 100644 --- a/tests/test_blueprint_tracker_hidden_selection.py +++ b/tests/test_blueprint_tracker_hidden_selection.py @@ -31,7 +31,7 @@ def _list_with(names, hidden=()) -> QListWidget: lst = QListWidget() - # Matches the real tab's lists (blueprint_tracker_tab.py:325,349) -- a + # Matches the real tab's lists (_blueprints_available_list, _blueprints_owned_list) -- a # bare QListWidget defaults to SingleSelection, under which selecting a # second item silently deselects the first, which made an earlier draft # of this test's _select_all() collapse to "only the last item" and fail diff --git a/tests/test_mission_variant_tuple.py b/tests/test_mission_variant_tuple.py index 47a2a747..79036625 100644 --- a/tests/test_mission_variant_tuple.py +++ b/tests/test_mission_variant_tuple.py @@ -33,11 +33,10 @@ def gen_module(): return mod -# Canonical mission-variant tuple — same shape as the one built at -# generate_enhancements_ini.py:2817 (``missions[title_key].append(...)``). -# Keeping this aligned with the writer site is the entire point: when the -# writer shape moves, this constant must move, and every destructure in -# ``_run_gen_missions`` must follow. +# Canonical mission-variant tuple — same shape as the ``ContractVariant`` that +# ``scan_contract_generators`` appends (``missions[title_key].append(...)``). +# Keeping this aligned with ``ContractVariant`` is the entire point: when that +# shape moves, this helper must move too. def _make_variant( system_name: str = "Stanton", success_xp: int = 100, From 40ee393ceb5563323e0c70b290a537d452e3a8b3 Mon Sep 17 00:00:00 2001 From: Stealrull <93751735+Stealrull@users.noreply.github.com> Date: Fri, 6 Nov 2026 19:23:39 +1000 Subject: [PATCH 05/32] Test the real ContractVariant, not a copy of its shape (#551) Every test in test_mission_variant_tuple.py built its tuple with the file's own _make_variant, so a 13th field in ContractVariant still passed all four. Three of them copied loops and v[N] reads that went in 415b0b47 (#240) and ran no generator code. One test now holds _make_variant against the real ContractVariant, field by field. Another runs the real scan_contract_generators and checks both of its ContractVariant(...) calls, the only code that still passes the fields by position. Also corrects the file's tests/CLAUDE.md entry, which had the 1.4.1 regression the wrong way round, and removes a clause from a generator docstring that claimed destructures that do not exist. The generator's code does not change: its AST is the same once docstrings are taken out. Fixes #551 Co-Authored-By: Claude Opus 5.5 --- scripts/generate_enhancements_ini.py | 3 +- tests/CLAUDE.md | 2 +- tests/test_mission_variant_tuple.py | 224 ++++++++++++++++----------- 3 files changed, 134 insertions(+), 95 deletions(-) diff --git a/scripts/generate_enhancements_ini.py b/scripts/generate_enhancements_ini.py index 472569c6..ed1e3f08 100644 --- a/scripts/generate_enhancements_ini.py +++ b/scripts/generate_enhancements_ini.py @@ -4573,8 +4573,7 @@ def _calculated_reward_titles_in(root: ET.Element, titles: set[str]) -> None: correct "nothing to show" case. A separate scan rather than widening ContractVariant (its 12-field - width is locked by tests/test_mission_variant_tuple.py, and several - ``for ... in variants:`` sites destructure it positionally) for a value + width is locked by tests/test_mission_variant_tuple.py) for a value only the end-of-run diagnostic needs. _run_gen_missions runs it on each root of the contractgen parse scan_contract_generators already makes, rather than walking the files again once a zero-XP title turns up diff --git a/tests/CLAUDE.md b/tests/CLAUDE.md index c7a18134..a7f65bff 100644 --- a/tests/CLAUDE.md +++ b/tests/CLAUDE.md @@ -11,7 +11,7 @@ Split by domain: - `test_mission_engagement.py` — FPS / Ship / FPS&Ship engagement classifier from CIG loc-key naming. - `test_mission_turrets.py` — turret detection from `SpawnDescription_ShipGroup Name="Turrets"` plus the `OverrideTurretHosility_BP` mission-variable signal. Fabricated XML, so no populated cache needed. - `test_spawn_classifier.py` — 1.4.1 4-bucket spawn classifier (Crew / Boarders / Wave Hostiles / Wave Allies) that replaced the flat "Enemies" tally on mission descriptions. -- `test_mission_variant_tuple.py` — 1.4.1 regression: a variant adding an 11th tuple field threw `ValueError: too many values to unpack (expected 10)`. Locks the variant-tuple shape. +- `test_mission_variant_tuple.py` — 1.4.1 regression: the spawn-classifier refactor shrank the variant tuple from 11 to 10 fields and missed one destructure, which threw `ValueError: not enough values to unpack (expected 11, got 10)`. Locks the variant-tuple shape: the real `ContractVariant` against the file's `_make_variant`, field by field and in order, and both `ContractVariant(...)` calls in `scan_contract_generators`, which pass every field by position, on a fabricated contract with two sub-contracts, the second with no description of its own, which takes the contract's (a wrong count at either call raises nothing: the scan logs the `TypeError` at DEBUG and goes on to the next contract, losing that variant and the contract's remaining sub-contracts). - `test_mission_desc_annotation_toggle.py` — 1.4.1 Tag-Builder toggle that suppresses `[CLASS-…]` component annotations (and since #492 armor weight tags, see `test_armor_blueprint_bullets.py`) inside mission descriptions while leaving the standalone component INIs annotated. - `test_missile_name_tag.py` — 1.4.1: bomb name tags carry their ordinance letter, not size-only, so the variant column distinguishes `[BOMB-X-…]` from `[BOMB-Y-…]`. - `test_commodity_journal.py` — 1.4.1 generator guard for the `commodity_journal` unpack crash when the crafting cache directory is missing. The SCO crafting index's material names follow the selected language (`ctx["loc"]`, not `tag_loc`), and its materials come in a fixed order, not the set order that changed with each run's string hashing (perf proposal review): the order they are first met, blueprint by blueprint, each blueprint's materials by name, so the index as a whole is not in name order. The test has eight materials in one blueprint, so a set order passes by chance about once in 40,000 runs. diff --git a/tests/test_mission_variant_tuple.py b/tests/test_mission_variant_tuple.py index 47a2a747..0ae85093 100644 --- a/tests/test_mission_variant_tuple.py +++ b/tests/test_mission_variant_tuple.py @@ -1,13 +1,16 @@ """Regression guard for the contract mission-variant tuple shape. -Contract generators store one tuple per mission variant under +Contract generators store one ``ContractVariant`` per mission variant under ``contractgen_missions[title_key]``. The 1.4.2 rep-tier-names change grew the tuple from 10 → 11 elements (adding ``rank_name`` at position 10). The 2.2.0 reputation-track change (issue #161) grew it again, 11 → 12 (adding ``rep_track`` at position 11) so a mission's body/title can show which reputation track (e.g. "Security" vs the generic Contractor/Standing track) -its XP feeds. These tests lock the tuple width at every consumer so the -next refactor can't slip a similar drift through. +its XP feeds. ``_run_gen_missions`` reads a variant by field name since #240, +but ``scan_contract_generators`` still builds it by position, so a field that +moves puts values in the wrong fields with no error. These tests lock the real +``ContractVariant`` and both calls that build it, so the next refactor can't +slip a similar drift through. """ from __future__ import annotations @@ -33,32 +36,31 @@ def gen_module(): return mod -# Canonical mission-variant tuple — same shape as the one built at -# generate_enhancements_ini.py:2817 (``missions[title_key].append(...)``). -# Keeping this aligned with the writer site is the entire point: when the -# writer shape moves, this constant must move, and every destructure in -# ``_run_gen_missions`` must follow. +# Canonical mission-variant tuple — same shape as the ``ContractVariant`` that +# ``scan_contract_generators`` appends (``missions[title_key].append(...)``). +# Keeping this aligned with ``ContractVariant`` is the entire point: when that +# shape moves, this helper must move too. def _make_variant( - system_name: str = "Stanton", - success_xp: int = 100, - failure_xp: int = -50, - desc_key: str = "test_desc_key", - contract_flags=None, - spawns=None, - contract_difficulty: str | None = "3", - contract_has_bp: bool = False, - contract_bp_chance: float = 0.0, - contract_bp_variant: str = "", - rank_name: str = "", - rep_track: str = "", + system_name: str, + success_xp: int, + failure_xp: int, + desc_key: str, + contract_flags: list[str], + spawns: dict, + contract_difficulty: str | None, + contract_has_bp: bool, + contract_bp_chance: float, + contract_bp_variant: str, + rank_name: str, + rep_track: str, ) -> tuple: return ( system_name, success_xp, failure_xp, desc_key, - contract_flags or [], - spawns if spawns is not None else {}, + contract_flags, + spawns, contract_difficulty, contract_has_bp, contract_bp_chance, @@ -69,81 +71,119 @@ def _make_variant( class TestMissionVariantTupleShape: - """The canonical variant tuple has exactly 12 positions.""" - - def test_variant_tuple_has_twelve_positions(self): - v = _make_variant() - assert len(v) == 12, ( - "The mission-variant tuple width is locked at 12 since the 2.2.0 " - "reputation-track change (#161). If you're changing this, also " - "update every destructure in _run_gen_missions and every v[N] " - "index access — see the comment block at the top of this test " - "file." + """``_make_variant`` holds the real ``ContractVariant``'s fields, in order.""" + + def test_helper_holds_each_field_at_its_position(self, gen_module): + """A field added to, dropped from, renamed in or moved within + ``ContractVariant`` fails here until ``_make_variant`` and the expected + dict below agree with it. Every value differs from the others, so a + field that moved cannot pass.""" + values = _make_variant( + system_name="Pyro", + success_xp=500, + failure_xp=-250, + desc_key="my_desc", + contract_flags=["Starter"], + spawns={"hostile": {"Pirates": 4}}, + contract_difficulty="2", + contract_has_bp=True, + contract_bp_chance=0.25, + contract_bp_variant="hard", + rank_name="Rookie", + rep_track="Security", + ) + fields = gen_module.ContractVariant._fields + assert len(values) == len(fields), ( + f"ContractVariant has {len(fields)} fields, _make_variant gives {len(values)}. " + "Change both, and both ContractVariant(...) calls in scan_contract_generators, " + "which pass every field by position." ) + assert gen_module.ContractVariant(*values)._asdict() == { + "system_name": "Pyro", + "success_xp": 500, + "failure_xp": -250, + "desc_key": "my_desc", + "flags": ["Starter"], + "spawns": {"hostile": {"Pirates": 4}}, + "difficulty": "2", + "has_bp": True, + "bp_chance": 0.25, + "bp_variant": "hard", + "rank_name": "Rookie", + "rep_track": "Security", + } + + +# One contract with two sub-contracts, the second with no description of its own. +# Each field of their variants gets a value of its own, so a value that lands in +# another field cannot pass. +_CONTRACT_GEN_XML = """\ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +""" + + +class TestScanContractGeneratorsFillsEachField: + """Both ``ContractVariant(...)`` calls in ``scan_contract_generators`` pass + every field by position. Two values swapped there raise nothing. Nor does a + wrong count: the scan logs the ``TypeError`` at DEBUG and goes on to the next + contract, losing that variant and the contract's remaining sub-contracts.""" + def test_each_value_lands_in_its_field(self, gen_module, tmp_path): + contractgen_dir = tmp_path / "contractgenerators" + contractgen_dir.mkdir() + (contractgen_dir / "contracts.xml").write_text(_CONTRACT_GEN_XML, encoding="utf-8") -class TestRunGenMissionsTupleConsumers: - """Both ``for ... in variants`` destructures in ``_run_gen_missions`` - must accept the canonical 12-tuple without raising. Pre-fix the head - destructure had 11 placeholders and exploded on the first call.""" - - def test_seen_tiers_destructure_accepts_canonical_tuple(self): - """Locks the ``for _, sxp, fxp, _, _, _, _, _, _, _, rname, rtrack in - variants:`` loop near the start of ``_run_gen_missions``.""" - variants = [ - _make_variant(success_xp=100, failure_xp=-50, rank_name="Rookie", rep_track="Security"), - _make_variant(success_xp=200, failure_xp=-100, rank_name="Junior", rep_track="Security"), - _make_variant(success_xp=300, failure_xp=-150, rank_name="Member", rep_track="Contractor"), - ] - seen_tiers: list[tuple[int, int, str, str]] = [] - for _, sxp, fxp, _, _, _, _, _, _, _, rname, rtrack in variants: - tier = (sxp, fxp, rname, rtrack) - if tier not in seen_tiers: - seen_tiers.append(tier) - assert seen_tiers == [ - (100, -50, "Rookie", "Security"), - (200, -100, "Junior", "Security"), - (300, -150, "Member", "Contractor"), - ] - - def test_desc_variants_destructure_accepts_canonical_tuple(self): - """Locks the ``for _, vsxp, vfxp, _, vflags, vspawns, vdiff, vhas_bp, - vbp_chance, vbp_variant, vrname, vrtrack in desc_variants:`` - aggregator loop.""" - desc_variants = [ - _make_variant( - success_xp=500, - contract_flags=["Starter"], - spawns={"hostile": {"Pirates": 4}}, - contract_difficulty="2", - contract_has_bp=True, - contract_bp_chance=0.25, - contract_bp_variant="hard", - rank_name="Rookie", - rep_track="Security", - ) - ] - rows: list[tuple] = [] - for _, vsxp, vfxp, _, vflags, vspawns, vdiff, vhas_bp, vbp_chance, vbp_variant, vrname, vrtrack in desc_variants: - rows.append((vsxp, vflags, vspawns, vdiff, vhas_bp, vbp_chance, vbp_variant, vrname, vrtrack)) - assert rows == [ - (500, ["Starter"], {"hostile": {"Pirates": 4}}, "2", True, 0.25, "hard", "Rookie", "Security") - ] - - def test_indexed_access_positions(self): - """v[3] = desc_key, v[7] = contract_has_bp, v[0] = system_name, - v[10] = rank_name, v[11] = rep_track — these are the exact ``v[N]`` - indices read inside _run_gen_missions (the destructure-free access - paths). Locks each.""" - v = _make_variant( + missions, _, _, _ = gen_module.scan_contract_generators( + contractgen_dir, + reputation_lookup={"gain": 500, "loss": -250}, + blueprint_pools={"pool": ["Some Armor"]}, + standings_lookup={"standing": "Rookie"}, + standing_track_lookup={"standing": "Security"}, + ) + + shared = dict( system_name="Pyro", - desc_key="my_desc", + success_xp=500, + failure_xp=-250, + contract_flags=["Starter"], + spawns={ + **gen_module._empty_spawn_breakdown(), + gen_module.SPAWN_HOSTILE: {"Pirates": 4}, + }, + contract_difficulty="Combat 5/7", contract_has_bp=True, + contract_bp_chance=0.25, + contract_bp_variant="Pyro_Intro_Test", rank_name="Rookie", rep_track="Security", ) - assert v[0] == "Pyro" - assert v[3] == "my_desc" - assert v[7] is True - assert v[10] == "Rookie" - assert v[11] == "Security" + assert missions["my_title"] == [_make_variant(desc_key="my_desc", **shared)] + assert missions["sub_title"] == [_make_variant(desc_key="sub_desc", **shared)] + assert missions["sub_no_desc_title"] == [_make_variant(desc_key="my_desc", **shared)] From e71811a6b9a92f3f7a3ee6d437f061f52ed8c690 Mon Sep 17 00:00:00 2001 From: Stealrull <93751735+Stealrull@users.noreply.github.com> Date: Fri, 6 Nov 2026 21:08:00 +1000 Subject: [PATCH 06/32] Stop contract scan errors passing silently (#553) scan_contract_generators built each variant with two calls passing all 12 fields by position, and logged a contract that raised at DEBUG, which a normal run does not show. On a LIVE cache, one wrong call skipped all 1,809 contracts it reached and lost 787 entries of mission_rewards_enhancements.ini, and the run still finished as a success. A variant is now built by keyword, and a sub-contract's comes from its contract's with _replace. A skipped contract gets a WARNING naming it and the error. More contracts skipped than giving a variant, or a walk that stops before any gives one, raise ContractScanFailed, so the run fails with the old INIs whole and the set not certified. The worker matches that class by name and words its box with two new strings, in all 12 languages. So game data cannot do that, a spawn count written as 4.0 now reads as 4, and any other that is not whole counts as 0 with a WARNING: 52% of LIVE's released contracts read a count. Older gaps in the same scan are fixed too, none hit on LIVE today: a bad count in a handler's own spawns ended the whole scan, a skipped contract left its blueprints and items behind, and PointsToAward="inf" lost the contract. On a LIVE cache, every file the generator writes matches the unchanged code byte for byte across 11 settings combinations. Built on #552, whose test wording this updates. Fixes #553 Co-Authored-By: Claude Opus 5.5 --- languages/chinese/ui.json | 8 + languages/chinese_traditional/ui.json | 8 + languages/english/ui.json | 8 + languages/french/ui.json | 8 + languages/german/ui.json | 8 + languages/italian/ui.json | 8 + languages/japanese/ui.json | 8 + languages/korean/ui.json | 8 + languages/portuguese_br/ui.json | 8 + languages/russian/ui.json | 8 + languages/spanish/ui.json | 8 + languages/turkish/ui.json | 8 + scripts/CLAUDE.md | 3 + scripts/generate_enhancements_ini.py | 189 ++++++++-- src/gui/CLAUDE.md | 2 +- src/gui/workers.py | 15 +- tests/CLAUDE.md | 3 +- tests/test_contract_scan.py | 511 ++++++++++++++++++++++++++ tests/test_generator_single_pass.py | 15 + tests/test_mission_variant_tuple.py | 19 +- 20 files changed, 802 insertions(+), 51 deletions(-) create mode 100644 tests/test_contract_scan.py diff --git a/languages/chinese/ui.json b/languages/chinese/ui.json index 918b3d6b..1bb3ba00 100644 --- a/languages/chinese/ui.json +++ b/languages/chinese/ui.json @@ -2992,6 +2992,14 @@ "ht": "", "at": "Smart Citizen 无法在以下位置使用工作文件夹:\n{folder}\n\n{reason}\n\n提取未开始,因此未做任何更改。" }, + "contract_scan_failed": { + "ht": "", + "at": "增强内容生成已停止,因为 {total} 份任务合约中有 {skipped} 份无法读取。你现有的增强内容保持不变。请报告此问题,这通常是 Smart Citizen 的错误。\n\n第一份合约:\n{first}" + }, + "contract_scan_stopped": { + "ht": "", + "at": "增强内容生成已停止,因为任务合约扫描因错误而中断。你现有的增强内容保持不变。请报告此问题,这通常是 Smart Citizen 的错误。\n\n错误:\n{first}" + }, "cache_folder_refused": { "ht": "", "at": "Windows 拒绝了 Smart Citizen 在以下位置创建 DataForge 缓存的权限:\n{folder}\n\n提取未开始,因此未做任何更改。" diff --git a/languages/chinese_traditional/ui.json b/languages/chinese_traditional/ui.json index 83286d55..2c70271e 100644 --- a/languages/chinese_traditional/ui.json +++ b/languages/chinese_traditional/ui.json @@ -2992,6 +2992,14 @@ "ht": "", "at": "Smart Citizen 無法在以下位置使用工作資料夾:\n{folder}\n\n{reason}\n\n提取未開始,因此未做任何更改。" }, + "contract_scan_failed": { + "ht": "", + "at": "增強內容生成已停止,因為 {total} 份任務合約中有 {skipped} 份無法讀取。你現有的增強內容保持不變。請回報此問題,這通常是 Smart Citizen 的錯誤。\n\n第一份合約:\n{first}" + }, + "contract_scan_stopped": { + "ht": "", + "at": "增強內容生成已停止,因為任務合約掃描因錯誤而中斷。你現有的增強內容保持不變。請回報此問題,這通常是 Smart Citizen 的錯誤。\n\n錯誤:\n{first}" + }, "cache_folder_refused": { "ht": "", "at": "Windows 拒絕了 Smart Citizen 在以下位置建立 DataForge 快取的權限:\n{folder}\n\n提取未開始,因此未做任何更改。" diff --git a/languages/english/ui.json b/languages/english/ui.json index 5fdd0359..5f837ac0 100644 --- a/languages/english/ui.json +++ b/languages/english/ui.json @@ -2992,6 +2992,14 @@ "ht": "Smart Citizen could not use a working folder in:\n{folder}\n\n{reason}\n\nThe extraction did not start, so nothing was changed.", "at": "" }, + "contract_scan_failed": { + "ht": "Enhancement generation stopped because {skipped} of {total} contracts could not be read. Your existing enhancements are unchanged. Please report this, as it is usually a fault in Smart Citizen.\n\nThe first contract:\n{first}", + "at": "" + }, + "contract_scan_stopped": { + "ht": "Enhancement generation stopped because the contract scan ended with an error. Your existing enhancements are unchanged. Please report this, as it is usually a fault in Smart Citizen.\n\nThe error:\n{first}", + "at": "" + }, "cache_folder_refused": { "ht": "Windows denied Smart Citizen permission to create the DataForge cache in:\n{folder}\n\nThe extraction did not start, so nothing was changed.", "at": "" diff --git a/languages/french/ui.json b/languages/french/ui.json index 9e4e31c7..1d1ae7fd 100644 --- a/languages/french/ui.json +++ b/languages/french/ui.json @@ -2992,6 +2992,14 @@ "ht": "", "at": "Smart Citizen n'a pas pu utiliser un dossier de travail dans :\n{folder}\n\n{reason}\n\nL'extraction n'a pas démarré, donc rien n'a été modifié." }, + "contract_scan_failed": { + "ht": "", + "at": "La génération des enrichissements s'est arrêtée, car {skipped} contrats sur {total} n'ont pas pu être lus. Vos enrichissements existants n'ont pas changé. Merci de le signaler : il s'agit généralement d'une erreur de Smart Citizen.\n\nPremier contrat :\n{first}" + }, + "contract_scan_stopped": { + "ht": "", + "at": "La génération des enrichissements s'est arrêtée, car la lecture des contrats a échoué. Vos enrichissements existants n'ont pas changé. Merci de le signaler : il s'agit généralement d'une erreur de Smart Citizen.\n\nErreur :\n{first}" + }, "cache_folder_refused": { "ht": "", "at": "Windows a refusé à Smart Citizen l'autorisation de créer le cache DataForge dans :\n{folder}\n\nL'extraction n'a pas démarré, donc rien n'a été modifié." diff --git a/languages/german/ui.json b/languages/german/ui.json index 125cff5e..c58ebdfe 100644 --- a/languages/german/ui.json +++ b/languages/german/ui.json @@ -2992,6 +2992,14 @@ "ht": "", "at": "Smart Citizen konnte in diesem Ordner keinen Arbeitsordner verwenden:\n{folder}\n\n{reason}\n\nDie Extraktion wurde nicht gestartet, daher wurde nichts geändert." }, + "contract_scan_failed": { + "ht": "", + "at": "Die Erzeugung der Erweiterungen wurde gestoppt, weil {skipped} von {total} Aufträgen nicht gelesen werden konnten. Deine vorhandenen Erweiterungen bleiben unverändert. Bitte melde das, denn meist liegt es an einem Fehler in Smart Citizen.\n\nDer erste Auftrag:\n{first}" + }, + "contract_scan_stopped": { + "ht": "", + "at": "Die Erzeugung der Erweiterungen wurde gestoppt, weil das Lesen der Aufträge mit einem Fehler abgebrochen ist. Deine vorhandenen Erweiterungen bleiben unverändert. Bitte melde das, denn meist liegt es an einem Fehler in Smart Citizen.\n\nDer Fehler:\n{first}" + }, "cache_folder_refused": { "ht": "", "at": "Windows hat Smart Citizen die Berechtigung verweigert, den DataForge-Cache in diesem Ordner zu erstellen:\n{folder}\n\nDie Extraktion wurde nicht gestartet, daher wurde nichts geändert." diff --git a/languages/italian/ui.json b/languages/italian/ui.json index 5bb04f1c..a5c24eb6 100644 --- a/languages/italian/ui.json +++ b/languages/italian/ui.json @@ -2992,6 +2992,14 @@ "ht": "", "at": "Smart Citizen non è riuscito a usare una cartella di lavoro in:\n{folder}\n\n{reason}\n\nL'estrazione non è stata avviata, quindi non è stato modificato nulla." }, + "contract_scan_failed": { + "ht": "", + "at": "La generazione dei miglioramenti si è interrotta perché {skipped} contratti su {total} non sono stati letti. I miglioramenti esistenti non sono cambiati. Segnalalo, perché di solito è un errore di Smart Citizen.\n\nIl primo contratto:\n{first}" + }, + "contract_scan_stopped": { + "ht": "", + "at": "La generazione dei miglioramenti si è interrotta perché la lettura dei contratti ha dato un errore. I miglioramenti esistenti non sono cambiati. Segnalalo, perché di solito è un errore di Smart Citizen.\n\nL'errore:\n{first}" + }, "cache_folder_refused": { "ht": "", "at": "Windows ha negato a Smart Citizen l'autorizzazione a creare la cache DataForge in:\n{folder}\n\nL'estrazione non è stata avviata, quindi non è stato modificato nulla." diff --git a/languages/japanese/ui.json b/languages/japanese/ui.json index 4668a06c..1fe9fcbd 100644 --- a/languages/japanese/ui.json +++ b/languages/japanese/ui.json @@ -2992,6 +2992,14 @@ "ht": "", "at": "Smart Citizen が次の場所で作業フォルダーを使用できませんでした:\n{folder}\n\n{reason}\n\n抽出は開始されなかったため、何も変更されていません。" }, + "contract_scan_failed": { + "ht": "", + "at": "{total} 件の契約のうち {skipped} 件を読み取れなかったため、拡張の生成を停止しました。既存の拡張は変更されていません。通常は Smart Citizen の不具合ですので、報告してください。\n\n最初の契約:\n{first}" + }, + "contract_scan_stopped": { + "ht": "", + "at": "契約の読み取りがエラーで止まったため、拡張の生成を停止しました。既存の拡張は変更されていません。通常は Smart Citizen の不具合ですので、報告してください。\n\nエラー:\n{first}" + }, "cache_folder_refused": { "ht": "", "at": "Windows によって、次の場所に DataForge キャッシュを作成するアクセス許可が Smart Citizen に対して拒否されました:\n{folder}\n\n抽出は開始されなかったため、何も変更されていません。" diff --git a/languages/korean/ui.json b/languages/korean/ui.json index bbde8269..fbcb53ac 100644 --- a/languages/korean/ui.json +++ b/languages/korean/ui.json @@ -2992,6 +2992,14 @@ "ht": "", "at": "Smart Citizen이 다음 위치에서 작업 폴더를 사용할 수 없었습니다:\n{folder}\n\n{reason}\n\n추출이 시작되지 않았으므로 아무것도 변경되지 않았습니다." }, + "contract_scan_failed": { + "ht": "", + "at": "계약 {total}개 중 {skipped}개를 읽을 수 없어 강화 생성이 중지되었습니다. 기존 강화는 변경되지 않았습니다. 대개 Smart Citizen의 오류이므로 신고해 주세요.\n\n첫 번째 계약:\n{first}" + }, + "contract_scan_stopped": { + "ht": "", + "at": "계약을 읽는 중 오류가 발생해 강화 생성이 중지되었습니다. 기존 강화는 변경되지 않았습니다. 대개 Smart Citizen의 오류이므로 신고해 주세요.\n\n오류:\n{first}" + }, "cache_folder_refused": { "ht": "", "at": "Windows가 다음 위치에 DataForge 캐시를 만드는 권한을 Smart Citizen에 거부했습니다:\n{folder}\n\n추출이 시작되지 않았으므로 아무것도 변경되지 않았습니다." diff --git a/languages/portuguese_br/ui.json b/languages/portuguese_br/ui.json index ef938354..df0fe2ac 100644 --- a/languages/portuguese_br/ui.json +++ b/languages/portuguese_br/ui.json @@ -2992,6 +2992,14 @@ "ht": "", "at": "O Smart Citizen não conseguiu usar uma pasta de trabalho em:\n{folder}\n\n{reason}\n\nA extração não foi iniciada, então nada foi alterado." }, + "contract_scan_failed": { + "ht": "", + "at": "A geração de aprimoramentos foi interrompida porque {skipped} de {total} contratos não puderam ser lidos. Seus aprimoramentos atuais não foram alterados. Informe isso, pois costuma ser um erro do Smart Citizen.\n\nPrimeiro contrato:\n{first}" + }, + "contract_scan_stopped": { + "ht": "", + "at": "A geração de aprimoramentos foi interrompida porque a leitura dos contratos parou com um erro. Seus aprimoramentos atuais não foram alterados. Informe isso, pois costuma ser um erro do Smart Citizen.\n\nO erro:\n{first}" + }, "cache_folder_refused": { "ht": "", "at": "O Windows negou ao Smart Citizen a permissão para criar o cache do DataForge em:\n{folder}\n\nA extração não foi iniciada, então nada foi alterado." diff --git a/languages/russian/ui.json b/languages/russian/ui.json index aaa19211..5f1d1992 100644 --- a/languages/russian/ui.json +++ b/languages/russian/ui.json @@ -2992,6 +2992,14 @@ "ht": "", "at": "Smart Citizen не удалось использовать рабочую папку в:\n{folder}\n\n{reason}\n\nИзвлечение не началось, поэтому ничего не изменено." }, + "contract_scan_failed": { + "ht": "", + "at": "Генерация улучшений остановлена: не удалось прочитать контракты ({skipped} из {total}). Ваши текущие улучшения не изменены. Сообщите об этом, так как обычно это ошибка Smart Citizen.\n\nПервый контракт:\n{first}" + }, + "contract_scan_stopped": { + "ht": "", + "at": "Генерация улучшений остановлена: чтение контрактов прервалось с ошибкой. Ваши текущие улучшения не изменены. Сообщите об этом, так как обычно это ошибка Smart Citizen.\n\nОшибка:\n{first}" + }, "cache_folder_refused": { "ht": "", "at": "Windows не разрешила Smart Citizen создать кэш DataForge в:\n{folder}\n\nИзвлечение не началось, поэтому ничего не изменено." diff --git a/languages/spanish/ui.json b/languages/spanish/ui.json index 65759fa3..a72df183 100644 --- a/languages/spanish/ui.json +++ b/languages/spanish/ui.json @@ -2992,6 +2992,14 @@ "ht": "", "at": "Smart Citizen no pudo usar una carpeta de trabajo en:\n{folder}\n\n{reason}\n\nLa extracción no se inició, así que no se cambió nada." }, + "contract_scan_failed": { + "ht": "", + "at": "La generación de mejoras se detuvo porque no se pudieron leer {skipped} de {total} contratos. Tus mejoras actuales no han cambiado. Infórmalo, ya que suele ser un error de Smart Citizen.\n\nPrimer contrato:\n{first}" + }, + "contract_scan_stopped": { + "ht": "", + "at": "La generación de mejoras se detuvo porque la lectura de contratos falló. Tus mejoras actuales no han cambiado. Infórmalo, ya que suele ser un error de Smart Citizen.\n\nEl error:\n{first}" + }, "cache_folder_refused": { "ht": "", "at": "Windows le denegó a Smart Citizen el permiso para crear la caché de DataForge en:\n{folder}\n\nLa extracción no se inició, así que no se cambió nada." diff --git a/languages/turkish/ui.json b/languages/turkish/ui.json index 8b20ae30..59992db6 100644 --- a/languages/turkish/ui.json +++ b/languages/turkish/ui.json @@ -2992,6 +2992,14 @@ "ht": "", "at": "Smart Citizen şu konumda bir çalışma klasörü kullanamadı:\n{folder}\n\n{reason}\n\nÇıkarma işlemi başlamadı, bu yüzden hiçbir şey değiştirilmedi." }, + "contract_scan_failed": { + "ht": "", + "at": "{total} sözleşmeden {skipped} tanesi okunamadığı için geliştirme oluşturma durduruldu. Mevcut geliştirmeleriniz değişmedi. Genellikle Smart Citizen hatası olduğundan lütfen bildirin.\n\nİlk sözleşme:\n{first}" + }, + "contract_scan_stopped": { + "ht": "", + "at": "Sözleşmeler okunurken bir hata oluştuğu için geliştirme oluşturma durduruldu. Mevcut geliştirmeleriniz değişmedi. Genellikle Smart Citizen hatası olduğundan lütfen bildirin.\n\nHata:\n{first}" + }, "cache_folder_refused": { "ht": "", "at": "Windows, Smart Citizen'ın şu konumda DataForge önbelleğini oluşturma iznini reddetti:\n{folder}\n\nÇıkarma işlemi başlamadı, bu yüzden hiçbir şey değiştirilmedi." diff --git a/scripts/CLAUDE.md b/scripts/CLAUDE.md index 143ded1e..eef3f3af 100644 --- a/scripts/CLAUDE.md +++ b/scripts/CLAUDE.md @@ -35,6 +35,9 @@ Closing Smart Citizen stops a running generation. `main()` takes `should_cancel` ### Recco Battaglia's Scan/Mining contracts — "[RS ####]" mission-title tag (4.9+) `Battaglia_RPT_Scan_*` / `Battaglia_RPT_ScanMine_*` contracts each target 1-3 mineable ores via sibling `MissionProperty` overrides with `extendedTextToken="ResourceType"`/`"ResourceType2"`/`"ResourceType3"` pointing at a `mineabletype_primary_` loc key. CIG doesn't expose a per-ore "RS" (resonance-signature) number anywhere in DataForge — checked the full PTU 4.9 cache for both the literal values and any per-mineable stat registry, found neither — so `MINEABLE_RS_VALUES` is a curated table sourced from community reference data (MrKraken/StarStrings, `ptu/4.9` branch `mining.ini`, cross-checked against starminersdepot.com's Alpha 4.8 scanner table). `_battaglia_mineable_rs_in()` scans one contractgen XML for matching titles and formats e.g. `[RS 3600/4300/3900]` (multiple ores slash-joined, in ResourceType/2/3 order) — every contract variant sharing a title targets the same ore set, so first-variant-wins. A generation runs it on each file of the contractgen parse it shares with `scan_contract_generators` (see the single-pass entry below), so there is no separate pre-pass. The tag itself is a hardcoded title-loop append gated by the `"rs"` key in `AppSettings.MISSION_TITLE_TAG_KEYS`/`_MISSION_TITLE_TAG_SETTING` (the "General Tags" checkbox pattern — same shape as `[BP]`/`[ACE]`/`[…Rep]`, not the `TagConfig`/`render_tag()` element system). Locked by `tests/test_battaglia_mineable_rs.py`, which reads the tags off a `_run_gen_missions` run. +### A contract the scan cannot read is skipped with a WARNING, and a scan that skips most fails +`scan_contract_generators` reads each contract in its own `try`: one that raises leaves nothing behind, since its blueprints and items are recorded only once its variant is built, and the scan goes on. The line names the contract and the error type at WARNING, as the ship and component generators log a bad file, and not at ERROR, which would open the error box over one odd contract. It used to be at DEBUG, which a normal run does not show (the app logs at INFO unless it starts with `--debug` or `LOG_LEVEL=DEBUG`), so a bug in that block lost everything quietly: one wrong `ContractVariant(...)` call, tried on LIVE, skipped all 1,809 contracts it reached and 787 entries of `mission_rewards_enhancements.ini`, and the run still finished as a success. So more contracts skipped than giving a variant raise `ContractScanFailed` once the walk ends, as does a walk that stops on an error before any contract gave one (a walk that stops later keeps what it read, with its WARNING, as before), which is meant to take a bug here: failing every category of the run is the deliberate trade. `main()` has written no INI yet, so the old set stays whole, the worker leaves `.dataforge_stamp` empty and the next start offers to generate again. The script stays free of `tr()`: the worker matches the class by name, as it does `GenerationStopped`, and words its box with `extract.contract_scan_failed`, or `extract.contract_scan_stopped` for a walk that stopped (a total of 0), so keep the name and its fields. A listing that fails part-way stops the walk too, and `_run_gen_missions` still ends the run with that error itself. Game data should not be able to do it. A spawn count that is not a whole number does not raise (`_spawn_count`): one written as `4.0` reads as 4, and anything else counts as 0 with a WARNING, so the mission loses that spawn line and keeps its variant, because 52% of LIVE's released contracts read a count, and one change to how counts are written would otherwise stop every generation. A failure in a handler's own spawns skips those spawns with a WARNING of its own and keeps its contracts: it used to end the whole scan, with the later files lost and the variants unsorted. A variant is built by keyword, and a sub-contract's comes from its contract's with `_replace(desc_key=...)`, so two values cannot swap and the two cannot drift apart. `tests/test_contract_scan.py` and `tests/test_mission_variant_tuple.py` lock this. + ### Mining Compendium journal — per-ore Base Resource Signature line (2.2.0) `MINEABLE_RS_VALUES` (see the Battaglia entry above for sourcing) has a second consumer: the "Mining Fundamentals #2: Where to Mine" journal entry (`Journal_General_Mining_Compendium_Content`), reformatted by `scan_crafting_blueprints`'s per-mineral block builder into `Name` / `Locations:` / `Used To Craft:` blocks. Each ore's block now gets a `Base Resource Signature: {value}` line (inserted right after the name header, before Locations) whenever `MINEABLE_RS_VALUES` has a confirmed value for it; ores without one (the journal lists 35, the table covers 26) are left unchanged, no placeholder. `_MINING_COMPENDIUM_ORE_ALIASES` bridges the one known spelling mismatch between the journal's own prose and the DataForge loc-key convention (`"Savrilium"` in-journal vs `mineabletype_primary_savrillium`). Locked by `TestMiningCompendiumRSTagging` in `tests/test_commodity_journal.py`. diff --git a/scripts/generate_enhancements_ini.py b/scripts/generate_enhancements_ini.py index ed1e3f08..a374a575 100644 --- a/scripts/generate_enhancements_ini.py +++ b/scripts/generate_enhancements_ini.py @@ -2881,6 +2881,27 @@ def _within_excluded_subtree(node: ET.Element, scope: ET.Element, return False +def _spawn_count(value: str, group: str, attribute: str) -> int: + """*value* as a whole number, also when it is written as "4.0". Anything else + counts as 0 with a WARNING, so a mission loses that spawn line and keeps its + variant: over half of LIVE's contracts read a count, and one change to how counts + are written would otherwise fail every generation (see ContractScanFailed). + """ + try: + return int(value) + except ValueError: + pass + try: + if float(value).is_integer(): + return int(float(value)) + except ValueError: + pass + logger.warning( + f"Spawn count is not a whole number, counted as 0: {group} {attribute}={value!r}" + ) + return 0 + + def _extract_spawn_counts(element: ET.Element, exclude_within: str | tuple[str, ...] | None = None) -> SpawnBreakdown: """Extract a per-bucket per-label breakdown of spawn descriptions. @@ -2916,7 +2937,10 @@ def _extract_spawn_counts(element: ET.Element, continue name = sg.get("Name", "") ships = sg.findall(".//SpawnDescription_Ship") - total = sum(int(s.get("concurrentAmount", "0")) for s in ships) + total = sum( + _spawn_count(s.get("concurrentAmount", "0"), name, "concurrentAmount") + for s in ships + ) if total <= 0: continue # Turret ship-groups are reported separately by _extract_turret_info; @@ -2941,11 +2965,13 @@ def _extract_spawn_counts(element: ET.Element, for auto in auto_settings: max_spawns = auto.get("maxSpawns", "0") if max_spawns != "-1": - total_npcs += max(int(max_spawns), 0) + total_npcs += max(_spawn_count(max_spawns, name, "maxSpawns"), 0) else: max_concurrent = auto.get("maxConcurrentSpawns", "0") if max_concurrent != "-1": - total_npcs += max(int(max_concurrent), 0) + total_npcs += max( + _spawn_count(max_concurrent, name, "maxConcurrentSpawns"), 0 + ) if total_npcs <= 0: m = re.search(r"x\s*(\d+)", name) @@ -4103,12 +4129,12 @@ class ContractVariant(NamedTuple): Was a bare positional tuple that grew to 12 fields (10 -> 11 for rank_name in 1.4.2, 11 -> 12 for rep_track in 2.2.0's reputation-track - change, #161); every growth risked an off-by-one at one of the many - ``v[N]`` reads and destructures below. A NamedTuple is still a plain - tuple — positional unpacking, indexing, and ``len()`` all still work - exactly as before (see tests/test_mission_variant_tuple.py, which locks - that contract) — so this is purely a self-documenting rename at the call - sites that read fields by name instead of by position. + change, #161); every growth risked an off-by-one at one of the ``v[N]`` + reads and destructures it had then. Now scan_contract_generators builds + it by keyword and _run_gen_missions reads it by field name, so a field + that moves or a new one cannot put a value in the wrong field. A + NamedTuple is still a plain tuple, and tests/test_mission_variant_tuple.py + locks its fields and their order. """ system_name: str success_xp: int @@ -4171,6 +4197,13 @@ def scan_contract_generators( reward pools sharing the same title/system. - mission_items: dict mapping title_key → list of reward item display names Sorted by system name for consistent output. + + A contract that raises is skipped with a WARNING and leaves nothing behind. + Raises ContractScanFailed when more contracts were skipped than gave a variant, + or when the walk stopped on an error before any gave one. Each is meant to take + a bug here, and game data should not be able to do it (see _spawn_count). Failing + every category of the run is the deliberate trade: a set without most of its + contracts must not be certified. """ if not contractgen_dir.exists(): return {}, {}, {}, {} @@ -4196,6 +4229,10 @@ def scan_contract_generators( mission_blueprints: dict[str, dict[str, dict]] = {} mission_bp_chance: dict[str, float] = {} mission_items: dict[str, list[str]] = {} + walk_error = "" + skipped_contracts = 0 + built_contracts = 0 + first_skip = "" # Build template lookup for contracts that inherit title/desc from templates if template_lookup is None: @@ -4275,9 +4312,18 @@ def _extract_system(name: str, fallback: str) -> str: # not a union of every sibling contract's roster (#186) — # including the one-time intro mission's own roster, which # uses the plain tag, not . - handler_spawns = _extract_spawn_counts( - handler, exclude_within=("CareerContract", "Contract") - ) + try: + handler_spawns = _extract_spawn_counts( + handler, exclude_within=("CareerContract", "Contract") + ) + except Exception as e: + # Outside the per-contract try, so a failure here used to end + # the whole scan. The contracts' own spawns still count. + logger.warning( + f"Skipped the shared spawns of handler {debug_name or '?'}: " + f"{type(e).__name__}: {e}" + ) + handler_spawns = _empty_spawn_breakdown() contracts = handler.findall(contract_xpath) @@ -4372,6 +4418,7 @@ def _extract_system(name: str, fallback: str) -> str: # the identical pool set. contract_has_bp = False contract_bp_chance = 0.0 + first_bp_chance = None contract_bp_variant = contract.get("debugName", "") # Accumulate this contract's own BlueprintRewards # locally, grouped by pool_label, before merging @@ -4417,18 +4464,13 @@ def _extract_system(name: str, fallback: str) -> str: contract_bp_chance = float(bp_elem.get("chance", "1")) except (ValueError, TypeError): contract_bp_chance = 1.0 - if title_key not in mission_bp_chance: - mission_bp_chance[title_key] = contract_bp_chance - for pool_label, (label_uuids, label_items) in contract_pools_by_label.items(): - _merge_blueprint_pool( - mission_blueprints, title_key, system_name, - tuple(sorted(label_uuids)), label_items, pool_label, - ) + if first_bp_chance is None: + first_bp_chance = contract_bp_chance # Extract item rewards null_uuid = "00000000-0000-0000-0000-000000000000" + item_names = [] if entity_names: - item_names = [] for item_elem in contract.findall(".//ContractResult_Item"): ec = item_elem.get("entityClass", "") if ec and ec != null_uuid and ec in entity_names: @@ -4441,8 +4483,6 @@ def _extract_system(name: str, fallback: str) -> str: name = entity_names[ec] if name not in item_names: item_names.append(name) - if item_names and title_key not in mission_items: - mission_items[title_key] = item_names # Extract XP from ContractResult_LegacyReputation blocks # First block with positive XP = success, first with negative = failure @@ -4476,7 +4516,7 @@ def _extract_system(name: str, fallback: str) -> str: continue try: val = int(float(points)) - except (ValueError, TypeError): + except (ValueError, TypeError, OverflowError): continue if val > 0: success_xp = val @@ -4515,9 +4555,40 @@ def _extract_system(name: str, fallback: str) -> str: rep_track = standing_track_lookup.get(min_standing, "") if min_standing else "" # Add all missions (not just those with XP/blueprint data) + variant = ContractVariant( + system_name=system_name, + success_xp=success_xp, + failure_xp=failure_xp, + desc_key=desc_key, + flags=contract_flags, + spawns=spawns, + difficulty=contract_difficulty, + has_bp=contract_has_bp, + bp_chance=contract_bp_chance, + bp_variant=contract_bp_variant, + rank_name=rank_name, + rep_track=rep_track, + ) + # After the call: a title with no variants would read + # as every variant having a blueprint (all([]) is True). if title_key not in missions: missions[title_key] = [] - missions[title_key].append(ContractVariant(system_name, success_xp, failure_xp, desc_key, contract_flags, spawns, contract_difficulty, contract_has_bp, contract_bp_chance, contract_bp_variant, rank_name, rep_track)) + missions[title_key].append(variant) + built_contracts += 1 + + # Only now, so a contract that raised above leaves no + # blueprints or items under its title. + for pool_label, (label_uuids, label_items) in ( + contract_pools_by_label.items() + ): + _merge_blueprint_pool( + mission_blueprints, title_key, system_name, + tuple(sorted(label_uuids)), label_items, pool_label, + ) + if first_bp_chance is not None and title_key not in mission_bp_chance: + mission_bp_chance[title_key] = first_bp_chance + if item_names and title_key not in mission_items: + mission_items[title_key] = item_names # Sub-contracts override title/desc but inherit # everything else from the parent contract. @@ -4532,7 +4603,9 @@ def _extract_system(name: str, fallback: str) -> str: sub_desc = "" if sub_title not in missions: missions[sub_title] = [] - missions[sub_title].append(ContractVariant(system_name, success_xp, failure_xp, sub_desc or desc_key, contract_flags, spawns, contract_difficulty, contract_has_bp, contract_bp_chance, contract_bp_variant, rank_name, rep_track)) + missions[sub_title].append( + variant._replace(desc_key=sub_desc or desc_key) + ) if contract_has_bp and sub_title != title_key: # Reuse contract_pools_by_label (already # scanned above) instead of re-iterating @@ -4549,7 +4622,13 @@ def _extract_system(name: str, fallback: str) -> str: mission_bp_chance[sub_title] = contract_bp_chance except Exception as e: - logger.debug(f"Skipped contract {contract.get('debugName', '?')}: {e}") + # WARNING, as the ship and component generators log a bad + # file. A normal run does not show DEBUG, and a bug in this + # block loses every contract's variants, not just one's. + skip = f"{contract.get('debugName', '?')}: {type(e).__name__}: {e}" + logger.warning(f"Skipped contract {skip}") + skipped_contracts += 1 + first_skip = first_skip or skip # Sort variants by system name for consistent output (Stanton first, then others alphabetically) for title_key in missions: @@ -4557,6 +4636,16 @@ def _extract_system(name: str, fallback: str) -> str: except Exception as e: logger.warning(f"Error scanning contract generators: {e}") + walk_error = f"{type(e).__name__}: {e}" + + # Outside the try above, which would only log it. main() has written no INI yet, + # so the old set stays whole and the worker leaves its stamp empty. + if skipped_contracts > built_contracts: + raise ContractScanFailed( + skipped_contracts, skipped_contracts + built_contracts, first_skip + ) + if walk_error and not built_contracts: + raise ContractScanFailed(0, 0, walk_error) logger.info(f"Contract generators: {len(missions)} missions, {len(mission_blueprints)} with blueprints, {len(mission_items)} with items") return missions, mission_blueprints, mission_bp_chance, mission_items @@ -4572,8 +4661,7 @@ def _calculated_reward_titles_in(root: ET.Element, titles: set[str]) -> None: (blueprint-only or item-only rewards), which is a separate, equally correct "nothing to show" case. - A separate scan rather than widening ContractVariant (its 12-field - width is locked by tests/test_mission_variant_tuple.py) for a value + A separate scan rather than a 13th ContractVariant field for a value only the end-of-run diagnostic needs. _run_gen_missions runs it on each root of the contractgen parse scan_contract_generators already makes, rather than walking the files again once a zero-XP title turns up @@ -7574,15 +7662,23 @@ def _contractgen_roots(xml_files): _contractgen_roots(_indexed_xml_files(contractgen_dir, xml_path_index, records)) if contractgen_dir.exists() else iter(()) ) - contractgen_missions, mission_blueprints, mission_bp_chance, mission_items = scan_contract_generators( - contractgen_dir, reputation_lookup, blueprint_pools, entity_names, - xml_path_index=xml_path_index, records_dir=records, - pool_names=pool_names, - standings_lookup=standings_lookup, - standing_track_lookup=standing_track_lookup, - template_lookup=_template_lookup, - roots=_roots, - ) + try: + contractgen_missions, mission_blueprints, mission_bp_chance, mission_items = ( + scan_contract_generators( + contractgen_dir, reputation_lookup, blueprint_pools, entity_names, + xml_path_index=xml_path_index, records_dir=records, + pool_names=pool_names, + standings_lookup=standings_lookup, + standing_track_lookup=standing_track_lookup, + template_lookup=_template_lookup, + roots=_roots, + ) + ) + except ContractScanFailed: + # A listing that failed part-way stops the scan too, and ends the run as itself. + if battaglia_error is not None: + raise battaglia_error + raise for _root in _roots: pass logger.info(f"Processed {len(contractgen_missions)} contract generator mission variants") @@ -8130,6 +8226,25 @@ class GenerationStopped(Exception): never checks for a stop once its write phase has begun.""" +class ContractScanFailed(Exception): + """More contracts were skipped than gave a variant, or the walk stopped on an + error before any gave one (a total of 0), so the run fails. + + EnhancementsGeneratorWorker matches it by class name, as it does + GenerationStopped, and words its own box from these fields in the selected + language, so keep the name and the fields. + """ + + def __init__(self, skipped: int, total: int, first: str): + super().__init__( + f"{skipped} of {total} contracts could not be read. The first: {first}" + if total else f"The contract scan stopped: {first}" + ) + self.skipped = skipped + self.total = total + self.first = first + + def main(base_ini_path: Path, forge_dir: Path | None = None, categories: set[str] | None = None, progress_callback: Optional[Callable[[int, int, str], None]] = None, diff --git a/src/gui/CLAUDE.md b/src/gui/CLAUDE.md index 28499d23..b83b4d86 100644 --- a/src/gui/CLAUDE.md +++ b/src/gui/CLAUDE.md @@ -62,7 +62,7 @@ A successful DataForge run's working folder is deleted after the run, in the bac A DataForge extraction started while such a delete still runs waits for it (perf proposal, the user's choice): `_run_dataforge_extraction` hands `DataForgeExtractWorker` the `done` event of every delete and sweep in `_working_folder_deleters` (`wait_for`), and the run waits for them on its own thread before `extract_dataforge`, so its sweep does not skip that folder and its free-space check does not count it as used. A delete lets go of its `WorkingFolder` before it sets `done`, so after one that failed with something other than an OSError (whose marker was never closed) the marker is closed by then and that sweep takes what is left. The GUI thread never waits, the dialog shows its busy bar with the extraction's label meanwhile (nothing is counted yet), and a Stop or a close ends the wait at once, as a stop. A crash or a kill during a delete leaves the working folder, with the old cache set aside in it, and before the perf proposal it was gone before generation began. `StaleWorkingFolderSweepWorker` sweeps such leftovers with pak_extractor's own `_sweep_stale_scratch` and `DATAFORGE_SCRATCH_PREFIX`, which leaves a folder whose marker is held, so a live run or delete in another window is safe: once a session at startup beside the active cache folder (`_sweep_stale_working_folders_at_startup`, first in `_continue_startup_after_update_gate`, reading the cache folder without creating it, #494, and skipped while an extraction runs, which sweeps there itself), and beside a queued old cache folder that `_cleanup_pending_old_cache` deletes or finds gone, where after a cache folder change no extraction sweeps any more (a cache in use gets none, because the extraction that just ran swept beside it, and a folder the check cannot resolve gets none, out of caution, as its cleanup is skipped and it cannot be compared with the cache in use). `_sweep_working_folders_in_background` starts it on its own thread, never the GUI's, and it first waits for the deletes already running (`after`), so it never works on a folder one of them is finishing. It is kept in `_working_folder_deleters` with them, so a new extraction waits for it the same way and it never runs beside an extraction's own sweep. Closing asks every running delete and sweep to stop (`requestInterruption`) before it waits: a sweep ends at once while it still waits for a delete, and before its next leftover folder while it sweeps (`_sweep_stale_scratch`'s `should_cancel`), finishing the folder in hand and leaving the rest for a later sweep, while a working folder's own delete has no stop and is waited for as before. A crash during the delete of a run that worked in `%TEMP%` (a OneDrive, network, removable or deep cache folder) still leaves that folder for the next DataForge extraction's sweep of `%TEMP%`. Anything that refuses to start work while a job runs, because the work closes the app (an update, Import Settings), must count `_working_folder_deleters` as well as the `_WORKERS_STOPPED_ON_CLOSE` workers: otherwise it can close the app while a delete or sweep runs, and the close then waits for it, hidden, for up to 180 s. `_closing_would_wait()` is that question (`_job_running() or bool(self._working_folder_deleters)`), asked by `_on_update_available` and `_handle_import_settings`, which say why in the status bar with the lock's own text (`extract.locked_while_job_runs`). `_job_running()` itself stays workers only: a generation is meant to run beside a delete, and a new DataForge run waits for one on its own thread, so neither asks the broader question. The two buttons are not locked for a delete (`_refresh_job_lock` follows the jobs, so it would need a refresh when each delete starts and ends), so the check at the relaunch is the whole guard there. `tests/test_extraction_stop.py` (`TestTheWorkerKeepsOnlyASuccessfulRunsFolder`, `TestTheWindowDeletesItInTheBackground`, `TestClosingWaitsForTheDelete`, `TestTheDeletesFinishedSlot`, `TestANewRunWaitsForTheDelete`, `TestLeftoversAreSweptAtStartup`), `tests/test_clear_cache.py` and `tests/test_pending_cache_cleanup.py` (`TestBesideTheBackgroundDelete`, `TestLeftoverWorkingFoldersBesideIt`) lock this. ### A cut-off generation is offered again at the next start (#476) -The startup check (`_check_enhancements_freshness`) used to ask only whether each enabled enhancement INI existed. A generation cut off part-way (the window closed, the app crashed, a write failed) left every INI there, each whole but the set a mix of old and new, and the next start stayed quiet. Now `EnhancementsGeneratorWorker` empties the `.dataforge_stamp` of the language it generates just before the generator's `main()` (after the patches and the module load, so a failure there keeps a good stamp) and writes it only once `main()` has returned. The check also asks when every file is there but `AppSettings.enhancements_built_from_current_dataforge()` says the set was not built from the current DataForge build: the last run was cut off or failed, or the set came from older game data. It asks that only when the DataForge cache is fresh (`_out_of_date_build_key`): an outdated cache is `_maybe_prompt_dataforge_refresh`'s question, so this one would stack on it or ask again after it was declined, and a cut-off extraction or a cache never extracted has nothing to generate from. In that case the category dialog offers every enabled category (less any the Enhancements tab has unticked and not saved), ticked with its plain label, under `extract.outdated_categories_body` ("out of date", with no count: the count's noun says "missing" in several languages). The answer holds for the rest of the session (`_enhancements_prompt_answer`, reset by a channel or data-folder change). Skip is never turned into a run by a later call, such as the one after a global.ini extraction, which used to run the pipeline silently, possibly a DataForge extraction the user had just declined. It is recorded before the dialog opens, because the dialog's event loop can deliver another load, and that call must start nothing. Nothing about the answer is kept on disk, so the question comes back at every start until the set is generated, like the missing-files question. The Generate button stays lit while a ticked category's set is not current, and the language switch (`_language_enhancements_fresh`, English included) uses the same test, so the three never disagree. A run that finishes re-reads this from disk (`set_operation_idle`), so one whose final stamp write failed (`set_enhancements_stamp` only logs that, and the run still succeeds) leaves the button lit over the set the next start will ask about. The worker also captures its folder, the build key and the tag fingerprint when its run starts: reading them when it ended let a channel, data-folder or cache-folder change during a hidden run stamp a set the run never wrote. Closing the window during a generation stops it at its next job boundary (see *Closing stops a running extraction* above), which after `main()` began is one of these cut-off runs, so the next start asks. `tests/test_enhancements_stale_startup.py` locks this. +The startup check (`_check_enhancements_freshness`) used to ask only whether each enabled enhancement INI existed. A generation cut off part-way (the window closed, the app crashed, a write failed) left every INI there, each whole but the set a mix of old and new, and the next start stayed quiet. Now `EnhancementsGeneratorWorker` empties the `.dataforge_stamp` of the language it generates just before the generator's `main()` (after the patches and the module load, so a failure there keeps a good stamp) and writes it only once `main()` has returned. The check also asks when every file is there but `AppSettings.enhancements_built_from_current_dataforge()` says the set was not built from the current DataForge build: the last run was cut off or failed, or the set came from older game data. It asks that only when the DataForge cache is fresh (`_out_of_date_build_key`): an outdated cache is `_maybe_prompt_dataforge_refresh`'s question, so this one would stack on it or ask again after it was declined, and a cut-off extraction or a cache never extracted has nothing to generate from. In that case the category dialog offers every enabled category (less any the Enhancements tab has unticked and not saved), ticked with its plain label, under `extract.outdated_categories_body` ("out of date", with no count: the count's noun says "missing" in several languages). The answer holds for the rest of the session (`_enhancements_prompt_answer`, reset by a channel or data-folder change). Skip is never turned into a run by a later call, such as the one after a global.ini extraction, which used to run the pipeline silently, possibly a DataForge extraction the user had just declined. It is recorded before the dialog opens, because the dialog's event loop can deliver another load, and that call must start nothing. Nothing about the answer is kept on disk, so the question comes back at every start until the set is generated, like the missing-files question. The Generate button stays lit while a ticked category's set is not current, and the language switch (`_language_enhancements_fresh`, English included) uses the same test, so the three never disagree. A run that finishes re-reads this from disk (`set_operation_idle`), so one whose final stamp write failed (`set_enhancements_stamp` only logs that, and the run still succeeds) leaves the button lit over the set the next start will ask about. The worker also captures its folder, the build key and the tag fingerprint when its run starts: reading them when it ended let a channel, data-folder or cache-folder change during a hidden run stamp a set the run never wrote. Closing the window during a generation stops it at its next job boundary (see *Closing stops a running extraction* above), which after `main()` began is one of these cut-off runs, so the next start asks. A run that skips most contracts (`ContractScanFailed`, see `scripts/CLAUDE.md`) ends the same way, and the worker matches that class by name and words its box with `extract.contract_scan_failed` or `extract.contract_scan_stopped`. `tests/test_enhancements_stale_startup.py` locks this. ### base.ini freshness follows Data.p4k's size (perf proposal) `_check_p4k_freshness` (startup, a channel switch, a data folder change) and the Config tab's `_refresh_p4k_status` (the Extract button's red nudge) compared Data.p4k's modified time with base.ini's own, and the RSI launcher's file verification moves that time on an ordinary game launch (#209), so playing was enough to be asked to extract global.ini again. Both now ask `pak_extractor.base_ini_is_fresh` (see `src/utils/CLAUDE.md`): a launch alone asks nothing and leaves the button alone, and a patch, which changes Data.p4k's size, still asks with the same `extract.p4k_prompt_p4k_newer` text. The language switch and Simple mode's Apply never checked base.ini and still don't; Simple mode meets the prompt only at startup, as Advanced does. `tests/test_base_ini_freshness.py` locks this. diff --git a/src/gui/workers.py b/src/gui/workers.py index aff0fa6e..58138314 100644 --- a/src/gui/workers.py +++ b/src/gui/workers.py @@ -801,8 +801,19 @@ def _on_progress(completed: int, total: int, message: str) -> None: self.stopped = True self.finished.emit(False) return - logger.exception(f"Enhancements generation failed: {e}") - self.error.emit(str(e)) + # The script stays free of tr(), and is loaded afresh for every run, + # so its ContractScanFailed is told apart by name and worded here. + if type(e).__name__ == "ContractScanFailed": + message = ( + tr("extract.contract_scan_failed", skipped=e.skipped, + total=e.total, first=e.first) + if e.total else tr("extract.contract_scan_stopped", first=e.first) + ) + logger.exception(message) + else: + message = str(e) + logger.exception(f"Enhancements generation failed: {e}") + self.error.emit(message) self.finished.emit(False) diff --git a/tests/CLAUDE.md b/tests/CLAUDE.md index a7f65bff..5958110e 100644 --- a/tests/CLAUDE.md +++ b/tests/CLAUDE.md @@ -11,7 +11,7 @@ Split by domain: - `test_mission_engagement.py` — FPS / Ship / FPS&Ship engagement classifier from CIG loc-key naming. - `test_mission_turrets.py` — turret detection from `SpawnDescription_ShipGroup Name="Turrets"` plus the `OverrideTurretHosility_BP` mission-variable signal. Fabricated XML, so no populated cache needed. - `test_spawn_classifier.py` — 1.4.1 4-bucket spawn classifier (Crew / Boarders / Wave Hostiles / Wave Allies) that replaced the flat "Enemies" tally on mission descriptions. -- `test_mission_variant_tuple.py` — 1.4.1 regression: the spawn-classifier refactor shrank the variant tuple from 11 to 10 fields and missed one destructure, which threw `ValueError: not enough values to unpack (expected 11, got 10)`. Locks the variant-tuple shape: the real `ContractVariant` against the file's `_make_variant`, field by field and in order, and both `ContractVariant(...)` calls in `scan_contract_generators`, which pass every field by position, on a fabricated contract with two sub-contracts, the second with no description of its own, which takes the contract's (a wrong count at either call raises nothing: the scan logs the `TypeError` at DEBUG and goes on to the next contract, losing that variant and the contract's remaining sub-contracts). +- `test_mission_variant_tuple.py` — 1.4.1 regression: the spawn-classifier refactor shrank the variant tuple from 11 to 10 fields and missed one destructure, which threw `ValueError: not enough values to unpack (expected 11, got 10)`. Locks the variant-tuple shape: the real `ContractVariant` against the file's `_make_variant`, field by field and in order, and both variants `scan_contract_generators` builds, one by keyword and one from it with `_replace`, on a fabricated contract with two sub-contracts, the second with no description of its own, which takes the contract's. What a wrong call logs is in `test_contract_scan.py`. - `test_mission_desc_annotation_toggle.py` — 1.4.1 Tag-Builder toggle that suppresses `[CLASS-…]` component annotations (and since #492 armor weight tags, see `test_armor_blueprint_bullets.py`) inside mission descriptions while leaving the standalone component INIs annotated. - `test_missile_name_tag.py` — 1.4.1: bomb name tags carry their ordinance letter, not size-only, so the variant column distinguishes `[BOMB-X-…]` from `[BOMB-Y-…]`. - `test_commodity_journal.py` — 1.4.1 generator guard for the `commodity_journal` unpack crash when the crafting cache directory is missing. The SCO crafting index's material names follow the selected language (`ctx["loc"]`, not `tag_loc`), and its materials come in a fixed order, not the set order that changed with each run's string hashing (perf proposal review): the order they are first met, blueprint by blueprint, each blueprint's materials by name, so the index as a whole is not in name order. The test has eight materials in one blueprint, so a set order passes by chance about once in 40,000 runs. @@ -103,6 +103,7 @@ Split by domain: - `test_backfill_new_elements.py`: `AppSettings._backfill_new_elements` upgrade path. Saved tag configs from older versions gain newly added element kinds disabled, so existing output is unchanged. - `test_discovered_items.py`: XML-based item discovery. Items whose loc key is absent from base.ini get synthesized descriptions and appear with status "New". - `test_not_for_release.py`: contracts/handlers with `notForRelease="1"` are skipped entirely by `scan_contract_generators` (the 1.4.2 dev-contract leak fix). +- `test_contract_scan.py`: `scan_contract_generators` on fabricated contracts. A contract that raises (made to, at its difficulty step) is skipped with one WARNING naming it and the error type, the others are kept, and it leaves no blueprints, chance or items behind, against a control that records them. A failed call leaves no title with no variants, which would read as every variant having a blueprint. A spawn count written as `4.0` reads as 4 with nothing logged. Any other that is not a whole number (`4.5`, `lots`, `inf`, `nan`), on a ship, an NPC or a handler's own spawns, counts as 0 with one WARNING, the contract stays, and every contract having one does not fail the run. A failure in a handler's own spawns gets a WARNING of its own: its contracts, the later files and the sort all still happen. `PointsToAward="inf"` skips only that reward. A title's drop chance stays its first pool's. A wrong `ContractVariant(...)` call (a stand-in with one more field) gets a WARNING for each contract, none at ERROR, and then `ContractScanFailed` with the counts and the first, and the real `main()`, on its serial and pool paths, then keeps the old INI, against a control that writes it. More skipped than built fails, half does not, and no contracts at all is not a failure. A walk that stops on an error before any contract gave a variant fails with a total of 0, and one that stops later returns what it read. A listing that fails before any variant still ends the run as its own error (in `test_generator_single_pass.py`). Through the real worker and the real script with one piece changed (a keyword taken out of its call, or a raise in the walk), the old INI stays, the stamp is empty and one ERROR and the `error` signal both carry `extract.contract_scan_failed` in the selected language, or `extract.contract_scan_stopped` for a walk that stops, with English asserted in full. Every language has both texts with their fields and the contract or error on a line of its own, English in `ht` and the rest in `at`. A contract that reads fine logs nothing. A sub-contract whose description is a sentinel keeps the contract's, and one with no title adds nothing. - `test_blueprint_list_type_tag.py`: #101 regression. The component Type element renders on component entries inside mission blueprint lists, not just standalone component names. - `test_armor_weight_tags.py`: #492 FPS armor weight-class tags (the Tag Builder's Armor category). `_armor_weight`: the stock "Item Type:" line wins when it names a class (only it knows Super Heavy and Utility), then an armor slot's AttachDef SubType (racing helmets, the "LightArmor" backpacks), then Clothing for any `Char_Clothing_*` or `Char_Accessory_*` slot; undersuits, flight suits, flight helmets with the plain "Helmet" SubType and ammo carriers get nothing, including the racing helmets with that plain SubType (most of them), while a flight or racing helmet in a Light slot (the real OMNI-AFS and Interceptor ones) gets Light. `build_wearable_index` and `_run_gen_armor` on fabricated entity XML: off by default with no scan, only `item_Name*` keys (a clothing record's shared "Body" item-port string stays), base.ini's own key casing for names and descriptions (the description test uses a class only the description has, so a SubType fallback cannot pass it), `_ARMOR_WEIGHT_PRECEDENCE` settling a shared name the same way in either file order, the English description for a translated base.ini, `_short` keys, and the user's style, mapping, enclosing and placement. The real `main()` writes the tags into `fps_weapons_desc_enhancements.ini` (left out with the tag off, nothing without Gear), and `TestSettingsWiring` pins that #492 adds no enhancement file, which would make the startup freshness check prompt every Gear user after upgrading. Also the Save Tag Changes fingerprint leaving the Armor config out while its tag is off (`test_tag_config_staleness_startup.py` adds that a stamp written before the Armor tab keeps the button grey, a saved, switched-on Armor tag lights it, and an unsaved Armor edit made with the tag off keeps it lit through a refresh), the tag config defaults (the issue's L/LT, M/MD, H/HV, S/SH, U/UT, C/CL codes), the Blueprint Tracker's enclosing set (armor's brackets count only while its tag is on, and its None style never, while another category's None still does; the Armor tab offers no None brackets and a stored one loads as Square) and the Armor page's standard controls, preview, tooltip and reset. - `test_armor_blueprint_bullets.py`: #492 armor weight tags on mission POTENTIAL BLUEPRINTS bullets. `armor_bullet_names` gives a bullet exactly its item name's tag (checked against `_run_gen_armor` in several styles), nothing when the tag is off, a hand-edited None bracket read as Square, the user's style and mapping, a bare bullet when one display name has two different tags (with the classes swapped between the files, since the read order is by file name), CIG's size prefix stripped and a non-breaking space kept. `_build_blueprint_body_parts` tags bullets only as they are written: no `bullet_names` leaves the output unchanged, colliding "X Set" headers, the Yormandi Eyes and Irradiated Valakkar Pearls override labels and the drift check all keep reading bare names, and bullet order and section order follow the bare names (several same-system sets come out in the same order tagged or not). `_run_gen_missions` end to end on a fabricated contract/pool/blueprint/wearable tree: tagged when on (Square for a hand-edited None), bare with the tag off or the annotate toggle off, no wearables scan with the tag off, English tags on a translated run, and three generations on one warm forge dir (off, Square, Round append) change the bullet while the stored `blueprint_pools` key stays the same and the pickle holds no armor tag. The real `main()` with Gear and missions loads the wearables once (spy on `build_wearable_index`), not at all with the tag off, and tags the bullets with only the missions enhancements selected (no Gear), as component tags do. `[Owned]` lands after a tagged bullet in every bracketed style and placement (idempotent, a longer name left alone), and a tagged bullet joins its bare Armor row in `build_blueprint_metadata`. diff --git a/tests/test_contract_scan.py b/tests/test_contract_scan.py new file mode 100644 index 00000000..4e33a01f --- /dev/null +++ b/tests/test_contract_scan.py @@ -0,0 +1,511 @@ +"""``scan_contract_generators``: what it does with data and code that go wrong. + +A contract that raises is skipped, leaves nothing behind and the rest of the scan goes +on, with one WARNING naming it. The line used to be at DEBUG, which the app does not +show in a normal run, so one wrong ``ContractVariant(...)`` call emptied every contract's +variants while the generation still finished as a success with nothing in the log. Now +a scan that skips more contracts than it builds fails the run, and the worker words the +failure in the selected language. Game data should not be able to do that, so a spawn +count that is not a whole number no longer raises. + +Every field of a contract's and a sub-contract's variant is checked in +``test_mission_variant_tuple.py``. +""" +from __future__ import annotations + +import collections +import importlib.util +import json +import logging +import string +import sys +from pathlib import Path + +import pytest +from PyQt6.QtCore import Qt + +from src.gui.workers import EnhancementsGeneratorWorker +from src.utils import i18n +from src.utils.pak_extractor import DATAFORGE_RECORDS_SUBPATH +from src.utils.settings import AppSettings + +pytestmark = [pytest.mark.unit, pytest.mark.regression] + +_SCRIPT = Path(__file__).resolve().parent.parent / "scripts" / "generate_enhancements_ini.py" +_LANGUAGES = Path(__file__).resolve().parent.parent / "languages" +_UI_FILES = sorted(_LANGUAGES.glob("*/ui.json")) +_KEY_FIELDS = { + "contract_scan_failed": {"skipped", "total", "first"}, + "contract_scan_stopped": {"first"}, +} +# The name EnhancementsGeneratorWorker.run loads the generator script under. +_WORKER_MODULE = "generate_enhancements_ini_worker" + + +@pytest.fixture(scope="module") +def gen_module(): + spec = importlib.util.spec_from_file_location( + "generate_enhancements_ini_contract_scan_test", _SCRIPT + ) + mod = importlib.util.module_from_spec(spec) + sys.modules[spec.name] = mod + spec.loader.exec_module(mod) + return mod + + +def _scan_all(gen_module, tmp_path, *xmls, **lookups): + contractgen_dir = tmp_path / "contractgenerators" + contractgen_dir.mkdir() + for number, xml in enumerate(xmls): + (contractgen_dir / f"contracts_{number}.xml").write_text(xml, encoding="utf-8") + return gen_module.scan_contract_generators(contractgen_dir, **lookups) + + +def _scan(gen_module, tmp_path, *xmls): + return _scan_all(gen_module, tmp_path, *xmls)[0] + + +def _warnings_and_up(caplog): + return [(r.levelname, r.getMessage()) for r in caplog.records + if r.levelno >= logging.WARNING] + + +def _generator(*contracts, handler_inside=""): + return ( + '\n \n' + + handler_inside + + "".join(contracts) + + " \n\n" + ) + + +def _contract(name, inside="", title=None): + return f"""\ + + + + {inside} + +""" + + +_REWARDS = """ + """ + + +def _raises_for(gen_module, monkeypatch, *names): + """Contracts with these debug names raise ``ValueError("odd")`` at the difficulty + step, after their rewards are read and before their variant is built.""" + real = gen_module._extract_difficulty + + def difficulty(element): + if element.get("debugName") in names: + raise ValueError("odd") + return real(element) + + monkeypatch.setattr(gen_module, "_extract_difficulty", difficulty) + + +def _wider_variant(gen_module): + """A ``ContractVariant`` with a field the real call does not pass.""" + return collections.namedtuple( + "ContractVariant", gen_module.ContractVariant._fields + ("extra",) + ) + + +class TestASkippedContract: + def test_it_gets_one_warning_and_the_others_are_kept( + self, gen_module, tmp_path, caplog, monkeypatch + ): + _raises_for(gen_module, monkeypatch, "odd") + xml = _generator(_contract("first"), _contract("odd"), _contract("last")) + with caplog.at_level(logging.WARNING): + missions = _scan(gen_module, tmp_path, xml) + + assert sorted(missions) == ["first_title", "last_title"] + assert _warnings_and_up(caplog) == [ + ("WARNING", "Skipped contract odd: ValueError: odd"), + ] + + @pytest.mark.parametrize("raises", [False, True], ids=["control", "raises"]) + def test_it_leaves_no_blueprints_or_items_behind( + self, gen_module, tmp_path, monkeypatch, raises + ): + """The control is the same contract, whose rewards are recorded.""" + if raises: + _raises_for(gen_module, monkeypatch, "rewards") + _, blueprints, chance, items = _scan_all( + gen_module, tmp_path, + _generator(_contract("good"), _contract("rewards", _REWARDS)), + blueprint_pools={"pool": ["Some Armor"]}, + entity_names={"item": "Some Item"}, + ) + assert ("rewards_title" in blueprints) is not raises + assert chance == ({} if raises else {"rewards_title": 0.25}) + assert items == ({} if raises else {"rewards_title": ["Some Item"]}) + + def test_a_failed_call_leaves_no_title_with_no_variants( + self, gen_module, tmp_path, monkeypatch + ): + """Such a title read as every variant having a blueprint (``all([])`` is True).""" + real = gen_module.ContractVariant + + def fails_for_one(**fields): + if fields["bp_variant"] == "odd": + raise TypeError("a wrong call") + return real(**fields) + + monkeypatch.setattr(gen_module, "ContractVariant", fails_for_one) + xml = _generator(_contract("first"), _contract("odd"), _contract("last")) + assert sorted(_scan(gen_module, tmp_path, xml)) == ["first_title", "last_title"] + + def test_a_titles_chance_stays_its_first_pools(self, gen_module, tmp_path): + """The chance is kept back until the variant is built, and still the first one.""" + pools = ( + '' + '' + ) + _, _, chance, _ = _scan_all( + gen_module, tmp_path, _generator(_contract("a", pools, title="shared")), + blueprint_pools={"pool": ["Some Armor"]}, + ) + assert chance == {"shared_title": 0.25} + + def test_one_that_reads_fine_logs_nothing(self, gen_module, tmp_path, caplog): + with caplog.at_level(logging.WARNING): + missions = _scan(gen_module, tmp_path, _generator(_contract("fine"))) + + assert list(missions) == ["fine_title"] + assert _warnings_and_up(caplog) == [] + + +_SHIPS = """ + + + """ + +_NPCS = """ + + + """ + + +class TestASpawnCountThatIsNotAWholeNumber: + """Over half of LIVE's contracts read a count, so one change to how counts are written + must not lose them and fail every generation.""" + + @pytest.mark.parametrize("inside, attribute, value", [ + (_SHIPS.format(first="4.5"), "Pirates concurrentAmount", "4.5"), + (_SHIPS.format(first="inf"), "Pirates concurrentAmount", "inf"), + (_SHIPS.format(first="nan"), "Pirates concurrentAmount", "nan"), + (_NPCS.format(spawns="lots", concurrent="1"), "Guards maxSpawns", "lots"), + (_NPCS.format(spawns="-1", concurrent="lots"), "Guards maxConcurrentSpawns", "lots"), + ], ids=["ship", "ship_inf", "ship_nan", "npc", "npc_concurrent"]) + def test_it_counts_as_nothing_and_the_contract_stays( + self, gen_module, tmp_path, caplog, inside, attribute, value + ): + with caplog.at_level(logging.WARNING): + missions = _scan(gen_module, tmp_path, _generator(_contract("odd", inside))) + + counts = {label: count for bucket in missions["odd_title"][0].spawns.values() + for label, count in bucket.items()} + assert list(counts.values()) == [2 if "Pirates" in attribute else 3] + assert _warnings_and_up(caplog) == [( + "WARNING", + f"Spawn count is not a whole number, counted as 0: {attribute}={value!r}", + )] + + def test_a_whole_number_written_with_a_point_is_read(self, gen_module, tmp_path, caplog): + with caplog.at_level(logging.WARNING): + missions = _scan(gen_module, tmp_path, _generator( + _contract("odd", _SHIPS.format(first="4.0")))) + + counts = {label: count for bucket in missions["odd_title"][0].spawns.values() + for label, count in bucket.items()} + assert list(counts.values()) == [6] + assert _warnings_and_up(caplog) == [] + + def test_every_contract_with_one_does_not_fail_the_run(self, gen_module, tmp_path): + inside = _SHIPS.format(first="4.5") + missions = _scan(gen_module, tmp_path, _generator( + _contract("a", inside), _contract("b", inside), _contract("c", inside))) + assert sorted(missions) == ["a_title", "b_title", "c_title"] + + def test_one_in_a_handlers_own_spawns_is_the_same(self, gen_module, tmp_path, caplog): + with caplog.at_level(logging.WARNING): + missions = _scan(gen_module, tmp_path, _generator( + _contract("kept"), handler_inside=_SHIPS.format(first="4.5"))) + + assert list(missions) == ["kept_title"] + assert [level for level, _ in _warnings_and_up(caplog)] == ["WARNING"] + + +class TestAFailureInAHandlersOwnSpawns: + def test_it_ends_nothing(self, gen_module, tmp_path, caplog, monkeypatch): + """Its contracts stay, with their own spawns or none, the file after it is read, + and the variants are still sorted with Stanton first.""" + real = gen_module._extract_spawn_counts + + def spawn_counts(element, exclude_within=None): + if exclude_within: + raise ValueError("odd") + return real(element) + + monkeypatch.setattr(gen_module, "_extract_spawn_counts", spawn_counts) + first_file = _generator( + _contract("Pyro_x", title="shared"), _contract("Stanton_x", title="shared")) + with caplog.at_level(logging.WARNING): + missions = _scan(gen_module, tmp_path, first_file, _generator(_contract("later"))) + + assert sorted(missions) == ["later_title", "shared_title"] + assert [v.system_name for v in missions["shared_title"]] == ["Stanton", "Pyro"] + assert missions["shared_title"][0].spawns == gen_module._empty_spawn_breakdown() + assert _warnings_and_up(caplog) == [ + ("WARNING", "Skipped the shared spawns of handler Test_Handler: ValueError: odd"), + ] * 2 + + +_SCENARIO_REWARD = """ + + """ + + +def test_points_that_are_infinite_skip_only_that_reward(gen_module, tmp_path, caplog): + inside = _SCENARIO_REWARD.format(points="inf") + _SCENARIO_REWARD.format(points="500") + with caplog.at_level(logging.WARNING): + missions = _scan(gen_module, tmp_path, _generator(_contract("odd", inside))) + + assert missions["odd_title"][0].success_xp == 500 + assert _warnings_and_up(caplog) == [] + + +class TestMostContractsSkipped: + def test_a_wrong_contractvariant_call_fails_the_scan( + self, gen_module, tmp_path, caplog, monkeypatch + ): + """It raises for every contract. Each gets a line naming the error, none at ERROR + here (the worker's failure path opens the error box), and the failure holds the + counts and the first.""" + monkeypatch.setattr(gen_module, "ContractVariant", _wider_variant(gen_module)) + with caplog.at_level(logging.WARNING), pytest.raises( + gen_module.ContractScanFailed + ) as failure: + _scan(gen_module, tmp_path, _generator(_contract("a"), _contract("b"))) + + assert (failure.value.skipped, failure.value.total) == (2, 2) + assert failure.value.first.startswith("a: TypeError: ") + assert "'extra'" in failure.value.first + assert str(failure.value).startswith("2 of 2 contracts could not be read. The first: a: ") + logged = _warnings_and_up(caplog) + assert [level for level, _ in logged] == ["WARNING", "WARNING"] + for (_, message), name in zip(logged, ("a", "b")): + assert message.startswith(f"Skipped contract {name}: TypeError: ") + + def test_more_skipped_than_built_fails(self, gen_module, tmp_path, monkeypatch): + _raises_for(gen_module, monkeypatch, "bad1", "bad2") + with pytest.raises(gen_module.ContractScanFailed) as failure: + _scan(gen_module, tmp_path, _generator( + _contract("good"), _contract("bad1"), _contract("bad2"))) + assert (failure.value.skipped, failure.value.total) == (2, 3) + + def test_half_does_not_fail(self, gen_module, tmp_path, monkeypatch): + _raises_for(gen_module, monkeypatch, "bad") + xml = _generator(_contract("good"), _contract("bad")) + assert list(_scan(gen_module, tmp_path, xml)) == ["good_title"] + + def test_no_contracts_at_all_is_not_a_failure(self, gen_module, tmp_path): + assert _scan(gen_module, tmp_path, _generator()) == {} + + def _roots(self, gen_module, *contracts): + """Roots that hold *contracts*, then raise, as a bug in the walk does.""" + for contract in contracts: + yield gen_module.ET.fromstring(_generator(contract)) + raise OSError("a bug") + + def test_a_walk_that_stops_before_any_variant_fails(self, gen_module, tmp_path, caplog): + with caplog.at_level(logging.WARNING), pytest.raises( + gen_module.ContractScanFailed + ) as failure: + gen_module.scan_contract_generators( + tmp_path, template_lookup={}, roots=self._roots(gen_module)) + + assert (failure.value.skipped, failure.value.total) == (0, 0) + assert failure.value.first == "OSError: a bug" + assert str(failure.value) == "The contract scan stopped: OSError: a bug" + assert _warnings_and_up(caplog) == [ + ("WARNING", "Error scanning contract generators: a bug"), + ] + + def test_a_walk_that_stops_later_keeps_what_it_read(self, gen_module, tmp_path): + missions, _, _, _ = gen_module.scan_contract_generators( + tmp_path, template_lookup={}, roots=self._roots(gen_module, _contract("read"))) + assert list(missions) == ["read_title"] + + @pytest.mark.parametrize("max_workers", [1, 6], ids=["serial", "pool"]) + @pytest.mark.parametrize("wrong_call", [False, True], ids=["control", "wrong_call"]) + def test_main_then_keeps_the_old_ini( + self, gen_module, tmp_path, monkeypatch, wrong_call, max_workers + ): + """A wrong call fails the run and leaves the old INI byte for byte, on the serial + path the GUI takes and the pool the CLI takes. The control is the same run with + the real class, which replaces it.""" + forge_dir = tmp_path / "forge" + contractgen_dir = forge_dir / DATAFORGE_RECORDS_SUBPATH / "contracts" / "contractgenerator" + contractgen_dir.mkdir(parents=True) + (contractgen_dir / "contracts.xml").write_text( + _generator(_contract("a")), encoding="utf-8" + ) + base_ini = tmp_path / "base.ini" + base_ini.write_text("a_title=A Title\na_desc=A description\n", encoding="utf-8") + old_ini = tmp_path / "mission_rewards_enhancements.ini" + old_ini.write_bytes(b"old=old\r\n") + + def generate(): + gen_module.main( + base_ini, forge_dir, categories={"mission_rewards", "medical_consumables"}, + max_workers=max_workers, + ) + + if wrong_call: + monkeypatch.setattr(gen_module, "ContractVariant", _wider_variant(gen_module)) + with pytest.raises(gen_module.ContractScanFailed): + generate() + assert old_ini.read_bytes() == b"old=old\r\n" + else: + generate() + assert old_ini.read_bytes() != b"old=old\r\n" + + +@pytest.fixture +def worker_loads_the_script_with(tmp_path, monkeypatch): + """The worker loads the real script with one piece of it changed.""" + script = tmp_path / "scripts" / "generate_enhancements_ini.py" + script.parent.mkdir() + real = importlib.util.spec_from_file_location + + def spec_from_file_location(name, location=None, *args, **kwargs): + if name == _WORKER_MODULE: + location = script + return real(name, location, *args, **kwargs) + + monkeypatch.setattr(importlib.util, "spec_from_file_location", spec_from_file_location) + monkeypatch.setitem(sys.modules, _WORKER_MODULE, None) + monkeypatch.setattr(sys, "path", list(sys.path)) + + def change(old, new): + text = _SCRIPT.read_text(encoding="utf-8") + assert text.count(old) == 1 + script.write_text(text.replace(old, new), encoding="utf-8") + + return change + + +class TestTheFailureReachesTheUser: + @pytest.mark.parametrize("old, new, language, opening, tail", [ + ("rep_track=rep_track,", "", "english", + "Enhancement generation stopped because 1 of 1 contracts could not be read.", + "\nodd: TypeError: "), + ("rep_track=rep_track,", "", "german", + "Die Erzeugung der Erweiterungen wurde gestoppt, weil 1 von 1 Aufträgen", + "\nodd: TypeError: "), + ("handler_flags = []", "raise OSError('a bug')", "english", + "Enhancement generation stopped because the contract scan ended with an error.", + "\nOSError: a bug"), + ("handler_flags = []", "raise OSError('a bug')", "german", + "Die Erzeugung der Erweiterungen wurde gestoppt, weil das Lesen der Aufträge", + "\nOSError: a bug"), + ], ids=["call_english", "call_german", "walk_english", "walk_german"]) + def test_in_the_selected_language_with_the_old_set_kept( + self, tmp_profile, qapp, english_ui, worker_loads_the_script_with, caplog, + old, new, language, opening, tail + ): + """The real worker running the script on a real cache folder and base.ini.""" + worker_loads_the_script_with(old, new) + i18n.set_language(language) + contractgen_dir = ( + AppSettings.get_dataforge_cache_leaf() / DATAFORGE_RECORDS_SUBPATH + / "contracts" / "contractgenerator" + ) + contractgen_dir.mkdir(parents=True) + (contractgen_dir / "contracts.xml").write_text( + _generator(_contract("odd")), encoding="utf-8" + ) + base_ini = AppSettings.get_base_ini_path() + base_ini.parent.mkdir(parents=True, exist_ok=True) + base_ini.write_text("odd_title=Odd\n", encoding="utf-8") + old_ini = AppSettings.get_enhancements_dir() / "mission_rewards_enhancements.ini" + old_ini.write_bytes(b"old=old\r\n") + AppSettings.set_enhancements_stamp("an old build") + + worker = EnhancementsGeneratorWorker( + categories={"mission_rewards"}, tag_configs=AppSettings.get_all_tag_configs() + ) + finished, errors = [], [] + worker.finished.connect(finished.append, Qt.ConnectionType.DirectConnection) + worker.error.connect(errors.append, Qt.ConnectionType.DirectConnection) + with caplog.at_level(logging.WARNING): + worker.start() + assert worker.wait(60000), "the generator worker did not finish" + + assert finished == [False] + assert len(errors) == 1 and errors[0].startswith(opening) + assert tail in errors[0] + assert [r.getMessage() for r in caplog.records if r.levelno >= logging.ERROR] == errors + assert old_ini.read_bytes() == b"old=old\r\n" + assert not AppSettings.get_enhancements_stamp() + + def test_english_in_full(self, english_ui): + assert i18n.tr("extract.contract_scan_failed", skipped=2, total=3, first="x: Error") == ( + "Enhancement generation stopped because 2 of 3 contracts could not be read. Your " + "existing enhancements are unchanged. Please report this, as it is usually a fault " + "in Smart Citizen.\n\nThe first contract:\nx: Error" + ) + assert i18n.tr("extract.contract_scan_stopped", first="OSError: x") == ( + "Enhancement generation stopped because the contract scan ended with an error. Your " + "existing enhancements are unchanged. Please report this, as it is usually a fault " + "in Smart Citizen.\n\nThe error:\nOSError: x" + ) + + @pytest.mark.parametrize("key", sorted(_KEY_FIELDS)) + @pytest.mark.parametrize("ui_json", _UI_FILES, ids=lambda p: p.parent.name) + def test_every_language_has_the_texts(self, ui_json, key): + """Its fields, the contract or error on a line of its own at the end, and the + provenance languages/TRANSLATIONS.md asks for.""" + node = json.loads(ui_json.read_text(encoding="utf-8"))["extract"][key] + if ui_json.parent.name == "english": + assert node["ht"] and node["at"] == "" + else: + assert node["ht"] == "" and node["at"] + text = node["ht"] or node["at"] + fields = {field for _, field, _, _ in string.Formatter().parse(text) if field} + assert fields == _KEY_FIELDS[key] + assert text.endswith("\n{first}") + formatted = text.format(skipped=1, total=2, first="x") + assert "{" not in formatted and "}" not in formatted + + +_SUB_CONTRACTS = """ + + + + + + + + """ + + +class TestSubContractVariants: + def _missions(self, gen_module, tmp_path): + return _scan(gen_module, tmp_path, _generator(_contract("parent", _SUB_CONTRACTS))) + + def test_a_sentinel_description_counts_as_none(self, gen_module, tmp_path): + """``@LOC_EMPTY`` is the game's placeholder, so the contract's description stays.""" + missions = self._missions(gen_module, tmp_path) + assert missions["sentinel_desc_title"][0].desc_key == "parent_desc" + assert missions["sentinel_desc_title"] == missions["parent_title"] + + def test_one_with_no_title_adds_nothing(self, gen_module, tmp_path): + assert sorted(self._missions(gen_module, tmp_path)) == [ + "parent_title", "sentinel_desc_title", + ] diff --git a/tests/test_generator_single_pass.py b/tests/test_generator_single_pass.py index 5eb020e0..470a212b 100644 --- a/tests/test_generator_single_pass.py +++ b/tests/test_generator_single_pass.py @@ -979,6 +979,21 @@ def fails_after_one(entity_dir, xml_path_index=None, records_dir=None): assert sum("Error scanning contract generators" in r.getMessage() for r in caplog.records) == 1 + def test_a_listing_that_fails_before_any_variant_ends_the_run_as_itself( + self, gen_module, tmp_path, monkeypatch): + """The scan has no variant then, so it raises ContractScanFailed. The run still + ends with the listing's own error, which is what went wrong.""" + real = gen_module._indexed_xml_files + + def fails_at_once(entity_dir, xml_path_index=None, records_dir=None): + if entity_dir.name == "contractgenerator": + raise OSError("listing failed at once") + yield from real(entity_dir, xml_path_index, records_dir) + + monkeypatch.setattr(gen_module, "_indexed_xml_files", fails_at_once) + with pytest.raises(OSError, match="at once"): + gen_module._run_gen_missions(_missions_ctx(_missions_tree(tmp_path), tmp_path)) + def test_a_listing_that_fails_outright_ends_the_run_before_the_scan( self, gen_module, tmp_path, monkeypatch, caplog): """scan_contract_generators asked for its file list outside its try, diff --git a/tests/test_mission_variant_tuple.py b/tests/test_mission_variant_tuple.py index 0ae85093..ce812eb4 100644 --- a/tests/test_mission_variant_tuple.py +++ b/tests/test_mission_variant_tuple.py @@ -7,10 +7,9 @@ ``rep_track`` at position 11) so a mission's body/title can show which reputation track (e.g. "Security" vs the generic Contractor/Standing track) its XP feeds. ``_run_gen_missions`` reads a variant by field name since #240, -but ``scan_contract_generators`` still builds it by position, so a field that -moves puts values in the wrong fields with no error. These tests lock the real -``ContractVariant`` and both calls that build it, so the next refactor can't -slip a similar drift through. +and ``scan_contract_generators`` builds it by keyword, a sub-contract's from its +contract's with ``_replace``. These tests lock the real ``ContractVariant`` and +both of those variants, so the next refactor can't slip a similar drift through. """ from __future__ import annotations @@ -95,8 +94,8 @@ def test_helper_holds_each_field_at_its_position(self, gen_module): fields = gen_module.ContractVariant._fields assert len(values) == len(fields), ( f"ContractVariant has {len(fields)} fields, _make_variant gives {len(values)}. " - "Change both, and both ContractVariant(...) calls in scan_contract_generators, " - "which pass every field by position." + "Change both, and the keywords of the ContractVariant(...) call in " + "scan_contract_generators." ) assert gen_module.ContractVariant(*values)._asdict() == { "system_name": "Pyro", @@ -150,10 +149,10 @@ def test_helper_holds_each_field_at_its_position(self, gen_module): class TestScanContractGeneratorsFillsEachField: - """Both ``ContractVariant(...)`` calls in ``scan_contract_generators`` pass - every field by position. Two values swapped there raise nothing. Nor does a - wrong count: the scan logs the ``TypeError`` at DEBUG and goes on to the next - contract, losing that variant and the contract's remaining sub-contracts.""" + """``scan_contract_generators`` builds a contract's variant by keyword and a + sub-contract's from it with ``_replace``. Two values swapped in that call raise + nothing. A missing keyword raises a ``TypeError`` for each contract, and the + scan then fails the run (see ``test_contract_scan.py``).""" def test_each_value_lands_in_its_field(self, gen_module, tmp_path): contractgen_dir = tmp_path / "contractgenerators" From 85ba02f4dd158a93f3d15e2298690fd398769eb1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 7 Nov 2026 14:27:03 +0000 Subject: [PATCH 07/32] Tag gem rock HUD names with their crafting tag A hand-mineable gem rock that has not been fractured shows its hud_mining_fps_rock_name_* string (_1 Aphorite, _2 Dolivine, _3 Hadanite), not its items_commodities_* name, so it showed no [CF] tag until it broke. The commodity crafting pass now gives each such HUD name the tag its commodity's name got. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011hqxFgHVmZmwQTwPYiS2Vi --- scripts/generate_enhancements_ini.py | 30 ++++++++++++++++++++++++ tests/test_commodity_journal.py | 35 ++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) diff --git a/scripts/generate_enhancements_ini.py b/scripts/generate_enhancements_ini.py index 472569c6..bad626d3 100644 --- a/scripts/generate_enhancements_ini.py +++ b/scripts/generate_enhancements_ini.py @@ -5048,6 +5048,29 @@ def _place_commodity_tag(base_name: str, tag: str, cfg) -> str: return join_tag(base_name, tag, placement) +_FPS_ROCK_NAME_PREFIX = "hud_mining_fps_rock_name_" + + +def _tag_fps_rock_names(out: dict[str, str], loc: dict[str, str], + tag_by_name: dict[str, str], cfg) -> None: + """Tag each ``hud_mining_fps_rock_name_*`` key in *loc* whose text is a + tagged commodity's name, ignoring case, with that commodity's tag. + + These keys are the names the HUD shows for a hand-mineable gem rock + before it is fractured (``_1`` Aphorite, ``_2`` Dolivine, ``_3`` + Hadanite). They name no commodity, so the crafting loop never reaches + them, and a gem rock showed no tag until it broke into commodities. + *tag_by_name* maps a lowercased commodity name to the tag its name key + got in the crafting loop. + """ + for key, base in loc.items(): + if not key.lower().startswith(_FPS_ROCK_NAME_PREFIX) or key in out: + continue + tag = tag_by_name.get(base.strip().lower()) + if tag: + out[key] = _place_commodity_tag(base, tag, cfg) + + def _parse_compendium_locations(base_content: str) -> dict: """Map a Mining Compendium mineral name (lowercased) to its sorted mining locations, parsed from the stock ``Name - loc, loc, ...`` paragraphs. @@ -5209,6 +5232,7 @@ def scan_crafting_blueprints( ) out: dict[str, str] = {} + tag_by_name: dict[str, str] = {} # lowercased commodity name -> its tag skipped_no_loc: list[str] = [] for commodity in sorted(commodity_items.keys()): pairs = _discover_commodity_loc_pairs(commodity, loc, commodity_keys) @@ -5234,6 +5258,8 @@ def scan_crafting_blueprints( usage_keys=usage_keys, ) out[name_key] = _place_commodity_tag(base_name, tag, cfg) if tag else base_name + if tag: + tag_by_name.setdefault(base_name.strip().lower(), tag) base_desc = loc.get(desc_key, "") if base_desc and desc_key not in out: @@ -5249,6 +5275,10 @@ def scan_crafting_blueprints( sections.append(enhancements_block) out[desc_key] = f"{base_desc}\\n\\n" + "\\n\\n".join(sections) + # A gem rock not yet fractured shows its HUD name, not its commodity + # name, so give those names the commodity's tag too. + _tag_fps_rock_names(out, loc, tag_by_name, tag_config or DEFAULT_TAG_CONFIGS.get("commodities")) + if skipped_no_loc: logger.warning( f"Crafting: {len(skipped_no_loc)} commodities had no matching loc keys " diff --git a/tests/test_commodity_journal.py b/tests/test_commodity_journal.py index 2ca6e305..27630093 100644 --- a/tests/test_commodity_journal.py +++ b/tests/test_commodity_journal.py @@ -139,6 +139,41 @@ def test_filename_regex_matches_harvestable_handheld(self, gen_module, tmp_path) assert "items_commodities_hadanite_desc" in commodities assert "BLUEPRINT DATA" in commodities["items_commodities_hadanite_desc"] + def test_fps_rock_hud_names_get_the_gem_tag(self, gen_module, tmp_path): + """A gem rock not yet fractured shows its ``hud_mining_fps_rock_name_*`` + string, so that name takes its commodity's tag; a HUD rock name that + is no crafting material stays out.""" + gem_uuid = "1234abcd-1234-abcd-1234-1234abcd1234" + carryables = tmp_path / "carryables" + carryables.mkdir() + (carryables / "harvestable_mineral_1h_hadanite.xml").write_text( + f'', encoding="utf-8" + ) + bp_dir = tmp_path / "bp" + bp_dir.mkdir() + (bp_dir / "bp_craft_test_weapon.xml").write_text( + f'' + f'' + f'', + encoding="utf-8", + ) + loc = { + "items_commodities_hadanite": "Hadanite", + "items_commodities_hadanite_desc": "A crystal oscillator.", + "hud_mining_fps_rock_name_1": "Aphorite", + "hud_mining_fps_rock_name_3": "Hadanite", + } + commodities, _journal = gen_module.scan_crafting_blueprints( + bp_dir=bp_dir, + carryables_dir=carryables, + entity_names={}, + loc=loc, + tag_config=_enabled_commodities_config(), + ) + assert commodities["hud_mining_fps_rock_name_3"] == commodities["items_commodities_hadanite"] + assert "[CF" in commodities["hud_mining_fps_rock_name_3"] + assert "hud_mining_fps_rock_name_1" not in commodities + def test_resource_and_item_paths_unify(self, gen_module, tmp_path): """When a single blueprint uses both element types, BOTH commodities get attributed to that blueprint.""" From 7bfb46e150983285dca1ac92323814702acd67af Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 8 Nov 2026 15:35:54 +0000 Subject: [PATCH 08/32] docs: Linux note that the overlay needs an X11 session, not Wayland Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011hqxFic6mjm78jYZMPtBoL --- README.md | 3 +++ docs/LINUX.md | 9 +++++++++ 2 files changed, 12 insertions(+) diff --git a/README.md b/README.md index a68b3b61..b820cc72 100644 --- a/README.md +++ b/README.md @@ -77,6 +77,9 @@ Download the **`SmartCitizen-{VERSION}-Setup.exe`** installer and run it. The ap Smart Citizen runs on Linux through the same Wine prefix as Star Citizen — see **[LINUX.md](docs/LINUX.md)** for the full guide (download the portable build, then point a launch script at the Wine runner your game uses). +> [!IMPORTANT] +> **The Smart Citizen Overlay needs an X11 session, not Wayland.** If you want the overlay on Linux, log out and pick the **X11** (or **Xorg**) session on your login screen. Steps are in [LINUX.md](docs/LINUX.md). + ### For Developers See **[docs/CONTRIBUTOR_GUIDE.md](docs/CONTRIBUTOR_GUIDE.md)** for local setup (prerequisites, clone, install, run). diff --git a/docs/LINUX.md b/docs/LINUX.md index c08e6ad4..21adb29e 100644 --- a/docs/LINUX.md +++ b/docs/LINUX.md @@ -4,6 +4,15 @@ This guide explains how to run Smart Citizen on Linux through the same Wine pref > Originally contributed by [@PinkFloyd1213](https://github.com/PinkFloyd1213) in [issue #33](https://github.com/Osiris-DevWorks/smart-citizen/issues/33#issuecomment-4460305053). +> [!IMPORTANT] +> **The Smart Citizen Overlay needs an X11 session, not Wayland.** It does not work on Wayland. If you want the overlay to work: +> +> 1. Log out of your desktop. +> 2. On the login screen, pick your user, then use the session menu (often a gear icon) to choose the **X11** or **Xorg** session (for example "Plasma (X11)" or "GNOME on Xorg"). +> 3. Log in and run Star Citizen and Smart Citizen as usual. +> +> To check which one you're on, run `echo $XDG_SESSION_TYPE` in a terminal: it prints `x11` or `wayland`. The rest of Smart Citizen works on either. + ## Prerequisites You need a working Star Citizen installation on Linux before installing Smart Citizen. If you haven't set that up yet, we strongly recommend the **[Star Citizen LUG](https://wiki.starcitizen-lug.org/)** community — they maintain scripts (notably the LUG-Helper) and have an active community that makes running SC on Linux relatively painless. From 5c68a1e15eafb9e393f3767377daac620b860736 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 8 Nov 2026 18:20:08 +0000 Subject: [PATCH 09/32] SCO: read the scan number at wide field of view and in small cockpits (#546) The readout panel is part of the 3D cockpit, so a wider field of view draws it smaller (7 px digits on a 2560x1600 screen at max, against the 11 px the reader expected) and nearer the middle of the screen (about 0.39 down, the edge of the old scan box). The Aurora's readout is as small. - The reader looks for digits at 1.0 and 0.7 of the expected size (DIGIT_SCALES), sharing one mask and blob pass per threshold. - SCAN_REGION reaches 0.45 down instead of 0.40. - A comma must sit on the row's baseline. The taller box took a small blob far below the digits for the comma and cut 21,200 short. - Reference glyphs from two max field of view Golem shots, and ALT_COUNT 4, since a new small 3 pushed a scaled-down 8 to fourth place. - Fixtures fov1-fov5 and tests for the small digits and the stray comma. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011hqxFinijz9yEfxUtqqe9g --- src/gui/CLAUDE.md | 2 +- src/gui/sco_overlay.py | 7 +++-- src/utils/CLAUDE.md | 2 +- src/utils/sco_digit_reader.py | 49 ++++++++++++++++++++++-------- src/utils/sco_digit_samples.py | 9 ++++++ tests/CLAUDE.md | 2 +- tests/fixtures/sco/fov1_2000.png | Bin 0 -> 7909 bytes tests/fixtures/sco/fov2_3170.png | Bin 0 -> 7757 bytes tests/fixtures/sco/fov3_3170.png | Bin 0 -> 7932 bytes tests/fixtures/sco/fov4_15360.png | Bin 0 -> 9030 bytes tests/fixtures/sco/fov5_21200.png | Bin 0 -> 10174 bytes tests/test_sco_digit_reader.py | 41 ++++++++++++++++++++++--- 12 files changed, 89 insertions(+), 23 deletions(-) create mode 100644 tests/fixtures/sco/fov1_2000.png create mode 100644 tests/fixtures/sco/fov2_3170.png create mode 100644 tests/fixtures/sco/fov3_3170.png create mode 100644 tests/fixtures/sco/fov4_15360.png create mode 100644 tests/fixtures/sco/fov5_21200.png diff --git a/src/gui/CLAUDE.md b/src/gui/CLAUDE.md index 28499d23..4149683f 100644 --- a/src/gui/CLAUDE.md +++ b/src/gui/CLAUDE.md @@ -22,7 +22,7 @@ GUI layer (PyQt6). See the root `CLAUDE.md` for project context and cross-cuttin - `test_plan_panel.py` — **Test Plan dock** (#144): `TestPlanPanel`, the right-side `QDockWidget` checklist testers use to verify a release. Checkboxes grouped by section, a live progress bar, a persisted tester name, and Copy Report / Submit to Discord / Reset. Modelled on the Help dock (`_ensure_help_dock` → `_ensure_test_plan_dock` / `show_test_plan`, opened from the **More** menu). Check-state, tester name, and webhook URL persist via `AppSettings`; the plan content and report formatting are Qt-free in `src/utils/test_plan.py`; the Discord POST runs off-thread in `TestPlanSubmitWorker` (`workers.py`). Submit is disabled with no webhook configured (`AppSettings.get_test_webhook_url()` → stored override, then `SMART_CITIZEN_TEST_WEBHOOK_URL` env); Copy Report always works. - `coach_mark.py` — `CoachMarkOverlay` plus `TutorialTour` for the in-app guided tour. Self-contained: the main window builds `CoachMarkStep` records (target widget, title, description, optional pre-action) and calls `tour.start()`. The overlay dims the window, spotlights the target, floats a callout with Back/Next/Skip; emits `finished(completed: bool)`. - `overlay_tab.py` — **Overlay Tab** (3.0.0): Start/Stop Overlay button plus one checkbox per `AppSettings.SCO_OPTION_SETTING` switch (names are `AppSettings.SCO_*` constants; a drawer section's switch is `SCO_DRAWER_SECTION_PREFIX` plus its `sco_overlay.SECTION_ICONS` name) (scan decoder, drawer, its Blueprints / Minerals / Crafting sections indented under it and greyed out while the drawer is off (names starting `drawer_`), mark new blueprints owned from Game.log) and a drawer position picker under the drawer switch (`AppSettings.SCO_DRAWER_POSITIONS`, default middle right, greyed out with the drawer). The scan-sample capture has no switch here (tuning only, see `sco_overlay.py`). #523 added, under the scan decoder switch and greyed out with it, a **Scan text size** picker (`AppSettings.SCO_LABEL_SIZES`, `get_/set_sco_label_size`, default medium) and, under an `overlay.alerts_title` line, one switch per watched-target alert (names `SCO_ALERT_PREFIX` plus a `sco_overlay.ALERTS` name: glow and flash on by default, color, bigger and sound off), and under the drawer's position picker, greyed out with the drawer, a **Drawer hotkey** box (`HotkeyEdit`, saved through `get_/set_sco_drawer_hotkey` as portable text; its hint's example is Alt+8 because Star Citizen's default bindings, `Data/Libs/Config/defaultProfile.xml` in Data.p4k, use every letter, F1 to F12, Home, End, Page Up and Down, Pause, Tab, Space, the arrows and `/`, and Left Ctrl and both Shifts on their own, but never 8, Insert, Delete or Alt on its own, read on LIVE 2026-10-23). Since #523 the tab is in titled boxes like the Config and Enhancements tabs (`_group`, titles `overlay.scan_group`, `drawer_group`, `blueprint_watch_group`): each part's switch first with its hint under it (`_add_switch`), what hangs under it set in (`_indented`), and the alerts and the drawer's sections side by side (`_switch_row`) with their hints as tooltips. `HotkeyEdit` (a `QLineEdit` nothing can be typed or pasted into, with a **Clear** button to its right, `clear_hotkey`, lit only while a hotkey is saved, since every key pressed in the box is taken as the hotkey) records one key combination from the Windows virtual key each press reports (`nativeVirtualKey`, through `global_hotkey.hotkey_text`), not from the character the keyboard layout makes of it: where Ctrl+Alt is AltGr, Ctrl+Alt+O makes a character such as "ó", which a `QKeySequenceEdit` takes for the key (so the combination could not be registered; found in testing), and Shift+1 makes "!". A press with no virtual key (another platform, a press sent by code) falls back to Qt's key. While only modifiers are held it shows them ("Ctrl+Alt+"; Qt leaves a modifier key's own modifier out of its press, so `_MODIFIER_KEYS` adds it back), it takes Tab and every key an app shortcut would (it accepts `ShortcutOverride`), and a combination `windows_hotkey` does not take (a key Windows has no code for, a dead key, a symbol another key also types, or a letter, digit, symbol, Space, Tab or navigation key without Ctrl, Alt or Win) is not saved: the box goes back to the saved one and says why (`refused`). `set_hotkey_problem()` shows why the hotkey is not in use (`global_hotkey.HOTKEY_TAKEN` or `HOTKEY_UNSUPPORTED`, worded by `hotkey_problem_text()` as `overlay.hotkey_taken` or `overlay.hotkey_unsupported`), which `MainWindow._on_sco_hotkey_problem` passes on from `ScoOverlay.hotkey_problem` along with the same text in the status bar. The text size and position pickers are both built by `_picker_row`, and their settings read and saved through `AppSettings._get_choice` / `_set_choice` (a value outside the choices reads, and saves, as the default). Checkboxes save straight to settings and emit `options_changed` → `MainWindow._apply_sco_options` (pushes them to `ScoOverlay.set_options` and starts/stops the live log watcher); the start button emits `overlay_toggled`, which drives the toolbar Overlay button so both go through `_toggle_sco_overlay`, and `set_running()` mirrors the overlay back without re-emitting; `set_option()` likewise mirrors a switch the SCO drawer changed. `refresh_styles()` re-applies the start button's theme colour; `MainWindow.refresh_action_buttons()` calls it on a live theme swap, with the toolbar's Overlay and Tutorial buttons. -- `sco_overlay.py` — **Smart Citizen Overlay (SCO)** (3.0.0): always-on-top windows over the game, toggled by the toolbar's **Overlay** button (`MainWindow._toggle_sco_overlay`). `ScanLabelWindow` is click-through and shows the decoded mining signature centred under the scanner readout, always in the ODW colours (`ODW_OVERLAY_COLORS`, see-through at `DRAWER_ALPHA`) whatever the app theme; `BlueprintDrawer` is a stack of three tapered icon tabs (`SectionTabBar`, one per section, icons from `SECTION_ICONS` painted as `QPainterPath`s and kept upright on every edge) docked at one of eight positions (`set_position()`; `drawer_edge()` maps a corner or middle position to its side edge and top/bottom centre to the top or bottom edge, and `SectionTabBar.set_edge()` turns the stack to match). Neighbouring tabs overlap by their taper (`DRAWER_HANDLE_OVERLAP`), like browser tabs; the open section's tab is painted last and near-solid so it sits in front, and `tab_at()` hit-tests front to back so a click in an overlap goes to the tab you see. A tab click (`open_section()`) opens the see-through (`DRAWER_ALPHA`) panel, a `QStackedWidget`, straight to that section, another tab switches section, and the open section's own tab closes it. A section switched off on the Overlay tab hides its tab (closing the drawer if it was open), and with all three off the drawer is hidden (`has_sections()`). The panel takes clicks and focus, but only a search box (`DrawerSearch`, focus on a click only) ever takes the keyboard: every other widget in the panel is `NoFocus`, and a search box ignores a held key's printable repeats. In game F is held to move the mouse, and a tab click gave the Blueprints or Crafting search focus, which then filled with "ffff" (#523). Its sections: a searchable blueprint list with owned checkboxes, a Minerals search (mineral → star system → location; a location finds its minerals) topped by a scan decoder checkbox (the same `scan_decoder` setting as the Overlay tab's: `scan_decoder_toggled` → `MainWindow._on_sco_scan_decoder_toggled` saves it and mirrors the tab via `OverlayTab.set_option`, while `set_options` mirrors the tab back with `set_scan_decoder`) and a **Clear all** button, with a watch checkbox right-aligned in column 1 of each ore's row (only ores `mining_catalog.ore_key` knows, since only those can decode) and of a **Salvage** row that ends the list whatever the catalog holds (`SALVAGE`, the `OTHER_SIGNATURES` key it watches, #523; `_add_watch_box` makes both kinds of box). Ticked ores and salvage go out through `watched_changed` and are saved by MainWindow (`AppSettings.get_/set_sco_watched_minerals`, fed back with `ScoOverlay.set_watched_minerals` on start); when a decode includes one, the label sets off the alerts switched on (`ScanLabelWindow.set_alerts`, from the Overlay tab's `alert_` switches; `set_watched` while it shows). The lasting ones: the glow (a pulsing `QGraphicsDropShadowEffect` in the theme accent, `GLOW_PULSE_MS`, with room for it from `GLOW_MARGIN`, less above (`GLOW_TOP_MARGIN`) so the masked label window never covers the readout), the text in `ALERT_COLOR` (a bright green, not the theme accent, which is dark blue in the Light theme) and the text `ALERT_TEXT_SCALE` times bigger, the label re-sized and re-centred under its anchor (`_place`). When a watched target is first read (`_on_reading`'s `fresh`: a value that was not showing), the label flashes, `FLASH_TOGGLES` swaps of its text and background colours `FLASH_MS` apart that change no size, and `play_alert_sound` starts the sound picked on the Overlay tab (`AppSettings.get_sco_alert_sound`, pushed with `ScoOverlay.set_alert_sound`; the tab's **Preview** calls it too) with winmm's `PlaySoundW`: "Smart Citizen Default" is a two-note chime made in memory (`alert_chime_wav`, `SND_MEMORY | SND_ASYNC`, so nothing waits and no sound file ships), the `voice_` choices are recorded WAVs shipped in `assets/sounds/` (`VOICE_ALERT_SOUNDS`, `voice_sound_path`, `SND_FILENAME`; bundled with the rest of `assets/`), and the others are Windows sound aliases (`WINDOWS_ALERT_SOUNDS`, `SND_ALIAS`, which follow the user's sound scheme); keep both in step with `AppSettings.SCO_ALERT_SOUNDS`; a steady repeat does neither. `set_text_size` (the Overlay tab's **Scan text size**) scales the label's font against the app font (`LABEL_SIZE_SCALE`). The third section is a Crafting search (a mineral lists the items it goes into, an item lists the materials it needs). Crafting rows lead back to their compendium mineral through `_mineral_by_material`, built in `set_catalog()` with `CraftingIndex.material_key_for`, the same link the mineral tooltips use, so CIG's spelling differences (Pressurized Ice / Ice, Aluminum / Aluminium, ...) resolve both ways. Minerals, locations and items carry tooltips (`mineral_tooltip`, `location_tooltip`, `_item_tooltip`). `BlueprintDrawer.retranslate_ui()` sets every fixed text and redraws the list and trees; it runs at construction and from `MainWindow.retranslate_ui()` after a language switch. The guided tour's `overlay` step (`assets/tutorial.json`, after `blueprint_tracker`) points at the Overlay tab's start button. `MainWindow._refresh_sco_catalog` feeds `set_catalog()` from the English base.ini's Mining Compendium and the `crafting_index.json` the generator writes (see `src/utils/mining_catalog.py`), re-reading only when either file changed. The drawer is coloured from the active theme via `OverlayColors.from_theme()` (window colour + title/tagline accents), re-read on every start. `ScoOverlay.set_options(dict)` applies the Overlay tab switches, a missing name taking its `SCO_OPTION_SETTING` default. `format_matches` lists a reading that fits salvage too (12,000 is 3 ROC, 4 FPS or 6 Salvage) with salvage first and the `SURFACE_ONLY` kinds marked `sco.surface_only`: the overlay cannot tell space from a planet, and in space salvage is the only answer (#523). A selected row in the drawer keeps the panel's text colour (`::item:selected { color }`): left to the palette it took `HighlightedText`, near black in the SCLE and ODW themes, on the dark see-through panel. **Drawer hotkey (#523).** `ScoOverlay` owns a `global_hotkey.GlobalHotkey`, given the Overlay tab's combination by `set_drawer_hotkey` and held only while the overlay runs with the drawer showing (`_sync_hotkey`, from `_sync_windows` and `stop`), since Windows keeps a held combination from the game. Each try ends in `hotkey_problem` ("" or why), and a refused combination is not tried again until it changes or the overlay stops (`_hotkey_refused`), so one start reports it once. The hotkey runs `toggle_drawer`: closed, it remembers the window that has focus (`foreground_window`, the game), opens the drawer on the section used last (`BlueprintDrawer.open_last_section`, the first shown one before any) and activates it. That takes focus only when the press came as `WM_HOTKEY` (Windows lets a process that just got its hotkey take the foreground), which frees the mouse over another program. In Star Citizen the press arrives through raw input alone, Windows keeps focus on the game, and the player still holds F to use the mouse in the drawer (found in testing; `_report_focus` logs it at INFO `FOCUS_CHECK_MS` after the open). Taking focus anyway would need `AttachThreadInput` with the game's thread, sent input or a changed Windows setting, which the anti-cheat rule below keeps out. Open, it closes it. Whenever the drawer closes (the hotkey or its own tab, `expanded_changed`) while it still has focus, the remembered window gets focus back (`give_focus_back`), so the mouse returns to the game; a drawer the player has clicked away from leaves focus alone, and a drawer hidden by its switch (`_sync_windows`) forgets the window, since it closed nothing the hotkey opened. `ScoOverlay` owns both plus the capture timer over `SCAN_REGION` (fractions of the primary screen); the reader is pluggable (`SignatureReader`); the default `hud_signature_reader` hands the capture's red, green and blue channels to `src/utils/sco_digit_reader.py` and maps the box back from device to logical pixels. A new reading only replaces the label after `READINGS_TO_CHANGE` identical captures in a row (`_on_reading`; any accepted or steady reading clears the pending candidate, so a misread that comes back later still has to repeat back to back), so a one-frame misread can't flash "Unidentified" over a steady signature. The confirming capture runs at once (`QTimer.singleShot(0, _capture_once)` on a first sighting) instead of waiting for the next tick, so a new signal shows up to a tick sooner; `_capture_once` drops a queued one once the timer has stopped. The label sits `LABEL_GAP` px below the readout, centred under the scanner's pin icon and the number together (the reader's box covers the digits only, so `_handle_found` widens it left by `READOUT_PIN_WIDTH` digit heights, #523), and `_mask_own_label` blacks it out of each capture so its own text is never read. The box comes to logical pixels with each edge rounded outward, so with display scaling the label never sits over the digits' bottom row (it did, and its blackout then made the next reads fail, 1080 px at 125%), and while the label shows, `_steady_anchor` keeps it where it is unless the readout has moved more than `ANCHOR_DEAD_ZONE` digit heights: captures of a still readout put its box a few pixels apart, which made the label twitch on every capture (#523). Each distinct reading's crop is saved once per session to `{logs_dir}/sco_samples/