From e4471d6c6f6d33c8e4584d89b11a22517b0d0e05 Mon Sep 17 00:00:00 2001 From: changbowei Date: Wed, 10 Jun 2026 21:04:31 -0400 Subject: [PATCH] Add reAM250 BOM research task pack --- .../ream250_bom_research/README.md | 197 ++++++++++ .../ream250_bom_research/README.zh.md | 187 +++++++++ .../research_instructions/agent.md | 170 ++++++++ .../research_result.schema.yaml | 52 +++ .../extract_step_materials.py | 154 ++++++++ .../research_scripts/generate_queue_tasks.py | 363 ++++++++++++++++++ .../research_scripts/run_codex_batches.sh | 337 ++++++++++++++++ .../research_scripts/validate_results.py | 164 ++++++++ .../ream250_bom/ream250_bom_row_0195_6Q.md | 48 +++ .../ream250_bom/ream250_bom_row_0308_174.md | 48 +++ .../ream250_bom/ream250_bom_row_0380_4122.md | 51 +++ 11 files changed, 1771 insertions(+) create mode 100644 queue_tasks/research_mission/ream250_bom_research/README.md create mode 100644 queue_tasks/research_mission/ream250_bom_research/README.zh.md create mode 100644 queue_tasks/research_mission/ream250_bom_research/research_instructions/agent.md create mode 100644 queue_tasks/research_mission/ream250_bom_research/research_schemas/research_result.schema.yaml create mode 100755 queue_tasks/research_mission/ream250_bom_research/research_scripts/extract_step_materials.py create mode 100755 queue_tasks/research_mission/ream250_bom_research/research_scripts/generate_queue_tasks.py create mode 100755 queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh create mode 100755 queue_tasks/research_mission/ream250_bom_research/research_scripts/validate_results.py create mode 100644 research/ream250_bom/ream250_bom_row_0195_6Q.md create mode 100644 research/ream250_bom/ream250_bom_row_0308_174.md create mode 100644 research/ream250_bom/ream250_bom_row_0380_4122.md diff --git a/queue_tasks/research_mission/ream250_bom_research/README.md b/queue_tasks/research_mission/ream250_bom_research/README.md new file mode 100644 index 00000000..e536244d --- /dev/null +++ b/queue_tasks/research_mission/ream250_bom_research/README.md @@ -0,0 +1,197 @@ +# reAM250 BOM Research Task Pack + +This folder contains one-off support files for the reAM250 BOM research run. It +is not part of the generic research queue system. + +## Files + +- `research_instructions/agent.md` - Prompt/instructions for a Codex agent processing + reAM250 BOM research queue items. +- `research_schemas/research_result.schema.yaml` - Expected structured result shape. +- `research_scripts/generate_queue_tasks.py` - Build queue items from the gold + CSV/manifest package, optionally extracting STEP metadata with FreeCAD. +- `research_scripts/validate_results.py` - Local validator for result Markdown/YAML/JSON + files. +- `research_scripts/run_codex_batches.sh` - Optional batch runner that repeatedly starts + fresh `codex exec` sessions. + +## Queue Requirements + +The queue should contain research tasks with: + +- `kind: research` +- `gap_type: research_task` +- IDs starting with `research_task:ream250_bom_row_` +- `context.output_path` under `research/ream250_bom/` +- `context.output_validator` pointing to this task pack validator + +Generate or refresh the 401 queue items from the gold CSV/manifest: + +```bash +.venv/bin/python queue_tasks/research_mission/ream250_bom_research/research_scripts/generate_queue_tasks.py \ + --replace-queue-prefix +``` + +This replaces only existing queue entries whose IDs start with +`research_task:ream250_bom_row_`. + +CAD geometry is intentionally read by the agent after it leases a specific row. +Use `--extract-cad-metadata` only for offline diagnostics, not for the normal +research queue run. + +Lease with hard filters: + +```bash +.venv/bin/python -m src.cli queue lease \ + --agent ream250-bom-agent-01 \ + --ttl 7200 \ + --kind research \ + --gap-type research_task \ + --id-prefix research_task:ream250_bom_row_ +``` + +## Agent Usage + +Open Codex from the repo root: + +```bash +cd /home/eastrolinux/seres +codex --search -C /home/eastrolinux/seres -s workspace-write -a on-request +``` + +Then tell the agent: + +```text +Read queue_tasks/research_mission/ream250_bom_research/research_instructions/agent.md and follow it as ream250-bom-agent-01. +``` + +Use a different agent name in each terminal, such as `ream250-bom-agent-02`. + +## Session Limit + +Each agent session should process at most 3 queue items. Restart or clear the +session for the next batch. This keeps web research context bounded and makes +failures easier to resume. + +## Automated Batch Runner + +For larger runs, use the task-local runner instead of manually clearing Codex or +opening new terminals. The runner starts a fresh `codex exec` session for each +small batch, so context does not accumulate across the whole BOM. + +Conservative default: + +```bash +queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh +``` + +Two workers, three rows per fresh Codex session: + +```bash +queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh \ + --workers 2 \ + --max-items 3 +``` + +Smoke test one Codex session: + +```bash +queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh \ + --max-batches 1 +``` + +Print the generated prompt without running Codex: + +```bash +queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh --dry-run +``` + +Logs are written to `out/ream250_bom_runner_logs/` by default. + +### Targeted Reruns + +The runner has an optional `--id-prefix` filter. If it is omitted, the runner +uses the normal broad prefix: + +```text +research_task:ream250_bom_row_ +``` + +That default means "any reAM250 BOM research row". Normal runs do not need to +pass `--id-prefix`. + +The option is named `--id-prefix` because the queue lease API filters with +`startswith(...)`, not exact-id matching. Passing a complete queue id still works +as an exact single-row filter because the complete id is also a valid prefix of +itself. + +To rerun a completed row, first release it back to `pending`, then run one +single-item batch with the complete queue id as the prefix: + +```bash +.venv/bin/python -m src.cli queue release \ + --id research_task:ream250_bom_row_0195_6Q \ + --agent rerun-targeted + +queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh \ + --workers 1 \ + --max-items 1 \ + --max-batches 1 \ + --id-prefix research_task:ream250_bom_row_0195_6Q +``` + +The runner prompt requires the agent to overwrite an existing output file after +re-checking evidence. If you are testing that behavior, verify the file mtime or +inspect the log for an actual file write. + +Rerun the first three completed smoke-test rows in one shell loop: + +```bash +for id in research_task:ream250_bom_row_0308_174 research_task:ream250_bom_row_0195_6Q research_task:ream250_bom_row_0380_4122; do .venv/bin/python -m src.cli queue release --id "$id" --agent rerun-targeted && queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh --workers 1 --max-items 1 --max-batches 1 --id-prefix "$id"; done +``` + +### Runner Risks + +- If a Codex session crashes after leasing an item, that item remains leased + until its TTL expires. Run `.venv/bin/python -m src.cli queue gc` after the TTL + to return expired leases to pending. +- Parallel workers increase web-search/API usage and can hit external rate + limits. Start with `--workers 1` or `--workers 2`. +- Do not run `python -m src.cli index` while the runner is active. This workflow + relies on the research queue as the state source. +- The runner does not guarantee research quality. It only bounds context and + automates fresh Codex sessions; use `research_scripts/validate_results.py` to check + required result structure and source fields. + +## Validate Results + +Validate one file: + +```bash +.venv/bin/python queue_tasks/research_mission/ream250_bom_research/research_scripts/validate_results.py \ + --file research/ream250_bom/ream250_bom_row_0001_11.md +``` + +Validate a directory: + +```bash +.venv/bin/python queue_tasks/research_mission/ream250_bom_research/research_scripts/validate_results.py \ + --dir research/ream250_bom +``` + +The validator checks that `function`, `mass`, `material`, and `how_to_make` +each have their own source object containing: + +- `url_or_path` +- `cited_fact_or_basis` +- `confidence` + +## Completion + +Complete research tasks without `--verify`: + +```bash +.venv/bin/python -m src.cli queue complete --id --agent --require-output --validate-output +``` + +Do not run `python -m src.cli index` during this one-off research workflow. diff --git a/queue_tasks/research_mission/ream250_bom_research/README.zh.md b/queue_tasks/research_mission/ream250_bom_research/README.zh.md new file mode 100644 index 00000000..688f9ded --- /dev/null +++ b/queue_tasks/research_mission/ream250_bom_research/README.zh.md @@ -0,0 +1,187 @@ +# reAM250 BOM 研究任務包 + +這個資料夾只服務 reAM250 BOM 這次單次研究任務，不是通用 research +queue 系統的一部分。 + +## 檔案 + +- `research_instructions/agent.md` - 給 Codex agent 的 reAM250 BOM 研究指令。 +- `research_schemas/research_result.schema.yaml` - 結果檔應符合的結構。 +- `research_scripts/generate_queue_tasks.py` - 從 gold CSV/manifest 產生 queue + items，可選擇用 FreeCAD 抽 STEP metadata。 +- `research_scripts/validate_results.py` - 檢查 Markdown/YAML/JSON 結果檔的本地驗證器。 +- `research_scripts/run_codex_batches.sh` - 選用的 batch runner，會反覆啟動新的 + `codex exec` session。 + +## Queue 條件 + +queue 裡的任務應該符合： + +- `kind: research` +- `gap_type: research_task` +- ID 以 `research_task:ream250_bom_row_` 開頭 +- `context.output_path` 位於 `research/ream250_bom/` +- `context.output_validator` 指到這個任務包的 validator + +從 gold CSV/manifest 產生或刷新 401 筆 queue items： + +```bash +.venv/bin/python queue_tasks/research_mission/ream250_bom_research/research_scripts/generate_queue_tasks.py \ + --replace-queue-prefix +``` + +這只會替換 ID 以 `research_task:ream250_bom_row_` 開頭的既有 queue entries。 + +CAD 幾何資料刻意由 agent 在 lease 到特定 row 後才讀取。`--extract-cad-metadata` +只用於離線診斷，不作為正常 research queue run 的流程。 + +租任務時使用 hard filters： + +```bash +.venv/bin/python -m src.cli queue lease \ + --agent ream250-bom-agent-01 \ + --ttl 7200 \ + --kind research \ + --gap-type research_task \ + --id-prefix research_task:ream250_bom_row_ +``` + +## Agent 使用方式 + +在 repo root 開 Codex： + +```bash +cd /home/eastrolinux/seres +codex --search -C /home/eastrolinux/seres -s workspace-write -a on-request +``` + +進入 Codex 後貼： + +```text +Read queue_tasks/research_mission/ream250_bom_research/research_instructions/agent.md and follow it as ream250-bom-agent-01. +``` + +不同 terminal 使用不同 agent 名稱，例如 `ream250-bom-agent-02`。 + +## 每個 Session 的工作上限 + +每個 agent session 最多處理 3 筆 queue item。做完 3 筆就停，下一輪重新開 +或 `/clear`。這可以降低 web research 把 context 撐爆的機率，也比較容易恢復。 + +## 自動 Batch Runner + +如果要跑大量 rows，不要手動一直 `/clear` 或開新 terminal。可以使用這個 +task-local runner；它每一小批都會啟動新的 `codex exec`，所以 context 不會 +在整份 BOM 期間持續累積。 + +保守預設： + +```bash +queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh +``` + +兩個 worker，每個新的 Codex session 最多處理 3 rows： + +```bash +queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh \ + --workers 2 \ + --max-items 3 +``` + +先測一個 Codex session： + +```bash +queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh \ + --max-batches 1 +``` + +只印出產生的 prompt，不實際執行 Codex： + +```bash +queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh --dry-run +``` + +log 預設會寫到 `out/ream250_bom_runner_logs/`。 + +### 指定 Row 重跑 + +runner 有選用的 `--id-prefix` filter。不加時會使用正常的寬 prefix： + +```text +research_task:ream250_bom_row_ +``` + +這個預設值代表「任何 reAM250 BOM research row」。正常批次執行不需要加 +`--id-prefix`。 + +它叫 `--id-prefix` 是因為 queue lease API 用的是 `startswith(...)` 過濾， +不是 exact-id matching。把完整 queue id 傳進去仍然等同於指定單一 row， +因為完整 id 也是它自己的 prefix。 + +如果要重跑已完成 row，先把該 queue item release 回 `pending`，再用完整 +queue id 當 prefix 跑一個單筆 batch： + +```bash +.venv/bin/python -m src.cli queue release \ + --id research_task:ream250_bom_row_0195_6Q \ + --agent rerun-targeted + +queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh \ + --workers 1 \ + --max-items 1 \ + --max-batches 1 \ + --id-prefix research_task:ream250_bom_row_0195_6Q +``` + +runner prompt 會要求 agent 在重新檢查證據後覆寫既有 output file。若你是在測試 +是否真的重寫，請檢查檔案 mtime，或看 log 裡是否有實際寫檔動作。 + +用一行 shell loop 重跑前三筆 smoke-test 結果： + +```bash +for id in research_task:ream250_bom_row_0308_174 research_task:ream250_bom_row_0195_6Q research_task:ream250_bom_row_0380_4122; do .venv/bin/python -m src.cli queue release --id "$id" --agent rerun-targeted && queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh --workers 1 --max-items 1 --max-batches 1 --id-prefix "$id"; done +``` + +### Runner 風險 + +- 如果 Codex session 在 lease 任務後中斷，該任務會維持 leased 到 TTL + 過期。TTL 到期後可跑 `.venv/bin/python -m src.cli queue gc` 回收。 +- 平行 worker 會增加 web search/API 使用量，也更容易遇到外部 rate limit。 + 建議先從 `--workers 1` 或 `--workers 2` 開始。 +- runner 執行時不要跑 `python -m src.cli index`。這個流程把 research queue + 當作狀態來源。 +- runner 不保證研究品質；它只負責限制 context 並自動啟動新的 Codex + session。結果格式與 source 欄位仍要用 `research_scripts/validate_results.py` 檢查。 + +## 驗證結果 + +驗證單一檔案： + +```bash +.venv/bin/python queue_tasks/research_mission/ream250_bom_research/research_scripts/validate_results.py \ + --file research/ream250_bom/ream250_bom_row_0001_11.md +``` + +驗證整個資料夾： + +```bash +.venv/bin/python queue_tasks/research_mission/ream250_bom_research/research_scripts/validate_results.py \ + --dir research/ream250_bom +``` + +驗證器會檢查 `function`、`mass`、`material`、`how_to_make` 是否各自有 +source object，且包含： + +- `url_or_path` +- `cited_fact_or_basis` +- `confidence` + +## 完成任務 + +research task 完成時不要加 `--verify`： + +```bash +.venv/bin/python -m src.cli queue complete --id --agent --require-output --validate-output +``` + +這個單次研究流程期間不要跑 `python -m src.cli index`。 diff --git a/queue_tasks/research_mission/ream250_bom_research/research_instructions/agent.md b/queue_tasks/research_mission/ream250_bom_research/research_instructions/agent.md new file mode 100644 index 00000000..85c2878e --- /dev/null +++ b/queue_tasks/research_mission/ream250_bom_research/research_instructions/agent.md @@ -0,0 +1,170 @@ +# Agent Instructions: reAM250 BOM Research + +You are processing one-off reAM250 BOM research tasks. + +## Lease Only Matching Research Tasks + +Lease with hard filters: + +```bash +.venv/bin/python -m src.cli queue lease \ + --agent \ + --ttl 7200 \ + --kind research \ + --gap-type research_task \ + --id-prefix research_task:ream250_bom_row_ +``` + +If the command returns `queue empty`, stop. + +If a leased item does not have `kind: research` or its `id` does not start with +`research_task:ream250_bom_row_`, release it immediately and stop. + +## Process A Small Batch + +Process at most the item limit given by the current user or runner prompt. If no +limit is provided, process at most 3 queue items in one Codex session. Stop when +that limit is reached even if more queue items remain. + +## Allowed Edits + +Only create or update result files under: + +```text +research/ream250_bom/ +``` + +Do not edit KB YAML, source code, docs, queue tasks, or generated index files. +Do not run: + +```bash +python -m src.cli index +``` + +## Research Requirements + +Use the leased item's `context` fields: + +- `source_csv` +- `manifest_csv` +- `bom_row_number` +- `source_row_number` +- `quantity` +- `cad_file` +- `canonical_step_path` +- `cad_export_status` +- `description_or_product_id` +- `manufacturer` +- `third_party_link_url` +- `output_path` + +After leasing a task, read the row's CAD file if `canonical_step_path` exists +and `cad_export_status` is not `missing_in_cad`. Use the local FreeCAD wrapper: + +```bash +.tools/freecad/freecadcmd -c "import Part; p=''; s=Part.Shape(); s.read(p); bb=s.BoundBox; print(len(s.Solids), s.Volume, s.Area, bb.XLength, bb.YLength, bb.ZLength)" +``` + +Use that geometry as row-specific evidence for mass and shape/function +inference. If `cad_export_status` is `assembly_only`, `ambiguous`, or +`missing_in_cad`, explain the CAD evidence limitation in `uncertainty_notes`. + +For material, use this evidence order: + +1. Check local CAD/STEP material metadata first. The per-part STEP often omits + material, but the full assembly may contain it. If + `design/real-mechanical/reAm250/reAM250_cad_gold_package/gold_export/assemblies/00_assembly.step` + exists, run: + + ```bash + .venv/bin/python queue_tasks/research_mission/ream250_bom_research/research_scripts/extract_step_materials.py \ + --step design/real-mechanical/reAm250/reAM250_cad_gold_package/gold_export/assemblies/00_assembly.step \ + --product-name "" + ``` + + Cite the material and density from this output when present. +2. Check the BOM row fields, manufacturer, `description_or_product_id`, and + `third_party_link_url`. +3. Use web search/product pages when a manufacturer, product ID, or vendor URL + is available and local CAD/BOM evidence does not state material or conflicts + with surrounding evidence. +4. If none of those sources states the material, do not name a specific metal + or polymer from function alone. Use a broader value such as + `unknown metal/alloy` with low confidence, and record any mass estimate as a + scenario in `mass.basis`. + +Do not write values like `steel, assumed`, `aluminum, assumed`, or +`stainless steel, assumed` in `material.primary_material`. A function-based +guess is an uncertainty note, not a sourced material. + +Keep research concise. Use at most 4 external sources per result unless the row +cannot be resolved without more. + +## Result Format + +Write each result to `context.output_path`. + +If `context.output_path` already exists, treat it as a stale prior draft. You +may read it for comparison, but you must re-check the leased row's current BOM, +CAD geometry, assembly material metadata, and web/vendor evidence as applicable, +then overwrite the result file. Do not complete a leased task by validating an +existing output file as-is. + +Each Markdown result must start with YAML frontmatter: + +```yaml +--- +function: + summary: + source: + url_or_path: + cited_fact_or_basis: + confidence: +mass: + value_kg: + basis: + source: + url_or_path: + cited_fact_or_basis: + confidence: +material: + primary_material: + source: + url_or_path: + cited_fact_or_basis: + confidence: +how_to_make: + summary: + manufacturing_steps: [] + source: + url_or_path: + cited_fact_or_basis: + confidence: +assumptions: [] +uncertainty_notes: [] +kb_implications: [] +--- +``` + +Hard source rule: `function`, `mass`, `material`, and `how_to_make` must each +have their own source object with: + +- `url_or_path` +- `cited_fact_or_basis` +- `confidence` + +## Validate Before Completion + +Run: + +```bash +.venv/bin/python queue_tasks/research_mission/ream250_bom_research/research_scripts/validate_results.py --file +``` + +Do not complete the queue item if validation fails. + +Complete without `--verify`: + +```bash +.venv/bin/python -m src.cli queue complete --id --agent --require-output --validate-output +``` diff --git a/queue_tasks/research_mission/ream250_bom_research/research_schemas/research_result.schema.yaml b/queue_tasks/research_mission/ream250_bom_research/research_schemas/research_result.schema.yaml new file mode 100644 index 00000000..3f7cab2b --- /dev/null +++ b/queue_tasks/research_mission/ream250_bom_research/research_schemas/research_result.schema.yaml @@ -0,0 +1,52 @@ +# Minimal structured result schema for this reAM250 BOM research task pack. +required: + - function + - mass + - material + - how_to_make + - assumptions + - uncertainty_notes + - kb_implications + +sections: + function: + required: + - summary + - source + source_required: + - url_or_path + - cited_fact_or_basis + - confidence + mass: + required: + - value_kg + - basis + - source + source_required: + - url_or_path + - cited_fact_or_basis + - confidence + material: + required: + - primary_material + - source + source_required: + - url_or_path + - cited_fact_or_basis + - confidence + how_to_make: + required: + - summary + - manufacturing_steps + - source + source_required: + - url_or_path + - cited_fact_or_basis + - confidence + +recommended: + confidence_values: + - low + - medium + - high + - unknown diff --git a/queue_tasks/research_mission/ream250_bom_research/research_scripts/extract_step_materials.py b/queue_tasks/research_mission/ream250_bom_research/research_scripts/extract_step_materials.py new file mode 100755 index 00000000..6af86dbf --- /dev/null +++ b/queue_tasks/research_mission/ream250_bom_research/research_scripts/extract_step_materials.py @@ -0,0 +1,154 @@ +#!/usr/bin/env python3 +"""Extract product material metadata from a STEP assembly file. + +This is intentionally small and STEP-text oriented. It targets the AP214-style +material properties exported in the reAM250 gold package, where product +definitions have PROPERTY_DEFINITION entries for "material name" and +"density of part". +""" +from __future__ import annotations + +import argparse +import json +import re +from pathlib import Path +from typing import Dict, Iterable, List + + +ENTITY_RE = re.compile(r"#(\d+)\s*=\s*(.*?);", re.DOTALL) +REF_RE = re.compile(r"#(\d+)") +STRING_RE = re.compile(r"'((?:[^']|'')*)'") +POSITIVE_RATIO_RE = re.compile(r"POSITIVE_RATIO_MEASURE\(([-+0-9.Ee]+)\)") + + +def load_entities(path: Path) -> Dict[str, str]: + text = path.read_text(encoding="utf-8", errors="replace") + return {match.group(1): normalize(match.group(2)) for match in ENTITY_RE.finditer(text)} + + +def normalize(value: str) -> str: + return " ".join(value.split()) + + +def step_strings(entity: str) -> List[str]: + return [value.replace("''", "'") for value in STRING_RE.findall(entity)] + + +def refs(entity: str) -> List[str]: + return REF_RE.findall(entity) + + +def first_ref(entity: str) -> str | None: + found = refs(entity) + return found[0] if found else None + + +def find_product_definitions(entities: Dict[str, str], product_name: str) -> List[str]: + needle = product_name.lower() + matches: List[str] = [] + for entity_id, body in entities.items(): + if not body.startswith("PRODUCT_DEFINITION("): + continue + strings = [value.lower() for value in step_strings(body)] + if any(needle == value or needle in value for value in strings): + matches.append(entity_id) + return matches + + +def property_definitions( + entities: Dict[str, str], product_definition_id: str, property_label: str +) -> Iterable[str]: + wanted_ref = f"#{product_definition_id}" + for entity_id, body in entities.items(): + if not body.startswith("PROPERTY_DEFINITION("): + continue + strings = [value.lower() for value in step_strings(body)] + if len(strings) < 2: + continue + if strings[0] == "material property" and strings[1] == property_label: + if refs(body)[-1:] == [product_definition_id] or wanted_ref in body: + yield entity_id + + +def representation_for_property(entities: Dict[str, str], property_definition_id: str) -> str | None: + for body in entities.values(): + if not body.startswith("PROPERTY_DEFINITION_REPRESENTATION("): + continue + entity_refs = refs(body) + if len(entity_refs) >= 2 and entity_refs[0] == property_definition_id: + return entity_refs[1] + return None + + +def representation_item(entities: Dict[str, str], representation_id: str | None) -> str | None: + if not representation_id: + return None + body = entities.get(representation_id) + if not body or not body.startswith("REPRESENTATION("): + return None + entity_refs = refs(body) + return entity_refs[0] if entity_refs else None + + +def descriptive_value(entities: Dict[str, str], item_id: str | None) -> str | None: + if not item_id: + return None + body = entities.get(item_id, "") + values = step_strings(body) + return values[0] if values else None + + +def density_value(entities: Dict[str, str], item_id: str | None) -> float | None: + if not item_id: + return None + body = entities.get(item_id, "") + match = POSITIVE_RATIO_RE.search(body) + if not match: + return None + return float(match.group(1)) + + +def extract_for_product(entities: Dict[str, str], product_name: str) -> List[dict]: + results: List[dict] = [] + for product_definition_id in find_product_definitions(entities, product_name): + material_prop_ids = list( + property_definitions(entities, product_definition_id, "material name") + ) + density_prop_ids = list( + property_definitions(entities, product_definition_id, "density of part") + ) + material_repr = representation_for_property( + entities, material_prop_ids[0] + ) if material_prop_ids else None + density_repr = representation_for_property( + entities, density_prop_ids[0] + ) if density_prop_ids else None + material_item = representation_item(entities, material_repr) + density_item = representation_item(entities, density_repr) + results.append( + { + "product_definition_id": f"#{product_definition_id}", + "material_property_id": f"#{material_prop_ids[0]}" if material_prop_ids else None, + "density_property_id": f"#{density_prop_ids[0]}" if density_prop_ids else None, + "material": descriptive_value(entities, material_item), + "density": density_value(entities, density_item), + "density_unit_note": "STEP positive ratio measure; reAM250 export uses kg/m^3-like material densities", + } + ) + return results + + +def main() -> int: + parser = argparse.ArgumentParser(description="Extract STEP material metadata for one product") + parser.add_argument("--step", required=True, type=Path, help="STEP assembly path") + parser.add_argument("--product-name", required=True, help="Product name to match") + args = parser.parse_args() + + entities = load_entities(args.step) + results = extract_for_product(entities, args.product_name) + print(json.dumps({"product_name": args.product_name, "matches": results}, indent=2)) + return 0 if results else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/queue_tasks/research_mission/ream250_bom_research/research_scripts/generate_queue_tasks.py b/queue_tasks/research_mission/ream250_bom_research/research_scripts/generate_queue_tasks.py new file mode 100755 index 00000000..57b35f8a --- /dev/null +++ b/queue_tasks/research_mission/ream250_bom_research/research_scripts/generate_queue_tasks.py @@ -0,0 +1,363 @@ +#!/usr/bin/env python3 +"""Generate reAM250 BOM research queue tasks from the gold CSV package.""" +from __future__ import annotations + +import argparse +import csv +import fcntl +import hashlib +import json +import re +import subprocess +import sys +import tempfile +import time +from pathlib import Path +from typing import Any + + +REPO_ROOT = Path(__file__).resolve().parents[4] +TASK_DIR = Path("queue_tasks/research_mission/ream250_bom_research") +PACKAGE_ROOT = Path("design/real-mechanical/reAm250/reAM250_cad_gold_package") +PACKAGE_ABS = REPO_ROOT / PACKAGE_ROOT +DEFAULT_BOM = PACKAGE_ROOT / "reAm250_BOM_gold.csv" +DEFAULT_MANIFEST = PACKAGE_ROOT / "gold_export/manifest.csv" +DEFAULT_OUTPUT = Path("out/ream250_bom_research_tasks.jsonl") +DEFAULT_QUEUE = Path("out/work_queue.jsonl") +DEFAULT_LOCK = Path("out/work_queue.lock") +DEFAULT_REGISTRY = Path("queue/gap_type_registry.json") +TASK_PREFIX = "research_task:ream250_bom_row_" +ITEM_PREFIX = "ream250_bom_row_" +OUTPUT_DIR = Path("research/ream250_bom") +VALIDATOR = TASK_DIR / "research_scripts/validate_results.py" +FREECADCMD = Path(".tools/freecad/freecadcmd") + + +def safe_token(value: str) -> str: + token = re.sub(r"[^A-Za-z0-9]+", "_", value.strip()).strip("_") + return token or "unknown" + + +def repo_relative(path: Path) -> str: + try: + return str(path.resolve().relative_to(REPO_ROOT)) + except ValueError: + return str(path) + + +def read_csv(path: Path) -> list[dict[str, str]]: + with path.open(newline="", encoding="utf-8-sig") as f: + return list(csv.DictReader(f)) + + +def manifest_key(row: dict[str, str]) -> tuple[str, str, str]: + return ( + (row.get("source_row_number") or "").strip(), + (row.get("item") or "").strip(), + (row.get("cad_file") or "").strip(), + ) + + +def bom_key(source_row_number: int, row: dict[str, str]) -> tuple[str, str, str]: + return ( + str(source_row_number), + (row.get("Item") or "").strip(), + (row.get("CAD file") or "").strip(), + ) + + +def extract_step_metadata(paths: list[Path], freecadcmd: Path) -> dict[str, dict[str, Any]]: + if not paths: + return {} + if not freecadcmd.exists(): + raise FileNotFoundError(f"FreeCADCmd wrapper not found: {freecadcmd}") + + script = r""" +import json +import sys + +import Part + +input_path = sys.argv[2] +output_path = sys.argv[3] + +with open(input_path, "r", encoding="utf-8") as f: + paths = json.load(f) + +result = {} +for path in paths: + try: + shape = Part.Shape() + shape.read(path) + bb = shape.BoundBox + result[path] = { + "read_ok": True, + "solid_count": len(shape.Solids), + "volume_mm3": shape.Volume, + "surface_area_mm2": shape.Area, + "bbox_mm": { + "x": bb.XLength, + "y": bb.YLength, + "z": bb.ZLength, + }, + } + except Exception as exc: + result[path] = { + "read_ok": False, + "error": str(exc), + } + +with open(output_path, "w", encoding="utf-8") as f: + json.dump(result, f, indent=2, sort_keys=True) +""" + with tempfile.TemporaryDirectory() as tmpdir: + tmp = Path(tmpdir) + script_path = tmp / "extract_step_metadata.py" + input_path = tmp / "paths.json" + output_path = tmp / "metadata.json" + script_path.write_text(script, encoding="utf-8") + input_path.write_text( + json.dumps([str(path) for path in paths], indent=2), + encoding="utf-8", + ) + proc = subprocess.run( + [str(freecadcmd), str(script_path), str(input_path), str(output_path)], + cwd=REPO_ROOT, + text=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + check=False, + ) + if proc.returncode != 0: + raise RuntimeError( + "FreeCAD metadata extraction failed\n" + f"stdout:\n{proc.stdout}\n" + f"stderr:\n{proc.stderr}" + ) + data = json.loads(output_path.read_text(encoding="utf-8")) + return {str(Path(path)): value for path, value in data.items()} + + +def build_tasks( + bom_path: Path, + manifest_path: Path, + extract_cad_metadata: bool, + freecadcmd: Path, +) -> list[dict[str, Any]]: + bom_rows = read_csv(bom_path) + manifest_rows = read_csv(manifest_path) + manifest = {manifest_key(row): row for row in manifest_rows} + + metadata_by_path: dict[str, dict[str, Any]] = {} + if extract_cad_metadata: + paths: list[Path] = [] + seen: set[Path] = set() + for row in manifest_rows: + rel = (row.get("canonical_step_path") or "").strip() + if not rel: + continue + step_path = (PACKAGE_ABS / rel).resolve() + if step_path.exists() and step_path not in seen: + paths.append(step_path) + seen.add(step_path) + metadata_by_path = extract_step_metadata(paths, freecadcmd.resolve()) + + tasks: list[dict[str, Any]] = [] + for ordinal, row in enumerate(bom_rows, start=1): + source_row_number = ordinal + 1 + item = (row.get("Item") or "").strip() + cad_file = (row.get("CAD file") or "").strip() + key = bom_key(source_row_number, row) + manifest_row = manifest.get(key) + if not manifest_row: + raise ValueError(f"no manifest match for BOM row {source_row_number}: {key}") + + item_id = f"{ITEM_PREFIX}{source_row_number:04d}_{safe_token(item)}" + output_path = OUTPUT_DIR / f"{item_id}.md" + canonical_rel = (manifest_row.get("canonical_step_path") or "").strip() + canonical_path_abs = (PACKAGE_ABS / canonical_rel).resolve() if canonical_rel else None + canonical_path = repo_relative(canonical_path_abs) if canonical_path_abs else "" + canonical_abs = str(canonical_path_abs) if canonical_path_abs else "" + cad_metadata = metadata_by_path.get(canonical_abs) + cad_hash = "" + if canonical_path_abs and canonical_path_abs.exists(): + cad_hash = hashlib.sha256(canonical_path_abs.read_bytes()).hexdigest() + + context: dict[str, Any] = { + "source_csv": repo_relative(bom_path), + "manifest_csv": repo_relative(manifest_path), + "bom_row_number": str(source_row_number), + "source_row_number": source_row_number, + "item": item, + "quantity": (row.get("Qty") or "").strip(), + "cad_file": cad_file, + "description_or_product_id": (row.get("Description / Product ID") or "").strip(), + "manufacturer": (row.get("Manufacturer") or "").strip(), + "third_party_link_url": (row.get("Link URL") or "").strip(), + "subsystem_suggested": (row.get("Subsystem (suggested)") or "").strip(), + "material_family_hint": (row.get("Material family") or "").strip(), + "specific_material_grade_hint": (row.get("Specific material / grade") or "").strip(), + "notes_from_gold_csv": (row.get("Notes") or "").strip(), + "raw_row_text": (row.get("Raw row text") or "").strip(), + "canonical_step_path": canonical_path, + "alternate_step_paths": (manifest_row.get("alternate_step_paths") or "").strip(), + "cad_export_status": (manifest_row.get("export_status") or "").strip(), + "cad_export_kind": (manifest_row.get("export_kind") or "").strip(), + "cad_source_object_label": (manifest_row.get("source_object_label") or "").strip(), + "cad_parent_assembly": (manifest_row.get("parent_assembly") or "").strip(), + "cad_instance_count_found": (manifest_row.get("instance_count_found") or "").strip(), + "cad_notes": (manifest_row.get("notes") or "").strip(), + "cad_sha256": cad_hash, + "cad_evidence_limited": (manifest_row.get("export_status") or "").strip() + not in {"matched_existing"}, + "required_outputs": ["function", "mass", "material", "how_to_make"], + "output_path": str(output_path), + "output_validator": str(VALIDATOR), + "done_criteria": ( + "Write the result to output_path. Include YAML frontmatter with " + "function, mass, material, and how_to_make sections; each section " + "must include its own source object. Validate the result and complete " + "the queue task with --require-output --validate-output." + ), + } + if cad_metadata is not None: + context["cad_metadata"] = cad_metadata + + tasks.append( + { + "kind": "research", + "gap_type": "research_task", + "item_id": item_id, + "source": "manual", + "description": ( + f"Research reAM250 BOM row {source_row_number} item {item} " + f"({cad_file}) for function, mass, material, and manufacturing." + ), + "context": context, + } + ) + return tasks + + +def write_jsonl(path: Path, tasks: list[dict[str, Any]]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + with path.open("w", encoding="utf-8") as f: + for task in tasks: + f.write(json.dumps(task, ensure_ascii=False, sort_keys=True) + "\n") + + +def queue_entry_for_task(task: dict[str, Any]) -> dict[str, Any]: + context = dict(task.get("context") or {}) + context["added_at"] = time.time() + gap_type = task["gap_type"] + item_id = task["item_id"] + if task.get("description") and "description" not in context: + context["description"] = task["description"] + return { + "id": f"{gap_type}:{item_id}", + "kind": task.get("kind", "gap"), + "reason": gap_type, + "gap_type": gap_type, + "item_id": item_id, + "source": task.get("source", "manual"), + "context": context, + "status": "pending", + } + + +def replace_queue_prefix(queue_path: Path, lock_path: Path, tasks: list[dict[str, Any]]) -> int: + queue_path.parent.mkdir(parents=True, exist_ok=True) + lock_path.parent.mkdir(parents=True, exist_ok=True) + with lock_path.open("w") as lockf: + fcntl.flock(lockf, fcntl.LOCK_EX) + try: + existing: list[dict[str, Any]] = [] + if queue_path.exists(): + for line in queue_path.read_text(encoding="utf-8").splitlines(): + if not line.strip(): + continue + try: + obj = json.loads(line) + except Exception: + continue + if not str(obj.get("id", "")).startswith(TASK_PREFIX): + existing.append(obj) + new_entries = [queue_entry_for_task(task) for task in tasks] + with queue_path.open("w", encoding="utf-8") as f: + for obj in existing + new_entries: + f.write(json.dumps(obj, ensure_ascii=False, sort_keys=True) + "\n") + return len(new_entries) + finally: + fcntl.flock(lockf, fcntl.LOCK_UN) + + +def ensure_research_task_registry(registry_path: Path, usage_count: int) -> None: + registry_path.parent.mkdir(parents=True, exist_ok=True) + if registry_path.exists(): + try: + registry = json.loads(registry_path.read_text(encoding="utf-8")) + except Exception: + registry = {} + else: + registry = {} + + entry = registry.setdefault( + "research_task", + { + "description": "Manual research task completed by writing task-specific artifacts.", + "created_by": "ream250_bom_research_task_pack", + "created_at": time.time(), + "usage_count": 0, + "source": "manual", + }, + ) + entry["description"] = entry.get("description") or "Manual research task completed by writing task-specific artifacts." + entry["source"] = entry.get("source") or "manual" + entry["usage_count"] = usage_count + registry_path.write_text(json.dumps(registry, indent=2), encoding="utf-8") + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--bom", type=Path, default=DEFAULT_BOM) + parser.add_argument("--manifest", type=Path, default=DEFAULT_MANIFEST) + parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT) + parser.add_argument("--extract-cad-metadata", action="store_true") + parser.add_argument("--freecadcmd", type=Path, default=FREECADCMD) + parser.add_argument( + "--replace-queue-prefix", + action="store_true", + help="Replace existing out/work_queue.jsonl entries with the reAM250 task prefix.", + ) + parser.add_argument("--queue", type=Path, default=DEFAULT_QUEUE) + parser.add_argument("--lock", type=Path, default=DEFAULT_LOCK) + parser.add_argument("--registry", type=Path, default=DEFAULT_REGISTRY) + args = parser.parse_args() + + bom_path = args.bom if args.bom.is_absolute() else REPO_ROOT / args.bom + manifest_path = args.manifest if args.manifest.is_absolute() else REPO_ROOT / args.manifest + output_path = args.output if args.output.is_absolute() else REPO_ROOT / args.output + freecadcmd = args.freecadcmd if args.freecadcmd.is_absolute() else REPO_ROOT / args.freecadcmd + queue_path = args.queue if args.queue.is_absolute() else REPO_ROOT / args.queue + lock_path = args.lock if args.lock.is_absolute() else REPO_ROOT / args.lock + registry_path = args.registry if args.registry.is_absolute() else REPO_ROOT / args.registry + + tasks = build_tasks( + bom_path, + manifest_path, + args.extract_cad_metadata, + freecadcmd, + ) + write_jsonl(output_path, tasks) + print(f"Wrote {len(tasks)} tasks to {repo_relative(output_path)}") + + if args.replace_queue_prefix: + added = replace_queue_prefix(queue_path, lock_path, tasks) + ensure_research_task_registry(registry_path, added) + print(f"Replaced {TASK_PREFIX} entries in {repo_relative(queue_path)} with {added} pending tasks") + + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh b/queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh new file mode 100755 index 00000000..47b2fa0a --- /dev/null +++ b/queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh @@ -0,0 +1,337 @@ +#!/usr/bin/env bash +set -euo pipefail + +TASK_DIR="queue_tasks/research_mission/ream250_bom_research" +TASK_INSTRUCTIONS="$TASK_DIR/research_instructions/agent.md" +TASK_VALIDATOR="$TASK_DIR/research_scripts/validate_results.py" +DEFAULT_REPO_ROOT="/home/eastrolinux/seres" + +repo_root="${REPO_ROOT:-$DEFAULT_REPO_ROOT}" +agent_prefix="ream250-bom-agent" +workers=1 +max_items=3 +max_batches=0 +ttl=7200 +log_dir="" +codex_bin="${CODEX_BIN:-codex}" +validate_at_end=0 +dry_run=0 +id_prefix="research_task:ream250_bom_row_" + +usage() { + cat <<'EOF' +Usage: + queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh [options] + +Runs short-lived Codex exec sessions for reAM250 BOM research queue items. +Each session gets a fresh context window and processes at most --max-items. + +Options: + --repo-root PATH Repository root. Default: /home/eastrolinux/seres + --agent-prefix NAME Agent name prefix. Default: ream250-bom-agent + --workers N Number of parallel worker loops. Default: 1 + --max-items N Max queue items per Codex exec session. Default: 3 + --max-batches N Max Codex exec sessions per worker. Default: 0, until queue empty + --ttl SECONDS Queue lease TTL passed to the agent prompt. Default: 7200 + --log-dir PATH Log directory. Default: out/ream250_bom_runner_logs + --codex-bin PATH Codex executable. Default: codex or $CODEX_BIN + --id-prefix PREFIX Queue id prefix for lease filtering. + Default: research_task:ream250_bom_row_ + --validate-at-end Validate research/ream250_bom after all workers exit + --dry-run Print the first prompt and command, then exit + -h, --help Show this help + +Examples: + # Conservative single-worker run. + queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh + + # Two workers, each Codex session handles at most 3 rows. + queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh --workers 2 --max-items 3 + + # Smoke test one fresh Codex session. + queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh --max-batches 1 + + # Rerun exactly one completed row after releasing it back to pending. + .venv/bin/python -m src.cli queue release --id research_task:ream250_bom_row_0195_6Q --agent rerun + queue_tasks/research_mission/ream250_bom_research/research_scripts/run_codex_batches.sh --id-prefix research_task:ream250_bom_row_0195_6Q --max-items 1 --max-batches 1 +EOF +} + +die() { + echo "error: $*" >&2 + exit 1 +} + +is_positive_int() { + [[ "${1:-}" =~ ^[1-9][0-9]*$ ]] +} + +is_nonnegative_int() { + [[ "${1:-}" =~ ^[0-9]+$ ]] +} + +while [[ $# -gt 0 ]]; do + case "$1" in + --repo-root) + repo_root="${2:-}" + shift 2 + ;; + --agent-prefix) + agent_prefix="${2:-}" + shift 2 + ;; + --workers) + workers="${2:-}" + shift 2 + ;; + --max-items) + max_items="${2:-}" + shift 2 + ;; + --max-batches) + max_batches="${2:-}" + shift 2 + ;; + --ttl) + ttl="${2:-}" + shift 2 + ;; + --log-dir) + log_dir="${2:-}" + shift 2 + ;; + --codex-bin) + codex_bin="${2:-}" + shift 2 + ;; + --id-prefix) + id_prefix="${2:-}" + shift 2 + ;; + --validate-at-end) + validate_at_end=1 + shift + ;; + --dry-run) + dry_run=1 + shift + ;; + -h|--help) + usage + exit 0 + ;; + *) + die "unknown option: $1" + ;; + esac +done + +is_positive_int "$workers" || die "--workers must be a positive integer" +is_positive_int "$max_items" || die "--max-items must be a positive integer" +is_nonnegative_int "$max_batches" || die "--max-batches must be a nonnegative integer" +is_positive_int "$ttl" || die "--ttl must be a positive integer" +[[ -n "$id_prefix" ]] || die "--id-prefix must not be empty" + +repo_root="$(cd "$repo_root" && pwd)" +[[ -d "$repo_root/.git" ]] || die "repo root does not contain .git: $repo_root" +[[ -f "$repo_root/$TASK_INSTRUCTIONS" ]] || die "missing task instructions: $TASK_INSTRUCTIONS" +[[ -f "$repo_root/$TASK_VALIDATOR" ]] || die "missing task validator: $TASK_VALIDATOR" +[[ -x "$repo_root/.venv/bin/python" ]] || die "missing .venv/bin/python; run uv sync first" + +if [[ -z "$log_dir" ]]; then + log_dir="$repo_root/out/ream250_bom_runner_logs" +elif [[ "$log_dir" != /* ]]; then + log_dir="$repo_root/$log_dir" +fi +mkdir -p "$log_dir" + +queue_gc() { + ( + cd "$repo_root" + .venv/bin/python -m src.cli queue gc >/dev/null + ) || true +} + +pending_count() { + ( + cd "$repo_root" + ID_PREFIX="$id_prefix" .venv/bin/python - <<'PY' +import fcntl +import json +import os +import time +from pathlib import Path + +id_prefix = os.environ["ID_PREFIX"] +queue_path = Path("out/work_queue.jsonl") +lock_path = Path("out/work_queue.lock") +lock_path.parent.mkdir(parents=True, exist_ok=True) +now = time.time() +count = 0 + +if not queue_path.exists(): + print(0) + raise SystemExit + +with lock_path.open("w") as lockf: + fcntl.flock(lockf, fcntl.LOCK_EX) + try: + with queue_path.open("r", encoding="utf-8") as f: + for line in f: + try: + obj = json.loads(line) + except Exception: + continue + if obj.get("kind") != "research": + continue + if (obj.get("gap_type") or obj.get("reason")) != "research_task": + continue + if not str(obj.get("id", "")).startswith(id_prefix): + continue + status = obj.get("status") + if status in (None, "pending"): + count += 1 + elif status == "leased" and obj.get("lease_expires_at", 0) < now: + count += 1 + finally: + fcntl.flock(lockf, fcntl.LOCK_UN) + +print(count) +PY + ) +} + +build_prompt() { + local agent_name="$1" + local item_limit="$2" + + cat < + +If already exists, treat it as a stale prior draft. You may read +it for comparison, but you must re-check the row's current BOM, CAD geometry, +assembly material metadata, and web/vendor evidence as applicable, then +overwrite the result file. Do not complete a leased task by validating an +existing output file as-is. + +Complete finished tasks without --verify: + +.venv/bin/python -m src.cli queue complete --id --agent ${agent_name} --require-output --validate-output + +Do not edit KB YAML, source code, docs, queue tasks, generated index files, or system files. +Only create or update the requested result files under research/ream250_bom/. +Do not run python -m src.cli index. +EOF +} + +run_one_batch() { + local worker_id="$1" + local batch_no="$2" + local agent_name + local log_file + local status + + agent_name="$(printf "%s-%02d" "$agent_prefix" "$worker_id")" + log_file="$log_dir/${agent_name}_batch_$(printf "%04d" "$batch_no")_$(date +%Y%m%d_%H%M%S).log" + + echo "[$agent_name] starting batch $batch_no; log: $log_file" + set +e + build_prompt "$agent_name" "$max_items" | ( + cd "$repo_root" && + "$codex_bin" --search -a on-request exec -C "$repo_root" -s workspace-write - + ) 2>&1 | tee "$log_file" + status=${PIPESTATUS[1]} + set -e + + if [[ "$status" -ne 0 ]]; then + echo "[$agent_name] codex exec failed with status $status; see $log_file" >&2 + return "$status" + fi +} + +worker_loop() { + local worker_id="$1" + local batch_no=1 + local pending + + while true; do + if [[ "$max_batches" -gt 0 && "$batch_no" -gt "$max_batches" ]]; then + echo "[$(printf "%s-%02d" "$agent_prefix" "$worker_id")] reached --max-batches $max_batches" + return 0 + fi + + queue_gc + pending="$(pending_count)" + if [[ "$pending" -eq 0 ]]; then + echo "[$(printf "%s-%02d" "$agent_prefix" "$worker_id")] no pending matching reAM250 research tasks" + return 0 + fi + + run_one_batch "$worker_id" "$batch_no" + batch_no=$((batch_no + 1)) + done +} + +if [[ "$dry_run" -eq 1 ]]; then + echo "Repository: $repo_root" + echo "Command: $codex_bin --search -a on-request exec -C $repo_root -s workspace-write -" + echo + build_prompt "$(printf "%s-%02d" "$agent_prefix" 1)" "$max_items" + exit 0 +fi + +command -v "$codex_bin" >/dev/null 2>&1 || die "codex executable not found: $codex_bin" + +echo "repo_root=$repo_root" +echo "workers=$workers max_items=$max_items max_batches=$max_batches ttl=$ttl" +echo "log_dir=$log_dir" + +if [[ "$workers" -eq 1 ]]; then + worker_loop 1 +else + pids=() + for worker_id in $(seq 1 "$workers"); do + worker_loop "$worker_id" & + pids+=("$!") + done + + failed=0 + for pid in "${pids[@]}"; do + if ! wait "$pid"; then + failed=1 + fi + done + [[ "$failed" -eq 0 ]] || die "one or more workers failed" +fi + +if [[ "$validate_at_end" -eq 1 ]]; then + ( + cd "$repo_root" + .venv/bin/python "$TASK_VALIDATOR" --dir research/ream250_bom + ) +fi + +echo "reAM250 BOM batch runner finished" diff --git a/queue_tasks/research_mission/ream250_bom_research/research_scripts/validate_results.py b/queue_tasks/research_mission/ream250_bom_research/research_scripts/validate_results.py new file mode 100755 index 00000000..aa28a701 --- /dev/null +++ b/queue_tasks/research_mission/ream250_bom_research/research_scripts/validate_results.py @@ -0,0 +1,164 @@ +#!/usr/bin/env python3 +"""Validate reAM250 BOM research result files. + +This is a one-off task-pack validator, intentionally kept outside the core queue +system. It accepts JSON, YAML, or Markdown files with YAML frontmatter. +""" +from __future__ import annotations + +import argparse +import json +import re +from pathlib import Path +from typing import Any, Dict, Iterable, List + +import yaml + + +REQUIRED_SECTIONS = ("function", "mass", "material", "how_to_make") +SOURCE_FIELDS = ("url_or_path", "cited_fact_or_basis", "confidence") +REQUIRED_LISTS = ("assumptions", "uncertainty_notes", "kb_implications") +CONFIDENCE_VALUES = {"low", "medium", "high", "unknown"} +ASSUMED_MATERIAL_RE = re.compile( + r"\b(assumed|assumption|guess|guessed|likely|suggests?|conservative)\b", + re.IGNORECASE, +) + + +def load_result(path: Path) -> Dict[str, Any]: + text = path.read_text(encoding="utf-8") + suffix = path.suffix.lower() + if suffix == ".json": + data = json.loads(text) + elif suffix in {".yaml", ".yml"}: + data = yaml.safe_load(text) + else: + data = load_markdown_frontmatter(text, path) + if not isinstance(data, dict): + raise ValueError(f"result must be a mapping: {path}") + return data + + +def load_markdown_frontmatter(text: str, path: Path) -> Dict[str, Any]: + if not text.startswith("---\n"): + raise ValueError(f"markdown result must start with YAML frontmatter: {path}") + end = text.find("\n---", 4) + if end == -1: + raise ValueError(f"markdown frontmatter is not closed: {path}") + data = yaml.safe_load(text[4:end]) or {} + if not isinstance(data, dict): + raise ValueError(f"frontmatter must be a mapping: {path}") + return data + + +def validate_result(data: Dict[str, Any]) -> List[str]: + issues: List[str] = [] + for section_name in REQUIRED_SECTIONS: + section = data.get(section_name) + if not isinstance(section, dict): + issues.append(f"missing required object section: {section_name}") + continue + issues.extend(validate_source(section, section_name)) + + mass = data.get("mass") if isinstance(data.get("mass"), dict) else {} + value_kg = mass.get("value_kg") + if value_kg in (None, ""): + issues.append("missing required field: mass.value_kg") + elif not isinstance(value_kg, (int, float)) or isinstance(value_kg, bool): + issues.append("mass.value_kg must be a number") + + function = data.get("function") if isinstance(data.get("function"), dict) else {} + material = data.get("material") if isinstance(data.get("material"), dict) else {} + how_to_make = data.get("how_to_make") if isinstance(data.get("how_to_make"), dict) else {} + + if not function.get("summary"): + issues.append("missing required field: function.summary") + if not mass.get("basis"): + issues.append("missing required field: mass.basis") + if not material.get("primary_material"): + issues.append("missing required field: material.primary_material") + else: + primary_material = str(material.get("primary_material")) + if ASSUMED_MATERIAL_RE.search(primary_material): + issues.append( + "material.primary_material must be sourced or broad/unknown; " + "do not encode assumed specific materials" + ) + if not how_to_make.get("summary"): + issues.append("missing required field: how_to_make.summary") + if "manufacturing_steps" not in how_to_make: + issues.append("missing required field: how_to_make.manufacturing_steps") + elif not isinstance(how_to_make.get("manufacturing_steps"), list): + issues.append("how_to_make.manufacturing_steps must be a list") + + for field in REQUIRED_LISTS: + if field not in data: + issues.append(f"missing required field: {field}") + elif not isinstance(data.get(field), list): + issues.append(f"{field} must be a list") + + return issues + + +def validate_source(section: Dict[str, Any], path: str) -> List[str]: + source = section.get("source") + if not isinstance(source, dict): + return [f"missing required object: {path}.source"] + issues = [] + for field in SOURCE_FIELDS: + if source.get(field) in (None, ""): + issues.append(f"missing required field: {path}.source.{field}") + confidence = source.get("confidence") + if isinstance(confidence, str) and confidence.strip().lower() not in CONFIDENCE_VALUES: + issues.append( + f"{path}.source.confidence must be one of: " + f"{', '.join(sorted(CONFIDENCE_VALUES))}" + ) + return issues + + +def iter_paths(files: Iterable[Path], directory: Path | None) -> List[Path]: + paths = list(files) + if directory: + if not directory.exists(): + raise FileNotFoundError(f"result directory does not exist: {directory}") + if not directory.is_dir(): + raise NotADirectoryError(f"result path is not a directory: {directory}") + paths.extend( + sorted( + path for path in directory.iterdir() + if path.is_file() and path.suffix.lower() in {".md", ".json", ".yaml", ".yml"} + ) + ) + return paths + + +def main() -> int: + parser = argparse.ArgumentParser(description="Validate reAM250 BOM research result files") + parser.add_argument("--file", action="append", default=[], type=Path, help="Result file to validate") + parser.add_argument("--dir", type=Path, help="Directory of result files to validate") + args = parser.parse_args() + + paths = iter_paths(args.file, args.dir) + if not paths: + parser.error("provide --file or --dir") + + failed = False + for path in paths: + try: + issues = validate_result(load_result(path)) + except Exception as exc: + issues = [str(exc)] + if issues: + failed = True + print(f"{path}: FAIL") + for issue in issues: + print(f" - {issue}") + else: + print(f"{path}: OK") + + return 1 if failed else 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/research/ream250_bom/ream250_bom_row_0195_6Q.md b/research/ream250_bom/ream250_bom_row_0195_6Q.md new file mode 100644 index 00000000..31e03ba7 --- /dev/null +++ b/research/ream250_bom/ream250_bom_row_0195_6Q.md @@ -0,0 +1,48 @@ +--- +function: + summary: "Small stainless-steel belt pulley or pulley mount component for the reAM250 belt path; the source name identifies a belt pulley without teeth, and the matched CAD file is a compact single solid." + source: + url_or_path: "design/real-mechanical/reAm250/reAM250_cad_gold_package/reAm250_BOM_gold.csv:195 and design/real-mechanical/reAm250/reAM250_cad_gold_package/gold_export/manifest.csv:195" + cited_fact_or_basis: "BOM row 195 lists item 6Q, quantity 9, as '6Q_mount_belt_pulley_without_teeth'; manifest row 195 maps it to a matched part STEP export with one instance found." + confidence: medium +mass: + value_kg: 0.014259 + basis: "Per unit mass from CAD volume and STEP material density: FreeCAD reports volume 1782.4339059137299 mm^3 for one solid. The assembly STEP material extractor reports Stainless Steel with density 8000.0 kg/m^3-like units, giving 0.014259 kg per part. BOM quantity 9 implies about 0.128335 kg total for the row." + source: + url_or_path: "design/real-mechanical/reAm250/reAM250_cad_gold_package/gold_export/parts/6Q_mount_belt_pulley_without_teeth.step" + cited_fact_or_basis: "FreeCAD Part.Shape read of the STEP file: 1 solid, volume 1782.4339059137299 mm^3, area 1184.884312061741 mm^2, bounding box 13.684210526315127 x 20.0 x 30.800000000000228 mm." + confidence: high +material: + primary_material: "Stainless Steel" + source: + url_or_path: "design/real-mechanical/reAm250/reAM250_cad_gold_package/gold_export/assemblies/00_assembly.step" + cited_fact_or_basis: "extract_step_materials.py for product '6Q_mount_belt_pulley_without_teeth' found material 'Stainless Steel' and density 8000.0." + confidence: high +how_to_make: + summary: "Make as a small stainless machined pulley or pulley-mount part: cut a blank, turn or mill the belt-contact geometry from the CAD model, finish any bore or mounting features, then deburr and inspect." + manufacturing_steps: + - "Cut stainless stock or near-net blank sized for the 13.7 x 20.0 x 30.8 mm CAD envelope." + - "Turn or mill the circular belt-contact/pulley surfaces and any central mounting geometry from the STEP model." + - "Drill, ream, or finish attachment features shown in the CAD model." + - "Deburr, clean, and inspect dimensions and surface finish before installation in the belt path." + source: + url_or_path: "design/real-mechanical/reAm250/reAM250_cad_gold_package/gold_export/parts/6Q_mount_belt_pulley_without_teeth.step" + cited_fact_or_basis: "Manufacturing route inferred from the matched single-solid stainless CAD geometry and compact pulley-like part name; no vendor process sheet or third-party part link is present in the BOM row." + confidence: medium +assumptions: + - "CAD units are interpreted as millimeters, consistent with the small 13.7 x 20.0 x 30.8 bounding box." + - "The mass value is per physical part, not the row total; multiply by BOM quantity 9 for row mass." + - "Because no manufacturer or third-party link is present, the manufacturing route is a practical process inference from CAD geometry and material metadata." +uncertainty_notes: + - "The BOM row has no manufacturer, product ID, material family hint, or third-party URL." + - "The part name says belt pulley without teeth, but the STEP file was not semantically annotated beyond the product name and geometry." + - "No local or vendor evidence identifies a specific stainless grade, heat treatment, bearing interface, or surface finish requirement." +kb_implications: + - "Model as a small discrete stainless pulley or pulley-mount part for belt guidance, with per-unit mass about 0.0143 kg." + - "Do not create a specific stainless alloy grade from this row alone; use generic stainless steel unless later evidence identifies a grade." + - "For a simplified KB BOM, this may fold into a belt pulley or motion-axis hardware group if fine pulley variants are below the needed modeling resolution." +--- + +# reAM250 BOM Row 195 - Item 6Q + +The current row evidence supports a concise component entry: quantity 9 of a small stainless belt pulley or pulley-mount part. CAD geometry is available and matched, and the full assembly STEP provides explicit Stainless Steel material metadata, so the mass estimate is stronger than rows with only generic material tags. diff --git a/research/ream250_bom/ream250_bom_row_0308_174.md b/research/ream250_bom/ream250_bom_row_0308_174.md new file mode 100644 index 00000000..ccd6475b --- /dev/null +++ b/research/ream250_bom/ream250_bom_row_0308_174.md @@ -0,0 +1,48 @@ +--- +function: + summary: "Dummy scanner flange for the reAM250 scanner/flange area; the BOM name and adjacent EOS scanner-flange row indicate a blank or substitute flange used where the scanner flange interface is present." + source: + url_or_path: "design/real-mechanical/reAm250/reAM250_cad_gold_package/reAm250_BOM_gold.csv:308 and :309" + cited_fact_or_basis: "Row 308 lists item 174 as '174_dummy_scanner_flange' from EOS; row 309 lists adjacent item 179 as '179_scanner_flange' with description 'EOS M270 flange'." + confidence: medium +mass: + value_kg: 0.760053 + basis: "FreeCAD geometry for the matched STEP file reports one solid with volume 760053.4076104617 mm^3. The full assembly STEP material extractor reports material 'Generic' with density 1000.0 kg/m^3-like units for this product, giving 0.760053 kg. Treat as CAD/export-metadata mass, not a verified vendor mass; if made from aluminum-like metal the same volume would be about 2.05 kg, and if made from steel-like metal about 5.97 kg." + source: + url_or_path: "design/real-mechanical/reAm250/reAM250_cad_gold_package/gold_export/parts/174_dummy_scanner_flange.step" + cited_fact_or_basis: "FreeCAD Part.Shape read of the STEP file: 1 solid, volume 760053.4076104617 mm^3, area 132298.02378840625 mm^2, bounding box 219.0 x 75.0 x 219.0 mm." + confidence: medium +material: + primary_material: "unspecified generic CAD material" + source: + url_or_path: "design/real-mechanical/reAm250/reAM250_cad_gold_package/gold_export/assemblies/00_assembly.step" + cited_fact_or_basis: "extract_step_materials.py for product '174_dummy_scanner_flange' found material 'Generic' and density 1000.0; BOM row gives EOS as manufacturer but no material family or grade." + confidence: low +how_to_make: + summary: "Manufacture as a simple blanking/substitute scanner-interface flange after selecting the engineering material; the CAD geometry is a single solid flange-sized part and is compatible with subtractive machining or additive/near-net fabrication followed by hole/face finishing." + manufacturing_steps: + - "Select the required engineering material from design or service constraints; the available evidence does not identify a real grade." + - "Create the flange blank to the CAD envelope and features from the STEP model." + - "Machine the mounting faces and holes/interfaces to final tolerance." + - "Deburr, clean, and inspect against the CAD model before installation." + source: + url_or_path: "design/real-mechanical/reAm250/reAM250_cad_gold_package/gold_export/manifest.csv:308" + cited_fact_or_basis: "Manifest row 308 maps item 174 to gold_export/parts/174_dummy_scanner_flange.step with export_status matched_existing, export_kind vendor_component, source_object_label 174_dummy_scanner_flange, and instance_count_found 1." + confidence: medium +assumptions: + - "The 'dummy' name means a blanking or substitute flange rather than the active scanner flange." + - "CAD units are interpreted as millimeters, consistent with the reported 219 x 75 x 219 bounding box." + - "The 1000 kg/m^3-like density is retained only because it is explicit STEP metadata; it may be a default placeholder." +uncertainty_notes: + - "No third-party link, EOS part number, material family, or material grade is present in the BOM row." + - "The assembly STEP material metadata says 'Generic', so the material is not resolved to steel, aluminum, polymer, or another specific material." + - "Web searches for EOS dummy scanner flange / M270 scanner flange did not produce a usable vendor material or mass source for this specific item." + - "Mass could vary by several times if the physical part uses a metal density rather than the exported Generic density." +kb_implications: + - "For KB modeling, treat as one discrete scanner-interface blank/flange component with unresolved material unless a vendor drawing or service manual identifies the grade." + - "Do not create a specific steel or aluminum material link from this research alone." +--- + +# reAM250 BOM Row 308 - Item 174 + +The current evidence supports a concise, conservative entry: one EOS dummy scanner flange, locally represented by a matched STEP file. The CAD geometry is available and usable for shape and volume, but the material metadata is generic and should not be treated as a real material grade. diff --git a/research/ream250_bom/ream250_bom_row_0380_4122.md b/research/ream250_bom/ream250_bom_row_0380_4122.md new file mode 100644 index 00000000..e7244289 --- /dev/null +++ b/research/ream250_bom/ream250_bom_row_0380_4122.md @@ -0,0 +1,51 @@ +--- +function: + summary: "SKF radial shaft seal for the reAM250 powder inlet/feedthrough area; the part designation 36X47X7 HMSA10 RG indicates a 36 mm shaft, about 47 mm outside diameter, and 7 mm axial-width rotary shaft seal." + source: + url_or_path: "design/real-mechanical/reAm250/reAM250_cad_gold_package/reAm250_BOM_gold.csv:380 and design/real-mechanical/reAm250/reAM250_cad_gold_package/gold_export/assemblies/00_assembly.step:4011172" + cited_fact_or_basis: "BOM row 380 lists quantity 2 of '4122_radial_shaft_seal_SKF_36X47X7 HMSA10 RG' from SKF; the assembly STEP product description says 'SAA_001-Radial shaft seals for general industrial applications'." + confidence: high +mass: + value_kg: 0.003361 + basis: "Per-unit mass from CAD volume and explicit STEP material density: FreeCAD reports volume 3360.6006267956573 mm^3 for one solid. The assembly STEP material extractor reports material 'Generic' with density 1000.0 kg/m^3-like units, giving 0.003361 kg per seal. BOM quantity 2 implies about 0.006721 kg total for the row. Treat this as CAD/export-metadata mass rather than verified SKF catalog mass." + source: + url_or_path: "design/real-mechanical/reAm250/reAM250_cad_gold_package/gold_export/parts/4122_radial_shaft_seal_SKF_36X47X7 HMSA10 RG.step" + cited_fact_or_basis: "FreeCAD Part.Shape read of the STEP file: 1 solid, volume 3360.6006267956573 mm^3, area 4353.102543451458 mm^2, bounding box 7.000000000000291 x 50.87243341374252 x 50.87243341374252 mm." + confidence: medium +material: + primary_material: "unspecified generic CAD material" + source: + url_or_path: "design/real-mechanical/reAm250/reAM250_cad_gold_package/gold_export/assemblies/00_assembly.step" + cited_fact_or_basis: "extract_step_materials.py for product '4122_radial_shaft_seal_SKF_36X47X7 HMSA10 RG' found material 'Generic' and density 1000.0; the BOM row provides SKF and product designation but no sourced material grade." + confidence: low +how_to_make: + summary: "Best modeled as a purchased standard SKF radial shaft seal. If localized manufacturing is required, first resolve the actual seal materials, then produce the seal as a small rotary sealing assembly with molded/final-formed sealing geometry, any reinforcing or spring elements required by the design, and dimensional inspection against the 36 x 47 x 7 mm class geometry." + manufacturing_steps: + - "Use the SKF/BOM designation to procure or specify the correct 36 x 47 x 7 mm HMSA10 RG radial shaft seal." + - "For local manufacture, resolve the real elastomer, reinforcement, and spring materials from a vendor drawing or data sheet before assigning KB material inputs." + - "Form or mold the sealing body and lip geometry to the STEP/CAD envelope, preserving the shaft bore and outside diameter." + - "Install any required reinforcing or spring elements only after their materials and dimensions are sourced." + - "Inspect outside diameter, shaft bore, axial width, concentricity, and sealing lip condition before installation." + source: + url_or_path: "design/real-mechanical/reAm250/reAM250_cad_gold_package/reAm250_BOM_gold.csv:380 and design/real-mechanical/reAm250/reAM250_cad_gold_package/gold_export/manifest.csv:380" + cited_fact_or_basis: "The BOM identifies an SKF standard radial-shaft-seal designation and vendor URL; the manifest maps it to a matched_existing vendor_component STEP export with one CAD instance found." + confidence: medium +assumptions: + - "The mass value is per physical seal, not the row total; multiply by BOM quantity 2 for row mass." + - "CAD units are interpreted as millimeters, consistent with the 36X47X7 product designation and the 7 mm axial bounding-box dimension." + - "The BOM vendor URL was treated as current row evidence, but live SKF lookup was not available in this environment because DNS resolution failed for www.skf.com." + - "The product designation appears to encode nominal seal dimensions, while the CAD bounding box reports an approximately 50.9 mm outer envelope; this may reflect model details or export geometry rather than catalog nominal outside diameter." +uncertainty_notes: + - "The full assembly STEP material metadata says 'Generic', so it should not be interpreted as a real material grade." + - "No local CAD metadata, BOM field, or accessible vendor page resolved the primary elastomer, metal case, garter spring, or coating material." + - "Do not record this as steel, aluminum, stainless steel, or NBR unless later vendor data or drawings identify those materials." + - "Mass could differ from the CAD/export estimate if the physical SKF seal includes denser reinforcement or spring elements not represented by the generic-density CAD solid." +kb_implications: + - "For KB modeling, represent this as a small purchased radial shaft seal with unresolved material and per-unit mass about 0.00336 kg unless a vendor catalog mass is later found." + - "Avoid creating a specific material recipe from this row alone; use an import or generic seal component until materials are sourced." + - "The item may be grouped with standard rotary seal hardware in a simplified reAM250 BOM if individual seal variants are below the desired modeling resolution." +--- + +# reAM250 BOM Row 380 - Item 4122 + +Current evidence supports a conservative COTS component entry: quantity 2 of an SKF 36X47X7 HMSA10 RG radial shaft seal. CAD geometry is matched and useful for shape and approximate mass, but material evidence is only generic STEP metadata, so the seal material should remain unresolved until a vendor data sheet or drawing is available.