diff --git a/docs/architecture/rfcs/typescript-control-plane-migration-v0.md b/docs/architecture/rfcs/typescript-control-plane-migration-v0.md index 055ed300a3..94da0f355d 100644 --- a/docs/architecture/rfcs/typescript-control-plane-migration-v0.md +++ b/docs/architecture/rfcs/typescript-control-plane-migration-v0.md @@ -2236,6 +2236,48 @@ The same real File/SQLite entrypoints must qualify functional parity and the original full-CLI cost gate together. Fewer crossings alone do not explain a whole-CLI tail regression or complete writer/backup/default-provider retirement. +Fresh public-boundary scanning retains its specialized Python regex / Host IO +owner. Characterization covers all eleven `splitlines` +separators, Unicode regex folding, casefold expansion false positives, ordered +multi-rule hits, credential references, the public-host exception and literal +metacharacters. These are scanner semantics, not a new decision owner or cache. + +A folded-literal union intervention reduced the exact quota scan population's +median wall / CPU cost by 8.5% / 6.9% with complete scanner payload parity. +Its fixed 32 alternating full-CLI pairs per provider still produced File p95 +976.33 → 986.84 ms (+10.52 ms / 1.08%) and SQLite p95 936.00 → 962.34 ms +(+26.33 ms / 2.81%). The latter misses the unchanged unexplained 25 ms target, +so the runtime candidate was withdrawn. All samples remain evidence; profiler +observations do not establish the tail cause or waive the qualification. +Further full-payload controls rejected folded-line reuse and literal regex +disjunctions; composing the existing direct-loop and line-union interventions +improved scanner wall time by only 3.6%. An independent fixed 32-pair A/A control +per provider, with identical runtime bytes and the same complete CLI workload, +still differed by -8.20 ms / -0.84% File p95 and -85.95 ms / -9.90% SQLite p95 +between checkout contexts. All 128 commands retained the expected selection and +frontier. This identifies a comparison confound, not its cause or an optimizer +qualification; the earlier failed candidate and all observations remain. +A same-checkout cold/warm control then isolated runtime startup: all 32 cold +samples changed the setup PID and were slower than their warm partner; warm +samples retained substantial variability. This explains a startup contribution, +not the historical A/A tail difference. + +The next intervention rejects unsupported regular filenames before repeated +canonical resolution in both Git enumeration and directory traversal. Symlinks +still resolve before suffix/local-name eligibility; tracked pruned-directory +files, explicit file roots and fresh retargets retain their original behavior. +On the unchanged package-root workload it removed 602 of 3,281 resolution calls +with full payload parity; eight alternating scan pairs improved median wall +time by 5.1%. Fixed 32-pair full quota CLI comparisons per provider used one +physical checkout and verified warm setup PID reuse in every sample. File +p50/p95 was 708.70/891.78 → 689.04/856.32 ms; SQLite was +758.55/1036.27 → 769.41/1026.34 ms. Both pass the original additive 25 ms and +5% tail budgets, while SQLite's slightly higher median limits the improvement +claim. This qualifies this enumeration intervention, not the withdrawn filter, +the historical frontier/direct-loop holds or a frozen multi-command deadline. +Preserve those separately from new-Goal defaults, recoverable imports, the +developer trial and last-caller retirement. + 2026-09-24: [Typed complete-source assembly and remaining delivery packages](ledger/shared-goal-authority-state-provider-v0/2026-09-24-source-capture.md) unify source construction, identity rejection and current-graph membership; L7/D2/D3 and provider defaults remain open. 2026-09-24: [Leased continuation and remaining local-default packages](ledger/shared-goal-authority-state-provider-v0/2026-09-24-leased-continuation.md). diff --git a/docs/architecture/rfcs/typescript-control-plane-migration-v0.zh-CN.md b/docs/architecture/rfcs/typescript-control-plane-migration-v0.zh-CN.md index 24b2b79339..8e5f29567e 100644 --- a/docs/architecture/rfcs/typescript-control-plane-migration-v0.zh-CN.md +++ b/docs/architecture/rfcs/typescript-control-plane-migration-v0.zh-CN.md @@ -1630,6 +1630,36 @@ TS 摘要批次;Python 保留旧格式解码、公开字段筛选及渲染。 查询命中情况覆盖。见[语义及回滚](../../reference/todo-work-counts.md)。这是 T3/L5 的 共享读取边界推进,不替代 D2/D3 或 provider 默认切换。 +fresh 公共边界扫描继续归现有专门 Python regex/Host IO owner。 +Characterization 已覆盖全部十一种 `splitlines` 分隔符、Unicode regex 折叠、 +casefold 扩展的假阳性、逐规则命中顺序、凭据引用、公共主机例外及字面量元字符; +这些是 scanner 语义,不新增决策 owner 或缓存。 + +折叠字面量合并的受控干预,在真实 quota 扫描集合上降低中位 wall/CPU 成本 +8.5%/6.9%,完整 scanner payload 相等。但每 provider 固定 32 对交替完整 CLI +比较仍得到 File p95 976.33 → 986.84 ms(+10.52 ms/1.08%)、SQLite p95 +936.00 → 962.34 ms(+26.33 ms/2.81%)。后者未通过既有未解释 25 ms 目标, +因此撤回运行时候选,保留全部样本;profile 不能证明尾延迟根因或豁免资格。 +后续完整 payload 对照淘汰了 folded-line 复用及 literal regex 合并;既有 +direct-loop 与 line-union 组合仅降低 scanner wall 3.6%。另一次独立固定每 provider +32 对 A/A 对照,在运行时字节及完整 CLI 负载相同的两个 checkout 间,仍得到 +File p95 -8.20 ms/-0.84%、SQLite p95 -85.95 ms/-9.90%;全部 128 次命令的 +selection/frontier 均符合预期。这证明比较存在混杂,尚未证明其根因或优化资格; +此前候选失败及全部观察继续保留。同一 checkout 的冷暖对照随后隔离了 runtime +启动:32 次冷态均更换 setup PID,且比暖态配对更慢;暖态仍明显波动。 +这证明启动成本贡献,未解释历史 A/A 尾差。 + +下一项干预在 Git 枚举和目录遍历中,先跳过后缀不支持的普通文件,再做 canonical +路径解析;符号链接仍先解析目标,再判断后缀及 local 名称。Git 跟踪的剪枝目录、 +显式文件 root 及链接改指向后的 fresh 读取保持原行为。同一 package-root 负载少做 +602/3,281 次解析,完整 payload 相等;8 对交替 scanner 中位 wall 改善 5.1%。 +每 provider 固定 32 对完整 quota CLI,在同一 physical checkout 逐次确认暖态 +setup PID 复用。File p50/p95 为 708.70/891.78 → 689.04/856.32 ms;SQLite 为 +758.55/1036.27 → 769.41/1026.34 ms,均通过原 25 ms 及 5% 尾延迟预算。 +SQLite 中位数略升,收益不能外推为全面加速。这只资格化本次枚举干预,不改写 +已撤回 filter 的失败,也不解除历史 frontier/direct-loop hold 或冻结多命令时限。 +这些与新 Goal 默认、可恢复导入、开发者试用及最后调用方退役分别保留。 + 2026-09-24:[完整源捕获的 TS 组装与剩余交付包](ledger/shared-goal-authority-state-provider-v0/2026-09-24-source-capture.zh-CN.md)统一源构造、身份拒绝和当前图成员规则;不关闭 L7/D2/D3 或启用默认 provider。 2026-09-24: [带租约接力与剩余本地默认交付包](ledger/shared-goal-authority-state-provider-v0/2026-09-24-leased-continuation.zh-CN.md). diff --git a/loopx/contract.py b/loopx/contract.py index 1cd68d6bf8..992f39746a 100644 --- a/loopx/contract.py +++ b/loopx/contract.py @@ -847,7 +847,10 @@ def _tracked_scan_files(scan_root: Path) -> list[Path]: for rel_path in tracked.stdout.split("\0"): if not rel_path: continue - path = (repo_root / rel_path).resolve() + path = repo_root / rel_path + if path.suffix not in DEFAULT_SCAN_SUFFIXES and not path.is_symlink(): + continue + path = path.resolve() if path.name.endswith(".local.json"): continue if path.is_file() and path.suffix in DEFAULT_SCAN_SUFFIXES: @@ -879,7 +882,10 @@ def iter_scan_files(scan_root: Path) -> list[Path]: and not os.path.isfile(current_dir / name / "pyvenv.cfg") ] for file_name in file_names: - path = (current_dir / file_name).resolve() + path = current_dir / file_name + if path.suffix not in DEFAULT_SCAN_SUFFIXES and not path.is_symlink(): + continue + path = path.resolve() if path.name.endswith(".local.json"): continue if not path.is_file(): diff --git a/loopx/semantics/project_registry_io_manifest_v1.json b/loopx/semantics/project_registry_io_manifest_v1.json index b3a360cedc..78f7daceb1 100644 --- a/loopx/semantics/project_registry_io_manifest_v1.json +++ b/loopx/semantics/project_registry_io_manifest_v1.json @@ -1039,7 +1039,7 @@ }, { "site": "loopx/contract.py::.check_contract::codec_read:load_registry#1", - "line": 1093, + "line": 1099, "column": 20, "kind": "codec_read", "api": "load_registry", diff --git a/tests/test_contract_public_boundary_prefilter.py b/tests/test_contract_public_boundary_prefilter.py index cf6380f8d9..d8ab82eeb7 100644 --- a/tests/test_contract_public_boundary_prefilter.py +++ b/tests/test_contract_public_boundary_prefilter.py @@ -1,6 +1,7 @@ from __future__ import annotations from pathlib import Path +import subprocess from typing import Any import pytest @@ -8,6 +9,54 @@ from loopx import contract +@pytest.mark.parametrize("tracked,pruned", [(False, False), (True, False), (True, True)]) +def test_scan_enumeration_uses_resolved_symlink_suffix_and_local_name( + tmp_path: Path, tracked: bool, pruned: bool +) -> None: + scan_root = tmp_path / ("node_modules" if pruned else "public") + scan_root.mkdir() + target = tmp_path / "target.md" + target.write_text("public", encoding="utf-8") + distinct = tmp_path / "distinct.md" + distinct.write_text("public", encoding="utf-8") + unsupported = tmp_path / "target.ts" + unsupported.write_text("not a directory scan input", encoding="utf-8") + local = tmp_path / "target.local.json" + local.write_text("local", encoding="utf-8") + regular = scan_root / "regular.md" + regular.write_text("public", encoding="utf-8") + (scan_root / "regular.ts").write_text("unsupported", encoding="utf-8") + (scan_root / "alias.ts").symlink_to(target) + (scan_root / "distinct.txt").symlink_to(distinct) + (scan_root / "alias.local.json").symlink_to(target) + (scan_root / "unsupported.md").symlink_to(unsupported) + (scan_root / "local.md").symlink_to(local) + (scan_root / "broken.ts").symlink_to(tmp_path / "absent.md") + if tracked: + subprocess.run(["git", "init", "-q", str(tmp_path)], check=True) + subprocess.run(["git", "-C", str(tmp_path), "add", "-f", "--", scan_root.name], check=True) + + # Git ownership includes tracked files under otherwise pruned directories. + # Directory eligibility follows the canonical target, not the alias name. + # The pruned-root fast path preserves Git's per-alias observations. + expected = [distinct, regular, target, target] if pruned else [distinct, regular, target] + assert contract.iter_scan_files(scan_root) == sorted(expected) + if tracked: + assert set(contract._tracked_scan_files(scan_root)) == {distinct, regular, target} + + # A later retarget must be observed; no eligibility result may be cached. + (scan_root / "alias.ts").unlink() + (scan_root / "alias.ts").symlink_to(local) + (scan_root / "alias.local.json").unlink() + assert contract.iter_scan_files(scan_root) == sorted([distinct, regular]) + + +def test_explicit_scan_file_keeps_unsupported_suffix(tmp_path: Path) -> None: + explicit = tmp_path / "explicit.ts" + explicit.write_text("explicit input", encoding="utf-8") + assert contract.iter_scan_files(explicit) == [explicit] + + @pytest.mark.parametrize( ("rule_name", "line"), [ @@ -110,3 +159,69 @@ def test_prefilter_preserves_all_boundary_hit_categories(tmp_path: Path) -> None "sample.md:4: internal_task_id", "sample.md:5: private_ip", ] + + +@pytest.mark.parametrize( + "separator", + [ + "\n", + "\r", + "\r\n", + "\v", + "\f", + "\x1c", + "\x1d", + "\x1e", + "\x85", + "\u2028", + "\u2029", + ], +) +def test_prefilter_preserves_unicode_line_numbers_and_multiple_rules( + tmp_path: Path, separator: str +) -> None: + # The folded necessary condition cannot change original line coordinates, + # rule order, the public-host exception, or credential-reference handling. + lines = [ + "ordinary " * 50, + "Author\u0131zation: literal host 10" + ".1.2.3", + "pa\u017f\u017fword=${EXAMPLE_KEY}", + "https://open.lark" + "office.com host 172" + ".31.2.3", + "pa\u00dfword=literal", # casefold expands; the authoritative regex refuses it + "ordinary again", + ] + (tmp_path / "sample.md").write_text(separator.join(lines), encoding="utf-8") + payload = contract.scan_public_boundary([tmp_path]) + assert payload["hits"] == [ + "sample.md:2: credential", + "sample.md:2: private_ip", + "sample.md:4: private_ip", + ] + assert payload["credential_reference_hits"] == ["sample.md:3: credential"] + + +def test_prefilter_literals_are_substrings_not_regex_syntax( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + class RecordingPattern: + def __init__(self) -> None: + self.lines: list[str] = [] + + def search(self, line: str) -> None: + self.lines.append(line) + + pattern = RecordingPattern() + monkeypatch.setattr( + contract, + "LEAK_RULES", + { + "private_ip": contract.LeakRule( + pattern=pattern, # type: ignore[arg-type] + required_literals=("a|b", "10."), + ), + }, + ) + lines = ["a or b", "10x", "actual a|b", "actual 10.", "ordinary"] + (tmp_path / "sample.md").write_text("\n".join(lines), encoding="utf-8") + assert contract.scan_public_boundary([tmp_path])["ok"] is True + assert pattern.lines == ["actual a|b", "actual 10."]