diff --git a/CHANGELOG.md b/CHANGELOG.md index 897528b..23e6861 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -89,6 +89,7 @@ First release as a standalone engine, extracted from [Agent-Notification](https: - The monitor's repository-root lookup no longer mangles directory names with trailing whitespace (B129): `_fingerprints` read `git rev-parse --show-toplevel` with `text=True` and applied `strip()` - universal-newline translation rewrote a real carriage return at the end of the repository directory name on top of Git's record separator, and `strip()` cut off genuine trailing spaces, tabs and newlines - so the sampled root landed on the trimmed sibling path. When that trimmed path did not exist, every status path was recorded as missing and a quiet executor rewriting the same non-ignored file stopped refreshing the activity clock (false stall); when it did exist, the sampler watched the wrong directory and real activity in this repository did not count. The `rev-parse` probe now captures bytes and removes exactly the one trailing `\n` that the Git output protocol appends; all other bytes decode into the root path verbatim, with no guessing, no fallback to a similarly named directory and no scanning of neighbouring directories, and with `cwd` inside a subdirectory `--show-toplevel` still resolves the exact root. B128's file-metadata sampling, `-z` and `-uall` status parsing, the all-intermediate-components and last-component symlink boundaries, symlink-itself activity and missing-file handling are preserved; no recursive scans, no reading file contents for hashing, no loosened `.git` or ignored-directory sampling. `run_monitored` and `PiHost` public signatures and return values, natural-exit `ok`/`error` and returncode, deadline/stop priority and the process-group termination policy are unchanged, and no configuration, dependencies or event fields are added. **Migration:** none. - Post-executor parsing no longer crashes on a top-level non-object JSON line in the event stream (B130): `PiHost.parse`, `parse_observability` and `parse_context` (through `_last_assistant_text`) called `event.get` directly after a successful `json.loads`, so one line of `null`, an array, boolean, number or bare string - the real host can emit one after otherwise normal events, with the subprocess already exited successfully - raised `AttributeError` out of `Dispatcher.run_executor`'s post-processing, and out of the standalone context/observability entries (which some callers downgrade to `unknown` or zero counts). Such values are not protocol events: all three parse entries now skip the whole line and continue with the surrounding object events, never expanding, re-parsing or trusting forged events or context markers embedded in strings/arrays. Model, token/cost usage, guard-denial reasons and tool-call counts of legal object events are unchanged, and the last legal assistant completion message remains the context authority - later non-object noise cannot overwrite, clear or forge it (a final message with no marker still reports `unknown`, an invalid report still `invalid`, tool-output markers are still not trusted, and an older report never stands in for the last message). A stream of only non-object noise returns the existing no-valid-event defaults: `parse` → `("", {}, {})`, context → `unknown`/empty items/`no_marker`, observability → zero tool counts/empty `missing_context`. Broken-JSON lines and unknown-typed objects are handled exactly as before, file read errors still propagate instead of being swallowed into empty results, and no nested field validation is added; `_context_item`/`_safe_context_ref`, context categories/short IDs/privacy filtering, guard decisions, usage/cost computation, the valid-event and report formats and the `PiHost`/`RunResult` public signatures are untouched. No configuration keys. **Migration:** none. - The `events` and `trace` queries now order events by the actual instant instead of the timestamp text (B131): both entry points and the segment list sorted by the raw `ts` string, so mixing legal ISO forms reordered history - `2026-10-16T10:00:00+08:00` (really 02:00Z) landed after `02:30:00Z` and `trace` misreported the later event as the first failure, second-precision and fractional-second Zulu strings could invert, and segment order plus segment start/end bounds followed the wrong order. Both queries now sort by the parsed actual moment (naive strings keep the existing UTC reading; aware values are compared as instants without re-normalizing the representation, so legal extreme-year stamps such as `0001-01-01T00:00:00+01:00` and `9999-12-31T23:59:59-01:00` - whose UTC conversion would leave `datetime`'s native range - order correctly instead of overflowing, and are neither dropped nor demoted to unparseable), with ties broken deterministically by `source` and then `seq`. Legacy timestamps that no longer parse sort after all parseable ones by `source`/`seq`, keep their original text verbatim and raise nothing. Segments are still assembled by the existing source/run-record window rules (attempt, head and count attribution unchanged; a moment that cannot be represented in UTC safely misses every representable window); the segment list is ordered by each segment's actual first instant, and `start`/`end` keep the original strings while representing the segment's real first and last events. First-failure detection still distinguishes `fail`/`deny`/`error`, and the longest stage, filter sets, count/total, every JSON field, the text layout, the stored timestamp format and the database/chain/export/import behaviour are unchanged; queries append no events, invalid arguments still exit 2 and empty queries exit 0. **Migration:** none. +- `events --since` no longer crashes with a traceback on values `datetime` parses natively but that cannot be represented in UTC (B132): an extreme-year stamp with an offset (`0001-01-01T00:00:00+01:00`, `9999-12-31T23:59:59-01:00` - legal ISO whose conversion to UTC would leave `datetime`'s native year range) raised `OverflowError` out of `astimezone`, and an oversized non-negative relative duration raised it out of the `timedelta` construction (`1000000000d` exceeds the `timedelta` cap) or out of the subtraction from the current moment (`999999999d` underflows below year 1) - all four escaped `events_main` as a crash instead of the documented argument error. The parser now converts exactly those input-parsing/time-arithmetic overflows into `ValueError` (one guard around the duration construction and subtraction, one around the UTC normalization, cause chained away), so the existing parameter-error path applies unchanged: text and `--json` both exit 2, stderr names `--since` and echoes the value, stdout stays empty, and the rejection happens before `events_io.query` runs. Unrepresentable values are rejected outright - no clamping, no substitution of `datetime` extremes, no silent fall-back to an unfiltered or empty query. Everything else keeps its interpretation: relative `d/h/m/s` durations, ISO with `Z`/naive/positive-or-negative offsets/surrounding whitespace, and the representable UTC boundaries of years 0001 and 9999 still filter as before with the boundary included, filter sets and count/total/stage/status counts are unchanged, `events_io`'s accepted range is not widened, and malformed inputs keep exiting 2. For valid input, downstream `OSError`/`OverflowError` from storage still propagate as-is (no broader exception swallowing, storage failures are not disguised as argument errors); B131's actual-instant ordering, first-failure reporting and segmentation, the stored timestamp text, every JSON field and the text layout, and the event/database/hash-chain/export/import behaviour are untouched, and no CLI parameter or public signature changes. **Migration:** none. - On the `pilot` branch only, `docs/backlog.md` and reserved `tests/test_pilot_*.py` files are exempt from Code Owner review so eligible pilot changes can merge through the approved host gate. Ordinary tests, workflow files, and CODEOWNERS itself retain owner review; executor edits to tests remain forbidden and task repro bytes remain checked against the owner-provided task base. **Migration:** none for engine users; this is a pilot repository policy change. diff --git a/docs/backlog.md b/docs/backlog.md index 9744fe7..745600e 100644 --- a/docs/backlog.md +++ b/docs/backlog.md @@ -134,3 +134,4 @@ B34–B41 的处理建议（B35 与 B36 先做、B41 放弃等）用户 2026-09- | B129 | 监测采样对仓库根路径的空白不保真（root-path-whitespace-false-stall）：`_fingerprints` 用 `text=True` 读 `git rev-parse --show-toplevel` 再 `strip()`——通用换行转换会把目录名末尾真实的回车连同 Git 追加的记录分隔符一起改写，`strip()` 则把名字末尾的空格、制表符、换行一并截掉，采样根随之落到被截短的同名路径上：截短路径不存在时每条状态路径都按缺失记，安静执行方持续改写同一非忽略文件不再刷新活动时钟，被误判 stall；截短路径存在时采的是别的目录，本仓库的真实活动也不计入。修法：`rev-parse` 改按字节取输出，只去掉 Git 输出协议在末尾追加的那一个 `\n` 记录分隔符，其余字节（含名字末尾空白与回车）原样解码为根路径；不猜测、不回退名字相似的目录、不扫描旁边目录修复，cwd 在仓库子目录时 `--show-toplevel` 照旧定位真实根。B128 已有的文件元数据采样、`-z` 与 `-uall`、全部中间目录与末分量符号链接边界、链接自身活动、文件消失按缺失记均保留，仍不递归扫描、不读文件正文做哈希、不放宽 .git 与忽略目录的采样；`run_monitored` 与 `PiHost` 公开签名/返回值、自然退出 ok/error 与 returncode、deadline/stop 优先级及进程组终止策略不变，不加配置、依赖或事件字段。验证：截短名字的旁边目录存在时，其中同名文件持续变化不计入本仓库活动，本仓库安静不动仍照常 stall | 2026-10-16，#237（宿主回填），pilot 分支试点 TASK-20261016-01 | | B130 | 顶层非对象 JSON 让已完成的执行在后处理崩溃（nonobject-json-postprocess-crash）：`PiHost.parse`、`parse_observability` 与 `parse_context`（经 `_last_assistant_text`）在 `json.loads` 成功后直接 `event.get`，事件流夹一行顶层 null、数组、布尔、数字或裸字符串（真实宿主输出正常事件后可以以此结束，子进程已正常退出）时抛 AttributeError——`Dispatcher.run_executor` 在执行成功后的解析阶段崩掉；单独调用上下文/观察入口同样抛异常，部分调用方降级 unknown 或零计数。修法：顶层非对象 JSON 值不是协议事件，三个解析入口统一整行忽略、继续处理其余对象事件；字符串/数组内伪造的事件或上下文标记不展开、不二次解析、不采信。合法对象事件的模型、usage/费用、守卫拒绝理由与工具调用计数不变；最后一条合法 assistant 完成消息仍是上下文权威来源，后续非对象噪声不覆盖、不清空、不伪造它（末条无标记仍 unknown、真实报告无效仍 invalid、工具输出标记仍不采信、不以旧报告代替）。全是非对象噪声时按无有效事件默认值返回：parse 为 `("", {}, {})`、context 为 unknown/空 items/no_marker、observability 为零工具计数/空 missing_context；坏 JSON 与未知类型对象的处理保持，文件读取异常原样上抛、不以吞异常的空值代替解析，不新增嵌套字段校验，不改其他错误处理；`_context_item`/`_safe_context_ref`、上下文类别/短 ID/隐私过滤、守卫判定、usage/费用计算、有效事件与报告格式、`PiHost`/`RunResult` 公开签名均不变。验证：真实 run_executor 在成功子进程输出混入五类非对象及空数组/伪造 assistant/伪造 tool_execution_end 数组噪声后仍返回 ok/0，且合法事件的 usage、guard 计数与 missing_context 保持 | 2026-10-16，#240（宿主回填），pilot 分支试点 TASK-20261016-02 | | B131 | events/trace 按时间戳原文排序使合法 ISO 时区/精度混用排错（trace-iso-time-order）：events_main 与 _timeline 的排序键、_segment_events 的分段序都是 ts 原文的字典序，实际 02:00Z 的 `2026-10-16T10:00:00+08:00` 被排到 `02:30Z` 之后，trace 把后者误报为首个失败；秒精度 Z 与小数秒 Z（`02:00:00Z` 对 `02:00:00.100Z`）也会倒序，分段列表顺序与段首尾随之错乱。修法：两个查询入口统一按解析后的实际时刻排序（naive 沿用既有解释按 UTC 对待；aware 直接按时刻比较、不规范化表示——0001/9999 极端年份带偏移换算到 UTC 会越过 datetime 原生范围，时刻比较不受影响），同一时刻沿用 source、seq 确定性次序；无法解析的历史旧串排在可解析之后、按 source/seq 定序，原文逐字保留、不抛异常。分段仍按既有来源/运行记录窗口规则组装（attempt/head/count 归属不变），分段列表改按段内实际起始时刻排序，start/end 因事件已按实际时刻排序而天然是该段实际首尾；运行记录窗口解析对不能表示到 UTC 的时刻安全返回无命中（不可能命中可表示的有效窗口），正常窗口归属不变。首个失败仍区分 fail/deny/error，最长阶段、筛选集合、count/total、JSON 字段与文本布局、时间戳保存格式、数据库/哈希链/导出/导入行为均不变，查询不追加事件、不触网，非法参数退出 2、空查询退出 0 保持。验证：+08:00 与 Z 混排的 events/trace 均按实际时刻排序且首失败取较早一条；同段内不同表示的段首尾取实际首尾；极端年份 0001-01-01T00:00:00+01:00、9999-12-31T23:59:59-01:00 的显示排序不溢出、不丢弃、保留 source/seq 与原文；合法串与多个不可解析历史串混排按合法在前、未知按 source/seq 定序 | 2026-10-16，#242（宿主回填），pilot 分支试点 TASK-20261016-03 | +| B132 | events `--since` 对 datetime 原生可读但换算到 UTC 越界的输入崩溃（since-input-overflow）：`_parse_since` 对极端年份带偏移的时间戳直接 `astimezone(UTC)`（`0001-01-01T00:00:00+01:00`、`9999-12-31T23:59:59-01:00`，datetime 能解析、换算越过原生 1–9999 年边界），对过大的非负相对时长直接构造 `timedelta` 并与当前时刻相减（`1000000000d` 溢出 timedelta 上限、`999999999d` 构造成功但相减下溢到年份 0 以下），四类输入的 OverflowError 都逃出 `events_main`，命令以 traceback 崩溃而不是按参数错误返回 2。修法：仅 `_parse_since` 内把输入解析/时间计算的溢出转 ValueError——相对时长的 timedelta 构造与相减一处、ISO 的 UTC 换算一处，`from None` 保留「直接原因」语义，沿用 `events_main` 既有参数错误路径（文本与 --json 均退出 2、stderr 写明 --since 与原输入、stdout 为空、无 traceback），且失败发生在 `events_io.query` 之前（不被调用的筛选不落库）；明确拒绝不可表示值，不钳制、不替换为 datetime 极值、不回退成无筛选或空结果。有效输入解释不变：d/h/m/s 相对时长、ISO 的 Z/naive/正负偏移/首尾空白、可表示的 0001/9999 UTC 边界（`0001-01-01T00:00:00Z`、`0001-01-01T01:00:00+01:00`、`9999-12-31T23:59:59Z`、`9999-12-31T22:59:59-01:00`）含义与含边界的筛选集合、count/total/stage/status 计数不变，不扩大 events_io 接受范围；畸形输入（not-a-time、1.5d、1w、2026-02-30）照旧退出 2；有效输入的下游 OSError/OverflowError 原样上抛，不新增宽泛吞异常、不把存储异常伪装成输入错误；B131 的显示排序/首失败/分段、时间戳原文、字段与文本布局、事件/数据库/哈希链/导出导入及只读查询保持，CLI 参数与公共签名不变。验证：四类超界输入文本与 --json 均退出 2 且 stderr 含 --since、无 traceback、`events_io.query` 未被调用、解析器抛 ValueError；1d/12h/60m/30s 含边界筛选与 8 事件计数、五种 ISO 表示的 deny 筛选集合与计数、可表示极端年份边界的命中/不命中、畸形输入退出 2、拒绝参数前后导出包与链核对不变、query 的 OSError/OverflowError 传播保持 | 2026-10-16，#243（宿主回填），pilot 分支试点 TASK-20261016-04 | diff --git a/engine/reports/trace.py b/engine/reports/trace.py index 099bb88..ccaae1f 100644 --- a/engine/reports/trace.py +++ b/engine/reports/trace.py @@ -39,15 +39,26 @@ def _parse_since(text: str) -> datetime: - """--since 的取值： 相对时长或 ISO 时间戳；不合法抛 ValueError（命令转退出码 2）。""" + """--since 的取值： 相对时长或 ISO 时间戳；不合法或超出 UTC 可表示范围抛 + ValueError（命令转退出码 2），不钳制、不回退。 + + datetime 原生可读但换算到 UTC 越界的值（0001/9999 极端年份带偏移）与过大的相对时长 + （timedelta 构造溢出、与当前时刻相减下溢）同样按参数错误拒绝。 + """ match = _DURATION_RE.fullmatch(text.strip()) if match: - return datetime.now(UTC) - timedelta(seconds=int(match[1]) * _UNITS[match[2]]) + try: + return datetime.now(UTC) - timedelta(seconds=int(match[1]) * _UNITS[match[2]]) + except OverflowError: + raise ValueError(f"--since 时长超出可表示的 UTC 范围：{text}") from None try: parsed = datetime.fromisoformat(text.strip()) except ValueError: raise ValueError(f"--since 无法解析为时长或 UTC 时间：{text}") from None - return parsed.replace(tzinfo=UTC) if parsed.tzinfo is None else parsed.astimezone(UTC) + try: + return parsed.replace(tzinfo=UTC) if parsed.tzinfo is None else parsed.astimezone(UTC) + except OverflowError: + raise ValueError(f"--since 时间换算到 UTC 超出可表示范围：{text}") from None def _parse_ts(value) -> datetime | None: diff --git a/tests/test_pilot_task_20261016_04.py b/tests/test_pilot_task_20261016_04.py new file mode 100644 index 0000000..451230a --- /dev/null +++ b/tests/test_pilot_task_20261016_04.py @@ -0,0 +1,131 @@ +"""B132候选：--since超界输入返回2；有效筛选与只读语义保持。""" +from __future__ import annotations + +from datetime import UTC, datetime +from unittest import TestCase, mock + +from engine.core import events, events_db, events_io +from engine.reports import trace +from tests import test_trace_events_cli as fixture + +MIN_OFFSET = "0001-01-01T00:00:00+01:00" +MAX_OFFSET = "9999-12-31T23:59:59-01:00" +BAD_DURATIONS = ("1000000000d", "999999999d") + + +class FrozenDatetime(datetime): + @classmethod + def now(cls, tz=None): + value = cls(2026, 10, 16, 12, tzinfo=UTC) + return value if tz is None else value.astimezone(tz) + + +class SinceInputBoundaryTest(TestCase): + fresh_repo = fixture.ObservabilityTaskTest.fresh_repo + run_cli = fixture.ObservabilityTaskTest.run_cli + json_cli = fixture.ObservabilityTaskTest.json_cli + + def setUp(self): + fixture.ObservabilityTaskTest.setUp(self) + for patch in [mock.patch.dict("os.environ", {"HARNESS_EVENTS": "on"}), + mock.patch.object(events, "_maybe_cleanup", return_value=None), + mock.patch.object(trace, "datetime", FrozenDatetime)]: + patch.start() + self.addCleanup(patch.stop) + + def emit_at(self, stamp, step, status="ok"): + with mock.patch.object(events_db, "_now", return_value=stamp): + self.assertIsNotNone(events.emit("verify", step, status, trace_id="task/777-since")) + + def reject_input(self, value): + for suffix in [(), ("--json",)]: + with (self.subTest(value=value, suffix=suffix), + mock.patch.object(events_io, "query", wraps=events_io.query) as query): + try: + code, out, err = self.run_cli("events", "--since", value, *suffix) + except OverflowError as error: + self.fail(f"--since输入超界异常逃出CLI：{error}") + self.assertEqual(code, 2) + self.assertEqual(out, "") + self.assertIn("--since", err) + self.assertNotIn("Traceback", err) + query.assert_not_called() + try: + trace._parse_since(value) + except OverflowError as error: + self.fail(f"解析器未将超界转换为ValueError：{error}") + except ValueError: + pass + else: + self.fail("不可表示的输入必须报ValueError") + + def test_minimum_year_offset_is_cli_error(self): + self.reject_input(MIN_OFFSET) + + def test_maximum_year_offset_is_cli_error(self): + self.reject_input(MAX_OFFSET) + + def test_duration_construction_overflow_is_cli_error(self): + self.reject_input(BAD_DURATIONS[0]) + + def test_duration_subtraction_underflow_is_cli_error(self): + self.reject_input(BAD_DURATIONS[1]) + + def test_valid_relative_units_keep_inclusive_boundary(self): + cases = [("1d", "2026-10-15T12:00:00Z", "2026-10-15T11:59:59Z"), + ("12h", "2026-10-16T00:00:00Z", "2026-10-15T23:59:59Z"), + ("60m", "2026-10-16T11:00:00Z", "2026-10-16T10:59:59Z"), + ("30s", "2026-10-16T11:59:30Z", "2026-10-16T11:59:29Z")] + for index, (value, boundary, earlier) in enumerate(cases): + self.emit_at(earlier, f"old-{index}") + self.emit_at(boundary, f"boundary-{index}") + for index, (value, boundary, _) in enumerate(cases): + with self.subTest(value=value): + view = self.json_cli("events", "--since", value) + self.assertIn(f"boundary-{index}", [row["step"] for row in view["events"]]) + self.assertNotIn(f"old-{index}", [row["step"] for row in view["events"]]) + self.assertEqual(view["total"], 8) + self.assertEqual(trace._parse_since(value).isoformat(), + datetime.fromisoformat(boundary).isoformat()) + + def test_valid_iso_representations_keep_filter_counts(self): + self.emit_at("2026-10-16T01:59:59Z", "old") + self.emit_at("2026-10-16T02:00:00Z", "boundary", "deny") + self.emit_at("2026-10-16T02:30:00Z", "later", "deny") + for value in ["2026-10-16T02:00:00Z", "2026-10-16T10:00:00+08:00", + "2026-10-15T22:00:00-04:00", "2026-10-16T02:00:00", + " 2026-10-16T02:00:00.000+00:00 "]: + with self.subTest(value=value): + view = self.json_cli("events", "--since", value, "--status", "deny") + self.assertEqual([row["step"] for row in view["events"]], ["boundary", "later"]) + self.assertEqual((view["count"], view["total"]), (2, 3)) + self.assertEqual(view["counts"], {"stage": {"verify": 2}, "status": {"deny": 2}}) + + def test_representable_native_year_boundaries_remain_valid(self): + self.emit_at("2026-10-16T02:00:00Z", "middle") + for value, count in [("0001-01-01T00:00:00Z", 1), + ("0001-01-01T01:00:00+01:00", 1), + ("9999-12-31T23:59:59Z", 0), + ("9999-12-31T22:59:59-01:00", 0)]: + with self.subTest(value=value): + self.assertEqual(self.json_cli("events", "--since", value)["count"], count) + + def test_malformed_inputs_keep_cli_error(self): + for value in ["not-a-time", "1.5d", "1w", "2026-02-30T00:00:00Z"]: + self.reject_input(value) + + def test_valid_and_invalid_queries_preserve_bundle_and_chain(self): + self.emit_at("2026-10-16T02:00:00Z", "keep", "fail") + before = events_io.export_bundle() + self.reject_input(MIN_OFFSET) + self.json_cli("events", "--since", "2026-10-16T02:00:00Z") + self.assertEqual(events_io.export_bundle(), before) + self.assertEqual(events.verify_chain("task/777-since", "local"), []) + + def test_query_failures_still_propagate(self): + for error in [OSError("query read failed"), OverflowError("downstream overflow")]: + with (self.subTest(error=type(error).__name__), + mock.patch.object(events_io, "query", side_effect=error)): + with self.assertRaises(type(error)) as caught: + self.run_cli("events", "--since", "2026-10-16T02:00:00Z") + self.assertIs(caught.exception, error)