Skip to content

pids.json 加锁读改写;派发与终态响应先落盘再返回 - #34

Merged
Fei2411 merged 2 commits into
mainfrom
fix/pids-lock-durable-status
Oct 7, 2026
Merged

Fei2411 merged 2 commits into
mainfrom
fix/pids-lock-durable-status

Conversation

@Fei2411

@Fei2411 Fei2411 commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Summary

Closes #26
Closes #27

Test plan

  • 3 个进程并发各登记 30 条,pids.json 保留全部 90 条
  • 读取持续失败时不改写表;瞬时失败重试后成功
  • reap 期间兄弟实例新增的登记被保留
  • 慢写盘下轮询 check_task 看到终态时,磁盘上已是终态
  • 慢写盘下 dispatch_task 返回时,任务行与 request_id 绑定已在磁盘
  • 写盘失败时终态响应 persisted=false,结果仍可读
  • pytest 674 passed;ruff、mypy 通过

Devin Review


Note

Medium Risk
Changes core dispatch persistence and concurrent pids.json behavior; mistakes could cause duplicate dispatches or orphan-worker handling issues, though extensive tests cover the main races.

Overview
Hardens multi-instance pids.json updates and makes task API responses honest about disk persistence.

record_pid / drop_pid / reap_orphans now do locked read-modify-write on pids.json (with retries on transient PermissionError). Unreadable tables are no longer treated as empty, and orphan reap only drops rows that still match the snapshot so concurrent sibling registrations are not wiped.

Task coordination: dispatch_task flushes state.json (task row + request_id binding) before the worker starts and returns persisted; the worker still starts if the MCP call is cancelled mid-flush. Terminal answers from wait_task, check_task, get_result, and cancel_task also include persisted after ensuring the terminal row is flushed (including the gap before _run_task’s final save). check_task and get_result are async through the MCP layer.

ORCHESTRATION (EN/zh) and the agent-bridge skill document the new persisted field.

Reviewed by Cursor Bugbot for commit 7dbd964. Bugbot is set up for automated code reviews on this repo. Configure here.

#26: record_pid/drop_pid/reap_orphans 在文件锁内读改写,PermissionError 退避重试;读取失败不再当空表写回,reap_orphans 不在持锁时杀进程。
#27: dispatch_task 在启动 worker 前落盘任务与 request_id 绑定;check_task/get_result/cancel_task 返回终态前等待写盘,统一带 persisted。
@cursor

cursor Bot commented Oct 6, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 08007008-d6e9-484c-a96a-9af31e1fd4eb)

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-06T15:59:42.835433Z 8c0d4b3 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

devin-ai-integration[bot]

This comment was marked as resolved.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8c0d4b389f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1371 to +1375
if task.status not in TERMINAL_STATUSES:
return None
if task.task_id not in self.tasks:
return True
return await self.flush_state()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Wait for finalization before declaring terminal state persisted

A terminal status does not mean the terminal snapshot has been queued for writing. For example, _stall_watch sets failed before awaiting worker cancellation, and _run_task sets failure states before awaiting final file-change collection; the only terminal save() occurs later in _run_task's finally. During that window, check_task, get_result, or cancel_task reaches this helper, flushes only the previously queued running snapshot, and returns persisted: true, so a crash can still lose the terminal state despite the new guarantee. Gate this verdict on _done/finalization or associate it with the generation created by the terminal save.

Useful? React with 👍 / 👎.

delay = PIDS_WRITE_RETRY_BASE_SEC
for attempt in range(PIDS_WRITE_ATTEMPTS):
try:
with file_lock(lock_path, timeout_sec=PIDS_LOCK_TIMEOUT_SEC):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Move PID lock waits off the event loop

When another Bridge instance holds pids.json.lock, file_lock polls with synchronous time.sleep for up to five seconds, and the PermissionError retry path sleeps synchronously as well. record_pid and drop_pid are called directly from async adapter startup/shutdown paths, so lock contention or transient Windows sharing failures freeze the server's entire event loop, delaying unrelated MCP calls and watchdogs. Run this locked read-modify-write in a worker thread or otherwise make the wait asynchronous.

Useful? React with 👍 / 👎.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

pids.json 加锁读改写与 PermissionError 重试看起来能挡住 #26 的丢更新。#27 这边 wait_task 在 _done.set() 之后同步 save(),再 flush_state(),这条路径是对的;但新的 _persisted() 只冲洗已经入队的 generation,check_task / get_result 仍可能在终态 save() 之前返回 persisted=true。

Open in Web View Automation 

Sent by Cursor Automation: GitHub PR 自动审核

Comment thread src/agent_bridge/registry.py Resolved
@cursor

cursor Bot commented Oct 6, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 5a4040da-70ce-47a6-ae55-1b1118d8b83a)

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

Devin Review

Comment on lines +806 to +812
try:
persisted = await self.flush_state()
finally:
# Even when the caller cancels mid-flush, the registered task
# must still start — a request_id retry would otherwise reuse
# a row that never runs.
self._bg[task.task_id] = asyncio.create_task(self._run_task(task.task_id), name=f"task-{task.task_id}")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Cancelled dispatch can run without durable binding

When flush_state is cancelled before writing, dispatch_task starts the worker without a durable request binding. A restart and request-id retry can dispatch the same prompt twice.

Learn more

The initial save contains both the queued task and its request-id binding. Cancellation ends the caller's wait on flush_state without guaranteeing the background write completed. The finally block starts _run_task before that initial generation is durable. If Bridge exits while the worker processes the prompt, start cannot recover the binding and a retry creates another task.

Example: A caller sends request ID R, then its MCP call times out during a slow initial write. The worker receives the prompt, but Bridge exits before state.json contains R. A retry of R after restart starts the prompt again.

Recommended fix: Keep the initial state write alive on cancellation and gate _run_task on its completion. Decide explicitly how to handle persistent write failure before execution, rather than treating cancellation as proof the binding was saved.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +1380 to +1389
done = self._done.get(task.task_id)
if done is None or not done.is_set():
# _run_task marks the status terminal in-memory, then collects
# final files_changed and flushes the transcript before its
# closing save. Done set means that save has been issued (it
# runs synchronously right after set()); before that, issue the
# save here so a read cannot answer persisted=True while disk
# still says running.
self.save()
return await self.flush_state()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Terminal answers can preserve unfinished file lists

When check_task sees a failed task during final file collection, _persisted saves its pending file list. It can return persisted=True before the final list reaches disk, leaving a restart with unavailable changes.

Learn more

On the failure path, _run_task marks a task terminal before its final workspace merge, finished_at update, transcript flush, and final save. _persisted now saves that intermediate row and waits only for its generation. Once finalization resumes, it issues another save, but the earlier flush can already report success; a crash before the second save leaves incomplete terminal metadata on disk. start converts a pending file list on a recovered terminal task into unavailable.

Example: A failed turn changed src/a.py. check_task saves status=failed and files_changed_state=pending while the final merge runs; it reports persisted=True. A crash before the closing save causes the recovered task to report files_changed_state=unavailable, losing src/a.py from the task metadata.

Recommended fix: Treat finalization as incomplete until _done is set and the closing save generation has been flushed. Wait for that generation before declaring the final terminal response persisted, rather than saving a partial terminal snapshot in _persisted.

Devin Review


Was this helpful? React with 👍 or 👎 to provide feedback.

@Fei2411

Fei2411 commented Oct 6, 2026

Copy link
Copy Markdown
Owner Author

自动审查结论处理:

  • 终态 persisted 误报(Codex P1 / Cursor / Devin 第一轮):已在 7dbd964 修复。_done 未 set 时先 save() 再 flush,回归测试 test_check_task_persists_terminal_row_during_finalize 在修复前失败。
  • dispatch 在 flush 期间被取消导致任务滞留:已在 7dbd964 修复,worker 在 finally 中启动。
  • 取消后 worker 在绑定落盘前启动(Devin 第二轮):不改。等待方被取消时,后台写盘任务不会被取消,仍会在一次写盘时间内完成。要出现重复派发,需要调用被取消、写盘尚未完成、Bridge 恰好崩溃、worker 在这段时间内已产生副作用,四件事同时发生。为它加延迟启动逻辑,复杂度不值得。
  • 中间态行的 files_changed 不完整(Devin 第二轮):不改。这次答复本身返回的就是 files_changed_state: pending,磁盘上的内容和答复一致;重启后 pending 会被如实转成 unavailable/collected,最终保存也会紧接着覆盖它。
  • PID 锁在事件循环上同步等待(Codex P2):不改。锁只在锁内读写期间持有(毫秒级),reap_orphans 不在持锁时杀进程;PermissionError 重试的总等待上限约 0.75 秒,只在杀毒软件短暂占用时出现。为此把 6 处调用改成线程调用不划算。

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

未发现高置信问题。

上一轮报的 _persisted 终态误报(磁盘仍是 running 却返回 persisted=true),以及 dispatch_task 在 flush_state 期间被取消后任务滞留,已在 7dbd964 修复:_done 未置位时先 save() 再 flush;_run_task 放到 finally 里启动。pids.json 加锁读改写与 reap_orphans 快照比对路径未见新的高置信正确性缺陷。

Open in Web View Automation 

Sent by Cursor Automation: GitHub PR 自动审核

@Fei2411
Fei2411 merged commit 71eda80 into main Oct 7, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant