Skip to content

fix: surface pm spawnSync stdout overruns as errors, not silent wrong answers - #59

Merged
unbraind merged 4 commits into
mainfrom
fix/pm-shellout-buffer-silent-degradation
Aug 9, 2026
Merged

unbraind merged 4 commits into
mainfrom
fix/pm-shellout-buffer-silent-degradation

Conversation

@unbraind

@unbraind unbraind commented Aug 8, 2026 •

Copy link
Copy Markdown
Owner

What

pm-csv shells out to pm through spawnSync in ~10 places. Node caps captured stdout at maxBuffer; when a tracker's output exceeds it the child is killed mid-capture and spawnSync returns status: null with an empty stderr and truncated stdout. Guards that branch on status !== 0 then read null !== 0 as truthy and route the overrun onto their failure branch — turning a buffer overflow into a wrong answer instead of an error:

  • itemStatus (pm get) mapped a null status onto undefined ("item does not exist"), so the compensation guard skipped closes it should have run.
  • loadAppliedByTransaction (pm list-all) swallowed a null/error result into an empty applied-rows map, so a resumed --atomic import silently re-imported every already-applied row.

This is silent data loss, not a crash — and pm-csv is the worst offender tracked in the pm-cli-website-wz6y epic.

Fix

A shared assertPmOutputFit(result, label) treats status === null as a hard, named error that names the buffer and the PM_LIST_MAX_BUFFER override, applied at every pm subprocess site that must not degrade. itemStatus additionally gains the maxBuffer it lacked (it was the only read still on Node's 1 MiB default); the write sites that previously had no buffer now carry one too.

Per-call-site tier assignment

Line Site Tier Change Reason
1016 loadKeyIndex — pm list-all (read) 2 unchanged Already capped (#41) and already throws on error; loud, not silent.
1290 loadAppliedByTransaction — pm list-all (read) 2 + assertPmOutputFit Silent bug: swallowed null/error into an empty "applied" map → resumed atomic import re-imported rows. Now throws.
1338 itemStatus — pm get (read) 2 + maxBuffer + assertPmOutputFit Silent bug: no buffer (1 MiB default) + status !== 0 → undefined ("item missing"). Now capped 16× and overruns throw.
1376 compensateCreate — pm update (strip markers) 2 + maxBuffer Best-effort rollback: tolerates a failed subprocess by design, so no assert.
1390 compensateCreate — pm close 2 + maxBuffer Best-effort rollback; existing status !== 0 && !== 4 log already covers null.
1843 upsertCreate — pm create 2 + maxBuffer + assertPmOutputFit Write; previously uncapped.
1870 upsertCreate — pm close (terminal apply) 2 + maxBuffer + assertPmOutputFit Write; previously uncapped.
1941 upsertUpdate — pm update 2 + maxBuffer + assertPmOutputFit Write; previously uncapped.
1947 upsertUpdate — pm close 2 + maxBuffer + assertPmOutputFit Write; previously uncapped.
2286 buildCsvExport — pm list-all --include-body (read) 2 unchanged Already capped (#41) and already names the ENOBUFS overrun.

Why not tier-1 (delete the subprocess, read the store in-process)?

A truly ceiling-free in-process read is not achievable in this package without regressing a documented property:

  • A static @unbrained/pm-cli value import breaks standalone-loadability. index.ts:23 documents that "standalone-installed extensions load only their own dist/, so @unbrained/pm-cli is not resolvable at runtime" — confirmed empirically: loading dist/index.js off the package resolution path throws ERR_MODULE_NOT_FOUND. (Every existing @unbrained/pm-cli import here is type-only for exactly this reason.)
  • Dynamic import() is disallowed by the task.
  • The host-injected ctx.sdk.client.list (PmClient) carries its own read-output budget, so it is not ceiling-free either.

Tier-2 therefore eliminates the silent wrong-answer (overruns now throw loudly with an actionable message) while preserving standalone-loadability and the sync architecture. Standalone load is verified preserved after the fix.

Tests

test/buffer-regression.test.ts (5 tests), each mutation-tested (revert the fix → the test fails; restore → passes):

  • itemStatus driven past Node's 1 MiB default (a 2 MiB body via --body-file) returns the real status, not undefined. (reverting the itemStatus fix fails this)
  • itemStatus and loadAppliedByTransaction under a capped PM_LIST_MAX_BUFFER surface the overrun as a named error, not an empty/absent result. (reverting each fix fails its test)
  • itemStatus still returns undefined for a genuinely absent item (non-zero exit), preserving the fake-get-fail compensation regression.
  • loadAppliedByTransaction returns the real applied rows under the default (uncapped) buffer — the production path.

Mutation-test results

  • Revert itemStatus (drop maxBuffer+assertPmOutputFit): the "exceeds 1 MiB" test and the "surfaces a buffer overrun" test FAIL; restore → pass.
  • Revert loadAppliedByTransaction assertPmOutputFit: the "surfaces a buffer overrun" test FAILS; restore → pass.

Gates

typecheck, coverage (index.ts 83.0/77.5/88.5; thresholds 81/74/88 unchanged, no c8 ignore), build, audit:prod (0 vulns), pack:dry-run, changelog:check — all green. 114 tests pass (109 prior + 5 new).

Tracking

pm-csv-02hc (companion epic pm-cli-website-wz6y).

Summary by Sourcery

Treat pm subprocess stdout buffer overruns as explicit errors instead of silent fallback results across critical read/write paths.

Bug Fixes:

  • Prevent loadAppliedByTransaction from interpreting buffer overruns as an empty applied-rows map that causes resumed atomic imports to re-import already-applied rows.
  • Prevent itemStatus from treating oversized pm get output as a missing item, ensuring compensation logic does not silently skip required closes.

Enhancements:

  • Introduce a shared assertPmOutputFit helper that detects spawnSync status=null conditions and throws a named error including buffer configuration guidance.
  • Standardize maxBuffer usage for pm spawnSync calls involved in create, update, close, and itemStatus operations to use pmListMaxBuffer and avoid uncapped stdout captures.
  • Export loadAppliedByTransaction and itemStatus for targeted regression testing of buffer overrun behavior.

Tests:

  • Add buffer-regression.test.ts to cover pm stdout overrun scenarios for itemStatus and loadAppliedByTransaction, including oversized bodies, capped buffers, genuine missing items, and normal applied-row detection.

Chores:

  • Record tracking metadata for issue pm-csv-02hc in the .agents/pm history and issues directories.

Summary by cubic

Surfaces pm shell-out stdout overruns and other spawnSync status: null cases as clear errors, hardens compensation/write paths so rollbacks don’t abort or strand orphans, and makes update failures actionable by naming the item and possible write state.

  • Bug Fixes
    • Added describePmNullStatus to tell ENOBUFS overruns from spawn/system errors and external signal kills; used by assertPmOutputFit and write-path errors.
    • Reads: itemStatus and loadAppliedByTransaction now set maxBuffer via PM_LIST_MAX_BUFFER and assert; no more silent “missing item” or empty applied map on overruns.
    • Writes: upsertCreate parses the id before any throw; on overrun/kill, best-effort closes the recovered orphan and reports the close’s real outcome (closed vs still open with cause). All create/update/close calls set maxBuffer, with asserts on hard-fail paths.
    • Updates: upsertUpdate errors include the item id and state the mutation may already be applied, with distinct messages for status: null vs non-zero exits.
    • Compensation: compensateCreate catches an overrun in its own itemStatus check and treats that row as a no-op so the sweep continues; compensation subprocesses add maxBuffer but keep tolerant behavior.
    • Tests: flush the fake pm create receipt with writeSync before SIGKILL to make the SIGKILL fixture deterministic and avoid false assertions.

Written for commit 7846ed4. Summary will update on new commits.

Review in cubic


Review round 3 (52f37e4)

Two Greptile P1s, both the same family as the defect this PR exists to remove — a message that
states something the code never established
.

# Finding Disposition
A Recovery close remains unchecked Fixed — the close result is inspected, not assumed
B Persisted updates remain unreconciled Accepted in part — errors now carry the id; update compensation deliberately not added

A — the recovery close was issued and its result discarded, while the thrown error claimed the
orphan "was best-effort closed". If that close itself failed or was killed, the operator was told
the orphan was handled and stopped looking for an item that is still open. Now: status 0 reports
the orphan closed; a null status or non-zero exit reports it still open, to be closed manually,
carrying the close's own cause. Attempted exactly once — under a capped buffer it fails identically
every time, so a retry loop would only turn a fast, actionable error into a slow one.

B — every pm update and terminal pm close failure now carries the item id and states
explicitly that the mutation may already have been applied. The null-status and non-zero arms are
separate, because a non-zero exit is not proof nothing was written; when the terminal close fails
after a successful update, one message reports both halves.

Deliberately not implemented: update compensation. An update has no inverse without the prior
field values, which this extension does not capture, so "compensating" one could only mean writing
some other value over it — a second unreviewed mutation on top of a failed one, which on a
partially-applied import is worse than a loud, identified failure. The docstrings therefore state
the guarantee in its honest, narrower form: no mutation is left un-identified, not no mutation
is left behind
. The capability that would widen it is filed as a real item rather than declined.

Evidence

  • 127 tests pass. Four new update-path regression tests, plus two tracker-scoped fake-pm
    markers (fake-update-fail, fake-update-kill) beside the existing close fakes.
  • Mutation-tested. Reverting the upsertUpdate fix fails exactly those four of 28 in
    atomic.test.ts; restoring it passes all 28.
  • Coverage 84.18 / 80.26 / 88.66, up from 83.62 / 78.42 / 88.66. Thresholds 81 / 74 / 88
    unchanged
    , no c8 ignore added.
  • typecheck clean · audit:prod 0 vulnerabilities · pack 9 files · changelog up to date.

pm items

  • pm-csv-158c — this round's fixes (closed, with the full evidence note)
  • pm-csv-71zg — deferred: capture prior field values so an update can be compensated (open)
  • pm-csv-02hc — rounds 1–2 (closed)

… answers

Root cause: pm-csv shells out to `pm` through spawnSync in ~11 places. Node
caps captured stdout at maxBuffer; when a tracker's output exceeds it the child
is killed mid-capture and spawnSync returns status=null with an EMPTY stderr and
truncated stdout. Guards that branch on `status !== 0` then read `null !== 0` as
truthy and route the overrun onto their failure branch, turning a buffer
overflow into a WRONG ANSWER instead of an error:

- itemStatus (pm get) had NO maxBuffer (Node's 1 MiB default) and mapped a null
  status onto `undefined` ("item does not exist"), so the compensation guard
  skipped closes it should have run.
- loadAppliedByTransaction (pm list-all) swallowed a null/error result into an
  EMPTY applied-rows map, so a resumed atomic import silently re-imported every
  already-applied row.

Fix: a shared assertPmOutputFit treats `status === null` as a hard, named error
that names the buffer and the PM_LIST_MAX_BUFFER override, applied at every pm
subprocess site that must not degrade. itemStatus additionally gains the
maxBuffer it lacked; the write sites (create/update/close) that previously had
no buffer now carry one too. The two best-effort compensation sites keep their
tolerant behavior (no assert) by design.

Tier-1 (delete the subprocess and read the store in-process, no ceiling) is not
used: a static `@unbrained/pm-cli` value import regresses this package's
documented standalone-loadability (dist/ must load with only its own files;
confirmed ERR_MODULE_NOT_FOUND when loaded off the package resolution path),
dynamic import() is disallowed, and the host-injected ctx.sdk.client.list
carries its own read-output budget. Standalone load is verified preserved.

Regression tests (test/buffer-regression.test.ts), each mutation-tested:
- itemStatus driven past Node's 1 MiB default (2 MiB body via --body-file)
  returns the real status, not undefined.
- itemStatus and loadAppliedByTransaction under a capped PM_LIST_MAX_BUFFER
  surface the overrun as a named error, not an empty/absent result.
- itemStatus still returns undefined for a genuinely absent item (preserves the
  fake-get-fail compensation regression).

Refs pm-csv-02hc (companion epic pm-cli-website-wz6y).

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @unbraind, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai

coderabbitai Bot commented Aug 8, 2026 •

Copy link
Copy Markdown

Review Change Stack

Summary by CodeRabbit

  • Bug Fixes

    • Prevented large PM command outputs from being silently truncated or misreported as missing data.
    • Added clear errors when output exceeds the configured buffer limit.
    • Improved reliability for transaction scans, item lookups, creates, updates, and closes.
    • Preserved correct handling for genuinely absent items and compensation operations.
    • Improved recovery of interrupted creates and reporting of updates or closes that may have partially applied.
  • Tests

    • Added regression coverage for oversized, capped, and uncapped PM subprocess output.
    • Added coverage for interrupted operations, compensation, and actionable error reporting.

Walkthrough

PM subprocess handling now bounds stdout, classifies null statuses, recovers created orphans, and reports uncertain mutations. Compensation remains tolerant. Regression tests cover reads, writes, closes, signals, overruns, and spawn errors.

Changes

PM subprocess hardening

Layer / File(s) Summary
Status classification and read handling
index.ts, test/buffer-regression.test.ts
Classified null statuses as buffer overruns, signal termination, or spawn errors. Applied bounded output handling to scans, lookups, and compensation reads.
Mutation recovery and diagnostics
index.ts, test/atomic.test.ts
Added orphan-ID recovery and close attempts after create failures. Updated close and update errors to report possible mutation application and affected item IDs.
Failure-mode regression coverage
test/atomic.test.ts, test/buffer-regression.test.ts
Added fake PM scenarios and tests for overruns, killed processes, recovery closes, compensation, missing items, and transaction scans.
Issue and feature records
.agents/pm/issues/*, .agents/pm/history/*, .agents/pm/features/*
Recorded remediation details, validation evidence, closure information, and update-compensation feature requirements.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main fix: surfacing pm spawnSync stdout overruns as errors instead of silent incorrect results.
Description check ✅ Passed The description directly explains the stdout overrun defect, implementation, recovery behavior, tests, and validation results.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/pm-shellout-buffer-silent-degradation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai

sourcery-ai Bot commented Aug 8, 2026

Copy link
Copy Markdown

Reviewer's Guide

Introduces a shared assertPmOutputFit helper to treat pm spawnSync stdout buffer overruns as explicit errors instead of silent wrong answers, wires it into critical pm subprocess call sites, adds maxBuffer caps consistently, exports internal read helpers for regression tests, and adds targeted tests to lock in the new behavior.

Sequence diagram for pm stdout overrun handling via assertPmOutputFit

sequenceDiagram
  participant pmCsv as pmCsvFunction
  participant spawnSync as spawnSync
  participant pmProcess as pmSubprocess
  participant assertPmOutputFit as assertPmOutputFit
  participant CommandError as CommandError

  pmCsv->>spawnSync: spawnSync("pm", args, { encoding, maxBuffer: pmListMaxBuffer })
  spawnSync->>pmProcess: run pm command
  pmProcess-->>spawnSync: stdout exceeds maxBuffer
  spawnSync-->>pmCsv: result { status: null, stdout: truncated, stderr: "" }
  pmCsv->>assertPmOutputFit: assertPmOutputFit(result, label)
  assertPmOutputFit->>pmListMaxBuffer: pmListMaxBuffer()
  assertPmOutputFit-->>CommandError: new CommandError(message)
  CommandError-->>pmCsv: throw
  pmCsv-->>pmCsv: abort instead of treating result as empty/absent

  alt status not null
    pmCsv->>assertPmOutputFit: assertPmOutputFit(result, label)
    assertPmOutputFit-->>pmCsv: return
    pmCsv-->>pmCsv: branch on result.status (0 vs non-zero)
  end
Loading

Flow diagram for spawnSync result handling with assertPmOutputFit

flowchart TD
  A["Call spawnSync with maxBuffer from pmListMaxBuffer"] --> B["Receive result from spawnSync"]
  B --> C{result.status === null?}
  C -->|Yes| D["assertPmOutputFit throws CommandError naming buffer overrun"]
  C -->|No| E{result.status !== 0?}
  E -->|Yes| F["Handle non-zero exit (e.g. empty map or undefined)"]
  E -->|No| G["Parse result.stdout and proceed with normal data path"]
Loading

File-Level Changes

Change Details Files
Introduce a shared helper to detect and fail on pm subprocess stdout buffer overruns.
  • Import SpawnSyncReturns type from node:child_process and define assertPmOutputFit(result, label).
  • Implement logic that throws CommandError when spawnSync returns status null, describing buffer size and PM_LIST_MAX_BUFFER override.
  • Document rationale for not using the helper at best-effort rollback sites in compensateCreate.
index.ts
Harden pm read paths by capping stdout buffers and asserting output validity to avoid silent wrong answers.
  • Update loadAppliedByTransaction to use pmListMaxBuffer in spawnSync, call assertPmOutputFit, and only return an empty map for genuine non-zero status exits.
  • Update itemStatus pm get call to use pmListMaxBuffer and assertPmOutputFit, preserving undefined for true non-zero exits while preventing buffer-overrun misclassification.
  • Add explanatory comments around tier-2 read behavior and buffer-overrun handling for both functions.
index.ts
Harden pm write/close paths with buffer caps and overrun assertions so write operations fail loudly on output truncation.
  • Add maxBuffer: pmListMaxBuffer() to upsertCreate pm create and close calls and invoke assertPmOutputFit before existing status checks.
  • Add maxBuffer and assertPmOutputFit to upsertUpdate pm update and close calls to surface ENOBUFS conditions as named errors.
  • Add maxBuffer options to compensateCreate pm update/close calls while intentionally skipping assertPmOutputFit to preserve best-effort rollback semantics.
index.ts
Expose internal read helpers and add regression tests to lock in buffer-overrun behavior and avoid silent data loss.
  • Export loadAppliedByTransaction and itemStatus from index.ts for use in tests, with commentary about their side-effectful read semantics.
  • Create buffer-regression.test.ts with utilities to spin up temporary pm trackers, create items, and temporarily cap PM_LIST_MAX_BUFFER.
  • Add tests covering oversized pm get output past Node’s 1 MiB default, buffer overruns surfaced as named errors for itemStatus and loadAppliedByTransaction, absent-item behavior, and normal applied-rows behavior under default buffer.
index.ts
test/buffer-regression.test.ts
Add tracking artifacts for the pm-csv-02hc issue.
  • Create pm-csv-02hc.jsonl history file for internal agent tracking.
  • Add pm-csv-02hc.toon issue descriptor aligned with the pm agents workflow.
.agents/pm/history/pm-csv-02hc.jsonl
.agents/pm/issues/pm-csv-02hc.toon

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@index.ts`:
- Around line 1325-1342: Update compensateCreate to catch CommandError thrown by
itemStatus during its compensation status lookups and return without aborting
the rollback sweep, while preserving the hard error for non-compensation
callers. Cover both compensation lookup sites and add a regression test for a
capped-buffer pm get result.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a782e85d-0724-4d61-8e63-c83fd355a39a

📥 Commits

Reviewing files that changed from the base of the PR and between 3dd822a and 02ee15c.

📒 Files selected for processing (4)
  • .agents/pm/history/pm-csv-02hc.jsonl
  • .agents/pm/issues/pm-csv-02hc.toon
  • index.ts
  • test/buffer-regression.test.ts

Comment thread index.ts
@greptile-apps

greptile-apps Bot commented Aug 8, 2026 •

Copy link
Copy Markdown

Greptile Summary

The PR prevents pm subprocess output overruns and other null-status terminations from degrading into silent wrong answers.

  • Classifies buffer overruns, spawn failures, and signal termination using the available spawnSync evidence.
  • Preserves create identifiers before failure handling, verifies recovery-close outcomes, and makes write failures explicitly reconcilable by item id.
  • Adds deterministic regression coverage for read overruns, create recovery, and update and terminal-close failures.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
index.ts Adds cause-specific null-status diagnostics, caps subprocess output, preserves create identity before error handling, verifies recovery-close outcomes, and identifies potentially persisted update mutations.
test/atomic.test.ts Adds deterministic create, recovery-close, update, and terminal-close failure coverage, including synchronous receipt emission before forced termination.
test/buffer-regression.test.ts Exercises oversized read output and each null-status diagnostic category while preserving genuine missing-item behavior.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Run pm subprocess] --> B{status}
  B -->|0| C[Continue with successful result]
  B -->|non-zero| D[Report operation and item id]
  B -->|null| E{error.code}
  E -->|ENOBUFS| F[Report stdout overrun and buffer override]
  E -->|other error| G[Report spawn or system failure]
  E -->|no error, signal set| H[Report external signal termination]
  F --> I{Create id recovered?}
  G --> I
  H --> I
  I -->|Yes| J[Attempt and inspect recovery close]
  I -->|No| K[Report title for manual reconciliation]
  J --> L{Close succeeded?}
  L -->|Yes| M[Report orphan closed]
  L -->|No| N[Report item still open with id and cause]
Loading

Reviews (4): Last reviewed commit: "test: flush the fake create receipt with..." | Re-trigger Greptile

Comment thread index.ts Outdated
Comment thread index.ts Outdated
…e orphans

Root cause: the shared assertPmOutputFit added to surface stdout overruns as
errors trades a silent wrong answer for an aborted rollback in three places
where a throw must not happen, because the same status===null signal marks
both a buffer overrun AND a child that was killed, and a write may already
have persisted by the time its receipt overruns.

1. compensateCreate calls itemStatus to decide whether to close, and itemStatus
   now throws on an overrun — so the compensation sweep aborted before reaching
   its own best-effort strip/close subprocesses, leaving every later applied
   create un-compensated. The docstring promised compensation was a no-op on
   partial failure; the code disagreed. compensateCreate now catches an overrun
   in its OWN status lookup and treats that row as a no-op, while the import
   read path (loadAppliedByTransaction, the verification itemStatus) keeps the
   hard error where a wrong answer is the real danger.

2. A write overrun stranded an un-identifiable orphan. `pm create` persists the
   item BEFORE its receipt overruns the buffer, and the guard threw before the
   id was parsed, so compensation could not identify what to roll back; the same
   guard on the close result bypassed the close-failure compensation branch.
   upsertCreate now parses the id FIRST (Node retains the full captured stdout
   for outputs under one read chunk, and pm create --json's flat receipt is
   always far under that, so the id is recoverable even on overrun), best-effort
   closes a recovered orphan directly, and throws a named, id-carrying error —
   never abandoning a write that may have taken effect without identifying it.
   The close-result guard is dropped so a null close status routes INTO the
   compensation branch (the id is already known, so nothing is stranded).

3. status===null is not only an overrun: a child terminated by a signal reports
   it too, and SIGTERM is the SAME signal a buffer-driven kill produces, so the
   signal alone cannot tell them apart. The old message asserted a buffer
   overrun and pointed the operator at PM_LIST_MAX_BUFFER even when the real
   cause was a kill. A shared describePmNullStatus now distinguishes the three
   causes from the evidence Node actually populates (verified against
   @types/node SpawnSyncReturns and empirically): error.code==="ENOBUFS" →
   stdout overrun; error set with another code → spawn/system error; error
   unset + signal → external signal kill. Each message is worded for its real
   cause, and is reused by both the read-path assert and the write-path errors.

Verified against node_modules/@types/node and confirmed empirically: a buffer
overrun sets error.code ENOBUFS and retains the full stdout for outputs under
one read chunk; an external signal kill leaves error unset; pm close under a
tiny maxBuffer still persists the close (the write precedes the receipt).

Gates: typecheck clean; coverage 83.62/78.42/88.66 (up vs 83.01/77.51/88.54,
above floors 83.0/77.5/88.5 and thresholds 81/74/88 UNCHANGED, no c8 ignore);
audit 0 vulns; pack 9 files; 121 tests pass (114 + 7 new). Each new test is
mutation-tested (revert its fix -> it fails). Standalone-loadability preserved
(dist/ loads with only node: builtins; @unbrained/pm-cli imports are type-only).

Refs pm-csv-02hc.
Comment thread index.ts Outdated
Comment thread index.ts Outdated
…ery update failure

Review round 3 on #59. Both findings are the same family as the defect this PR
exists to remove: a message that states something the code never established.

Finding A (Greptile P1, upsertCreate) - the recovery close was issued and its
result discarded, while the thrown error claimed the orphan "was best-effort
closed". If that close itself failed or was killed, the operator was told the
orphan was handled and stopped looking for an item still open. The result is now
inspected: status 0 reports the orphan closed; a null status or non-zero exit
reports it STILL OPEN, to be closed manually, carrying the close's own cause
(describePmNullStatus for a null status, the close stderr otherwise). The close
is attempted exactly once - under a capped buffer it fails identically every
time, so a retry loop would only turn a fast actionable error into a slow one.

Finding B (Greptile P1, upsertUpdate) - a pm update or terminal pm close could
persist its mutation and then return a null status or non-zero exit, and the
error named neither the item nor the possibility that the write had landed. Every
such failure now carries the item id and states explicitly that the mutation may
already have been applied. The null-status and non-zero arms are separate,
because a non-zero exit is not proof that nothing was written; when the terminal
close fails after a successful update, the message reports both halves.

Deliberately NOT implemented: update compensation in the atomic path. An update
has no inverse without the prior field values, which this extension does not
capture, so "compensating" it could only mean writing some other value over it -
a second unreviewed mutation on top of a failed one, which on a partially applied
import is worse than a loud identified failure. The guarantee is therefore stated
in the docstrings in its honest, narrower form: no mutation is left
UN-IDENTIFIED, not no mutation is left behind. The capability that would widen it
is tracked as pm-csv-71zg.

Tests: four new update-path regression tests, plus two tracker-scoped fake pm
markers (fake-update-fail, fake-update-kill) alongside the existing close fakes.
Mutation-tested - reverting the upsertUpdate fix fails exactly those four of 28
in atomic.test.ts, and restoring it passes all 28.

Gates: typecheck clean; 127 tests pass; coverage 84.18 / 80.26 / 88.66, up from
83.62 / 78.42 / 88.66, thresholds 81 / 74 / 88 unchanged and no c8 ignore added;
audit:prod 0 vulnerabilities; pack 9 files; changelog up to date.
@unbraind

unbraind commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai full review
@greptileai review

Round 3 is pushed as 52f37e4. Both of the previous round's P1s are answered — details and evidence
are in the updated PR description, and each thread has an in-line reply.

Context worth carrying into this review, because it defines what a correct finding looks like here:

The invertibility test decides whether compensation is sound. A create has a true inverse
(close it — the item did not exist before). An update does not, without a snapshot of prior
field values, which this extension does not capture. So in this codebase "compensate" is only ever
a legitimate ask for creates. For updates the honest ceiling is identification, not rollback, and
the docstrings now say so in those words. If a future change adds update compensation without also
adding prior-value capture to the transaction record, that is a real finding — it would be a second
unreviewed mutation wearing a rollback's name.

The defect class this PR is about is a message that asserts an unverified outcome. Round 1 fixed
a guard that turned a buffer overrun into a wrong answer. Round 2 found that the fix had moved the
failure — it created an un-identifiable orphan. Round 3 found that the recovery step itself claimed
an outcome it never checked. Each round found the same shape one level deeper, so the most valuable
thing to look for here is any remaining place where the code states a result it did not
observe — not style, not naming.

One implementation note that will look like a gap and is not: compensateCreate catches
itemStatus with a bare catch rather than narrowing to CommandError and rethrowing.
itemStatus's only throw is that CommandError, so the narrowing branch would be permanently
unreachable and permanently uncovered. The constraint is written into the docstring at the call
site; if itemStatus ever gains a second failure mode, the narrowing must come back.

Everything green: 127 tests, coverage 84.18 / 80.26 / 88.66 (thresholds 81 / 74 / 88 unchanged),
audit 0 vulnerabilities, mutation tests confirm all four new tests are load-bearing.

@coderabbitai

coderabbitai Bot commented Aug 9, 2026 •

Copy link
Copy Markdown

@unbraind I will perform a full review of PR #59.

I will focus on messages that state subprocess or compensation outcomes without observing those outcomes. I will also verify that create compensation remains limited to invertible create operations.


✏️ Learnings added
Learnt from: unbraind
URL: https://github.com/unbraind/pm-csv/pull/59

Timestamp: 2026-08-09T00:16:55.263Z
Learning: In `unbraind/pm-csv`, compensation is sound only for create operations because a created item can be inverted by closing the item. Update operations do not have prior field-value snapshots in the transaction record, so they must only be identified and must not be presented or implemented as rollback compensation. A future update compensation feature requires prior-value capture.

Learnt from: unbraind
URL: https://github.com/unbraind/pm-csv/pull/59

Timestamp: 2026-08-09T00:16:55.263Z
Learning: In `unbraind/pm-csv`, review subprocess failure handling for messages that assert outcomes without observing them. In particular, `compensateCreate` intentionally uses a bare `catch` around `itemStatus` because `itemStatus` currently throws only `CommandError`; if `itemStatus` gains another failure mode, restore narrowing and rethrow non-`CommandError` failures.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

⚠️ Action not completed

Review rate limited.


Your included review limit is currently reached under our Fair Usage Limits Policy. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 59 minutes.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
index.ts (1)

2009-2018: 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

Wrap the verification itemStatus so a null-status lookup still reports the item id.

Line 2009 calls itemStatus directly. Since this PR, itemStatus throws a CommandError when its pm get returns status: null (overrun or signal kill). That throw replaces the message built on Lines 2015-2017, so the failure loses the item id and the title. The comment on Lines 2003-2008 still assumes the lookup only returns undefined when it fails.

This is reachable in the same conditions this PR targets: a small PM_LIST_MAX_BUFFER makes the pm close receipt overrun, compensateCreate no-ops on its own overrun, and the verification lookup then throws a raw buffer message with no id. The created item is left open and un-identified, which contradicts the stated guarantee that every possibly-applied mutation is reported with its id.

Catch the throw and treat it as an unverified compensation, so the existing message is still produced.

🛠️ Proposed fix
-      const compensatedStatus = itemStatus(pmRoot, id);
+      // The verification lookup can itself return `status: null` (overrun or
+      // signal kill) and throw. Treat that as "unverified" so the failure still
+      // names the id and title, instead of surfacing a raw buffer error that
+      // identifies nothing.
+      let compensatedStatus: ItemStatus | undefined;
+      try {
+        compensatedStatus = itemStatus(pmRoot, id);
+      } catch {
+        compensatedStatus = undefined;
+      }
       if (compensatedStatus !== "closed") {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@index.ts` around lines 2009 - 2018, Wrap the verification call to itemStatus
in the compensation failure path so CommandError or other lookup failures are
converted to an unverified status instead of escaping. Preserve the existing
compensatedStatus message branch, ensuring failures still report the item title
and id while retaining the existing handling for returned statuses.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.agents/pm/history/pm-csv-158c.jsonl:
- Around line 1-4: Regenerate the PM audit records using actual UTC event times
no later than August 8, 2026, preserving append-only history by adding
corrective entries rather than rewriting persisted events. Update
.agents/pm/history/pm-csv-158c.jsonl lines 1-4 with corrected timestamps and
matching hashes; regenerate .agents/pm/history/pm-csv-71zg.jsonl lines 1-1 with
its actual creation time; then regenerate the materialized metadata in
.agents/pm/features/pm-csv-71zg.toon lines 8-9 and
.agents/pm/issues/pm-csv-158c.toon lines 8-17 from the corrected histories,
including dependency and note timestamps.

In `@test/atomic.test.ts`:
- Around line 189-193: Replace the asynchronous process.stdout.write call in the
fake-create-overrun handling block with a synchronous fs.writeSync(1, ...)
write, preserving the existing stdout content and ensuring it completes before
process.kill sends SIGKILL.

---

Outside diff comments:
In `@index.ts`:
- Around line 2009-2018: Wrap the verification call to itemStatus in the
compensation failure path so CommandError or other lookup failures are converted
to an unverified status instead of escaping. Preserve the existing
compensatedStatus message branch, ensuring failures still report the item title
and id while retaining the existing handling for returned statuses.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0fd994ab-9481-4874-ac93-82d720b3353e

📥 Commits

Reviewing files that changed from the base of the PR and between 02ee15c and 52f37e4.

📒 Files selected for processing (9)
  • .agents/pm/features/pm-csv-71zg.toon
  • .agents/pm/history/pm-csv-02hc.jsonl
  • .agents/pm/history/pm-csv-158c.jsonl
  • .agents/pm/history/pm-csv-71zg.jsonl
  • .agents/pm/issues/pm-csv-02hc.toon
  • .agents/pm/issues/pm-csv-158c.toon
  • index.ts
  • test/atomic.test.ts
  • test/buffer-regression.test.ts

Comment thread .agents/pm/history/pm-csv-158c.jsonl
Comment thread test/atomic.test.ts
process.stdout.write on a pipe is asynchronous, and the SIGKILL on the next
statement cannot be handled, so a queued chunk is discarded. When that happened
the receipt never reached the parent, the id was not recovered, and the test
silently exercised the no-id branch while asserting the recovered-id one - a
non-deterministic test that fails INTO THE WRONG ASSERTION rather than into an
error, which would have read as flake and been 'fixed' by a retry.

writeSync(1, ...) returns only once the bytes are handed to the fd. The hazard
is named at the call site so the more idiomatic form is not restored.

Found by CodeRabbit. Notable that the harness had the same failure mode as the
code under test: this PR is about a subprocess killed after its side effect but
before its output is readable, and the fixture reproduces that by killing a
subprocess.
@unbraind

unbraind commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai full review
@greptileai review

7846ed4 fixes CodeRabbit's async-write finding (writeSync(1, …) before the SIGKILL in the
create-overrun fixture) — a real non-determinism that would have failed into the wrong assertion
rather than into an error. 28/28 in atomic.test.ts still pass.

The other finding in that batch (pm ledger timestamps read as future-dated) was declined with
evidence
: the review ran at 00:18:58Z on 2026-08-09 and every ledger event predates it
(00:13:19Z–00:14:03Z). It rested on an assumed "today" during a UTC midnight crossing. Full
reasoning is in the thread — acting on it would have rewritten an append-only audit trail, which is
the defect the check exists to prevent.

Nothing else changed since the round-3 review.

@coderabbitai

coderabbitai Bot commented Aug 9, 2026 •

Copy link
Copy Markdown

@unbraind I will perform a full review of PR #59.

The ledger timestamp finding does not apply. The event timestamps precede the review timestamp on 2026-08-09. I withdraw that finding.

⚠️ Action not completed

Review rate limited.


Your included review limit is currently reached under our Fair Usage Limits Policy. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 49 minutes.

@unbraind

unbraind commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Merging. All checks green (test 22/26, CodeRabbit, Greptile, DeepScan, Hound, Sourcery, semgrep,
cubic), MERGEABLE/CLEAN, every thread answered and every bot comment read and voted.

@coderabbitai — thank you for withdrawing the timestamp finding explicitly rather than letting
it lapse. A retracted finding with the reason stated is more useful than one that quietly stops
being mentioned, because the next reader of the thread can tell the difference between "resolved"
and "wrong". Upvoted, same as the correct one.

Three rounds, one defect class. Worth stating plainly for both of you, because it is the most
transferable thing here:

Round What was found
1 A guard turned a buffer overrun into a wrong answer
2 The fix moved the failure — it produced an un-identifiable orphan
3 The recovery step claimed an outcome it never checked

Every round found the same shape one level deeper: code asserting a result it did not observe.
Round 3's writeSync finding was the sharpest instance, because the test fixture had the same
failure mode as the code under test — it reproduced "killed after the side effect, before the
output is readable" by killing a subprocess, and could therefore lose the receipt itself and assert
the wrong branch while looking like flake.

The guarantee this PR ships is deliberately narrower than it could have been made to sound: no
mutation is left un-identified
, not no mutation is left behind. Update compensation was declined
because an update has no inverse without prior field values, and filed as
pm-csv-71zg
rather than waved away. If a future PR adds update compensation without prior-value capture in the
transaction record, that is a finding, not a fix — @greptileai has this saved as a rule.

@unbraind
unbraind merged commit f5ac1d5 into main Aug 9, 2026
9 checks passed
@unbraind
unbraind deleted the fix/pm-shellout-buffer-silent-degradation branch August 9, 2026 00:48
@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

@unbraind Thank you for the clear review record.

The stated guarantee is correct: the code identifies uncertain mutations. It does not claim that every mutation was reversed.

I will apply this review rule in future subprocess and recovery paths: code and test fixtures must not assert an outcome unless they observed that outcome.

🐇 ✅


✏️ Learnings added
Learnt from: unbraind
URL: https://github.com/unbraind/pm-csv/pull/59

Timestamp: 2026-08-09T00:48:58.607Z
Learning: In `unbraind/pm-csv`, review subprocess, recovery, and test-fixture paths for assertions of outcomes that the code did not observe. Treat an unobserved result as a defect even when the path reports an error, because it can produce a wrong answer or select the wrong assertion branch.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

You are interacting with an AI system.

@greptile-apps

greptile-apps Bot commented Aug 9, 2026

Copy link
Copy Markdown

Confirmed — the rule is saved and will flag any future PR that adds update compensation without prior-value capture in the transaction record.

The three-round progression you described is a clean example of the pattern worth naming: each round the defect moved one step closer to the claim site rather than to the actual observed state. Round 3 being sharpest because the fixture had the same failure mode as the code under test is the kind of thing that usually only gets noticed in retrospect. Worth the write-up.

Good luck with the merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant