Skip to content

fix: exclude same-millisecond replayed usage from scan budgets - #631

Open
mldangelo-oai wants to merge 40 commits into
mainfrom
mdangelo/codex/fix-scan-budget-attribution
Open

fix: exclude same-millisecond replayed usage from scan budgets#631
mldangelo-oai wants to merge 40 commits into
mainfrom
mdangelo/codex/fix-scan-budget-attribution

Conversation

@mldangelo-oai

@mldangelo-oai mldangelo-oai commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

Exclude replayed parent usage from a child scan's budget when the parent turn and child thread were created within the same millisecond.

Changes

  • Retarget this PR to main, preserving the original branch history without force-pushing. With fix(sdk): preserve sealed artifacts after post-scan failure #626's restoration change now merged, this seven-file diff carries only attribution changes and does not carry the closed fix(plugin): preserve committed diff scan integrity #623/fix(scan): confine local diff inputs to the selected target #630 implementations.
  • Compare complete UUIDv7 values in the live tracker and Python collector, preserving the existing non-UUID fallback and inherited-usage baseline.
  • Keep the producer assumption narrow: pinned Codex 0.149.1's fresh-worker path snapshots parent history, allocates the child ID, then submits its own turn in the same process using a monotonic generator. The existing producer evidence does not establish ordering for imported histories, custom ID factories, unrelated processes, or every external fork caller.
  • Update the source plugin and SDK bundle markers to 0.1.80. The SDK bundle is generated from plugins/codex-security; generated files are not edited or committed separately.
  • Extend the existing real cache-upgrade/restoration fixture for 0.1.60 and 0.1.79: replace a stale collector, verify its installed bytes, and execute its usage calculation. Retain the restoration, stale MCP configuration, safety forwarding, credentials, login, and unrelated-state checks.

Testing

  • Main's collector overcounts the focused inherited-plus-owned fixture as 1,210 tokens; the patched collector reports the owned 110 tokens. The owned-only and non-UUID controls pass.
  • Combined attribution/restoration checks with Bun 1.3.14, seed 12345: 390 passed, 15 platform skips, no failures across cost, runtime, post-scan, API, contract, and bundle-generation suites.
  • Existing Python collector suite: 20 passed. The integration did not change the already-tested collector or ownership fixture.
  • Node 22.15.1: bundle generation, generated-model check, MCP and SDK types, build, and plugin-source check passed. Changed-file formatting, Ruff, and diff checks passed.
  • Fresh integrated npm package verification passed for all 282 entries.
  • Full current-head hosted CI is pending this push. No local native Windows, live-model/billing execution, real account refresh, or release/publish run is claimed. The upstream producer was checked from pinned source, not rebuilt locally.

Risk and rollout

Pricing, public APIs, and CLI options are unchanged. The ownership rule depends on the pinned fresh-worker producer behavior described above, not on UUIDv7 in general. The earlier review thread remains open for its participants; this restack does not resolve another reviewer's thread or reopen either closed parent.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

Historical commit contact metadata and restricted links in automated comments remain, so the second attestation is unchecked. This update uses synthetic fixtures and a GitHub noreply commit identity; other authors' comments are unchanged.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 24, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-27T09:04:50.008582Z f3d1782 New commits
🔒 Security Review Completed 2026-08-25T05:19:19.831535Z adb1201 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: cb3e851450

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kmbroai kmbroai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critical review

Reviewed cb3e851450b603fddecdb1d8fe0a72f07285f9a3, against its declared #630 base. The same-millisecond attribution fix is justified; no blocking correctness issue found.

Necessity and correctness

Counting replayed parent usage as new child usage can terminate a scan against a user-requested cost limit. Comparing only the UUIDv7 timestamp cannot distinguish two IDs created within one millisecond. The change keeps the full ordering in both the live tracker and the bundled collector, so those paths agree about the inherited baseline. BigInt in TypeScript avoids losing the low ordering bits through Number conversion.

There is an important compatibility assumption: UUIDv7 format alone does not promise ordering within a millisecond. This relies on the producer's monotonic generator. I checked current public upstream source: Codex thread IDs and submission IDs use Uuid::now_v7, whose documented guarantee is creation order within the same process. Keep that producer relationship explicit; do not broaden this comparison to unrelated imported UUIDs or assume it proves cross-process chronology. This was a current-source check, not a rebuild of the pinned Codex runtime.

Simplification and scope

The production change is small and appropriately avoids a new attribution ledger. Keep the existing non-UUID fallback rather than adding arbitrary ID rejection. The bundle-version change is necessary for the Python collector to reach cached installations; coordinate it with the #623#626#630 stack rather than treating this as an independent version bump on main.

The test setup is much larger than the logic. The lower/higher same-millisecond controls and cross-runtime usage comparison are useful. The mock marketplace installer and exact bundle-version expectations would fit better in the existing runtime-upgrade fixture than being duplicated inside cost tests. The CODEX_SAFETY_IDENTIFIER assertion is unrelated to replay attribution and can stay in the package/environment tests. Preserve the installed-collector check, but reuse the existing setup where possible.

Verification

Ran cost.test.ts and diff-rank-input.test.ts with seed 12345: 64 passed, 0 failed, Bun 1.3.14/Linux with locally available dependencies. This includes the cached-collector and cross-runtime controls. No real billing, live model call, full clean install, or native platform matrix was exercised.

@faizan-oai faizan-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed this PR's replay-attribution delta against #630. All 60 cost tests, typecheck, and current-head CI pass. No blocking findings in this delta; #630's separate review discussion remains unresolved.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex security review

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 7cc7c65b9c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: 7cc7c65b9c

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: adb12010bf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: adb12010bf

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@faizan-oai faizan-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed adb12010bf4676cb9ff9eb0c78fdb56d42a39ac2 against declared base 446ff8fdbe08bdcb9a341a25fe79c867f1ed9e03 (#630). No blocking concern in this PR's replay-attribution delta; its own patch is unchanged from the previously reviewed restack.

The 67 focused cost/diff-input/cache tests and incremental formatting, syntax, version, and diff checks passed on this head. Full current-head node-ci 32812126779, attempt 2 now passes all 25 jobs, including native Windows. Head, base, CI, and discussions were rechecked before approval; no unchanged local suite was rerun for this CI-only follow-up. My earlier CI-hold comment concerned 5483c74b, not this head.

Approval covers only this PR's delta, not its parents. The existing within-process monotonic UUIDv7 qualification remains; upstream producer behavior and live billing/model execution were not revalidated.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e7afffc779

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +480 to +487
const threadOrder = uuid7Order(session.threadId);
const turnOrder = uuid7Order(payload["turn_id"]);
const owned =
threadOrder === null
? typeof payload["started_at"] === "number" &&
session.startedAt !== null &&
payload["started_at"] >= Math.floor(session.startedAt / 1_000)
: turnOrder !== null && turnOrder >= threadOrder;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid ordering UUIDv7 values beyond their timestamps

When a child thread and its first turn share a millisecond, UUIDv7 does not generally guarantee that the random tail of the later-created turn is greater than the thread's tail. A genuine lower-valued turn therefore leaves a one-turn worker permanently in replay mode and omits all its usage, while a higher-valued replayed turn can be accepted as owned; either case corrupts accounting, and the former can prevent --max-cost from being enforced. Use an actual ownership boundary rather than comparing the complete UUID values.

AGENTS.md reference: sdk/typescript/AGENTS.md:L25-L25

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UUIDv7 alone would not establish this ordering, but the pinned Codex 0.149.1 producer is more specific: thread IDs and turn/submission IDs both use Uuid::now_v7(), whose uuid 1.20.0 implementation guarantees same-process creation order.

Core's fresh-worker path captures parent history before allocating the child ID, then submits its first input afterward in that process (fork path; no reserved child ID). That path does not produce the proposed lower-valued own turn. This does not establish order for arbitrary imported histories, custom ID factories, or unrelated processes; a supported counterexample would change the assessment. This is source verification, not a live-model test. Approval of e7afffc7 still waits for the failed Windows CI run.

@faizan-oai faizan-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed f3d17829017ee7f4a67ff2f96a9b3fdcd775a780 against declared #630 base 059e7b9de1a9dfc780330e1f9b8b144edc6b471a. No blocking findings in this attribution delta. Production behavior is unchanged from the prior source review; shared fixtures and installed-collector upgrade coverage preserve the relevant checks. The UUID ordering assumption remains limited to Core's fresh-worker, same-process path.

Full current-head CI passes all 27 jobs, including Windows. Local checks were source/diff/AST verification, not a Bun suite.

This approval covers #631's own changes only; it does not approve #630 or other parent changes.

Retain only replay-attribution changes after the restoration fix. Preserve branch history without including the closed-parent implementations.

Keep the pinned fresh-worker UUID ordering qualification and combined installed-cache regression coverage.
@faizan-oai
faizan-oai changed the base branch from mdangelo/codex/fix-local-diff-input-confinement to main August 27, 2026 21:08
@github-actions github-actions Bot added the bug Something isn't working label Aug 27, 2026

@faizan-oai faizan-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed ccd0f7241abcc05b20a4b6d04a4509d66c675adc targeting main. The seven-file attribution diff is unchanged against current main 6ec373cb502511f221766afb9381465f8f9301e6 (merge base 562db120abacdfb8fe2746f0b5ef2935d632153d); the intervening main change is documentation-only and does not change the tested runtime inputs. No blocking findings in this isolated delta.

Full current-head CI passes all 28 jobs, including Windows. The older same-head runs were cancelled during retargeting and do not override this complete matrix. Local integrated verification passed 390 Bun tests (15 platform skips), 20 Python collector tests, types/build/formatting, and the package check. No local native Windows or live-model/billing run is claimed.

The existing producer qualification remains limited to the pinned Core fresh-worker, same-process path. This approval does not resolve the existing review thread; its participants still need to close that discussion before merge. The closed #623/#630 implementations are not included in this main-based delta.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants