Skip to content

Certify complete local reads before Jira export and import - #130

Merged
unbraind merged 2 commits into
mainfrom
fix/complete-local-reads
Oct 7, 2026
Merged

unbraind merged 2 commits into
mainfrom
fix/complete-local-reads

Conversation

@unbraind

@unbraind unbraind commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

Fixes #128

The previous exporter read pm list --full --include-body --limit 10000 and decoded items, results, or an empty fallback as the whole corpus. That excluded terminal work, capped active rows, and accepted partial or omitted answers. Export now certifies its complete local input before planning or pushing. Both import entrypoints also certify their existing-item index before local writes and skip previously imported issues across every lifecycle state.

The issue's October 5 correction is respected: the old bounded reader had only an export caller. Import matching and zero-create re-import behavior are explicit additions required by this task, rather than a claim that the old importer called that reader.

Mechanism and design

  • Use the public in-process listAllComplete({ includeBody: true }, { pmRoot, cwd, noExtensions: true }) API. Resolve relative tracker roots once so root and working-directory resolution agree. No local-read subprocess, rendered JSON buffer, or 10,000-row ceiling remains.
  • Re-certify with public certifyCompleteListResult. Its certificate establishes complete source scanning, all statuses, full metadata, unique identifiers, no pagination, and no field/output omission or compaction. Additionally require body inclusion and validate every consumed payload/provenance field, because the SDK certificate does not validate those row field types or body inclusion.
  • Raise semantic CommandError with a tracker/SDK recovery hint and the full strict/all-status/unbounded inspection command. A certified empty tracker succeeds; an absent/uninitialized tracker fails.
  • Index Jira browse URLs, including terminal work and description/body provenance or URL metadata. Preserve existing items without updating or reopening them.
  • Keep the atomic transaction identity based on its original issue set and retain its already-applied mutation IDs in the SDK retry plan. Recovery reports only newly applied creates; complete re-import reports zero.
  • Align npm peer and extension manifest minimums at the known-supported pinned SDK/CLI version, 2026.10.4. Include README, verification documentation, generated distribution, tracker items, and matching history streams. No threshold or source exclusion changed.

Refusal table

Answer Enforcement before writes
Missing/null/array envelope; missing or non-array items; results-only alias Public complete-list envelope validation
Partial/unchecked/missing source proof; unreadable source artifacts Strict SDK scan and source completeness validation
Filtered corpus, excluded terminal items, missing strict-read proof Public scope validation
truncated, has_more, next_cursor, applied limit, missing pagination proof, count mismatch Public pagination/count validation
Compact projection or unproven/missing bodies Public projection validation plus integration body checks
Missing, contradictory, or affirmative field-omission receipt Public field-omission validation
Missing/malformed read-output receipt or incomplete dimensions Public universal receipt validation
Compacted strings/rows, output truncation, result/budget omission, session projection Public intact/unbounded delivery validation
Null rows; missing/blank/duplicate identifiers Public item identity validation
Missing/blank/malformed title/status/type/body; invalid descriptions/provenance/tags/priority Integration payload-field validation

Real regression tests and revert proof

test/complete-local-reads.test.ts creates disposable trackers through the real installed pm CLI and generates real TOON items plus hashed history with public SDK serialization/history/file-writer APIs in process. A sampled history is verified through the CLI. No SDK is mocked, and no real Jira/Linear service or live provider credential is used.

The focused suite passes 60 tests, covering 10,003 items (10,001 active plus closed/canceled work), retained final bodies, 51 refusal cases derived from a real SDK envelope, corrupt-tracker refusal before export push/import writes, durable no-write snapshots, terminal matching, both import modes, zero-create repeat imports, URL/body provenance, duplicate local identities, empty/missing trackers, relative roots, and real SDK interruption recovery. Existing exporter runtime tests now populate real trackers; config-driven sync also verifies a zero-create repeat import.

Negative control: restore the baseline CLI reader and original runImport; expose the old envelope decode expression through the validation test entrypoint while retaining test exports. The unchanged selected regressions exit 1 with 57 failures, including all 51 refusal cases and both terminal-matching modes (3 creates instead of 1). A separate reader-only revert against the final large fixture returns exactly 10,000 instead of 10,003 and exits 1. Restore the implementation: the focused suite and full gates pass.

Gates and scope

Exact commit: 3465e21

  • node --test test/complete-local-reads.test.ts: 60 passed; also passed through pm test --run --match complete-local-reads --progress.
  • npm run release:check: 254 passed; exact configured 100% lines, branches, and functions.
  • bun run release:check: 254 passed; same unchanged thresholds.
  • Both full gates include typecheck, build, docstrings, production audit, package dry run, changelog freshness, changelog-date verification, and publish-attestation workflow verification.
  • npm run changelog:full ran after tracker writes. No version/tag/release/publish action was performed.

Coverage's configured executable inventory is index.ts and sdk-importer.ts; this does not claim operational-script coverage. The Bun release command uses the repository's Node coverage runner and is not independent native-Bun SDK-runtime certification. Test-result tracking remains disabled by existing policy; test results and decisions are recorded in tracker comments.

Out-of-scope follow-up pm-jira-d3tp records the existing misleading compensation message on a resumable SDK interruption. Recovery is preserved and tested; that diagnostic remains open. The SDK payload/body certificate gap is handled locally; no upstream issue was filed.

Work item: pm-jira-sdwp. Claim released without closing.

Detailed contract and evidence: docs/complete-local-reads.md.

Summary by Sourcery

Certify complete local tracker data before Jira export or import and make imports idempotent across all item lifecycle states.

New Features:

  • Certify complete local tracker reads through the public SDK before export planning or import writes, including all lifecycle states, bodies, and required payload metadata.
  • Skip Jira issues already represented locally by browse URL across sequential, atomic, and sync imports, including terminal work and repeat imports.

Bug Fixes:

  • Prevent partial, truncated, malformed, compacted, or incomplete local data from being exported or written to before certification.
  • Preserve atomic transaction identity and applied mutations so interrupted imports recover correctly and report only newly created items.

Enhancements:

  • Replace the bounded CLI JSON reader with an in-process, unbounded complete-read workflow and actionable recovery errors.
  • Resolve Jira provenance by full browse URL and preserve existing local items without updating or reopening them.

Build:

  • Raise the minimum supported pm-cli dependency and extension version to 2026.10.4.

Documentation:

  • Document the complete local-read contract, refusal matrix, recovery behavior, and verification evidence.

Tests:

  • Add real-tracker regression coverage for large corpora, terminal items, malformed read refusal, no-write guarantees, matching, repeat imports, and atomic recovery.

Chores:

  • Regenerate distribution artifacts and record related tracker work and diagnostic follow-up.

Summary by cubic

Fixes #128. The exporter previously read pm list --full --include-body --limit 10000 and decoded items, results, or an empty fallback as the whole corpus, which dropped terminal work, capped active rows, and accepted partial output. Export and both import entrypoints now certify a complete local read through the public SDK listAllComplete API before planning or writing anything.

  • Malformed, partial, truncated, or uncompact results are refused with a CommandError naming the full strict all-status inspection command; a certified empty tracker still succeeds.
  • Import indexes Jira browse URLs including terminal work and skips already-imported issues without updating or reopening them; re-import reports zero new creates.
  • The original transaction identity and applied mutation IDs are retained in the SDK retry plan, so recovery reports only newly applied creates.
  • The SDK is loaded once through importPmSdk at the complete read; a missing SDK is refused there before any planning or writing.
  • Added a regression suite using real disposable trackers through the installed PM CLI, covering 10,003 items, 51 refusal cases, terminal matching, both import modes, and interruption recovery.

Migration

  • The npm peer dependency and manifest.json minimum move to @unbrained/pm-cli 2026.10.4.

Written for commit b139f9f. Summary will update on new commits.

View guided diff Turn on auto-fix

Comment thread dist/index.js
@sourcery-ai

sourcery-ai Bot commented Oct 7, 2026

Copy link
Copy Markdown

Sorry @unbraind, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 2 days and 7 hours by commenting @sourcery-ai review. Upgrade to get a review now.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@coderabbitai

coderabbitai Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: b8de7fc5-a0e4-4c95-a00e-27917ab50d51
📥 Commits

Reviewing files that changed from the base of the PR and between df351b0 and b139f9f.

⛔ Files ignored due to path filters (5)
  • dist/index.d.ts is excluded by !**/dist/**
  • dist/index.d.ts.map is excluded by !**/dist/**, !**/*.map
  • dist/index.js is excluded by !**/dist/**
  • dist/index.js.map is excluded by !**/dist/**, !**/*.map
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (12)
  • .agents/pm/history/pm-jira-d3tp.jsonl
  • .agents/pm/history/pm-jira-sdwp.jsonl
  • .agents/pm/issues/pm-jira-d3tp.toon
  • .agents/pm/issues/pm-jira-sdwp.toon
  • README.md
  • docs/complete-local-reads.md
  • index.ts
  • manifest.json
  • package.json
  • test/atomic.test.ts
  • test/complete-local-reads.test.ts
  • test/coverage-runtime.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.


Summary by CodeRabbit

  • New Features
    • Export previews and pushes now read the complete local tracker, including trackers with more than 10,000 items.
    • Imports and Jira sync skip issues already present, preventing duplicate items. Re-importing matched issues reports zero new items, and existing matches remain unchanged.
  • Bug Fixes
    • Incomplete, inconsistent, or malformed tracker data now stops exports and imports before writes begin, with recovery guidance.
  • Documentation
    • Updated setup requirements and documented complete-read checks, import behavior, and refusal conditions.

Walkthrough

The change replaces subprocess-based item reads with certified SDK reads. Import and sync paths use existing Jira URLs to skip duplicates, while atomic imports retain a transaction ID for recovery. Tests and documentation cover complete reads, matching, refusal cases, and retries.

Changes

Certified local reads and Jira synchronization

Layer / File(s) Summary
Certify complete local reads and export
index.ts, test/complete-local-reads.test.ts, test/coverage-runtime.test.ts, README.md, docs/complete-local-reads.md, manifest.json, package.json, .agents/pm/issues/pm-jira-sdwp.toon, .agents/pm/history/pm-jira-sdwp.jsonl
The SDK reader requests complete local items, certifies the result, and validates item fields before export or import proceeds. Tests cover large trackers, invalid reads, and refusal before writes. The documented and package minimum CLI version changes to 2026.10.4.
Match existing Jira imports and recover atomic runs
index.ts, test/atomic.test.ts, test/complete-local-reads.test.ts, test/coverage-runtime.test.ts, README.md, docs/complete-local-reads.md, .agents/pm/issues/pm-jira-d3tp.toon, .agents/pm/history/pm-jira-d3tp.jsonl
Import paths skip Jira URLs already present locally, including matches in terminal states. Atomic imports pass a stable transaction ID and can resume transaction-owned work. Tests cover repeated imports and interrupted recovery; the atomic diagnostic issue is recorded as deferred.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant runImport
  participant readPmItems
  participant existingJiraUrls
  participant importJiraAtomic
  runImport->>readPmItems: Read and certify local items
  readPmItems-->>runImport: Return complete PmItem list
  runImport->>existingJiraUrls: Index Jira URLs and item IDs
  existingJiraUrls-->>runImport: Return existing item index
  runImport->>importJiraAtomic: Submit pending issues and transaction ID
Loading

Merge Risk: ⚪ Minimal · up to b139f

Export and import refuse failed complete reads before writing. No merge-blocking issue was established; the change is ready for normal merge checks.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: certifying complete local reads before Jira export and import.
Description check ✅ Passed The description directly explains the changes, implementation, tests, and scope of the pull request.
Linked Issues check ✅ Passed Issue #128 requires complete, certified local reads before export or import writes, refusal of incomplete or malformed data, and regression coverage for large trackers, terminal matching, and repeat i…
Out of Scope Changes check ✅ Passed The source changes, tests, SDK minimum updates, and documentation support Issue #128's complete-read, import-matching, or recovery requirements. The tracker records a deferred diagnostic follow-up rel…
Docstring Coverage ✅ Passed Docstring coverage is 91.67% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 4 files. (8 skipped: 8 …
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai

sourcery-ai Bot commented Oct 7, 2026

Copy link
Copy Markdown

Reviewer's Guide

The PR makes Jira export and both import paths depend on a certified, in-process, whole-tracker read rather than a bounded CLI JSON decode, adds strict pre-write validation and recovery errors, makes imports skip existing Jira identities across all lifecycle states with resumable atomic semantics, and backs the behavior with real-tracker integration tests and aligned SDK packaging.

Sequence diagram for certified Jira export

sequenceDiagram
    participant Export as JiraExport
    participant Reader as readPmItems
    participant SDK as pmCliSDK
    participant Certifier as certifyPmItems
    participant Jira as JiraAPI

    Export->>Reader: readPmItems(pmRoot)
    Reader->>SDK: listAllComplete({ includeBody: true }, { pmRoot, cwd, noExtensions: true })
    SDK-->>Reader: complete-list result with certificate
    Reader->>Certifier: certifyPmItems(candidate)
    Certifier->>SDK: certifyCompleteListResult(candidate)
    alt certificate and payload fields valid
        Certifier-->>Export: certified items
        Export->>Jira: create issue payloads
    else incomplete or malformed local read
        Certifier-->>Export: CommandError with recovery command
    end
Loading

Sequence diagram for duplicate-safe Jira import

sequenceDiagram
    participant Import as JiraImportOrSync
    participant Reader as readPmItems
    participant SDK as pmCliSDK
    participant Index as existingJiraUrls
    participant Tracker as LocalTracker

    Import->>Index: existingJiraUrls(pmRoot)
    Index->>Reader: readPmItems(pmRoot)
    Reader->>SDK: listAllComplete({ includeBody: true }, { pmRoot, cwd, noExtensions: true })
    SDK-->>Reader: certified complete local corpus
    Reader-->>Index: all local items
    Index-->>Import: Jira browse URL index
    Import->>Import: filter pending issues by Jira URL
    alt pending issues exist
        Import->>Tracker: createPmItem(pmRoot, item)
    else all issues already imported
        Import-->>Import: report zero newly imported items
    end
Loading

State diagram for atomic Jira import recovery

stateDiagram-v2
    [*] --> ReadCertified
    ReadCertified --> ExistingIndexed
    ExistingIndexed --> PendingPlanned
    ExistingIndexed --> CompleteReplay: all Jira URLs match
    PendingPlanned --> AtomicCommit
    AtomicCommit --> Imported
    AtomicCommit --> Interrupted
    Interrupted --> AtomicCommit: importJiraAtomic recovery
    Imported --> [*]
    CompleteReplay --> [*]
Loading

File-Level Changes

Change Details Files
Replace bounded CLI/JSON local reads with certified, in-process complete-corpus reads.
  • Use the public SDK complete-list API with resolved tracker roots, bodies, all lifecycle states, strict reads, no extensions, and unbounded output.
  • Certify SDK completeness metadata and validate all payload/provenance fields before planning or writing.
  • Return semantic recovery errors for malformed, incomplete, missing, or uninitialized trackers while accepting certified empty trackers.
index.ts
dist/index.js
dist/index.d.ts
docs/complete-local-reads.md
README.md
Make import matching lifecycle-complete and re-imports non-duplicating.
  • Index existing Jira browse URLs from metadata and provenance across terminal and active local items, preserving host distinctions.
  • Skip matched issues in sequential, atomic, and config-driven sync imports without updating or reopening existing items.
  • Preserve atomic transaction identity and applied mutation IDs for interruption recovery while reporting only newly created items.
index.ts
dist/index.js
test/atomic.test.ts
test/coverage-runtime.test.ts
test/complete-local-reads.test.ts
README.md
Align supported SDK versions and ship regenerated distribution/documentation artifacts.
  • Raise npm peer and extension minimums to 2026.10.4.
  • Update generated distribution files, package lock, README, and complete-read verification documentation.
package.json
package-lock.json
manifest.json
dist/index.js
dist/index.d.ts
dist/index.js.map
dist/index.d.ts.map
README.md
docs/complete-local-reads.md
Add real integration and regression coverage for completeness, refusal, matching, and recovery.
  • Exercise real installed CLI trackers and public SDK serialization, certification, history, and transaction APIs without SDK mocks or live Jira credentials.
  • Cover large unbounded exports, terminal work, body retention, 51 refusal cases, corrupt trackers, zero-create re-imports, distinct hosts, relative roots, and atomic interruption recovery.
  • Replace fake CLI-reader coverage fixtures with real tracker fixtures and verify release gates and revert behavior.
test/complete-local-reads.test.ts
test/coverage-runtime.test.ts
test/atomic.test.ts
Record work-item history and the deferred interruption-diagnostic follow-up.
  • Add tracker issue and history artifacts for the implementation and out-of-scope diagnostic.
.agents/pm/issues/pm-jira-sdwp.toon
.agents/pm/history/pm-jira-sdwp.jsonl
.agents/pm/issues/pm-jira-d3tp.toon
.agents/pm/history/pm-jira-d3tp.jsonl

Assessment against linked issues

Issue Objective Addressed Explanation
#128 Replace the bounded, permissive local item reader with a complete, strict, all-status, unbounded read that uses a public SDK completeness certificate and avoids accepting missing, partial, malformed, compacted, or omitted results. ✅
#128 Refuse uncertified or malformed local reads before export planning/pushing, import reconciliation, or local writes, while validating the payload fields consumed by the Jira integration. ✅
#128 Ensure reconciliation and re-import account for all lifecycle states and preserve already imported Jira issues, including terminal-item matching, distinct Jira hosts, atomic recovery, and zero-create idempotent re-imports. ✅

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@unbraind

unbraind commented Oct 7, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
✅ Action performed

Full review finished.

@unbraind
unbraind merged commit 28f312b into main Oct 7, 2026
10 checks passed
@unbraind
unbraind deleted the fix/complete-local-reads branch October 7, 2026 13:09
unbraind added a commit that referenced this pull request Oct 7, 2026
Co-authored-by: SteveBot <1153461+unbraind@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refuse incomplete local item reads before Jira reconciliation and export

1 participant