Skip to content

refactor(import): project CSV previews without retaining raw row matrices - #201

Merged
johnny4young merged 5 commits into
mainfrom
refactor/csv-preview-projection
Oct 8, 2026
Merged

johnny4young merged 5 commits into
mainfrom
refactor/csv-preview-projection

Conversation

@johnny4young

@johnny4young johnny4young commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Review follow-ups

Reviewed; no changes needed. Header selection, error precedence (unclosedQuotedField over an invalid header), short-row/blank-text counting and pinned parsing are identical to main, and no other caller used the removed parseCSV.

Merge order

Independent of every other open PR; merge before the v0.10.0 release PR #193.


Original description:

Finding

CSV preview first collected the entire raw row matrix, shifted off its header, and for Gancho exports created another complete formula-decoded matrix before constructing candidates. Projection did not need either retained intermediary collection.

Refactor

  • Visit each completed row directly and interpret the header once.
  • Project candidate text, optional title/pin state and unsupported counts without retaining the full raw matrix.
  • Keep the character scanner and formula-guard policy unchanged.
  • Complete scanning before reporting invalid headers or returning a document. An unterminated tail still takes precedence and never exposes partial candidates.
  • Remove the now-unused internal collecting parser; the public readCSV/Data and Document APIs are unchanged.
  • Document the preview/persistence boundary in English architecture documentation.

The full input string and final candidate document still remain in memory. This is not a bounded-memory file-streaming API, and no measured memory or latency improvement is claimed.

Regression coverage

Six new native tests cover BOM/normalized and duplicate headers, Gancho versus foreign apostrophes, quoted CRLF/multiline and escaped quotes, short optional fields and unsupported rows, malformed-tail atomicity/error precedence, and ordered repeatable 5,000-row previews. Existing importer/export roundtrip and migration tests remain intact.

Verification

Published source head: ca93a55
Uploaded tree: f51b961003bc89fee3b20c3a5bd4c9ced1a23f22
Explicit base: main at ccc4403

  • Local git diff --check passed.
  • make format, make lint and make test were attempted; this Linux executor lacks Swift/Xcode/xcode-select, so they are blocked, not passed.
  • Exact-head native CI 37405620189 completed successfully. Existing formatter, test, coverage and app-build gates are unchanged.

Scope and integration

Independent main-based PR; no dependency on the open Keychain, license, archive, recovery, ownership or MCP repairs. Storage ownership/actual-connection lifetime protocols are untouched. Physical iOS/App Group lifecycle, quiescent upgraded cohort and combined native MCP qualification remain outside this refactor and unresolved. No merge, release, deployment, live library or credential access.

Exact-head hosted evidence

Corrected source head: ca93a55
Verified remote/local tree: f51b961003bc89fee3b20c3a5bd4c9ced1a23f22
Verified synthetic PR merge checkout: cf4d528

  • The first head failed only strict formatting of the new large-preview assertion. The normal forward follow-up corrects that assertion's line break/indentation; production behavior and test expectations are unchanged.
  • Strict swift-format, SwiftLint and warning checks pass.
  • All six new CSV projection regressions pass natively, including malformed-tail atomicity and repeated 5,000-row previews.
  • 1,506 package tests in 233 suites pass; production-source coverage is 83.9% above the unchanged 80% floor.
  • StoreKit purchase/entitlement tests and macOS/iOS app builds pass.
  • The non-blocking beta canary has its expected skip; both required jobs complete successfully.
  • Source-only pairwise merge checks were conflict-free with recorded existing repair heads. This does not qualify their combined native integration or physical App Group lifecycle.

Ready for human review; remains main-based and mergeable. No gates were relaxed, and no merge, release or deployment occurred.

Human-review readiness

Scoped CSV implementation and regression coverage are complete at unchanged head ca93a55. Fresh verification confirms both required native jobs and GitGuardian successful, with the expected non-blocking beta-canary skip, and no pending discussion comments, submitted reviews or unresolved inline threads. No physical lifecycle or combined-integration qualification is inferred for other open PRs.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ExmnfUngrDYjAheGvYwMnn


Generated by Claude Code

@johnny4young
johnny4young marked this pull request as ready for review October 6, 2026 03:47

@johnny4young johnny4young left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary: ClipImporter.readCSV now runs header interpretation and candidate projection inside a row visitor (forEachCSVRow), so it no longer keeps the full [[String]] matrix plus a formula-guard-mapped copy in memory.

Verdict: ready. Nothing blocking.

What I checked against main:

  • Header selection is the same: it is the first non-empty row, because empty rows are still filtered in endRow. text still wins over contenttext, the first duplicate column still wins, and isGanchoExport still keys on contenttext.
  • Error precedence is the same. The old code parsed everything before validating the header, and the new code finishes the scan before the emptyCSV / missingTextColumn guards, so unclosedQuotedField still wins over a bad header. No partial Document can escape, because the visitor only appends to a local array and the function throws before returning.
  • Short-row and blank-text handling (unsupportedCount), and the pinned values true/1/yes, are unchanged.
  • The deleted parseCSV had no other callers.
  • CI is green: lint, package tests, app builds.

🟡 Nit, optional: when the header has no text column, the remaining rows are scanned only to keep syntax-error precedence. That is the right call, and the closure already returns early, so this is fine as is.


Generated by Claude Code

claude added 3 commits October 8, 2026 04:23
…w tests

- Strip Gancho formula guards from the text, title and pinned fields only,
  through one bounds-checked helper, instead of copying every column per row
- Drop a malformed-tail assertion that could never fail and compare the large
  preview against one expected array
- State the real column precedence in the architecture notes
@johnny4young
johnny4young merged commit 51e8a83 into main Oct 8, 2026
4 checks passed
@johnny4young
johnny4young deleted the refactor/csv-preview-projection branch October 8, 2026 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants