Skip to content

fix(embeddings): reject background vectors for superseded content - #204

Merged
johnny4young merged 5 commits into
mainfrom
fix/content-bound-embeddings
Oct 8, 2026
Merged

johnny4young merged 5 commits into
mainfrom
fix/content-bound-embeddings

Conversation

@johnny4young

@johnny4young johnny4young commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

Finding

Capture enrichment keeps the captured body across asynchronous title work; model-version refresh reads a body before computing its vector. Both later used an unconditional ID-only upsert. A concurrent text edit could delete its old vector, then an older background producer could put that stale vector back.

Correction

  • Add the separate optional ContentBoundEmbeddingStoring capability. The existing ClipEnriching and EmbeddingRefreshSource requirements and unconditional methods remain source-compatible.
  • Persist with one INSERT ... SELECT in a GRDB write transaction: the complete source body must still match, and the row must remain present, unexpired, unarchived, non-sensitive and text-backed.
  • Route both capture enrichment and model refresh through the guarded capability. Unsupported custom stores skip vectors instead of using an unsafe fallback; production GRDB supports the capability.
  • Count only accepted refresh writes as progress and retain zero-progress termination.
  • Inject an internal embedder factory for deterministic caller tests, without model downloads.
  • Explain compatibility, eligibility, full-body comparison and ABA semantics in docs/CONTENT_BOUND_EMBEDDINGS.md.

Identity and risk

Title/keyword-only edits do not invalidate a body vector. A content-equal ABA change, including deletion/recreation under the same ID with the same eligible body, may accept it: the vector describes the current body. Different replacement text or changed privacy eligibility is rejected. The content hash is deliberately not used because curation preserves it for deduplication. Full-body comparison is stricter than the model's 1,000-character input truncation.

This does not introduce a reindexing scheduler, retroactively repair all existing vectors, or change title/OCR behavior. Review the optional-capability behavior for custom stores and transaction predicate.

Regression evidence

  • Executed the original SQL upsert against an edit-between-read-and-write fixture: it stores the stale vector (red).
  • Executed the new extracted SQL: old body is rejected and current body accepted (green). This is portable SQLite evidence, not Swift execution.
  • Four store tests cover changed bodies, deletion/ID reuse, title/content-equal changes and altered privacy/expiry.
  • Three caller tests use continuation barriers at the write boundary, with changed-body capture and refresh cases plus an unchanged-body positive control. No timing sleeps or on-device models.
  • Existing refresh tests now implement the guarded capability.
  • git diff --check passes; staged local tree equals uploaded tree.

Exact source and limits

Head: 919acb3
Tree: 4f974d5947577ac0fb326de9370e58609438ea46
Base: main at ccc4403

make format, make lint and make test were attempted, but this Linux executor lacks Swift/Xcode. Initial CI 37661464708 failed strict swift-format on one trailing comma in ContentBoundEmbeddingTests.swift before package tests executed. The forward-only follow-up removes that comma, with no behavior, assertion or gate change. Exact-head CI 37690654827 passed: strict swift-format, SwiftLint, build-warning checks, 1,507 package tests in 234 suites, 83.8% production-source coverage above the unchanged 80% floor, StoreKit entitlement tests and both macOS/iOS app builds. All seven new embedding regressions passed natively. GitGuardian passed; the non-blocking beta canary was skipped as expected. Scoped gates are complete and this PR is ready for review. No UI flow was changed; the separate full UI workflow was not dispatched.

All earlier open repairs were inventoried before editing. No open PR addresses these producer/persistence paths. #203 fixes legacy snippet-edit invalidation; this PR has no overlapping files, is independently main-based, and also protects current draft/text-edit paths. No stack dependency or combined-native qualification is claimed.

No public API removal, schema migration, new dependency, log content, license removal, threshold/workflow change, manual CI dispatch, merge, release or deployment.

claude added 3 commits October 8, 2026 04:38
- Accept a guarded vector write for any row the store still shows, using the
  shared unexpired predicate, so an expired pinned, boarded or snippet row
  stays indexable as it was before
- Share the ineligible-kind list between the guarded write and the stale
  refresh queue, so rows the write always rejects cannot fill a batch and
  stop the refresh for every row behind them
- Bound the write-race tests with a time limit so a regression fails instead
  of hanging, and cover both rules
@johnny4young
johnny4young merged commit 84ab7a4 into main Oct 8, 2026
4 checks passed
@johnny4young
johnny4young deleted the fix/content-bound-embeddings branch October 8, 2026 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants