redact: entropy pass eats long camelCase identifiers (14 false positives on a 50-note claude-memory import)

[sergiuszm]

Environment: agentcairn 0.26.1 (PyPI, uv tool install), Linux x86_64, Python 3.11.

What I did: cairn import claude-memory --source ~/.claude/projects/<slug>/memory --vault <trial> - 50 notes, 14 redactions. Zero of the 14 were secrets. All 14 are over-redactions by the Pass 3 entropy heuristic.

Repro corpus (worst cases). Full before/after pairs were recovered by comparing the untouched CC source store against the imported notes, so all cases below are exact and testable. 14 sites, 3 classes:

Class 1 (11/14 - 8 distinct tokens) - camelCase identifiers eaten by the entropy pass.
All are prose identifiers, no digits, no separators, >=24 chars of [A-Za-z0-9], Shannon entropy ~3.9 over the char distribution:

setDefaultModelAndProvider          # 26 chars - survived in CC source, pair verified
normalizeToolResultImages
getAvailableProviderCount
copySelectionToClipboard            # exactly _ENTROPY_MIN_LEN = 24, boundary case
maxActiveAsyncRunsPerSession
piInfrastructureReadPaths           # x2 in one note
verifyAutoModeGateAccess
cudaErrorCompatNotSupportedOnDevice # 35 chars - a CUDA error enum in prose

Context examples from my store:

before:  (Ctrl+S is what persists (`setDefaultModelAndProvider()` -> settings.json))
after:   (Ctrl+S is what persists (`[REDACTED:high_entropy]()` -> settings.json))

before:  TuiAltScreen.prototype.copySelectionToClipboard, dedents the shared indent
after:   TuiAltScreen.prototype.[REDACTED:high_entropy], dedents the shared indent

Class 2 (1/14) - a named-pattern pass splits an identifier at a word boundary and redacts the tail with its numeric value:

before:  DEFAULT_THINKING_TOKEN_BUDGET=32768
after:   DEFAULT_THINKING_[REDACTED:secret_assignment]

Class 3 (2/14) - correctly-shaped catches, policy question. A fragment of an ed25519 fingerprint (SHA256:vUz1U5S2DHxNkK/[REDACTED:...]) and a 64-hex sha256 of a GGUF model file. Genuinely secret-shaped, but in infra-diary notes file digests are routine, non-secret content. Worth a policy note, not a bug.

A machine-readable corpus (agentcairn-redact-corpus.json) with all 14 before/after pairs is attached - it can drive redactor regression tests directly. Reproduce from source with the one-way importer against any identifier-dense Claude Code memory store.

Root cause (reading cairn/ingest/redact.py @ 0.26.1):

1. _TOKEN_RE = [A-Za-z0-9+]{24,} deliberately excludes / - _ (good - that was the 0.7.1 fix), but leaves unseparated camelCase in the candidate set.
2. _looks_secret(): tokens with upper+lower+digit are gated at _ENTROPY_BITS (3.5); tokens without digits fall to the 3.8 branch. A long letter-rich camelCase word (no digits) has ~3.9-4.0 bits of Shannon entropy over its char distribution, above 3.8, so setDefaultModelAndProvider (26 chars) qualifies as structureless secret material.

Suggested direction: require a digit in both entropy branches. Secrets are effectively always alphanumeric-mixed; camelCase prose identifiers rarely are. This preserves the intentional catches - 40-char AWS-style runs have their own guarded pass, and 64-hex digests stay caught even when mixed-case.

Context that makes this worse than cosmetic: the one-way importer re-redacts a note whenever its source changes (version++ gets a fresh redaction pass), so manually repairing an over-redacted note regresses on the next source change. A toggle or a repaired-anchor rule would make local repair durable; a per-import opt-out would help trusted-home vaults.

agentcairn-redact-corpus.json