Add optional format support to the re-identify tool - #28
Conversation
Adds an optional `format` input to the re-identify tool so the calling agent can specify, per entity type, how tokens are rendered on the way out: fully restored (plaintext), partially masked, or fully redacted. This mirrors the Skyflow Detect API `format` object and maps to the skyflow-node SDK's `ReidentifyTextOptions`. Entity types not listed default to full plaintext restoration, preserving existing behavior. - handler: build `ReidentifyTextOptions` from the format and echo the applied format back in the output - server: add `format` to the re-identify input/output Zod schemas and thread it through to the handler - types: add `ReIdentifyFormat`, extend `ReIdentifyOutput` - UI: summarize the applied format treatment in the re-identify app - tests: 8 new cases (entity routing, echo-back, empty/invalid buckets, backward compatibility) - docs: CLAUDE.md, README, wrapping guide, CHANGELOG Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AZZMCL2fjoR3FvvoM3vofZ
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
✅ Gitleaks Findings: No secrets detected. Safe to proceed! |
Review: Add optional
|
- Reject a format where the same entity type appears in more than one bucket (redacted/masked/plaintext), which would otherwise forward the entity to multiple SDK setters with undefined last-wins behavior. - Normalize the echoed-back format to omit empty buckets so the response reflects only what was actually applied. - UI: drop the redundant `.toUpperCase()` before `getEntityClass` (which lowercases internally) and give the format-summary badge dot a neutral color fallback so entity types without a dedicated CSS class still show. - Docs/tests: note the overlap rejection in CLAUDE.md; add unit tests for overlap rejection, empty-bucket normalization, and intra-bucket dups. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AZZMCL2fjoR3FvvoM3vofZ
|
✅ Gitleaks Findings: No secrets detected. Safe to proceed! |
|
Thanks for the review — addressed the actionable points in
#1 (unlisted-entities → plaintext default) — flagging for a human/live check. I couldn't verify this end-to-end from here: the hosted Detect API reference returned 403 to automated fetch, and there's no live vault in this environment for a round-trip. My confidence rests on (a) the no-format baseline already restores everything to plaintext (verified by the existing tool), and (b) the SDK models Generated by Claude Code |
|
Code Review — Add optional Nicely scoped, well-tested PR. The handler-first pattern is respected, both server-side Zod ( Correctness / behavior
Minor / style
Tests Coverage is strong: routing to each setter, single-bucket, empty/absent buckets, no-op forwarding, echo-back, empty-bucket normalization, intra-bucket dups, cross-bucket overlap rejection, invalid entity, and the omitted- Security No concerns. Entity strings are enum-constrained at the schema layer, error messages only echo those constrained values, and the UI escapes labels via Review based on reading the diff against main; I was unable to run the suite in this environment, but the PR reports 174 tests green and a clean |
Follow-up to the second review pass:
- Refactor buildReidentifyOptions / normalizeFormat / findFormatOverlaps
around a shared FORMAT_BUCKETS list so the SDK options and the echoed
format derive from one normalization step (no duplicated per-bucket
logic that could drift).
- normalizeFormat now de-duplicates entity types within a bucket, so a
value listed twice (masked: ["ssn","ssn"]) reaches the SDK and the echo
once.
- Omit `format` from the response entirely when the normalized result is
empty, instead of echoing `{}`.
- Reword docs/schema/type comments from "applied" to "requested": the
echo reflects the caller's requested (normalized) format, not what the
SDK verified it rendered.
- Tests: dedupe now expects a single entry; empty-format case asserts the
field is omitted; add a test that the overlap guard short-circuits
before entity-name validation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AZZMCL2fjoR3FvvoM3vofZ
|
✅ Gitleaks Findings: No secrets detected. Safe to proceed! |
Code Review — Add optional
|
Third review pass (doc accuracy): - Reword the "unlisted entities → plaintext" behavior across the tool description, output schema, README, CLAUDE.md, CHANGELOG, and wrapping guide to attribute it to the Detect API's default rather than stating it as a verified invariant. The unit tests mock the SDK and can't exercise the real defaulting semantics, so the docs no longer over-claim. - Decouple the (verified) backward-compat claim — omitting `format` behaves exactly as before — from the API-governed default for unlisted entities. - Soften the partial-mask example: drop the specific "***-**-6789" shape in favor of noting the masked form depends on the vault's masking configuration. No behavior change — the handler still forwards the format to the SDK unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AZZMCL2fjoR3FvvoM3vofZ
|
Thanks — addressed the doc-accuracy points in
On actually verifying the default (the 🔴 ask): I can't close it from here, and want to be transparent about why rather than imply otherwise. I inspected the 🟡 badge-dot fallback and 🟢 dual error shapes: left as-is, per your note that both are acceptable/non-blocking. Generated by Claude Code |
|
✅ Gitleaks Findings: No secrets detected. Safe to proceed! |
|
Code Review — PR 28: Add optional Overall this is a clean, well-scoped, and genuinely well-hardened change. The multi-pass iteration visible in the commit history (overlap rejection, normalize-once refactor, doc-accuracy softening) resolved most of what I would otherwise have flagged. Backward compatibility is preserved correctly (no I could not run the suite locally (sandbox blocked Strengths
Minor findings / nits (none blocking)
Security / performance No concerns. Happy to see this merge once the (cosmetic) heading wording in item 1 is addressed — the rest are optional. |
Fourth review pass: - UI: rename the re-identify summary heading "Format Applied" -> "Requested Format" and the shared-type comment to match the "requested" framing used in the schema, type docs, and CHANGELOG (the echo reflects what the caller asked for, not what the SDK verified it rendered). - Handler: validate entity names before the try, alongside the overlap guard, so an invalid entity type returns a distinct client-side validation error (no HTTP `code`) instead of surfacing through the generic catch looking like a Skyflow API error. Overlap still checked first (ordering test preserved). - Test: assert the invalid-entity error carries no `code`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AZZMCL2fjoR3FvvoM3vofZ
|
Addressed in
#3 (lenient output schema) and #4 (live-vault sanity check of the REST/default behavior) left as-is — #3 is the intended input-strict/output-lenient split, and #4 is the same pre-merge live-vault check already flagged above. Thanks for the thorough passes. Generated by Claude Code |
|
✅ Gitleaks Findings: No secrets detected. Safe to proceed! |
PR Review: Add optional
|
Fifth review pass (clarity only, no behavior change): - Note in the `plaintext` schema description that it is redundant with the default treatment for unlisted entities and exists only for explicitness, so callers don't assume they must list entities to get restoration. - Comment renderFormatSummary's least-to-most-restrictive group ordering, which intentionally differs from the handler's FORMAT_BUCKETS order. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AZZMCL2fjoR3FvvoM3vofZ
|
✅ Gitleaks Findings: No secrets detected. Safe to proceed! |
|
Review: Add optional format support to the re-identify tool Thorough, well-executed change. The six-commit progression shows genuine iterative hardening (overlap rejection, single-normalization lockstep, doc-accuracy corrections, distinct client-side validation errors), and the result is clean and consistent across handler, schemas, types, UI, docs, and tests. It also follows the "Modifying Tools" checklist in CLAUDE.md end-to-end. A few notes below, mostly minor. Strengths
Minor observations (non-blocking)
Verification Relied on the PR-stated results (174 tests passing, pnpm build clean) plus a read of the final state — the code type-checks by inspection (FORMAT_BUCKETS typing, format[bucket] indexing, and the conditional format spread all line up with ReIdentifyFormat), and the test mock lowercase DetectEntities proxy correctly round-trips through ENTITY_MAP / getEntityEnum. I was not able to run the suite in this sandbox to independently confirm. Overall: ship it. The observations above are polish, not blockers. |
Sixth review pass (comment only): note that handleReIdentify re-validates entity names even though the server inputSchema already enum-constrains them, because the handler is a standalone reusable function — so a future reader doesn't remove the guard as "redundant." Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AZZMCL2fjoR3FvvoM3vofZ
|
✅ Gitleaks Findings: No secrets detected. Safe to proceed! |
Review: Add optional
|
main advanced after this PR was opened: PR #28 (optional output-format support for the re-identify text tool) merged. This merge brings the PR branch up to date so it stays cleanly mergeable. Conflicts resolved in CHANGELOG.md and README.md by unioning both features' documentation (file de-identify/re-identify tools + re-identify format control), keeping the re-identify format example in the re-identify section and the file-tool sections after it. Source files (src/lib/tools/types.ts, src/server.ts, ui/shared/types.ts) auto-merged with no conflicts. Verified: full build green (tsc + 4 UI apps) and the complete unit suite passes (270 tests) on the merged tree. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nv6ppGHwgQFFZSd32FBiZA
Summary
Adds an optional
formatinput to the re-identify tool so the calling agent can specify, per entity type, how tokens are rendered on the way out — fully restored (plaintext), partially masked (masked), or fully redacted (redacted). This mirrors the Skyflow Detect APIformatobject and maps to theskyflow-nodeSDK'sReidentifyTextOptions(setRedactedEntities/setMaskedEntities/setPlainTextEntities).Entity types not listed in any bucket default to full plaintext restoration, so existing calls that omit
formatbehave exactly as before.Example
{ "inputString": "email [EMAIL_ADDRESS_a1b2], ssn [SSN_c3d4], name [NAME_e5f6]", "format": { "masked": ["ssn"], "redacted": ["email_address"], "plaintext": ["name"] } }SSNs come back partially masked, emails fully redacted, names restored in full, and any other detected entity type defaults to plaintext.
Changes
src/lib/tools/reIdentify.ts) — new optionalformatparameter; buildsReidentifyTextOptions(only for non-empty buckets, falling back to no options otherwise) and echoes the appliedformatback in the output. Invalid entity types surface as a clean error via the existinggetEntityEnummapping.src/server.ts) —formatadded to the re-identify input and output Zod schemas (input uses the sharedENTITY_KEYSenum for validation) and threaded through to the handler.src/lib/tools/types.ts) — newReIdentifyFormatinterface;ReIdentifyOutputextended with an optionalformat.ui/re-identify/main.ts,ui/shared/types.ts,ui/shared/styles.css) — the re-identify app now renders a "Format Applied" summary grouping entity types by treatment when a format was provided.CLAUDE.md,README.md, and the wrapping guide (docs/wrapping-mcp-tools-with-skyflow.md, both SDK and REST sections) updated;CHANGELOG.mdentry added.Testing
tests/unit/tools/reIdentify.test.tscover entity routing to the correct setters, echo-back, single-bucket usage, empty-bucket/no-op handling, invalid entity errors, and backward compatibility (no options passed whenformatis omitted).pnpm build(UI + server) passes with no type errors.🤖 Generated with Claude Code
https://claude.ai/code/session_01AZZMCL2fjoR3FvvoM3vofZ
Generated by Claude Code