Repository navigation
Release 1.5.0: FRS-1 specification, Python/Go/Rust SDKs and the sidecar gateway - #12
Merged
Merged
Conversation
…ar gateway Add FRS-1 (spec/SPEC.md), a normative description of detector packs and the scan algorithm precise enough that independent engines produce identical output, together with spec/detectors.json, its JSON schema, and a 121-case conformance corpus that every engine's test suite runs. Add Python, Go and Rust SDKs implementing FRS-1 against that corpus, each with the engine, vaults, sessions and the logging and HTTP integrations native to its ecosystem. Add the sidecar gateway and its Docker image: a reverse proxy that redacts request bodies on the way out and restores the originals on the way back, so an application integrates by changing one base URL. It also serves /v1/redact, /v1/scan, /v1/detectors and server-side /v1/sessions for languages without a native engine, plus health and Prometheus endpoints. Add flare-redact/pack so a pack loads into the JavaScript engine as ordinary detectors, RedactOptions.detectors to replace the built-in set, and the portable `boundary` and `reject` detector fields. Verified: 341 Node tests, 87 Python, 18 Go, 19 Rust, go vet, clippy -D warnings, the spec drift check, all three example smokes, npm pack and a clean install of both the npm tarball and the Python wheel.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
FRS-1, a portable detector specification.
spec/SPEC.mddescribes how adetector pack is written and executed — the pattern subset, the scan algorithm,
overlap resolution, confidence, replacement modes, keyed transforms, vault
semantics and failure behaviour — precisely enough that independent engines
produce identical output. It ships with
spec/detectors.json, its JSON schema,and a 121-case conformance corpus that every engine's suite runs.
The pattern subset forbids every construct whose meaning differs across
JavaScript, Python, Go and Rust — lookaround, backreferences,
\b \d \w \s,\p{…}, inline flags and anchors — replacing them with explicitboundary,reject, named validators and the{{ANY}}/{{L}}tokens. It is RE2-safe byconstruction, so a pack cannot introduce catastrophic backtracking into a host.
Python, Go and Rust SDKs. Each implements FRS-1 against the shared corpus,
with the engine, vaults, sessions, and the logging and HTTP integrations native
to its ecosystem — a
loggingfilter and formatter for Python, anslog.Handlerandhttp.RoundTripperfor Go,serde_json::Valuetraversal forRust.
The sidecar gateway. A reverse proxy that redacts request bodies on the way
out and restores the originals on the way back, so an application integrates by
changing one base URL. Streamed responses are restored incrementally. It also
serves
/v1/redact,/v1/scan,/v1/detectorsand server-side/v1/sessionsfor languages without a native engine, plus health and Prometheus endpoints, and
ships a multi-stage non-root Docker image.
Library additions.
flare-redact/packloads an FRS-1 document as ordinarydetectors,
RedactOptions.detectorsreplaces the built-in set, andboundary/rejectgive custom detectors portable spellings for\band a trailingnegative lookahead.
Verification
Run locally on macOS arm64 against freshly installed Go 1.25.5 and Rust 1.97.1:
go vetcleanclippy --all-targets -D warningscleanspec:sync --check, conformance build, Bun/Deno runtime smokenpm ci,npm audit,npm pack, clean install of the npm tarball and the Python wheel/healthz,/v1/detectors,/v1/redactand/metricsThe real
docker buildis left to thegateway-imagejob in this PR's CI; itwas not run locally.
Notes for review
Three fixes in here are worth a second look:
/v1/prefix for its own API, which meant it404'd every vendor path it exists to proxy —
/v1/chat/completions,/v1/models,/v1/messages. Only the eight control paths shadow the upstreamnow; a regression test covers it.
could not tell a refusal from a network failure.
synthetic strings shaped like real provider credentials, which is the point.
Six detections were reviewed and allowed as test data.