Repository navigation
feat(f5): the masking prototype (Gate 0) - #213
Merged
Merged
Conversation
bb_identity 0.2 adds bbid_mask_hmac(domain, value): HMAC-SHA-256 under a key domain's key, a 32-byte file at /run/bb-mask/<domain>.key that must be the server's own and readable by nobody else; never in a table or a message; read again when the file is replaced; never granted to PUBLIC. An upgrade from 0.1 is included. make integration-f5 builds the image and, on throwaway containers and a ZFS pool of its own, proves the design's claims on PostgreSQL 18 and 16 (57/57): each rule's mask; what a masked reader can and cannot reach; the lookup; the application unaffected; key domains and rotation; and a masked branch on a ZFS clone left with no real value in its files. It found and fixed two things, now in the design (section 17): a refused lookup put its value in the server log, so masked logins get log_min_error_statement = panic; and a masked branch's VACUUM steps must each run alone. It measured the cost over the 10% budget: writes to a marked table 21-30% fewer, bulk updates 3.6x with a statement trigger, masked reads 25% fewer.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
F5 Gate 0: masking's prototype, proving the claims of the design agreed in #212. The findings are in
docs/FOX_F5_Masking_Design.md, section 17. There is no product code yet; Gate 1 follows.What it adds
bb_identity0.2:bbid_mask_hmac(domain, value). It is HMAC-SHA-256 under a key domain's key./run/bb-mask/<domain>.key.a-z 0-9 _ -.make integration-f5(scripts/integration_f5.sh). It builds the image under its own tag, and runs throwaway containers plus a ZFS pool of its own made from a file. The shared stack is never touched.docker/postgres/f5_test.go: the function is never public, the key is read from its one fixed place with its checks, and no message can carry key bytes.What it proves, on PostgreSQL 18 and 16 (57/57)
Each rule's mask:
Out of a masked reader's reach: the real table, the shadows, the hash, the fake function and the key file.
Works on masks: filtering, grouping and joining (a customer's orders by the hashed email), and
EXPLAIN.pg_statsis empty for the real table, andCOPYof a query copies masks.Writes: none to marked tables. Whatever a masked reader makes runs as them, so their own definer function reading the real table is refused.
The lookup: returns masked rows, allows 30 a minute, and records every call without the value.
The application: writes, key changes, deletes and TRUNCATE all work as before, and the shadows keep up.
The key:
A masked branch on a real ZFS clone, after:
VACUUM FULLof the table and ofpg_statistic;The branch has no real value left in any of its files, tables, statistics and WAL alike. A fresh cluster of only the masked rows also has none, in 0.3 s for 20,000 rows.
What it found and fixed (now in the design)
log_min_error_statement. Every masked login now getslog_min_error_statement = panic, which only a superuser can set or undo.VACUUM FULL pg_statisticsent with another statement runs in one transaction and is refused, silently leaving the old statistics in place. Each masked-branch step now runs as its own statement.Measured (PostgreSQL 18, 16 clients, medians of three)
For 1,000,000 rows:
Over the 10% budget. The design's decision 9 recommends accepting the cost, as for the audit log:
That decision is the founders'.
Verified
Those last three ran on the image with
bb_identity0.2.go test ./docker/postgresand the licence-header test pass.