From 1ea2f8bdab544c5024fc28078be36c30d2fb31e5 Mon Sep 17 00:00:00 2001 From: Lindsay Holmwood Date: Wed, 7 Oct 2026 09:06:34 +1100 Subject: [PATCH 1/4] docs(plans): walk through each plan-builder Go example in its directory Add a README.md to every example directory that contains code. Each one walks through its files concept by concept, with excerpts copied from the source and linked to their lines, and ends with the rows of the index's "What was checked" table that apply to it. The prose follows ASD-STE100 and ISO 24495-1. It says plainly that the SDK and stashgen do not exist and that the _stash.go files are written by hand. Claude-Session: https://claude.ai/code/session_016Usf7cQGY6ULWGyZ4NHQEB --- docs/plans/2026-10-04-plan-builder/README.md | 2 + .../accounts/README.md | 176 +++++++++ .../cmd/genencrypt/README.md | 68 ++++ .../contacts/README.md | 265 +++++++++++++ .../2026-10-04-plan-builder/crm/README.md | 84 ++++ .../documents/README.md | 149 +++++++ .../individuals/README.md | 230 +++++++++++ .../internal/pb/README.md | 126 ++++++ .../internal/userdb/README.md | 115 ++++++ .../2026-10-04-plan-builder/proto/README.md | 94 +++++ .../2026-10-04-plan-builder/rules/README.md | 123 ++++++ .../2026-10-04-plan-builder/sqlc/README.md | 114 ++++++ .../2026-10-04-plan-builder/users/README.md | 362 ++++++++++++++++++ 13 files changed, 1908 insertions(+) create mode 100644 docs/plans/2026-10-04-plan-builder/accounts/README.md create mode 100644 docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md create mode 100644 docs/plans/2026-10-04-plan-builder/contacts/README.md create mode 100644 docs/plans/2026-10-04-plan-builder/crm/README.md create mode 100644 docs/plans/2026-10-04-plan-builder/documents/README.md create mode 100644 docs/plans/2026-10-04-plan-builder/individuals/README.md create mode 100644 docs/plans/2026-10-04-plan-builder/internal/pb/README.md create mode 100644 docs/plans/2026-10-04-plan-builder/internal/userdb/README.md create mode 100644 docs/plans/2026-10-04-plan-builder/proto/README.md create mode 100644 docs/plans/2026-10-04-plan-builder/rules/README.md create mode 100644 docs/plans/2026-10-04-plan-builder/sqlc/README.md create mode 100644 docs/plans/2026-10-04-plan-builder/users/README.md diff --git a/docs/plans/2026-10-04-plan-builder/README.md b/docs/plans/2026-10-04-plan-builder/README.md index b67641fe4..f8ef3d259 100644 --- a/docs/plans/2026-10-04-plan-builder/README.md +++ b/docs/plans/2026-10-04-plan-builder/README.md @@ -6,6 +6,8 @@ The module path is `example.com/app`. ## What each file shows +Each directory has a `README.md` that walks through its code. + | File | What it shows | |---|---| | [`main.go`](main.go) | A client, one cipher for each tenant, and a batch of two types in one request | diff --git a/docs/plans/2026-10-04-plan-builder/accounts/README.md b/docs/plans/2026-10-04-plan-builder/accounts/README.md new file mode 100644 index 000000000..e7735a017 --- /dev/null +++ b/docs/plans/2026-10-04-plan-builder/accounts/README.md @@ -0,0 +1,176 @@ +# Preserve GORM fields and redact an account + +This directory shows an embedded GORM model, copied tags, and generated print methods. +It also distinguishes an unexported field from an omitted field. +The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The [Go example index](../README.md) lists every example and what was checked. + +`account_stash.go` is written by hand. +It shows what `stashgen` is planned to write. + +## What you will learn + +- Tag an embedded GORM model and distinguish two fields that are not stored. +- Inspect the encrypted type, copied tags, warnings, and declaration. +- Add redacted print methods to the tagged struct. +- Give the encrypted type to GORM. + +## Tag the account fields + +- **Purpose:** Understand how one tag covers an embedded struct from another package. +- **Related:** [The plan's embedded struct design](../../2026-10-04-plan-builder.md#embedded-structs-unexported-fields-and-other-tags), [the plan's printing design](../../2026-10-04-plan-builder.md#printing) +- **Needs:** Nothing + +The `stash:",passthrough"` tag stores every `gorm.Model` field as a passthrough field. +`Email` becomes one Encrypt Query Language (EQL) column with the `TextEq` type. +`stashgen` copies the `gorm` and `json` tags of `Email` to the same field of `EncryptedAccount`. + +[`account.go`, lines 12 to 21](account.go#L12-L21) + +```go +//go:generate go tool stashgen -type Account -redact + +type Account struct { + _ struct{} `stash:"context=accounts"` + + // One tag decides for every field of gorm.Model, which cannot carry tags. + gorm.Model `stash:",passthrough"` + + // stashgen copies the gorm and json tags onto EncryptedAccount. + Email string `stash:"email,encrypt_into=TextEq" gorm:"uniqueIndex" json:"email"` +``` + +The unexported `cache` field has no tag, so the generator ignores it with three notices: + +- `stashgen` prints a warning. +- The generated file names `cache` in a comment. +- The program prints a warning to stderr once for each type, on first use. + +The `token` field uses `stash:"-"`, so it is an omitted field and produces no warning. + +[`account.go`, lines 23 to 29](account.go#L23-L29) + +```go + // An unexported field with no stash tag is ignored, and stashgen and the + // running program both warn about it. + cache string + + // The tag acknowledges the skip, so nothing warns. + token string `stash:"-"` +} +``` + +## Inspect the encrypted account + +- **Purpose:** Follow copied tags and fields that are not stored into the generated file. +- **Related:** [The plan's generated file design](../../2026-10-04-plan-builder.md#what-stashgen-writes), [the plan's embedded struct design](../../2026-10-04-plan-builder.md#embedded-structs-unexported-fields-and-other-tags) +- **Needs:** Understand the account tags. + +The generated comment names `cache`, and `EncryptedAccount` embeds the same `gorm.Model`. +The encrypted `Email` field retains its GORM and JSON tags. + +[`account_stash.go`, lines 19 to 25](account_stash.go#L19-L25) + +```go +// Not encrypted and not stored: the unexported field "cache". Tag it +// `stash:"-"` to confirm that. + +type EncryptedAccount struct { + gorm.Model + Email eql.TextEq `gorm:"uniqueIndex" json:"email"` +} +``` + +The codec's `Unexported` list enables the planned runtime warning for `cache`. +The declaration omits `token` and does not include `cache`. + +[`account_stash.go`, lines 55 to 66](account_stash.go#L55-L66) + +```go +var declaration = gensupport.Declare("accounts"). + Passthrough("id"). + Passthrough("created_at"). + Passthrough("updated_at"). + Passthrough("deleted_at"). + EncryptInto("email", "TextEq"). + Omit("token") + +var codec = gensupport.New(gensupport.Generated[Account, EncryptedAccount]{ + TypeName: "Account", + Declaration: declaration, + Unexported: []string{"cache"}, +``` + +## Redact the tagged struct + +- **Purpose:** See what the `-redact` flag adds to `Account`. +- **Related:** [The plan's printing design](../../2026-10-04-plan-builder.md#printing) +- **Needs:** Understand that the tagged struct contains plaintext values. + +`-redact` asks `stashgen` to add `String` and `LogValue` methods to `Account`. +These methods show `gorm.Model` and hide `Email`. + +[`account_stash.go`, lines 35 to 42](account_stash.go#L35-L42) + +```go +// Written because of -redact: Account no longer prints its sealed fields. +func (a Account) String() string { + return gensupport.Redacted("Account", map[string]any{"Model": a.Model}, "Email") +} + +func (a Account) LogValue() slog.Value { + return gensupport.RedactedLog(map[string]any{"Model": a.Model}, "Email") +} +``` + +## Give encrypted accounts to GORM + +- **Purpose:** Follow a slice through encryption and into GORM. +- **Related:** [The plan's database design](../../2026-10-04-plan-builder.md#databases) +- **Needs:** Understand `EncryptedAccount`. + +`EncryptedAccount.TableName` maps the encrypted type to the `accounts` table. +`Create` encrypts the whole slice before GORM receives it. + +[`account.go`, lines 31 to 39](account.go#L31-L39) + +```go +func (EncryptedAccount) TableName() string { return "accounts" } + +func Create(ctx context.Context, db *gorm.DB, cipher *encrypt.Cipher, accounts []Account) error { + encrypted, err := Encrypt(ctx, cipher, accounts) + if err != nil { + return err + } + return db.WithContext(ctx).Create(&encrypted).Error +} +``` + +## What was checked + +- **Every Go file type-checks:** + Run. + `go vet ./...` passes against a stub of the SDK. + The stub is not in this repository, and it has signatures only. +- **A change to a tagged struct, a model or a type in another package stops the build:** + Run. + Each change fails `go build` with "cannot convert". +- **A generated file from another version does not compile:** + Run. + A file that names an unknown version constant fails `go build`. +- **The files `stashgen` writes:** + Not run. + `stashgen` does not exist, and the five `_stash.go` files are written by hand. +- **The SDK can be built with these signatures:** + Not run. +- **The guest returns an EQL value for a field with `encrypt_into`:** + Not run. + The target form and the dispatch do not exist. +- **`stashgen` checks a declaration with the embedded guest:** + Not run. +- **The code works with a database, GORM or pgx:** + Not run. + Nothing here has connected to a database. +- **The EQL types, and the value the guest returns for them:** + Not run. + `eql-codegen` does not write Go yet, and the examples use a stub of `eql.TextEq`. diff --git a/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md b/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md new file mode 100644 index 000000000..459b62f6a --- /dev/null +++ b/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md @@ -0,0 +1,68 @@ +# Generate encryption code from protobuf rules + +This directory contains the command that is planned to turn protobuf rules into a generated file. +It connects the protobuf source, `rules.Individuals`, and the requested output path. +The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The [Go example index](../../README.md) lists every example and what was checked. + +## What you will learn + +- Run `stashgen.Generate` with a protobuf source and policy. +- Review rule changes in the generated file. + +## Connect the source, rules, and output + +- **Purpose:** Follow the three inputs to the planned library generator. +- **Related:** [The plan's policy declarations](../../../2026-10-04-plan-builder.md#declarations-from-a-policy), [the rules walkthrough](../../rules/README.md), [the individuals walkthrough](../../individuals/README.md) +- **Needs:** Read the `rules.Individuals` policy first. + +`protosource.New()` supplies protobuf descriptors and field options. +`rules.Individuals` supplies the field decisions. +`stashgen.Output` selects the generated file in the `individuals` package. +The output goes in your own `individuals` package, not `internal/pb`, which holds the generated type. +This separation stops the generate program from importing a file that it writes. + +[`main.go`, lines 12 to 18](main.go#L12-L18) + +```go +func main() { + err := stashgen.Generate(protosource.New(), rules.Individuals, + stashgen.Output("../individuals/individual_stash.go")) + if err != nil { + log.Fatal(err) + } +} +``` + +The `stashgen`, `protosource`, and policy packages do not exist yet. + +## Review rule changes in the generated file + +- **Purpose:** See how reviewers inspect policy changes without running rules in the application. +- **Related:** [The generated individuals walkthrough](../../individuals/README.md), [the rules walkthrough](../../rules/README.md) +- **Needs:** Understand the generator inputs. + +The [`rules` package](../../rules/README.md) runs this command through `go generate`. +The application calls the resulting `individuals.Encrypt` and `individuals.Decrypt` functions. +A rule change changes `individual_stash.go`, which gives a reviewer a concrete generated diff. +The application never runs the rules. + +## What was checked + +- **Every Go file type-checks:** + Run. + `go vet ./...` passes against a stub of the SDK. + The stub is not in this repository, and it has signatures only. +- **The policy example compiles against real protobuf code:** + Run. + buf v1.50.0 and `protoc-gen-go` wrote `internal/pb/`, and `go vet` passes. +- **The protobuf source reads the field options, and the rules run:** + Not run. + Neither the source nor the generator exists. +- **The files `stashgen` writes:** + Not run. + `stashgen` does not exist, and the five `_stash.go` files are written by hand. +- **The SDK can be built with these signatures:** + Not run. +- **`stashgen` checks a declaration with the embedded guest:** + Not run. diff --git a/docs/plans/2026-10-04-plan-builder/contacts/README.md b/docs/plans/2026-10-04-plan-builder/contacts/README.md new file mode 100644 index 000000000..cf3e7bf19 --- /dev/null +++ b/docs/plans/2026-10-04-plan-builder/contacts/README.md @@ -0,0 +1,265 @@ +# Encrypt a contact type from another package into separate columns + +This directory declares encryption for `crm.Contact`, a type from another package. +It stores ciphertexts and search terms in separate columns, with a model for GORM. +An output is a ciphertext or one search term. +The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The [Go example index](../README.md) lists every example and what was checked. + +`contactstash_stash.go` is written by hand. +It shows what `stashgen` is planned to write. + +## What you will learn + +- Declare tags for a type in another package. +- Read the nested encrypted field types and available indexes. +- Map separate outputs into a model with a shape check. +- Insert contacts through `database/sql` or GORM. +- Search a stored search term by phone number. + +## Declare tags for a type in another package + +- **Purpose:** Connect a tag declaration in your own package to `crm.Contact`. +- **Related:** [The `crm.Contact` type in another package](../crm/README.md), [the plan's design for types in another package](../../2026-10-04-plan-builder.md#types-in-another-package) +- **Needs:** Understand that the program does not own `crm.Contact`. + +The `-for crm.Contact` flag makes `contactStash` declare tags for a type in another package. +The generator is planned to match fields by name and type. + +[`contacts.go`, lines 14 to 25](contacts.go#L14-L25) + +```go +//go:generate go tool stashgen -type contactStash -for crm.Contact -model Rows=ContactRow + +// contactStash declares the tags for crm.Contact, which cannot carry them. +// stashgen matches each field to the crm.Contact field with the same name and +// type, and refuses a crm.Contact field this struct does not name. +type contactStash struct { + _ struct{} `stash:"context=contacts"` + ID int64 `stash:"id,passthrough"` + Email string `stash:"email,encrypt,index=equality;match"` + PhoneNumber string `stash:"phone_number,encrypt,index=equality"` + Internal string `stash:"-"` +} +``` + +`Email` declares the `equality` and `match` indexes. +`PhoneNumber` declares only the `equality` index. + +The engine produces only the `TextEq` Encrypt Query Language (EQL) type today. +No EQL type is built from match search terms yet. +The plan therefore uses separate columns for `Email`, which needs a match search. +One EQL column for each field is the layout to use when a suitable EQL type exists. + +The generated file uses a shape check for `crm.Contact`. +A change to `crm.Contact` stops the contacts package from building. + +[`contactstash_stash.go`, lines 45 to 53](contactstash_stash.go#L45-L53) + +```go +// Stops compiling when crm.Contact gains, loses, reorders or retypes a field. +var _ = contactShape(crm.Contact{}) + +type contactShape struct { + ID int64 + Email string + PhoneNumber string + Internal string +} +``` + +## Read the separate encrypted outputs + +- **Purpose:** See how each encrypted field holds a ciphertext and its declared search terms. +- **Related:** [The plan's column layouts](../../2026-10-04-plan-builder.md#columns), [the plan's generated file design](../../2026-10-04-plan-builder.md#what-stashgen-writes) +- **Needs:** Understand the `contactStash` tags. + +`EncryptedContact` nests one encrypted type for each encrypted field. + +[`contactstash_stash.go`, lines 20 to 30](contactstash_stash.go#L20-L30) + +```go +type EncryptedContact struct { + ID int64 + Email EncryptedContactEmail + PhoneNumber EncryptedContactPhoneNumber +} + +type EncryptedContactEmail struct { + Ciphertext encrypt.Ciphertext + Equality encrypt.EqualityTerm + Match encrypt.MatchTerm +} +``` + +[`contactstash_stash.go`, lines 32 to 35](contactstash_stash.go#L32-L35) + +```go +type EncryptedContactPhoneNumber struct { + Ciphertext encrypt.Ciphertext + Equality encrypt.EqualityTerm +} +``` + +The declaration uses `EncryptIndex` to request those separate outputs. + +[`contactstash_stash.go`, lines 55 to 59](contactstash_stash.go#L55-L59) + +```go +var declaration = gensupport.Declare("contacts"). + Passthrough("id"). + EncryptIndex("email", encrypt.Equality, encrypt.Match()). + EncryptIndex("phone_number", encrypt.Equality). + Omit("internal") +``` + +The generated file can add print methods to `EncryptedContact`. +It cannot add methods to `crm.Contact` because Go requires methods to belong to the type's package. + +[`contactstash_stash.go`, lines 17 to 18](contactstash_stash.go#L17-L18) + +```go +// crm.Contact prints its sealed fields in the clear, and stashgen cannot add +// print methods to a type from another package. +``` + +## Map outputs into a model + +- **Purpose:** Flatten nested encrypted outputs into one field for each database column. +- **Related:** [The plan's model design](../../2026-10-04-plan-builder.md#models) +- **Needs:** Understand the separate encrypted outputs. + +`-model Rows=ContactRow` asks for `EncryptRows` and `DecryptRows`. +`ContactRow` is a model with one tagged field for each column. +`EncryptedContact` nests structs, but GORM maps one struct field to one column. +This separate-column layout therefore needs `ContactRow` for GORM. + +[`contacts.go`, lines 27 to 36](contacts.go#L27-L36) + +```go +// ContactRow is a model: one field for each column. Each tag names the output +// the field holds. +type ContactRow struct { + ID int64 `stash:"id"` + Email encrypt.Ciphertext `stash:"email"` + EmailEq encrypt.EqualityTerm `stash:"email,equality"` + EmailMatch encrypt.MatchTerm `stash:"email,match"` + PhoneNumber encrypt.Ciphertext `stash:"phone_number"` + PhoneNumberEq encrypt.EqualityTerm `stash:"phone_number,equality"` +} +``` + +`rowsShape` is the model's shape check. +The conversions compile only while the fields, types, and order match. + +[`contactstash_stash.go`, lines 155 to 164](contactstash_stash.go#L155-L164) + +```go +// The -model flag: ContactRow converts to and from its shape only while the +// two have the same fields, with the same types, in the same order. +type rowsShape struct { + ID int64 + Email encrypt.Ciphertext + EmailEq encrypt.EqualityTerm + EmailMatch encrypt.MatchTerm + PhoneNumber encrypt.Ciphertext + PhoneNumberEq encrypt.EqualityTerm +} +``` + +[`contactstash_stash.go`, lines 187 to 193](contactstash_stash.go#L187-L193) + +```go +func EncryptRows(ctx context.Context, cipher *encrypt.Cipher, contacts []crm.Contact) ([]ContactRow, error) { + return rowsCodec.Encrypt(ctx, cipher, contacts) +} + +func DecryptRows(ctx context.Context, d encrypt.Decrypter, rows []ContactRow) ([]crm.Contact, error) { + return rowsCodec.Decrypt(ctx, d, rows) +} +``` + +## Insert contacts through two database APIs + +- **Purpose:** Compare direct column writes with a model-based GORM write. +- **Related:** [The plan's database design](../../2026-10-04-plan-builder.md#databases) +- **Needs:** Understand both encrypted layouts. + +`Create` passes each nested output to `database/sql`. + +[`contacts.go`, lines 41 to 51](contacts.go#L41-L51) + +```go +func Create(ctx context.Context, db *sql.DB, cipher *encrypt.Cipher, list []crm.Contact) error { + encrypted, err := Encrypt(ctx, cipher, list) + if err != nil { + return err + } + for _, e := range encrypted { + _, err := db.ExecContext(ctx, ` + INSERT INTO contacts (id, email, email_eq, email_match, phone_number, phone_number_eq) + VALUES ($1, $2, $3, $4, $5, $6)`, + e.ID, e.Email.Ciphertext, e.Email.Equality, e.Email.Match, + e.PhoneNumber.Ciphertext, e.PhoneNumber.Equality) +``` + +`CreateWithGORM` asks `EncryptRows` for the flat model before GORM receives it. + +[`contacts.go`, lines 59 to 66](contacts.go#L59-L66) + +```go +// CreateWithGORM writes the model, which GORM maps one field to one column. +func CreateWithGORM(ctx context.Context, db *gorm.DB, cipher *encrypt.Cipher, list []crm.Contact) error { + rows, err := EncryptRows(ctx, cipher, list) + if err != nil { + return err + } + return db.WithContext(ctx).Create(&rows).Error +} +``` + +## Search by the phone search term + +- **Purpose:** Derive and use the equality search term for a separate column. +- **Related:** [The plan's call design](../../2026-10-04-plan-builder.md#calls), [the plan's terminology](../../2026-10-04-plan-builder.md#terminology) +- **Needs:** Understand the `phone_number_eq` column. + +`Fields.PhoneNumber.Equality` derives an `encrypt.EqualityTerm` search term. +The query compares that search term with the `phone_number_eq` column. + +[`contacts.go`, lines 68 to 76](contacts.go#L68-L76) + +```go +func IDByPhone(ctx context.Context, db *sql.DB, cipher *encrypt.Cipher, phone string) (int64, error) { + term, err := Fields.PhoneNumber.Equality(ctx, cipher, phone) + if err != nil { + return 0, err + } + var id int64 + err = db.QueryRowContext(ctx, `SELECT id FROM contacts WHERE phone_number_eq = $1`, term).Scan(&id) + return id, err +} +``` + +## What was checked + +- **Every Go file type-checks:** + Run. + `go vet ./...` passes against a stub of the SDK. + The stub is not in this repository, and it has signatures only. +- **A change to a tagged struct, a model or a type in another package stops the build:** + Run. + Each change fails `go build` with "cannot convert". +- **A generated file from another version does not compile:** + Run. + A file that names an unknown version constant fails `go build`. +- **The files `stashgen` writes:** + Not run. + `stashgen` does not exist, and the five `_stash.go` files are written by hand. +- **The SDK can be built with these signatures:** + Not run. +- **`stashgen` checks a declaration with the embedded guest:** + Not run. +- **The code works with a database, GORM or pgx:** + Not run. + Nothing here has connected to a database. diff --git a/docs/plans/2026-10-04-plan-builder/crm/README.md b/docs/plans/2026-10-04-plan-builder/crm/README.md new file mode 100644 index 000000000..1edada18f --- /dev/null +++ b/docs/plans/2026-10-04-plan-builder/crm/README.md @@ -0,0 +1,84 @@ +# Stand in for a type in another package + +This directory stands in for a package that the program does not own. +Its `crm.Contact` type has no `stash` tags because another package controls its source. +`crm` imports no Go software development kit (SDK) code. + +The SDK is planned and does not exist yet. +The [`contacts` package](../contacts/README.md) uses SDK code, so that package does not build in this repository. +The [Go example index](../README.md) lists every example and what was checked. + +## What you will learn + +- Declare encryption tags for a type in another package from your own package. +- Check changes to `crm.Contact` at compile time. + +## Declare tags in your own package + +- **Purpose:** Understand why `crm.Contact` stays free of encryption tags. +- **Related:** [The plan's design for types in another package](../../2026-10-04-plan-builder.md#types-in-another-package), [the contacts walkthrough](../contacts/README.md) +- **Needs:** Nothing + +`crm.Contact` contains only the fields supplied by the stand-in package. + +[`contact.go`, lines 1 to 9](contact.go#L1-L9) + +```go +// Package crm stands in for a package this program does not own. +package crm + +type Contact struct { + ID int64 + Email string + PhoneNumber string + Internal string +} +``` + +The [`contacts` package](../contacts/README.md) uses `contactStash` to declare tags for `crm.Contact`. +`contactStash` leaves out `Internal` with `stash:"-"`. + +[`contacts.go`, lines 19 to 25](../contacts/contacts.go#L19-L25) + +```go +type contactStash struct { + _ struct{} `stash:"context=contacts"` + ID int64 `stash:"id,passthrough"` + Email string `stash:"email,encrypt,index=equality;match"` + PhoneNumber string `stash:"phone_number,encrypt,index=equality"` + Internal string `stash:"-"` +} +``` + +## Check the type's shape at compile time + +- **Purpose:** See how a change to `crm.Contact` stops the contacts package from building. +- **Related:** [The contacts walkthrough](../contacts/README.md), [the plan's design for types in another package](../../2026-10-04-plan-builder.md#types-in-another-package) +- **Needs:** Understand the fields of `crm.Contact`. + +The generated file converts `crm.Contact` to `contactShape` at compile time. +The conversion fails when `crm.Contact` gains, loses, reorders, or retypes a field. + +[`contactstash_stash.go`, lines 45 to 53](../contacts/contactstash_stash.go#L45-L53) + +```go +// Stops compiling when crm.Contact gains, loses, reorders or retypes a field. +var _ = contactShape(crm.Contact{}) + +type contactShape struct { + ID int64 + Email string + PhoneNumber string + Internal string +} +``` + +## What was checked + +- **Every Go file type-checks:** + Run. + `go vet ./...` passes against a stub of the SDK. + The stub is not in this repository, and it has signatures only. +- **A change to a tagged struct, a model or a type in another package stops the build:** + Run. + Each change fails `go build` with "cannot convert". diff --git a/docs/plans/2026-10-04-plan-builder/documents/README.md b/docs/plans/2026-10-04-plan-builder/documents/README.md new file mode 100644 index 000000000..5f327e054 --- /dev/null +++ b/docs/plans/2026-10-04-plan-builder/documents/README.md @@ -0,0 +1,149 @@ +# Encrypt a document as one opaque value + +This directory encrypts each `Document` as one ciphertext column. +Its fields have no tags because the opaque layout does not expose or index them separately. +The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The [Go example index](../README.md) lists every example and what was checked. + +`document_stash.go` is written by hand. +It shows what `stashgen` is planned to write. + +## What you will learn + +- Declare an opaque tagged struct and its encryption context. +- Read the opaque declaration and single ciphertext field. +- Save with a cipher and load with the client. + +## Declare one opaque value + +- **Purpose:** Understand why the document fields have no individual tags. +- **Related:** [The plan's struct tags](../../2026-10-04-plan-builder.md#struct-tags), [the plan's column layouts](../../2026-10-04-plan-builder.md#columns) +- **Needs:** Nothing + +The `opaque` option is part of the encryption context tag. +It tells the planned generator to encrypt the complete `Document` as one value. +`Title`, `Body`, and `Tags` therefore have no `stash` tags. + +[`documents.go`, lines 12 to 19](documents.go#L12-L19) + +```go +//go:generate go tool stashgen -type Document + +type Document struct { + _ struct{} `stash:"context=documents/v2/body,opaque"` + Title string + Body string + Tags []string +} +``` + +Nothing inside the document can be read or searched independently through this layout. + +## Read the single ciphertext layout + +- **Purpose:** Follow the opaque tag into the encrypted type and declaration. +- **Related:** [The plan's generated file design](../../2026-10-04-plan-builder.md#what-stashgen-writes) +- **Needs:** Understand the opaque tagged struct. + +`EncryptedDocument` has one `Sealed` ciphertext field. + +[`document_stash.go`, lines 19 to 29](document_stash.go#L19-L29) + +```go +type EncryptedDocument struct { + Sealed encrypt.Ciphertext +} + +func (e EncryptedDocument) String() string { + return gensupport.Redacted("EncryptedDocument", nil, "Sealed") +} + +func (e EncryptedDocument) LogValue() slog.Value { + return gensupport.RedactedLog(nil, "Sealed") +} +``` + +`DeclareOpaque` builds the declaration for the complete value. +The codec maps all three document fields into `gensupport.OpaqueField`. + +[`document_stash.go`, lines 41 to 49](document_stash.go#L41-L49) + +```go +var declaration = gensupport.DeclareOpaque("documents/v2/body") + +var codec = gensupport.New(gensupport.Generated[Document, EncryptedDocument]{ + TypeName: "Document", + Declaration: declaration, + PrintsPlaintext: true, + Source: func(v Document) gensupport.Values { + return gensupport.Values{gensupport.OpaqueField: map[string]any{"title": v.Title, "body": v.Body, "tags": v.Tags}} + }, +``` + +## Save with a cipher and load with the client + +- **Purpose:** See why `Save` uses a cipher and `Load` uses the client. +- **Related:** [The plan's call design](../../2026-10-04-plan-builder.md#calls), [the root example program](../main.go) +- **Needs:** Understand `EncryptedDocument.Sealed`. + +`Save` encrypts one document by passing a one-element slice. +It stores `encrypted[0].Sealed` in the `body` column. + +[`documents.go`, lines 21 to 28](documents.go#L21-L28) + +```go +func Save(ctx context.Context, db *sql.DB, cipher *encrypt.Cipher, id int64, doc Document) error { + encrypted, err := Encrypt(ctx, cipher, []Document{doc}) + if err != nil { + return err + } + _, err = db.ExecContext(ctx, `INSERT INTO documents (id, body) VALUES ($1, $2)`, id, encrypted[0].Sealed) + return err +} +``` + +`Load` decrypts with the client, not the cipher. +A `*Client` decrypts each value under the keyset that encrypted it. +A `*Cipher` refuses a value from another keyset with `ErrForeignKeyset`. + +[`documents.go`, lines 30 to 43](documents.go#L30-L43) + +```go +// Load decrypts with the client, which opens each value under the keyset that +// sealed it. A *encrypt.Cipher would also refuse a value from another +// keyset. +func Load(ctx context.Context, db *sql.DB, client *encrypt.Client, id int64) (Document, error) { + var encrypted EncryptedDocument + if err := db.QueryRowContext(ctx, `SELECT body FROM documents WHERE id = $1`, id).Scan(&encrypted.Sealed); err != nil { + return Document{}, err + } + docs, err := Decrypt(ctx, client, []EncryptedDocument{encrypted}) + if err != nil { + return Document{}, err + } + return docs[0], nil +} +``` + +## What was checked + +- **Every Go file type-checks:** + Run. + `go vet ./...` passes against a stub of the SDK. + The stub is not in this repository, and it has signatures only. +- **A change to a tagged struct, a model or a type in another package stops the build:** + Run. + Each change fails `go build` with "cannot convert". +- **A generated file from another version does not compile:** + Run. + A file that names an unknown version constant fails `go build`. +- **The files `stashgen` writes:** + Not run. + `stashgen` does not exist, and the five `_stash.go` files are written by hand. +- **The SDK can be built with these signatures:** + Not run. +- **`stashgen` checks a declaration with the embedded guest:** + Not run. +- **The code works with a database, GORM or pgx:** + Not run. + Nothing here has connected to a database. diff --git a/docs/plans/2026-10-04-plan-builder/individuals/README.md b/docs/plans/2026-10-04-plan-builder/individuals/README.md new file mode 100644 index 000000000..545cd9e4a --- /dev/null +++ b/docs/plans/2026-10-04-plan-builder/individuals/README.md @@ -0,0 +1,230 @@ +# Store a protobuf message with mixed encrypted layouts + +This directory shows the generated file planned from the `Individual` policy rules. +One message combines passthrough fields, separate columns, and one Encrypt Query Language (EQL) column. +The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The [Go example index](../README.md) lists every example and what was checked. + +`individual_stash.go` is written by hand. +It shows what `stashgen.Generate` is planned to write. + +## What you will learn + +- Read the mixed storage layouts chosen by the policy rules. +- Follow pointer messages through field-by-name reads instead of a shape check. +- Use the generated helpers and let the compiler reject an undeclared query. +- Insert encrypted fields and search the Medicare EQL column. + +## Read the mixed storage layouts + +- **Purpose:** Connect policy decisions to the encrypted type and declaration. +- **Related:** [The rules walkthrough](../rules/README.md), [the plan's policy declarations](../../2026-10-04-plan-builder.md#declarations-from-a-policy) +- **Needs:** Understand `rules.Individuals`. + +`Id` and `Nickname` are passthrough fields. +`Name` has one ciphertext, while `Email` has a ciphertext and two search terms. +`MedicareNo` uses one `TextEq` EQL column named `medicare_number` in the declaration. + +[`individual_stash.go`, lines 25 to 31](individual_stash.go#L25-L31) + +```go +type EncryptedIndividual struct { + Id int64 + Name EncryptedIndividualName + Email EncryptedIndividualEmail + MedicareNo eql.TextEq + Nickname string +} +``` + +[`individual_stash.go`, lines 33 to 41](individual_stash.go#L33-L41) + +```go +type EncryptedIndividualName struct { + Ciphertext encrypt.Ciphertext +} + +type EncryptedIndividualEmail struct { + Ciphertext encrypt.Ciphertext + Equality encrypt.EqualityTerm + Match encrypt.MatchTerm +} +``` + +[`individual_stash.go`, lines 51 to 56](individual_stash.go#L51-L56) + +```go +var declaration = gensupport.Declare("individuals"). + Passthrough("id"). + Encrypt("name"). + EncryptIndex("email", encrypt.Equality, encrypt.Match()). + EncryptInto("medicare_number", "TextEq"). + Passthrough("nickname") +``` + +## Read pointer messages by field name + +- **Purpose:** Understand why protobuf messages do not use the usual shape check. +- **Related:** [The generated protobuf walkthrough](../internal/pb/README.md), [the plan's design for types in another package](../../2026-10-04-plan-builder.md#types-in-another-package) +- **Needs:** Know that `pb.Individual` contains unexported protobuf fields. + +The codec takes `*pb.Individual` values. +It reads each field through a generated getter because direct struct conversion cannot include unexported fields. + +[`individual_stash.go`, lines 58 to 69](individual_stash.go#L58-L69) + +```go +var codec = gensupport.New(gensupport.Generated[*pb.Individual, EncryptedIndividual]{ + TypeName: "pb.Individual", + Declaration: declaration, + PrintsPlaintext: true, + Source: func(v *pb.Individual) gensupport.Values { + return gensupport.Values{ + "id": v.GetId(), + "name": v.GetName(), + "email": v.GetEmail(), + "medicare_number": v.GetMedicareNo(), + "nickname": v.GetNickname(), + } +``` + +The compiler finds a removed or retyped field through these named reads. +Adding a field to `Individual` does not stop the build. +Continuous integration (CI) runs `go generate ./...` and fails when a generated file differs from the committed file. + +> **Take care** +> +> The [`String` method of `pb.Individual`](../internal/pb/individual.pb.go#L44-L46) prints every field in plaintext. +> `stashgen` cannot replace that method, because `pb.Individual` is a type in another package. +> Do not print or log `pb.Individual` values. + +## Use the generated helpers + +- **Purpose:** Follow protobuf pointers and the Medicare field through the generated API. +- **Related:** [The plan's call design](../../2026-10-04-plan-builder.md#calls), [the generation command walkthrough](../cmd/genencrypt/README.md) +- **Needs:** Understand the codec's pointer type. + +`Encrypt` accepts a slice of `*pb.Individual` pointers. +`Decrypt` returns the same pointer form. + +[`individual_stash.go`, lines 117 to 123](individual_stash.go#L117-L123) + +```go +func Encrypt(ctx context.Context, cipher *encrypt.Cipher, individuals []*pb.Individual) ([]EncryptedIndividual, error) { + return codec.Encrypt(ctx, cipher, individuals) +} + +func Decrypt(ctx context.Context, d encrypt.Decrypter, encrypted []EncryptedIndividual) ([]*pb.Individual, error) { + return codec.Decrypt(ctx, d, encrypted) +} +``` + +`Name` declares encryption without an index, so `NameField` has only an `Encrypt` method. + +[`individual_stash.go`, lines 135 to 142](individual_stash.go#L135-L142) + +```go +type NameField struct { + field gensupport.Field[string] +} + +func (f NameField) Encrypt(ctx context.Context, c *encrypt.Cipher, v string) (EncryptedIndividualName, error) { + out, err := f.field.Encrypt(ctx, c, v) + return EncryptedIndividualName{Ciphertext: out.Ciphertext}, err +} +``` + +`Fields.Name.Equality` does not compile because `NameField` does not declare that method. + +`Fields.MedicareNo.Query` returns an `eql.TextEqQuery` query value. + +[`individual_stash.go`, lines 168 to 176](individual_stash.go#L168-L176) + +```go +func (f MedicareNoField) Encrypt(ctx context.Context, c *encrypt.Cipher, v string) (eql.TextEq, error) { + out, err := f.field.Encrypt(ctx, c, v) + return eql.TextEq(out.EQL), err +} + +func (f MedicareNoField) Query(ctx context.Context, c *encrypt.Cipher, v string) (eql.TextEqQuery, error) { + out, err := f.field.Query(ctx, c, v) + return eql.TextEqQuery(out.EQL), err +} +``` + +## Insert fields and search Medicare numbers + +- **Purpose:** Map each storage layout to its database columns and query. +- **Related:** [The plan's database design](../../2026-10-04-plan-builder.md#databases) +- **Needs:** Understand the mixed encrypted type. + +`Create` passes the passthrough fields, ciphertexts, search terms, and EQL value to their columns. + +[`store.go`, lines 14 to 23](store.go#L14-L23) + +```go +func Create(ctx context.Context, db *sql.DB, cipher *encrypt.Cipher, people []*pb.Individual) error { + encrypted, err := Encrypt(ctx, cipher, people) + if err != nil { + return err + } + for _, e := range encrypted { + _, err := db.ExecContext(ctx, ` + INSERT INTO individuals (id, nickname, name, email, email_eq, email_match, medicare_number) + VALUES ($1, $2, $3, $4, $5, $6, $7)`, + e.Id, e.Nickname, e.Name.Ciphertext, e.Email.Ciphertext, e.Email.Equality, e.Email.Match, e.MedicareNo) +``` + +`IDByMedicare` creates a query value and casts it to `eql_v3.query_text_eq` in SQL. + +[`store.go`, lines 31 to 40](store.go#L31-L40) + +```go +func IDByMedicare(ctx context.Context, db *sql.DB, cipher *encrypt.Cipher, medicareNo string) (int64, error) { + query, err := Fields.MedicareNo.Query(ctx, cipher, medicareNo) + if err != nil { + return 0, err + } + var id int64 + err = db.QueryRowContext(ctx, + `SELECT id FROM individuals WHERE medicare_number = $1::eql_v3.query_text_eq`, query).Scan(&id) + return id, err +} +``` + +## What was checked + +- **Every Go file type-checks:** + Run. + `go vet ./...` passes against a stub of the SDK. + The stub is not in this repository, and it has signatures only. +- **The policy example compiles against real protobuf code:** + Run. + buf v1.50.0 and `protoc-gen-go` wrote `internal/pb/`, and `go vet` passes. +- **A field added to the protobuf message does not stop the build:** + Run. + It builds, as the plan says: CI finds that change. +- **A struct from another package with an unexported field cannot convert:** + Run, with `sync.Once`. +- **A generated file from another version does not compile:** + Run. + A file that names an unknown version constant fails `go build`. +- **The protobuf source reads the field options, and the rules run:** + Not run. + Neither the source nor the generator exists. +- **The files `stashgen` writes:** + Not run. + `stashgen` does not exist, and the five `_stash.go` files are written by hand. +- **The SDK can be built with these signatures:** + Not run. +- **The guest returns an EQL value for a field with `encrypt_into`:** + Not run. + The target form and the dispatch do not exist. +- **`stashgen` checks a declaration with the embedded guest:** + Not run. +- **The code works with a database, GORM or pgx:** + Not run. + Nothing here has connected to a database. +- **The EQL types, and the value the guest returns for them:** + Not run. + `eql-codegen` does not write Go yet, and the examples use a stub of `eql.TextEq`. diff --git a/docs/plans/2026-10-04-plan-builder/internal/pb/README.md b/docs/plans/2026-10-04-plan-builder/internal/pb/README.md new file mode 100644 index 000000000..b5bd0556d --- /dev/null +++ b/docs/plans/2026-10-04-plan-builder/internal/pb/README.md @@ -0,0 +1,126 @@ +# Use generated protobuf messages and field options + +This directory contains real `protoc-gen-go` output for the `Individual` message and its field option. +The rules use `pb.Individual`, while the individuals code uses its getters and exported fields. +The planned `protosource` package reads the descriptors and `E_DataCategories`. +The Go software development kit (SDK) is planned and does not exist yet, so those examples do not build in this repository. +The [Go example index](../../README.md) lists every example and what was checked. + +## What you will learn + +- Generate the protobuf package again without editing generated code. +- Use `pb.Individual` and its getters from generated encryption code. +- Read `E_DataCategories` through the planned protobuf source. + +## Generate the protobuf package again + +- **Purpose:** Identify the generators, versions, and the command that generates the package again. +- **Related:** [The protobuf source walkthrough](../../proto/README.md) +- **Needs:** Install buf and `protoc-gen-go`. + +buf v1.50.0 ran `protoc-gen-go` v1.26.0 to write these files. +The generated header reports the `protoc` version as unknown. +Run `buf generate` in [`proto/`](../../proto/) to generate the files again. +A person must not edit them. + +Adding a field to `Individual` does not stop the build. +Continuous integration (CI) runs `go generate ./...` and fails when a generated file differs from the committed file. + +[`individual.pb.go`, lines 1 to 7](individual.pb.go#L1-L7) + +```go +// Code generated by protoc-gen-go. DO NOT EDIT. +// versions: +// protoc-gen-go v1.26.0 +// protoc (unknown) +// source: individual.proto + +package pb +``` + +## Use the message and getters + +- **Purpose:** Identify the generated values that the individuals example reads. +- **Related:** [The generated individuals walkthrough](../../individuals/README.md), [the plan's design for types in another package](../../../2026-10-04-plan-builder.md#types-in-another-package) +- **Needs:** Understand the `Individual` message fields. + +`pb.Individual` includes three unexported protobuf fields before its exported data fields. + +[`individual.pb.go`, lines 23 to 33](individual.pb.go#L23-L33) + +```go +type Individual struct { + state protoimpl.MessageState + sizeCache protoimpl.SizeCache + unknownFields protoimpl.UnknownFields + + Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"` + Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"` + Email string `protobuf:"bytes,3,opt,name=email,proto3" json:"email,omitempty"` + MedicareNo string `protobuf:"bytes,4,opt,name=medicare_no,json=medicareNo,proto3" json:"medicare_no,omitempty"` + Nickname string `protobuf:"bytes,5,opt,name=nickname,proto3" json:"nickname,omitempty"` +} +``` + +Those unexported fields prevent a shape check through direct struct conversion. +`individuals/individual_stash.go` reads each field through `GetId`, `GetName`, `GetEmail`, `GetMedicareNo`, or `GetNickname`. + +[`individual.pb.go`, lines 88 to 100](individual.pb.go#L88-L100) + +```go +func (x *Individual) GetMedicareNo() string { + if x != nil { + return x.MedicareNo + } + return "" +} + +func (x *Individual) GetNickname() string { + if x != nil { + return x.Nickname + } + return "" +} +``` + +The examples do not use protobuf internals such as `file_individual_proto_rawDesc`. + +## Read the field option identifier + +- **Purpose:** Connect the protobuf option to the planned policy source. +- **Related:** [The rules walkthrough](../../rules/README.md), [the plan's policy declarations](../../../2026-10-04-plan-builder.md#declarations-from-a-policy) +- **Needs:** Understand the `data_categories` option. + +`classification.pb.go` exports `E_DataCategories` for the repeated field option. + +[`classification.pb.go`, lines 34 to 40](classification.pb.go#L34-L40) + +```go +// Extension fields to descriptorpb.FieldOptions. +var ( + // The Fideslang data categories of a field. + // + // repeated string data_categories = 50001; + E_DataCategories = &file_classification_proto_extTypes[0] +) +``` + +The planned `protosource` package reads this option through protobuf descriptors. + +## What was checked + +- **Every Go file type-checks:** + Run. + `go vet ./...` passes against a stub of the SDK. + The stub is not in this repository, and it has signatures only. +- **The policy example compiles against real protobuf code:** + Run. + buf v1.50.0 and `protoc-gen-go` wrote `internal/pb/`, and `go vet` passes. +- **A field added to the protobuf message does not stop the build:** + Run. + It builds, as the plan says: CI finds that change. +- **A struct from another package with an unexported field cannot convert:** + Run, with `sync.Once`. +- **The protobuf source reads the field options, and the rules run:** + Not run. + Neither the source nor the generator exists. diff --git a/docs/plans/2026-10-04-plan-builder/internal/userdb/README.md b/docs/plans/2026-10-04-plan-builder/internal/userdb/README.md new file mode 100644 index 000000000..e5bc522d3 --- /dev/null +++ b/docs/plans/2026-10-04-plan-builder/internal/userdb/README.md @@ -0,0 +1,115 @@ +# Use the generated sqlc package with the encrypted type + +This directory contains sqlc's generated database package for the `users` example. +Its structs match `users.EncryptedUser`, so the store uses Go conversions instead of field copies. +The generated fields use planned Encrypt Query Language (EQL) types. +The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The [Go example index](../../README.md) lists every example and what was checked. + +## What you will learn + +- Generate the sqlc package again without editing generated code. +- Convert sqlc's row struct and parameters to the encrypted type. +- Use the generated methods from the users store. + +## Generate the sqlc package again + +- **Purpose:** Identify the generator, its version, and the command that generates the package again. +- **Related:** [The sqlc inputs](../../sqlc/README.md) +- **Needs:** Nothing + +sqlc v1.31.1 generated `db.go`, `models.go`, and `query.sql.go`. +Run `sqlc generate` in [`sqlc/`](../../sqlc/) to generate them again. +A person must not edit these files. + +[`db.go`, lines 1 to 5](db.go#L1-L5) + +```go +// Code generated by sqlc. DO NOT EDIT. +// versions: +// sqlc v1.31.1 + +package userdb +``` + +## Convert generated structs by shape + +- **Purpose:** See why direct Go conversions connect sqlc to the encrypted type. +- **Related:** [The plan's model design](../../../2026-10-04-plan-builder.md#models), [the users sqlc store](../../users/README.md) +- **Needs:** Know that an encrypted type has one field for each stored field. + +sqlc's row struct `userdb.User` has the same fields as `users.EncryptedUser`. + +[`models.go`, lines 11 to 15](models.go#L11-L15) + +```go +type User struct { + ID int64 + Email eql.TextEq + Name eql.TextEq +} +``` + +`CreateUserParams` has the same shape too. + +[`query.sql.go`, lines 18 to 22](query.sql.go#L18-L22) + +```go +type CreateUserParams struct { + ID int64 + Email eql.TextEq + Name eql.TextEq +} +``` + +Go ignores struct tags during struct conversion in Go 1.8 and later. +The `db` and `gorm` tags on `users.EncryptedUser` therefore do not stop either conversion. +The users store converts with `EncryptedUser(row)` and `userdb.CreateUserParams(e)`. +A field change stops either conversion from compiling until both shapes agree. + +## Use the generated methods + +- **Purpose:** Connect generated identifiers to the handwritten users store. +- **Related:** [The users sqlc store](../../users/README.md), [the sqlc queries](../../sqlc/query.sql) +- **Needs:** Understand the matching struct shapes. + +`userdb.New` creates `Queries`, and `WithTx` binds the methods to a transaction. +The users store calls `CreateUser`, `GetUser`, and `FindUsersByEmail`. + +[`db.go`, lines 19 to 31](db.go#L19-L31) + +```go +func New(db DBTX) *Queries { + return &Queries{db: db} +} + +type Queries struct { + db DBTX +} + +func (q *Queries) WithTx(tx *sql.Tx) *Queries { + return &Queries{ + db: tx, + } +} +``` + +The examples do not use the generated `ListUsers` method. + +## What was checked + +- **Every Go file type-checks:** + Run. + `go vet ./...` passes against a stub of the SDK. + The stub is not in this repository, and it has signatures only. +- **sqlc's row struct converts to the generated type:** + Run. + The conversions in `users/sqlcstore.go` compile against real sqlc output. +- **The SDK can be built with these signatures:** + Not run. +- **The code works with a database, GORM or pgx:** + Not run. + Nothing here has connected to a database. +- **The EQL types, and the value the guest returns for them:** + Not run. + `eql-codegen` does not write Go yet, and the examples use a stub of `eql.TextEq`. diff --git a/docs/plans/2026-10-04-plan-builder/proto/README.md b/docs/plans/2026-10-04-plan-builder/proto/README.md new file mode 100644 index 000000000..b94c92366 --- /dev/null +++ b/docs/plans/2026-10-04-plan-builder/proto/README.md @@ -0,0 +1,94 @@ +# Annotate protobuf fields for encryption rules + +This directory defines a protobuf field option and applies it to an `Individual` message. +The option carries Fideslang data categories that the planned rules read during generation. +`buf generate` works for these protobuf files. + +The Go software development kit (SDK) is planned and does not exist yet. +The rules that read the field options belong to the planned SDK. +The [Go example index](../README.md) lists every example and what was checked. + +## What you will learn + +- Define the `data_categories` field option. +- Annotate message fields while leaving one field uncategorised. +- Generate the Go protobuf package in `internal/pb`. + +## Define the data category option + +- **Purpose:** See how protobuf fields carry facts for encryption rules. +- **Related:** [The plan's policy declarations](../../2026-10-04-plan-builder.md#declarations-from-a-policy), [the rules walkthrough](../rules/README.md) +- **Needs:** Nothing + +`classification.proto` extends `google.protobuf.FieldOptions` with `data_categories`. +Its values use Fideslang data categories. +Fideslang is a published taxonomy of personal data categories, such as `user.contact.email`. + +[`classification.proto`, lines 5 to 12](classification.proto#L5-L12) + +```proto +import "google/protobuf/descriptor.proto"; + +option go_package = "example.com/app/internal/pb"; + +extend google.protobuf.FieldOptions { + // The Fideslang data categories of a field. + repeated string data_categories = 50001; +} +``` + +## Annotate the message fields + +- **Purpose:** Follow each protobuf field into the policy rules. +- **Related:** [The rules walkthrough](../rules/README.md), [the generated individuals walkthrough](../individuals/README.md) +- **Needs:** Understand the `data_categories` option. + +`name`, `email`, and `medicare_no` each carry a category. +`id` and `nickname` have no category, so message-specific rules must decide them. + +[`individual.proto`, lines 9 to 15](individual.proto#L9-L15) + +```proto +message Individual { + int64 id = 1; + string name = 2 [(classification.data_categories) = "user.name"]; + string email = 3 [(classification.data_categories) = "user.contact.email"]; + string medicare_no = 4 [(classification.data_categories) = "user.government_id"]; + string nickname = 5; +} +``` + +## Generate the Go protobuf package + +- **Purpose:** Understand where buf writes the generated Go files. +- **Related:** [The generated protobuf walkthrough](../internal/pb/README.md) +- **Needs:** Install buf and `protoc-gen-go`. + +`buf.gen.yaml` writes source-relative Go files into `../internal/pb`. + +[`buf.gen.yaml`, lines 1 to 5](buf.gen.yaml#L1-L5) + +```yaml +version: v2 +plugins: + - local: protoc-gen-go + out: ../internal/pb + opt: paths=source_relative +``` + +Run `buf generate` in this directory to generate the package again. +The committed files came from buf v1.50.0 and `protoc-gen-go` v1.26.0. +Adding a field to `Individual` does not stop the build. +Continuous integration (CI) runs `go generate ./...` and fails when a generated file differs from the committed file. + +## What was checked + +- **The policy example compiles against real protobuf code:** + Run. + buf v1.50.0 and `protoc-gen-go` wrote `internal/pb/`, and `go vet` passes. +- **A field added to the protobuf message does not stop the build:** + Run. + It builds, as the plan says: CI finds that change. +- **The protobuf source reads the field options, and the rules run:** + Not run. + Neither the source nor the generator exists. diff --git a/docs/plans/2026-10-04-plan-builder/rules/README.md b/docs/plans/2026-10-04-plan-builder/rules/README.md new file mode 100644 index 000000000..74b113fd7 --- /dev/null +++ b/docs/plans/2026-10-04-plan-builder/rules/README.md @@ -0,0 +1,123 @@ +# Decide protobuf encryption from data categories + +This directory defines planned policy rules for `pb.Individual` fields. +The first matching rule chooses each field's storage layout. +The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The [Go example index](../README.md) lists every example and what was checked. + +## What you will learn + +- Match Fideslang data categories with shared rules. +- Add message-specific decisions and a column name. +- Run the generation command without using rules at application runtime. + +## Match data categories with shared rules + +- **Purpose:** Understand how `Base` decides fields from protobuf annotations. +- **Related:** [The plan's policy declarations](../../2026-10-04-plan-builder.md#declarations-from-a-policy), [the protobuf source walkthrough](../proto/README.md) +- **Needs:** Understand the `data_categories` field option. + +`policy.Key` names the field option by its full name, `classification.data_categories`. +`policy.FirstOf` evaluates each `policy.When` in order, and the first match decides. + +[`rules.go`, lines 13 to 21](rules.go#L13-L21) + +```go +// The key is the full name of the field option in proto/classification.proto. +var category = policy.Key("classification.data_categories") + +// Base applies to every message. The first rule that matches a field decides it. +var Base = policy.FirstOf( + policy.When(category.Under("user.government_id"), policy.EncryptInto("TextEq")), + policy.When(category.Under("user.contact.email"), policy.EncryptIndex(encrypt.Equality, encrypt.Match())), + policy.When(category.Under("user"), policy.Encrypt()), +) +``` + +Each rule in `Base` makes one decision: + +- `user.government_id` uses `EncryptInto("TextEq")` and stores one Encrypt Query Language (EQL) column. +- `user.contact.email` uses `EncryptIndex` and stores a ciphertext with equality and match search terms in separate columns. +- `user` uses `Encrypt()` and stores a ciphertext without an index. + +`user.contact.email` is under `user`, so its rule must come before the broader `user` rule. +If the `user` rule came first, email would use `Encrypt()` without an index. +The policy API also defines `Index`, `Omit`, and `Fail` decisions. + +## Add message-specific decisions + +- **Purpose:** See how `Individuals` handles fields before it falls back to `Base`. +- **Related:** [The generated individuals walkthrough](../individuals/README.md), [the protobuf source walkthrough](../proto/README.md) +- **Needs:** Understand the shared rules. + +`policy.ForMessage` binds the rules to `pb.Individual` and the `individuals` encryption context. +`policy.Field` handles uncategorised fields and overrides a categorised field. +`policy.Name("medicare_number")` changes the database column name for `medicare_no`. + +[`rules.go`, lines 23 to 31](rules.go#L23-L31) + +```go +// Individuals adds the rules for one message. A field with no data category +// needs a rule too: with none, the generator stops. +var Individuals = policy.ForMessage(&pb.Individual{}, policy.Context("individuals"), + policy.FirstOf( + policy.When(policy.Field("medicare_no"), policy.EncryptInto("TextEq"), policy.Name("medicare_number")), + policy.When(policy.Field("id"), policy.Passthrough()), + policy.When(policy.Field("nickname"), policy.Passthrough()), + ).OrElse(Base), +) +``` + +`OrElse(Base)` applies the shared rules when no message-specific rule matches. +A field with no matching rule stops the planned generator. +Adding a field to `Individual` does not stop the build. +Continuous integration (CI) runs `go generate ./...` and fails when a generated file differs from the committed file. + +## Run rules during generation + +- **Purpose:** Keep policy evaluation out of the application runtime. +- **Related:** [The generation command walkthrough](../cmd/genencrypt/README.md), [the plan's policy declarations](../../2026-10-04-plan-builder.md#declarations-from-a-policy) +- **Needs:** Understand the complete `Individuals` policy. + +The `go:generate` line runs `cmd/genencrypt`. + +[`rules.go`, lines 1 to 11](rules.go#L1-L11) + +```go +// Package rules decides what to encrypt from the data categories that the +// protobuf schema gives each field. Only the generate program runs it. +package rules + +import ( + "example.com/app/internal/pb" + "github.com/cipherstash/stack/languages/golang/encrypt" + "github.com/cipherstash/stack/languages/golang/encrypt/policy" +) + +//go:generate go run ../cmd/genencrypt +``` + +The application uses the generated functions and never runs these rules. + +## What was checked + +- **Every Go file type-checks:** + Run. + `go vet ./...` passes against a stub of the SDK. + The stub is not in this repository, and it has signatures only. +- **The policy example compiles against real protobuf code:** + Run. + buf v1.50.0 and `protoc-gen-go` wrote `internal/pb/`, and `go vet` passes. +- **A field added to the protobuf message does not stop the build:** + Run. + It builds, as the plan says: CI finds that change. +- **The protobuf source reads the field options, and the rules run:** + Not run. + Neither the source nor the generator exists. +- **The files `stashgen` writes:** + Not run. + `stashgen` does not exist, and the five `_stash.go` files are written by hand. +- **The SDK can be built with these signatures:** + Not run. +- **`stashgen` checks a declaration with the embedded guest:** + Not run. diff --git a/docs/plans/2026-10-04-plan-builder/sqlc/README.md b/docs/plans/2026-10-04-plan-builder/sqlc/README.md new file mode 100644 index 000000000..36af82160 --- /dev/null +++ b/docs/plans/2026-10-04-plan-builder/sqlc/README.md @@ -0,0 +1,114 @@ +# Generate sqlc code for Encrypt Query Language columns + +This directory configures sqlc for a `users` table with two Encrypt Query Language (EQL) columns. +It also defines the queries that generate the `userdb` package. +`sqlc generate` ran for these SQL and YAML files. + +The Go software development kit (SDK) is planned and does not exist yet. +The generated package names the planned `encrypt/eql` package, so it does not build in this repository. +The [Go example index](../README.md) lists every example and what was checked. + +## What you will learn + +- Define the EQL domains and table columns that sqlc reads. +- Map each EQL domain spelling to its planned Go type. +- Generate typed methods from queries that use EQL columns. + +## Define the EQL domains and table columns + +- **Purpose:** Understand the schema files that sqlc reads. +- **Related:** [The plan's database design](../../2026-10-04-plan-builder.md#databases), [the generated `userdb` package](../internal/userdb/README.md) +- **Needs:** Nothing + +`eql-domains.sql` gives sqlc small domain declarations because sqlc cannot parse the EQL installation bundle. +The domain spelling must match `schema.sql` and the matching override. + +[`eql-domains.sql`, lines 1 to 6](eql-domains.sql#L1-L6) + +```sql +-- For sqlc only: sqlc cannot parse the EQL install bundle. Spell each domain +-- exactly as schema.sql and query.sql do, or its db_type override never matches. +CREATE SCHEMA eql_v3; + +CREATE DOMAIN public.eql_v3_text_eq AS jsonb; +CREATE DOMAIN eql_v3.query_text_eq AS jsonb; +``` + +`schema.sql` uses one `public.eql_v3_text_eq` EQL column for each encrypted field. + +[`schema.sql`, lines 1 to 5](schema.sql#L1-L5) + +```sql +CREATE TABLE users ( + id bigint PRIMARY KEY, + email public.eql_v3_text_eq NOT NULL, + name public.eql_v3_text_eq NOT NULL +); +``` + +Read the index's [three rules for sqlc with EQL columns](../README.md#use-sqlc-with-eql-columns) before you change these spellings. + +## Map each domain to its Go type + +- **Purpose:** Follow the sqlc inputs and type overrides into the generated package. +- **Related:** [The plan's model design](../../2026-10-04-plan-builder.md#models), [the generated `userdb` package](../internal/userdb/README.md) +- **Needs:** Read the EQL domain and table files first. + +`sqlc.yaml` reads the domain file before the table schema. +It writes package `userdb` into `../internal/userdb`. + +[`sqlc.yaml`, lines 3 to 16](sqlc.yaml#L3-L16) + +```yaml + - engine: postgresql + schema: + - eql-domains.sql + - schema.sql + queries: query.sql + gen: + go: + package: userdb + out: ../internal/userdb + overrides: + - db_type: "public.eql_v3_text_eq" + go_type: { import: github.com/cipherstash/stack/languages/golang/encrypt/eql, type: TextEq } + - db_type: "eql_v3.query_text_eq" + go_type: { import: github.com/cipherstash/stack/languages/golang/encrypt/eql, type: TextEqQuery } +``` + +Run `sqlc generate` in this directory to generate the package again with sqlc v1.31.1. + +## Generate typed query methods + +- **Purpose:** See how the SQL produces typed create, read, list, and search methods. +- **Related:** [The users sqlc store](../users/README.md), [the generated query methods](../internal/userdb/README.md) +- **Needs:** Understand the domain overrides. + +`query.sql` defines four methods through sqlc's query annotations. +The email search casts its parameter directly to `eql_v3.query_text_eq`. + +[`query.sql`, lines 7 to 13](query.sql#L7-L13) + +```sql +-- name: ListUsers :many +SELECT * FROM users ORDER BY id; + +-- One cast, straight to the query domain: sqlc types a parameter by its first +-- cast, so ::jsonb::eql_v3.query_text_eq would generate json.RawMessage. +-- name: FindUsersByEmail :many +SELECT * FROM users WHERE email = sqlc.arg(email)::eql_v3.query_text_eq; +``` + +The direct cast lets the override generate an `eql.TextEqQuery` parameter. + +## What was checked + +- **sqlc's row struct converts to the generated type:** + Run. + The conversions in `users/sqlcstore.go` compile against real sqlc output. +- **The code works with a database, GORM or pgx:** + Not run. + Nothing here has connected to a database. +- **The EQL types, and the value the guest returns for them:** + Not run. + `eql-codegen` does not write Go yet, and the examples use a stub of `eql.TextEq`. diff --git a/docs/plans/2026-10-04-plan-builder/users/README.md b/docs/plans/2026-10-04-plan-builder/users/README.md new file mode 100644 index 000000000..87dc02522 --- /dev/null +++ b/docs/plans/2026-10-04-plan-builder/users/README.md @@ -0,0 +1,362 @@ +# Encrypt and store users with generated helpers + +This directory shows the planned path from a tagged struct to three database stores. +Each encrypted field uses one Encrypt Query Language (EQL) column. +The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The [Go example index](../README.md) lists every example and what was checked. + +`user_stash.go` is written by hand. +It shows what `stashgen` is planned to write. + +## What you will learn + +- Declare stored fields and create a tenant cipher. +- Read what the generated file holds. +- Insert and search users with `database/sql`. +- Batch encryption before GORM receives values. +- Convert between sqlc structs and update one EQL column. + +## Declare fields and create a tenant cipher + +- **Purpose:** Connect the tagged struct to its generator command and runtime cipher. +- **Related:** [The plan's struct tags](../../2026-10-04-plan-builder.md#struct-tags), [the root example program](../main.go) +- **Needs:** Nothing + +The `go:generate` line asks `stashgen` to read `User`. +The encryption context is `users`, and every exported field has a `stash` tag. +`ID` is a passthrough field, while `Internal` is an omitted field. +`Email` and `Name` each become one `TextEq` EQL column. + +[`model.go`, lines 3 to 13](model.go#L3-L13) + +```go +//go:generate go tool stashgen -type User + +// Every exported field needs a stash tag, so a new field cannot reach the +// database unencrypted by accident. +type User struct { + _ struct{} `stash:"context=users"` + ID int64 `stash:"id,passthrough" db:"id" gorm:"primaryKey"` + Email string `stash:"email,encrypt_into=TextEq" db:"email"` + Name string `stash:"name,encrypt_into=TextEq" db:"name"` + Internal string `stash:"-"` +} +``` + +The root program creates the client and derives one cipher for tenant `tenant-42`. +The cipher holds the tenant's keyset and its extension to each encryption context. +`Extend` adds the tenant segment to each field's declared encryption context. +No call takes a keyset or an encryption context, so the write, query, and read cannot use different ones. + +[`../main.go`, lines 25 to 30](../main.go#L25-L30) + +```go +func run(ctx context.Context) error { + client, err := encrypt.NewClient(ctx, encrypt.WithCredentials(encrypt.AutoCredentials())) + if err != nil { + return err + } + defer client.Close() +``` + +[`../main.go`, lines 38 to 41](../main.go#L38-L41) + +```go + // One cipher for each tenant: its keyset, and its part of every field's + // context. Every call through this cipher carries both. + cipher := client.Keyset(encrypt.KeysetName("tenant-42")).Extend("tenant-42") + store := users.NewSQLStore(db) +``` + +## Read what the generated file holds + +- **Purpose:** Understand the main parts that `stashgen` is planned to write. +- **Related:** [The plan's generated file design](../../2026-10-04-plan-builder.md#what-stashgen-writes), [the plan's call design](../../2026-10-04-plan-builder.md#calls) +- **Needs:** Understand the tagged struct. + +`EncryptedUser` keeps `ID` and replaces `Email` and `Name` with `eql.TextEq`. +Its `String` and `LogValue` methods hide the encrypted fields when code prints the encrypted type. + +[`user_stash.go`, lines 20 to 32](user_stash.go#L20-L32) + +```go +type EncryptedUser struct { + ID int64 `db:"id" gorm:"primaryKey"` + Email eql.TextEq `db:"email"` + Name eql.TextEq `db:"name"` +} + +func (e EncryptedUser) String() string { + return gensupport.Redacted("EncryptedUser", map[string]any{"ID": e.ID}, "Email", "Name") +} + +func (e EncryptedUser) LogValue() slog.Value { + return gensupport.RedactedLog(map[string]any{"ID": e.ID}, "Email", "Name") +} +``` + +The `userShape(User{})` conversion is the shape check. +It stops compilation when the tagged struct and generated file have different fields. +The declaration crosses to the engine with the struct. +It names each field's encryption context, indexes, and EQL type. + +[`user_stash.go`, lines 35 to 49](user_stash.go#L35-L49) + +```go +var _ = userShape(User{}) + +type userShape struct { + _ struct{} + ID int64 + Email string + Name string + Internal string +} + +var declaration = gensupport.Declare("users"). + Passthrough("id"). + EncryptInto("email", "TextEq"). + EncryptInto("name", "TextEq"). + Omit("internal") +``` + +The codec maps the tagged struct to named values before the engine encrypts them. + +[`user_stash.go`, lines 51 to 57](user_stash.go#L51-L57) + +```go +var codec = gensupport.New(gensupport.Generated[User, EncryptedUser]{ + TypeName: "User", + Declaration: declaration, + PrintsPlaintext: true, + Source: func(v User) gensupport.Values { + return gensupport.Values{"id": v.ID, "email": v.Email, "name": v.Name} + }, +``` + +`Encrypt` and `Decrypt` process slices through that codec. +ZeroKMS is the CipherStash key service. +`Encrypt` and `Decrypt` send one ZeroKMS request for the whole slice. +`Fields.Email.Query` creates the query value for a `WHERE` clause. + +[`user_stash.go`, lines 88 to 97](user_stash.go#L88-L97) + +```go +// Encrypt seals every user in one ZeroKMS request. The result has one element +// for each user, in the same order. +func Encrypt(ctx context.Context, cipher *encrypt.Cipher, users []User) ([]EncryptedUser, error) { + return codec.Encrypt(ctx, cipher, users) +} + +// Decrypt opens every value in one ZeroKMS request. +func Decrypt(ctx context.Context, d encrypt.Decrypter, encrypted []EncryptedUser) ([]User, error) { + return codec.Decrypt(ctx, d, encrypted) +} +``` + +[`user_stash.go`, lines 99 to 105](user_stash.go#L99-L105) + +```go +var Fields = struct { + Email EmailField + Name NameField +}{ + Email: EmailField{gensupport.NewField[string](declaration, "email")}, + Name: NameField{gensupport.NewField[string](declaration, "name")}, +} +``` + +> **Take care** +> +> `User` has no `String` or `LogValue` method, so a program that prints or logs it shows the plaintext of `Email` and `Name`. +> [`main.go`, line 76](../main.go#L76) prints only IDs and counts for that reason. +> Write both methods, or run `stashgen` with `-redact`. +> The [accounts walkthrough](../accounts/README.md) shows `-redact`. + +## Insert and search with database/sql + +- **Purpose:** Follow encrypted values through a transaction and an EQL equality query. +- **Related:** [The plan's database design](../../2026-10-04-plan-builder.md#databases), [the sqlc schema walkthrough](../sqlc/README.md) +- **Needs:** Understand `EncryptedUser` and `Fields.Email.Query`. + +`SQLStore.Import` encrypts the whole input slice before it starts inserting rows. +It passes each `EncryptedUser` field directly to `ExecContext`. + +[`sqlstore.go`, lines 27 to 39](sqlstore.go#L27-L39) + +```go +// Import encrypts every user in one ZeroKMS request, then inserts them in one +// transaction. +func (s *SQLStore) Import(ctx context.Context, cipher *encrypt.Cipher, people []User) error { + encrypted, err := Encrypt(ctx, cipher, people) + if err != nil { + return fmt.Errorf("encrypt %d users: %w", len(people), err) + } + + tx, err := s.db.BeginTx(ctx, nil) + if err != nil { + return err + } + defer tx.Rollback() +``` + +[`sqlstore.go`, lines 47 to 52](sqlstore.go#L47-L52) + +```go + for _, e := range encrypted { + if _, err := stmt.ExecContext(ctx, e.ID, e.Email, e.Name); err != nil { + return fmt.Errorf("insert user %d: %w", e.ID, err) + } + } + return tx.Commit() +``` + +`FindByEmail` creates an `eql.TextEqQuery` query value through `Fields.Email.Query`. +The SQL casts `$1` directly to `eql_v3.query_text_eq`. + +[`sqlstore.go`, lines 59 to 70](sqlstore.go#L59-L70) + +```go +// FindByEmail matches the whole address. Postgres compares the encrypted +// column through EQL's = operator. +func (s *SQLStore) FindByEmail(ctx context.Context, cipher *encrypt.Cipher, email string) ([]User, error) { + query, err := Fields.Email.Query(ctx, cipher, email) + if err != nil { + return nil, err + } + rs, err := s.db.QueryContext(ctx, selectUser+` WHERE email = $1::eql_v3.query_text_eq`, query) + if err != nil { + return nil, err + } + defer rs.Close() +``` + +## Batch before GORM receives values + +- **Purpose:** See why encryption happens before GORM maps the encrypted type. +- **Related:** [The plan's database design](../../2026-10-04-plan-builder.md#databases), [the plan's call design](../../2026-10-04-plan-builder.md#calls) +- **Needs:** Understand the generated `Encrypt` function. + +`EncryptedUser.TableName` sends GORM to `users` instead of `encrypted_users`. +A `driver.Valuer` hook cannot batch because it receives one field and no `context.Context`. +It also cannot observe cancellation through `ctx`. + +[`gormstore.go`, lines 10 to 19](gormstore.go#L10-L19) + +```go +// TableName lets GORM use the generated type as the model. Without it GORM +// would look for a table named encrypted_users. +func (EncryptedUser) TableName() string { return "users" } + +// GormStore encrypts before GORM sees a value. driver.Valuer gets no +// context.Context and runs one field at a time, so it cannot batch a ZeroKMS +// request or stop when the request is cancelled. +type GormStore struct { + db *gorm.DB +} +``` + +`GormStore.Create` encrypts the slice first, then gives GORM the encrypted values. + +[`gormstore.go`, lines 25 to 31](gormstore.go#L25-L31) + +```go +func (s *GormStore) Create(ctx context.Context, cipher *encrypt.Cipher, people ...User) error { + encrypted, err := Encrypt(ctx, cipher, people) + if err != nil { + return err + } + return s.db.WithContext(ctx).CreateInBatches(encrypted, 500).Error +} +``` + +## Convert sqlc structs and update one column + +- **Purpose:** Connect generated sqlc shapes and field-level updates to the encrypted type. +- **Related:** [The generated `userdb` walkthrough](../internal/userdb/README.md), [the sqlc inputs](../sqlc/README.md) +- **Needs:** Understand `EncryptedUser` and `Fields.Email.Encrypt`. + +`userdb.User` and `userdb.CreateUserParams` have the same fields as `EncryptedUser`. +Go therefore converts these structs directly, and a change to either shape stops the build. +The store converts `EncryptedUser` to `userdb.CreateUserParams` before each insert. + +[`sqlcstore.go`, lines 39 to 45](sqlcstore.go#L39-L45) + +```go + queries := s.queries.WithTx(tx) + for _, e := range encrypted { + if err := queries.CreateUser(ctx, userdb.CreateUserParams(e)); err != nil { + return err + } + } + return tx.Commit() +``` + +It converts each sqlc row back with `EncryptedUser(row)` before decryption. + +[`sqlcstore.go`, lines 68 to 76](sqlcstore.go#L68-L76) + +```go + rows, err := s.queries.FindUsersByEmail(ctx, query) + if err != nil { + return nil, err + } + encrypted := make([]EncryptedUser, len(rows)) + for i, row := range rows { + encrypted[i] = EncryptedUser(row) + } + return Decrypt(ctx, cipher, encrypted) +``` + +`ChangeEmail` encrypts one field and updates the single EQL column. +That column holds the ciphertext and its search term together. + +[`sqlcstore.go`, lines 79 to 88](sqlcstore.go#L79-L88) + +```go +// ChangeEmail rewrites one field. One EQL column holds the ciphertext and +// its term, so they cannot go out of step. +func (s *SQLCStore) ChangeEmail(ctx context.Context, cipher *encrypt.Cipher, id int64, email string) error { + sealed, err := Fields.Email.Encrypt(ctx, cipher, email) + if err != nil { + return err + } + _, err = s.db.ExecContext(ctx, `UPDATE users SET email = $2 WHERE id = $1`, id, sealed) + return err +} +``` + +## What was checked + +- **Every Go file type-checks:** + Run. + `go vet ./...` passes against a stub of the SDK. + The stub is not in this repository, and it has signatures only. +- **A change to a tagged struct, a model or a type in another package stops the build:** + Run. + Each change fails `go build` with "cannot convert". +- **sqlc's row struct converts to the generated type:** + Run. + The conversions in `users/sqlcstore.go` compile against real sqlc output. +- **A generated file from another version does not compile:** + Run. + A file that names an unknown version constant fails `go build`. +- **The files `stashgen` writes:** + Not run. + `stashgen` does not exist, and the five `_stash.go` files are written by hand. +- **The SDK can be built with these signatures:** + Not run. +- **The guest returns an EQL value for a field with `encrypt_into`:** + Not run. + The target form and the dispatch do not exist. +- **`stashgen` checks a declaration with the embedded guest:** + Not run. +- **The code works with a database, GORM or pgx:** + Not run. + Nothing here has connected to a database. +- **The EQL types, and the value the guest returns for them:** + Not run. + `eql-codegen` does not write Go yet, and the examples use a stub of `eql.TextEq`. +- **The steps in "Use the SDK":** + Not run. + Nobody has followed them. From 779828f8477ee8af96c24f5d0e954450b7225acd Mon Sep 17 00:00:00 2001 From: Lindsay Holmwood Date: Wed, 7 Oct 2026 09:06:40 +1100 Subject: [PATCH 2/4] docs(plans): name the Golang SDK, say "example", link sqlc and Fideslang Open each walkthrough with "This example" rather than "This directory", call the SDK "the Golang SDK for CipherStash Stack", and write CI without expanding it, because every reader knows both terms. Link sqlc, Fideslang and the Fideslang data categories where a walkthrough first names them. Claude-Session: https://claude.ai/code/session_016Usf7cQGY6ULWGyZ4NHQEB --- docs/plans/2026-10-04-plan-builder/accounts/README.md | 4 ++-- .../2026-10-04-plan-builder/cmd/genencrypt/README.md | 4 ++-- docs/plans/2026-10-04-plan-builder/contacts/README.md | 4 ++-- docs/plans/2026-10-04-plan-builder/crm/README.md | 4 ++-- .../plans/2026-10-04-plan-builder/documents/README.md | 4 ++-- .../2026-10-04-plan-builder/individuals/README.md | 6 +++--- .../2026-10-04-plan-builder/internal/pb/README.md | 6 +++--- .../2026-10-04-plan-builder/internal/userdb/README.md | 4 ++-- docs/plans/2026-10-04-plan-builder/proto/README.md | 11 +++++------ docs/plans/2026-10-04-plan-builder/rules/README.md | 7 ++++--- docs/plans/2026-10-04-plan-builder/sqlc/README.md | 4 ++-- docs/plans/2026-10-04-plan-builder/users/README.md | 4 ++-- 12 files changed, 31 insertions(+), 31 deletions(-) diff --git a/docs/plans/2026-10-04-plan-builder/accounts/README.md b/docs/plans/2026-10-04-plan-builder/accounts/README.md index e7735a017..71195093b 100644 --- a/docs/plans/2026-10-04-plan-builder/accounts/README.md +++ b/docs/plans/2026-10-04-plan-builder/accounts/README.md @@ -1,8 +1,8 @@ # Preserve GORM fields and redact an account -This directory shows an embedded GORM model, copied tags, and generated print methods. +This example shows an embedded GORM model, copied tags, and generated print methods. It also distinguishes an unexported field from an omitted field. -The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The Golang SDK for CipherStash Stack is planned and does not exist yet, so this code does not build in this repository. The [Go example index](../README.md) lists every example and what was checked. `account_stash.go` is written by hand. diff --git a/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md b/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md index 459b62f6a..c27cd5a11 100644 --- a/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md +++ b/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md @@ -1,8 +1,8 @@ # Generate encryption code from protobuf rules -This directory contains the command that is planned to turn protobuf rules into a generated file. +This example contains the command that is planned to turn protobuf rules into a generated file. It connects the protobuf source, `rules.Individuals`, and the requested output path. -The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The Golang SDK for CipherStash Stack is planned and does not exist yet, so this code does not build in this repository. The [Go example index](../../README.md) lists every example and what was checked. ## What you will learn diff --git a/docs/plans/2026-10-04-plan-builder/contacts/README.md b/docs/plans/2026-10-04-plan-builder/contacts/README.md index cf3e7bf19..e9cbb0b0e 100644 --- a/docs/plans/2026-10-04-plan-builder/contacts/README.md +++ b/docs/plans/2026-10-04-plan-builder/contacts/README.md @@ -1,9 +1,9 @@ # Encrypt a contact type from another package into separate columns -This directory declares encryption for `crm.Contact`, a type from another package. +This example declares encryption for `crm.Contact`, a type from another package. It stores ciphertexts and search terms in separate columns, with a model for GORM. An output is a ciphertext or one search term. -The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The Golang SDK for CipherStash Stack is planned and does not exist yet, so this code does not build in this repository. The [Go example index](../README.md) lists every example and what was checked. `contactstash_stash.go` is written by hand. diff --git a/docs/plans/2026-10-04-plan-builder/crm/README.md b/docs/plans/2026-10-04-plan-builder/crm/README.md index 1edada18f..7d10e5604 100644 --- a/docs/plans/2026-10-04-plan-builder/crm/README.md +++ b/docs/plans/2026-10-04-plan-builder/crm/README.md @@ -1,8 +1,8 @@ # Stand in for a type in another package -This directory stands in for a package that the program does not own. +This example stands in for a package that the program does not own. Its `crm.Contact` type has no `stash` tags because another package controls its source. -`crm` imports no Go software development kit (SDK) code. +`crm` imports no code from the Golang SDK for CipherStash Stack. The SDK is planned and does not exist yet. The [`contacts` package](../contacts/README.md) uses SDK code, so that package does not build in this repository. diff --git a/docs/plans/2026-10-04-plan-builder/documents/README.md b/docs/plans/2026-10-04-plan-builder/documents/README.md index 5f327e054..aeef48217 100644 --- a/docs/plans/2026-10-04-plan-builder/documents/README.md +++ b/docs/plans/2026-10-04-plan-builder/documents/README.md @@ -1,8 +1,8 @@ # Encrypt a document as one opaque value -This directory encrypts each `Document` as one ciphertext column. +This example encrypts each `Document` as one ciphertext column. Its fields have no tags because the opaque layout does not expose or index them separately. -The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The Golang SDK for CipherStash Stack is planned and does not exist yet, so this code does not build in this repository. The [Go example index](../README.md) lists every example and what was checked. `document_stash.go` is written by hand. diff --git a/docs/plans/2026-10-04-plan-builder/individuals/README.md b/docs/plans/2026-10-04-plan-builder/individuals/README.md index 545cd9e4a..af3ffa1e9 100644 --- a/docs/plans/2026-10-04-plan-builder/individuals/README.md +++ b/docs/plans/2026-10-04-plan-builder/individuals/README.md @@ -1,8 +1,8 @@ # Store a protobuf message with mixed encrypted layouts -This directory shows the generated file planned from the `Individual` policy rules. +This example shows the generated file planned from the `Individual` policy rules. One message combines passthrough fields, separate columns, and one Encrypt Query Language (EQL) column. -The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The Golang SDK for CipherStash Stack is planned and does not exist yet, so this code does not build in this repository. The [Go example index](../README.md) lists every example and what was checked. `individual_stash.go` is written by hand. @@ -90,7 +90,7 @@ var codec = gensupport.New(gensupport.Generated[*pb.Individual, EncryptedIndivid The compiler finds a removed or retyped field through these named reads. Adding a field to `Individual` does not stop the build. -Continuous integration (CI) runs `go generate ./...` and fails when a generated file differs from the committed file. +CI runs `go generate ./...` and fails when a generated file differs from the committed file. > **Take care** > diff --git a/docs/plans/2026-10-04-plan-builder/internal/pb/README.md b/docs/plans/2026-10-04-plan-builder/internal/pb/README.md index b5bd0556d..cd6672a52 100644 --- a/docs/plans/2026-10-04-plan-builder/internal/pb/README.md +++ b/docs/plans/2026-10-04-plan-builder/internal/pb/README.md @@ -1,9 +1,9 @@ # Use generated protobuf messages and field options -This directory contains real `protoc-gen-go` output for the `Individual` message and its field option. +This example contains real `protoc-gen-go` output for the `Individual` message and its field option. The rules use `pb.Individual`, while the individuals code uses its getters and exported fields. The planned `protosource` package reads the descriptors and `E_DataCategories`. -The Go software development kit (SDK) is planned and does not exist yet, so those examples do not build in this repository. +The Golang SDK for CipherStash Stack is planned and does not exist yet, so those examples do not build in this repository. The [Go example index](../../README.md) lists every example and what was checked. ## What you will learn @@ -24,7 +24,7 @@ Run `buf generate` in [`proto/`](../../proto/) to generate the files again. A person must not edit them. Adding a field to `Individual` does not stop the build. -Continuous integration (CI) runs `go generate ./...` and fails when a generated file differs from the committed file. +CI runs `go generate ./...` and fails when a generated file differs from the committed file. [`individual.pb.go`, lines 1 to 7](individual.pb.go#L1-L7) diff --git a/docs/plans/2026-10-04-plan-builder/internal/userdb/README.md b/docs/plans/2026-10-04-plan-builder/internal/userdb/README.md index e5bc522d3..a06ecebd8 100644 --- a/docs/plans/2026-10-04-plan-builder/internal/userdb/README.md +++ b/docs/plans/2026-10-04-plan-builder/internal/userdb/README.md @@ -1,9 +1,9 @@ # Use the generated sqlc package with the encrypted type -This directory contains sqlc's generated database package for the `users` example. +This example is the database package that [sqlc](https://github.com/sqlc-dev/sqlc) generates for the [`users` example](../../users/README.md). Its structs match `users.EncryptedUser`, so the store uses Go conversions instead of field copies. The generated fields use planned Encrypt Query Language (EQL) types. -The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The Golang SDK for CipherStash Stack is planned and does not exist yet, so this code does not build in this repository. The [Go example index](../../README.md) lists every example and what was checked. ## What you will learn diff --git a/docs/plans/2026-10-04-plan-builder/proto/README.md b/docs/plans/2026-10-04-plan-builder/proto/README.md index b94c92366..565bac6bd 100644 --- a/docs/plans/2026-10-04-plan-builder/proto/README.md +++ b/docs/plans/2026-10-04-plan-builder/proto/README.md @@ -1,10 +1,10 @@ # Annotate protobuf fields for encryption rules -This directory defines a protobuf field option and applies it to an `Individual` message. -The option carries Fideslang data categories that the planned rules read during generation. +This example defines a protobuf field option and applies it to an `Individual` message. +The option carries [data categories](https://ethyca.github.io/fideslang/taxonomy/data_categories/) from [Fideslang](https://ethyca.github.io/fideslang/), a published taxonomy of personal data. `buf generate` works for these protobuf files. -The Go software development kit (SDK) is planned and does not exist yet. +The Golang SDK for CipherStash Stack is planned and does not exist yet. The rules that read the field options belong to the planned SDK. The [Go example index](../README.md) lists every example and what was checked. @@ -21,8 +21,7 @@ The [Go example index](../README.md) lists every example and what was checked. - **Needs:** Nothing `classification.proto` extends `google.protobuf.FieldOptions` with `data_categories`. -Its values use Fideslang data categories. -Fideslang is a published taxonomy of personal data categories, such as `user.contact.email`. +Each value names one Fideslang data category, such as `user.contact.email`. [`classification.proto`, lines 5 to 12](classification.proto#L5-L12) @@ -79,7 +78,7 @@ plugins: Run `buf generate` in this directory to generate the package again. The committed files came from buf v1.50.0 and `protoc-gen-go` v1.26.0. Adding a field to `Individual` does not stop the build. -Continuous integration (CI) runs `go generate ./...` and fails when a generated file differs from the committed file. +CI runs `go generate ./...` and fails when a generated file differs from the committed file. ## What was checked diff --git a/docs/plans/2026-10-04-plan-builder/rules/README.md b/docs/plans/2026-10-04-plan-builder/rules/README.md index 74b113fd7..cd4dbe94c 100644 --- a/docs/plans/2026-10-04-plan-builder/rules/README.md +++ b/docs/plans/2026-10-04-plan-builder/rules/README.md @@ -1,8 +1,8 @@ # Decide protobuf encryption from data categories -This directory defines planned policy rules for `pb.Individual` fields. +This example defines planned policy rules for `pb.Individual` fields. The first matching rule chooses each field's storage layout. -The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The Golang SDK for CipherStash Stack is planned and does not exist yet, so this code does not build in this repository. The [Go example index](../README.md) lists every example and what was checked. ## What you will learn @@ -17,6 +17,7 @@ The [Go example index](../README.md) lists every example and what was checked. - **Related:** [The plan's policy declarations](../../2026-10-04-plan-builder.md#declarations-from-a-policy), [the protobuf source walkthrough](../proto/README.md) - **Needs:** Understand the `data_categories` field option. +A protobuf field can carry [data categories](https://ethyca.github.io/fideslang/taxonomy/data_categories/) from [Fideslang](https://ethyca.github.io/fideslang/), such as the category `user.contact.email`. `policy.Key` names the field option by its full name, `classification.data_categories`. `policy.FirstOf` evaluates each `policy.When` in order, and the first match decides. @@ -71,7 +72,7 @@ var Individuals = policy.ForMessage(&pb.Individual{}, policy.Context("individual `OrElse(Base)` applies the shared rules when no message-specific rule matches. A field with no matching rule stops the planned generator. Adding a field to `Individual` does not stop the build. -Continuous integration (CI) runs `go generate ./...` and fails when a generated file differs from the committed file. +CI runs `go generate ./...` and fails when a generated file differs from the committed file. ## Run rules during generation diff --git a/docs/plans/2026-10-04-plan-builder/sqlc/README.md b/docs/plans/2026-10-04-plan-builder/sqlc/README.md index 36af82160..f0fa5968e 100644 --- a/docs/plans/2026-10-04-plan-builder/sqlc/README.md +++ b/docs/plans/2026-10-04-plan-builder/sqlc/README.md @@ -1,10 +1,10 @@ # Generate sqlc code for Encrypt Query Language columns -This directory configures sqlc for a `users` table with two Encrypt Query Language (EQL) columns. +This example configures [sqlc](https://github.com/sqlc-dev/sqlc) for a `users` table with two Encrypt Query Language (EQL) columns. It also defines the queries that generate the `userdb` package. `sqlc generate` ran for these SQL and YAML files. -The Go software development kit (SDK) is planned and does not exist yet. +The Golang SDK for CipherStash Stack is planned and does not exist yet. The generated package names the planned `encrypt/eql` package, so it does not build in this repository. The [Go example index](../README.md) lists every example and what was checked. diff --git a/docs/plans/2026-10-04-plan-builder/users/README.md b/docs/plans/2026-10-04-plan-builder/users/README.md index 87dc02522..8fa0e7a01 100644 --- a/docs/plans/2026-10-04-plan-builder/users/README.md +++ b/docs/plans/2026-10-04-plan-builder/users/README.md @@ -1,8 +1,8 @@ # Encrypt and store users with generated helpers -This directory shows the planned path from a tagged struct to three database stores. +This example shows the planned path from a tagged struct to three database stores: `database/sql`, GORM and [sqlc](https://github.com/sqlc-dev/sqlc). Each encrypted field uses one Encrypt Query Language (EQL) column. -The Go software development kit (SDK) is planned and does not exist yet, so this code does not build in this repository. +The Golang SDK for CipherStash Stack is planned and does not exist yet, so this code does not build in this repository. The [Go example index](../README.md) lists every example and what was checked. `user_stash.go` is written by hand. From 3dbc1cc381f3704dce243ac98e07c98e72a7858f Mon Sep 17 00:00:00 2001 From: Lindsay Holmwood Date: Wed, 7 Oct 2026 09:06:48 +1100 Subject: [PATCH 3/4] docs(plans): drop "What was checked" and "Needs" from the walkthroughs Each walkthrough repeated rows of the index's "What was checked" table, and nothing kept the copies in step with it. The opening line of each page still links the index, which has the one table. A "Needs" line mostly pointed back at the section before it, which the order of the page already shows. Also stop saying "carries" in prose. Claude-Session: https://claude.ai/code/session_016Usf7cQGY6ULWGyZ4NHQEB --- .../accounts/README.md | 33 --------------- .../cmd/genencrypt/README.md | 22 ---------- .../contacts/README.md | 28 ------------- .../2026-10-04-plan-builder/crm/README.md | 12 ------ .../documents/README.md | 26 ------------ .../individuals/README.md | 41 ------------------- .../internal/pb/README.md | 21 ---------- .../internal/userdb/README.md | 21 ---------- .../2026-10-04-plan-builder/proto/README.md | 21 ++-------- .../2026-10-04-plan-builder/rules/README.md | 28 +------------ .../2026-10-04-plan-builder/sqlc/README.md | 15 ------- .../2026-10-04-plan-builder/users/README.md | 40 ------------------ 12 files changed, 4 insertions(+), 304 deletions(-) diff --git a/docs/plans/2026-10-04-plan-builder/accounts/README.md b/docs/plans/2026-10-04-plan-builder/accounts/README.md index 71195093b..32daa0802 100644 --- a/docs/plans/2026-10-04-plan-builder/accounts/README.md +++ b/docs/plans/2026-10-04-plan-builder/accounts/README.md @@ -19,7 +19,6 @@ It shows what `stashgen` is planned to write. - **Purpose:** Understand how one tag covers an embedded struct from another package. - **Related:** [The plan's embedded struct design](../../2026-10-04-plan-builder.md#embedded-structs-unexported-fields-and-other-tags), [the plan's printing design](../../2026-10-04-plan-builder.md#printing) -- **Needs:** Nothing The `stash:",passthrough"` tag stores every `gorm.Model` field as a passthrough field. `Email` becomes one Encrypt Query Language (EQL) column with the `TextEq` type. @@ -64,7 +63,6 @@ The `token` field uses `stash:"-"`, so it is an omitted field and produces no wa - **Purpose:** Follow copied tags and fields that are not stored into the generated file. - **Related:** [The plan's generated file design](../../2026-10-04-plan-builder.md#what-stashgen-writes), [the plan's embedded struct design](../../2026-10-04-plan-builder.md#embedded-structs-unexported-fields-and-other-tags) -- **Needs:** Understand the account tags. The generated comment names `cache`, and `EncryptedAccount` embeds the same `gorm.Model`. The encrypted `Email` field retains its GORM and JSON tags. @@ -105,7 +103,6 @@ var codec = gensupport.New(gensupport.Generated[Account, EncryptedAccount]{ - **Purpose:** See what the `-redact` flag adds to `Account`. - **Related:** [The plan's printing design](../../2026-10-04-plan-builder.md#printing) -- **Needs:** Understand that the tagged struct contains plaintext values. `-redact` asks `stashgen` to add `String` and `LogValue` methods to `Account`. These methods show `gorm.Model` and hide `Email`. @@ -127,7 +124,6 @@ func (a Account) LogValue() slog.Value { - **Purpose:** Follow a slice through encryption and into GORM. - **Related:** [The plan's database design](../../2026-10-04-plan-builder.md#databases) -- **Needs:** Understand `EncryptedAccount`. `EncryptedAccount.TableName` maps the encrypted type to the `accounts` table. `Create` encrypts the whole slice before GORM receives it. @@ -145,32 +141,3 @@ func Create(ctx context.Context, db *gorm.DB, cipher *encrypt.Cipher, accounts [ return db.WithContext(ctx).Create(&encrypted).Error } ``` - -## What was checked - -- **Every Go file type-checks:** - Run. - `go vet ./...` passes against a stub of the SDK. - The stub is not in this repository, and it has signatures only. -- **A change to a tagged struct, a model or a type in another package stops the build:** - Run. - Each change fails `go build` with "cannot convert". -- **A generated file from another version does not compile:** - Run. - A file that names an unknown version constant fails `go build`. -- **The files `stashgen` writes:** - Not run. - `stashgen` does not exist, and the five `_stash.go` files are written by hand. -- **The SDK can be built with these signatures:** - Not run. -- **The guest returns an EQL value for a field with `encrypt_into`:** - Not run. - The target form and the dispatch do not exist. -- **`stashgen` checks a declaration with the embedded guest:** - Not run. -- **The code works with a database, GORM or pgx:** - Not run. - Nothing here has connected to a database. -- **The EQL types, and the value the guest returns for them:** - Not run. - `eql-codegen` does not write Go yet, and the examples use a stub of `eql.TextEq`. diff --git a/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md b/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md index c27cd5a11..667df7f70 100644 --- a/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md +++ b/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md @@ -14,7 +14,6 @@ The [Go example index](../../README.md) lists every example and what was checked - **Purpose:** Follow the three inputs to the planned library generator. - **Related:** [The plan's policy declarations](../../../2026-10-04-plan-builder.md#declarations-from-a-policy), [the rules walkthrough](../../rules/README.md), [the individuals walkthrough](../../individuals/README.md) -- **Needs:** Read the `rules.Individuals` policy first. `protosource.New()` supplies protobuf descriptors and field options. `rules.Individuals` supplies the field decisions. @@ -40,29 +39,8 @@ The `stashgen`, `protosource`, and policy packages do not exist yet. - **Purpose:** See how reviewers inspect policy changes without running rules in the application. - **Related:** [The generated individuals walkthrough](../../individuals/README.md), [the rules walkthrough](../../rules/README.md) -- **Needs:** Understand the generator inputs. The [`rules` package](../../rules/README.md) runs this command through `go generate`. The application calls the resulting `individuals.Encrypt` and `individuals.Decrypt` functions. A rule change changes `individual_stash.go`, which gives a reviewer a concrete generated diff. The application never runs the rules. - -## What was checked - -- **Every Go file type-checks:** - Run. - `go vet ./...` passes against a stub of the SDK. - The stub is not in this repository, and it has signatures only. -- **The policy example compiles against real protobuf code:** - Run. - buf v1.50.0 and `protoc-gen-go` wrote `internal/pb/`, and `go vet` passes. -- **The protobuf source reads the field options, and the rules run:** - Not run. - Neither the source nor the generator exists. -- **The files `stashgen` writes:** - Not run. - `stashgen` does not exist, and the five `_stash.go` files are written by hand. -- **The SDK can be built with these signatures:** - Not run. -- **`stashgen` checks a declaration with the embedded guest:** - Not run. diff --git a/docs/plans/2026-10-04-plan-builder/contacts/README.md b/docs/plans/2026-10-04-plan-builder/contacts/README.md index e9cbb0b0e..8e0485efe 100644 --- a/docs/plans/2026-10-04-plan-builder/contacts/README.md +++ b/docs/plans/2026-10-04-plan-builder/contacts/README.md @@ -21,7 +21,6 @@ It shows what `stashgen` is planned to write. - **Purpose:** Connect a tag declaration in your own package to `crm.Contact`. - **Related:** [The `crm.Contact` type in another package](../crm/README.md), [the plan's design for types in another package](../../2026-10-04-plan-builder.md#types-in-another-package) -- **Needs:** Understand that the program does not own `crm.Contact`. The `-for crm.Contact` flag makes `contactStash` declare tags for a type in another package. The generator is planned to match fields by name and type. @@ -72,7 +71,6 @@ type contactShape struct { - **Purpose:** See how each encrypted field holds a ciphertext and its declared search terms. - **Related:** [The plan's column layouts](../../2026-10-04-plan-builder.md#columns), [the plan's generated file design](../../2026-10-04-plan-builder.md#what-stashgen-writes) -- **Needs:** Understand the `contactStash` tags. `EncryptedContact` nests one encrypted type for each encrypted field. @@ -127,7 +125,6 @@ It cannot add methods to `crm.Contact` because Go requires methods to belong to - **Purpose:** Flatten nested encrypted outputs into one field for each database column. - **Related:** [The plan's model design](../../2026-10-04-plan-builder.md#models) -- **Needs:** Understand the separate encrypted outputs. `-model Rows=ContactRow` asks for `EncryptRows` and `DecryptRows`. `ContactRow` is a model with one tagged field for each column. @@ -183,7 +180,6 @@ func DecryptRows(ctx context.Context, d encrypt.Decrypter, rows []ContactRow) ([ - **Purpose:** Compare direct column writes with a model-based GORM write. - **Related:** [The plan's database design](../../2026-10-04-plan-builder.md#databases) -- **Needs:** Understand both encrypted layouts. `Create` passes each nested output to `database/sql`. @@ -222,7 +218,6 @@ func CreateWithGORM(ctx context.Context, db *gorm.DB, cipher *encrypt.Cipher, li - **Purpose:** Derive and use the equality search term for a separate column. - **Related:** [The plan's call design](../../2026-10-04-plan-builder.md#calls), [the plan's terminology](../../2026-10-04-plan-builder.md#terminology) -- **Needs:** Understand the `phone_number_eq` column. `Fields.PhoneNumber.Equality` derives an `encrypt.EqualityTerm` search term. The query compares that search term with the `phone_number_eq` column. @@ -240,26 +235,3 @@ func IDByPhone(ctx context.Context, db *sql.DB, cipher *encrypt.Cipher, phone st return id, err } ``` - -## What was checked - -- **Every Go file type-checks:** - Run. - `go vet ./...` passes against a stub of the SDK. - The stub is not in this repository, and it has signatures only. -- **A change to a tagged struct, a model or a type in another package stops the build:** - Run. - Each change fails `go build` with "cannot convert". -- **A generated file from another version does not compile:** - Run. - A file that names an unknown version constant fails `go build`. -- **The files `stashgen` writes:** - Not run. - `stashgen` does not exist, and the five `_stash.go` files are written by hand. -- **The SDK can be built with these signatures:** - Not run. -- **`stashgen` checks a declaration with the embedded guest:** - Not run. -- **The code works with a database, GORM or pgx:** - Not run. - Nothing here has connected to a database. diff --git a/docs/plans/2026-10-04-plan-builder/crm/README.md b/docs/plans/2026-10-04-plan-builder/crm/README.md index 7d10e5604..5ed583424 100644 --- a/docs/plans/2026-10-04-plan-builder/crm/README.md +++ b/docs/plans/2026-10-04-plan-builder/crm/README.md @@ -17,7 +17,6 @@ The [Go example index](../README.md) lists every example and what was checked. - **Purpose:** Understand why `crm.Contact` stays free of encryption tags. - **Related:** [The plan's design for types in another package](../../2026-10-04-plan-builder.md#types-in-another-package), [the contacts walkthrough](../contacts/README.md) -- **Needs:** Nothing `crm.Contact` contains only the fields supplied by the stand-in package. @@ -54,7 +53,6 @@ type contactStash struct { - **Purpose:** See how a change to `crm.Contact` stops the contacts package from building. - **Related:** [The contacts walkthrough](../contacts/README.md), [the plan's design for types in another package](../../2026-10-04-plan-builder.md#types-in-another-package) -- **Needs:** Understand the fields of `crm.Contact`. The generated file converts `crm.Contact` to `contactShape` at compile time. The conversion fails when `crm.Contact` gains, loses, reorders, or retypes a field. @@ -72,13 +70,3 @@ type contactShape struct { Internal string } ``` - -## What was checked - -- **Every Go file type-checks:** - Run. - `go vet ./...` passes against a stub of the SDK. - The stub is not in this repository, and it has signatures only. -- **A change to a tagged struct, a model or a type in another package stops the build:** - Run. - Each change fails `go build` with "cannot convert". diff --git a/docs/plans/2026-10-04-plan-builder/documents/README.md b/docs/plans/2026-10-04-plan-builder/documents/README.md index aeef48217..22cfcb3e9 100644 --- a/docs/plans/2026-10-04-plan-builder/documents/README.md +++ b/docs/plans/2026-10-04-plan-builder/documents/README.md @@ -18,7 +18,6 @@ It shows what `stashgen` is planned to write. - **Purpose:** Understand why the document fields have no individual tags. - **Related:** [The plan's struct tags](../../2026-10-04-plan-builder.md#struct-tags), [the plan's column layouts](../../2026-10-04-plan-builder.md#columns) -- **Needs:** Nothing The `opaque` option is part of the encryption context tag. It tells the planned generator to encrypt the complete `Document` as one value. @@ -43,7 +42,6 @@ Nothing inside the document can be read or searched independently through this l - **Purpose:** Follow the opaque tag into the encrypted type and declaration. - **Related:** [The plan's generated file design](../../2026-10-04-plan-builder.md#what-stashgen-writes) -- **Needs:** Understand the opaque tagged struct. `EncryptedDocument` has one `Sealed` ciphertext field. @@ -84,7 +82,6 @@ var codec = gensupport.New(gensupport.Generated[Document, EncryptedDocument]{ - **Purpose:** See why `Save` uses a cipher and `Load` uses the client. - **Related:** [The plan's call design](../../2026-10-04-plan-builder.md#calls), [the root example program](../main.go) -- **Needs:** Understand `EncryptedDocument.Sealed`. `Save` encrypts one document by passing a one-element slice. It stores `encrypted[0].Sealed` in the `body` column. @@ -124,26 +121,3 @@ func Load(ctx context.Context, db *sql.DB, client *encrypt.Client, id int64) (Do return docs[0], nil } ``` - -## What was checked - -- **Every Go file type-checks:** - Run. - `go vet ./...` passes against a stub of the SDK. - The stub is not in this repository, and it has signatures only. -- **A change to a tagged struct, a model or a type in another package stops the build:** - Run. - Each change fails `go build` with "cannot convert". -- **A generated file from another version does not compile:** - Run. - A file that names an unknown version constant fails `go build`. -- **The files `stashgen` writes:** - Not run. - `stashgen` does not exist, and the five `_stash.go` files are written by hand. -- **The SDK can be built with these signatures:** - Not run. -- **`stashgen` checks a declaration with the embedded guest:** - Not run. -- **The code works with a database, GORM or pgx:** - Not run. - Nothing here has connected to a database. diff --git a/docs/plans/2026-10-04-plan-builder/individuals/README.md b/docs/plans/2026-10-04-plan-builder/individuals/README.md index af3ffa1e9..35dffde4f 100644 --- a/docs/plans/2026-10-04-plan-builder/individuals/README.md +++ b/docs/plans/2026-10-04-plan-builder/individuals/README.md @@ -19,7 +19,6 @@ It shows what `stashgen.Generate` is planned to write. - **Purpose:** Connect policy decisions to the encrypted type and declaration. - **Related:** [The rules walkthrough](../rules/README.md), [the plan's policy declarations](../../2026-10-04-plan-builder.md#declarations-from-a-policy) -- **Needs:** Understand `rules.Individuals`. `Id` and `Nickname` are passthrough fields. `Name` has one ciphertext, while `Email` has a ciphertext and two search terms. @@ -66,7 +65,6 @@ var declaration = gensupport.Declare("individuals"). - **Purpose:** Understand why protobuf messages do not use the usual shape check. - **Related:** [The generated protobuf walkthrough](../internal/pb/README.md), [the plan's design for types in another package](../../2026-10-04-plan-builder.md#types-in-another-package) -- **Needs:** Know that `pb.Individual` contains unexported protobuf fields. The codec takes `*pb.Individual` values. It reads each field through a generated getter because direct struct conversion cannot include unexported fields. @@ -102,7 +100,6 @@ CI runs `go generate ./...` and fails when a generated file differs from the com - **Purpose:** Follow protobuf pointers and the Medicare field through the generated API. - **Related:** [The plan's call design](../../2026-10-04-plan-builder.md#calls), [the generation command walkthrough](../cmd/genencrypt/README.md) -- **Needs:** Understand the codec's pointer type. `Encrypt` accepts a slice of `*pb.Individual` pointers. `Decrypt` returns the same pointer form. @@ -156,7 +153,6 @@ func (f MedicareNoField) Query(ctx context.Context, c *encrypt.Cipher, v string) - **Purpose:** Map each storage layout to its database columns and query. - **Related:** [The plan's database design](../../2026-10-04-plan-builder.md#databases) -- **Needs:** Understand the mixed encrypted type. `Create` passes the passthrough fields, ciphertexts, search terms, and EQL value to their columns. @@ -191,40 +187,3 @@ func IDByMedicare(ctx context.Context, db *sql.DB, cipher *encrypt.Cipher, medic return id, err } ``` - -## What was checked - -- **Every Go file type-checks:** - Run. - `go vet ./...` passes against a stub of the SDK. - The stub is not in this repository, and it has signatures only. -- **The policy example compiles against real protobuf code:** - Run. - buf v1.50.0 and `protoc-gen-go` wrote `internal/pb/`, and `go vet` passes. -- **A field added to the protobuf message does not stop the build:** - Run. - It builds, as the plan says: CI finds that change. -- **A struct from another package with an unexported field cannot convert:** - Run, with `sync.Once`. -- **A generated file from another version does not compile:** - Run. - A file that names an unknown version constant fails `go build`. -- **The protobuf source reads the field options, and the rules run:** - Not run. - Neither the source nor the generator exists. -- **The files `stashgen` writes:** - Not run. - `stashgen` does not exist, and the five `_stash.go` files are written by hand. -- **The SDK can be built with these signatures:** - Not run. -- **The guest returns an EQL value for a field with `encrypt_into`:** - Not run. - The target form and the dispatch do not exist. -- **`stashgen` checks a declaration with the embedded guest:** - Not run. -- **The code works with a database, GORM or pgx:** - Not run. - Nothing here has connected to a database. -- **The EQL types, and the value the guest returns for them:** - Not run. - `eql-codegen` does not write Go yet, and the examples use a stub of `eql.TextEq`. diff --git a/docs/plans/2026-10-04-plan-builder/internal/pb/README.md b/docs/plans/2026-10-04-plan-builder/internal/pb/README.md index cd6672a52..77ee577c7 100644 --- a/docs/plans/2026-10-04-plan-builder/internal/pb/README.md +++ b/docs/plans/2026-10-04-plan-builder/internal/pb/README.md @@ -16,7 +16,6 @@ The [Go example index](../../README.md) lists every example and what was checked - **Purpose:** Identify the generators, versions, and the command that generates the package again. - **Related:** [The protobuf source walkthrough](../../proto/README.md) -- **Needs:** Install buf and `protoc-gen-go`. buf v1.50.0 ran `protoc-gen-go` v1.26.0 to write these files. The generated header reports the `protoc` version as unknown. @@ -42,7 +41,6 @@ package pb - **Purpose:** Identify the generated values that the individuals example reads. - **Related:** [The generated individuals walkthrough](../../individuals/README.md), [the plan's design for types in another package](../../../2026-10-04-plan-builder.md#types-in-another-package) -- **Needs:** Understand the `Individual` message fields. `pb.Individual` includes three unexported protobuf fields before its exported data fields. @@ -89,7 +87,6 @@ The examples do not use protobuf internals such as `file_individual_proto_rawDes - **Purpose:** Connect the protobuf option to the planned policy source. - **Related:** [The rules walkthrough](../../rules/README.md), [the plan's policy declarations](../../../2026-10-04-plan-builder.md#declarations-from-a-policy) -- **Needs:** Understand the `data_categories` option. `classification.pb.go` exports `E_DataCategories` for the repeated field option. @@ -106,21 +103,3 @@ var ( ``` The planned `protosource` package reads this option through protobuf descriptors. - -## What was checked - -- **Every Go file type-checks:** - Run. - `go vet ./...` passes against a stub of the SDK. - The stub is not in this repository, and it has signatures only. -- **The policy example compiles against real protobuf code:** - Run. - buf v1.50.0 and `protoc-gen-go` wrote `internal/pb/`, and `go vet` passes. -- **A field added to the protobuf message does not stop the build:** - Run. - It builds, as the plan says: CI finds that change. -- **A struct from another package with an unexported field cannot convert:** - Run, with `sync.Once`. -- **The protobuf source reads the field options, and the rules run:** - Not run. - Neither the source nor the generator exists. diff --git a/docs/plans/2026-10-04-plan-builder/internal/userdb/README.md b/docs/plans/2026-10-04-plan-builder/internal/userdb/README.md index a06ecebd8..2393ce3a3 100644 --- a/docs/plans/2026-10-04-plan-builder/internal/userdb/README.md +++ b/docs/plans/2026-10-04-plan-builder/internal/userdb/README.md @@ -16,7 +16,6 @@ The [Go example index](../../README.md) lists every example and what was checked - **Purpose:** Identify the generator, its version, and the command that generates the package again. - **Related:** [The sqlc inputs](../../sqlc/README.md) -- **Needs:** Nothing sqlc v1.31.1 generated `db.go`, `models.go`, and `query.sql.go`. Run `sqlc generate` in [`sqlc/`](../../sqlc/) to generate them again. @@ -36,7 +35,6 @@ package userdb - **Purpose:** See why direct Go conversions connect sqlc to the encrypted type. - **Related:** [The plan's model design](../../../2026-10-04-plan-builder.md#models), [the users sqlc store](../../users/README.md) -- **Needs:** Know that an encrypted type has one field for each stored field. sqlc's row struct `userdb.User` has the same fields as `users.EncryptedUser`. @@ -71,7 +69,6 @@ A field change stops either conversion from compiling until both shapes agree. - **Purpose:** Connect generated identifiers to the handwritten users store. - **Related:** [The users sqlc store](../../users/README.md), [the sqlc queries](../../sqlc/query.sql) -- **Needs:** Understand the matching struct shapes. `userdb.New` creates `Queries`, and `WithTx` binds the methods to a transaction. The users store calls `CreateUser`, `GetUser`, and `FindUsersByEmail`. @@ -95,21 +92,3 @@ func (q *Queries) WithTx(tx *sql.Tx) *Queries { ``` The examples do not use the generated `ListUsers` method. - -## What was checked - -- **Every Go file type-checks:** - Run. - `go vet ./...` passes against a stub of the SDK. - The stub is not in this repository, and it has signatures only. -- **sqlc's row struct converts to the generated type:** - Run. - The conversions in `users/sqlcstore.go` compile against real sqlc output. -- **The SDK can be built with these signatures:** - Not run. -- **The code works with a database, GORM or pgx:** - Not run. - Nothing here has connected to a database. -- **The EQL types, and the value the guest returns for them:** - Not run. - `eql-codegen` does not write Go yet, and the examples use a stub of `eql.TextEq`. diff --git a/docs/plans/2026-10-04-plan-builder/proto/README.md b/docs/plans/2026-10-04-plan-builder/proto/README.md index 565bac6bd..d84da4107 100644 --- a/docs/plans/2026-10-04-plan-builder/proto/README.md +++ b/docs/plans/2026-10-04-plan-builder/proto/README.md @@ -1,7 +1,7 @@ # Annotate protobuf fields for encryption rules This example defines a protobuf field option and applies it to an `Individual` message. -The option carries [data categories](https://ethyca.github.io/fideslang/taxonomy/data_categories/) from [Fideslang](https://ethyca.github.io/fideslang/), a published taxonomy of personal data. +The option holds [data categories](https://ethyca.github.io/fideslang/taxonomy/data_categories/) from [Fideslang](https://ethyca.github.io/fideslang/), a published taxonomy of personal data. `buf generate` works for these protobuf files. The Golang SDK for CipherStash Stack is planned and does not exist yet. @@ -16,9 +16,8 @@ The [Go example index](../README.md) lists every example and what was checked. ## Define the data category option -- **Purpose:** See how protobuf fields carry facts for encryption rules. +- **Purpose:** See how a field option labels protobuf fields for encryption rules. - **Related:** [The plan's policy declarations](../../2026-10-04-plan-builder.md#declarations-from-a-policy), [the rules walkthrough](../rules/README.md) -- **Needs:** Nothing `classification.proto` extends `google.protobuf.FieldOptions` with `data_categories`. Each value names one Fideslang data category, such as `user.contact.email`. @@ -40,9 +39,8 @@ extend google.protobuf.FieldOptions { - **Purpose:** Follow each protobuf field into the policy rules. - **Related:** [The rules walkthrough](../rules/README.md), [the generated individuals walkthrough](../individuals/README.md) -- **Needs:** Understand the `data_categories` option. -`name`, `email`, and `medicare_no` each carry a category. +`name`, `email`, and `medicare_no` each have a category. `id` and `nickname` have no category, so message-specific rules must decide them. [`individual.proto`, lines 9 to 15](individual.proto#L9-L15) @@ -61,7 +59,6 @@ message Individual { - **Purpose:** Understand where buf writes the generated Go files. - **Related:** [The generated protobuf walkthrough](../internal/pb/README.md) -- **Needs:** Install buf and `protoc-gen-go`. `buf.gen.yaml` writes source-relative Go files into `../internal/pb`. @@ -79,15 +76,3 @@ Run `buf generate` in this directory to generate the package again. The committed files came from buf v1.50.0 and `protoc-gen-go` v1.26.0. Adding a field to `Individual` does not stop the build. CI runs `go generate ./...` and fails when a generated file differs from the committed file. - -## What was checked - -- **The policy example compiles against real protobuf code:** - Run. - buf v1.50.0 and `protoc-gen-go` wrote `internal/pb/`, and `go vet` passes. -- **A field added to the protobuf message does not stop the build:** - Run. - It builds, as the plan says: CI finds that change. -- **The protobuf source reads the field options, and the rules run:** - Not run. - Neither the source nor the generator exists. diff --git a/docs/plans/2026-10-04-plan-builder/rules/README.md b/docs/plans/2026-10-04-plan-builder/rules/README.md index cd4dbe94c..895edff7f 100644 --- a/docs/plans/2026-10-04-plan-builder/rules/README.md +++ b/docs/plans/2026-10-04-plan-builder/rules/README.md @@ -15,9 +15,8 @@ The [Go example index](../README.md) lists every example and what was checked. - **Purpose:** Understand how `Base` decides fields from protobuf annotations. - **Related:** [The plan's policy declarations](../../2026-10-04-plan-builder.md#declarations-from-a-policy), [the protobuf source walkthrough](../proto/README.md) -- **Needs:** Understand the `data_categories` field option. -A protobuf field can carry [data categories](https://ethyca.github.io/fideslang/taxonomy/data_categories/) from [Fideslang](https://ethyca.github.io/fideslang/), such as the category `user.contact.email`. +A protobuf field can name [data categories](https://ethyca.github.io/fideslang/taxonomy/data_categories/) from [Fideslang](https://ethyca.github.io/fideslang/), such as the category `user.contact.email`. `policy.Key` names the field option by its full name, `classification.data_categories`. `policy.FirstOf` evaluates each `policy.When` in order, and the first match decides. @@ -49,7 +48,6 @@ The policy API also defines `Index`, `Omit`, and `Fail` decisions. - **Purpose:** See how `Individuals` handles fields before it falls back to `Base`. - **Related:** [The generated individuals walkthrough](../individuals/README.md), [the protobuf source walkthrough](../proto/README.md) -- **Needs:** Understand the shared rules. `policy.ForMessage` binds the rules to `pb.Individual` and the `individuals` encryption context. `policy.Field` handles uncategorised fields and overrides a categorised field. @@ -78,7 +76,6 @@ CI runs `go generate ./...` and fails when a generated file differs from the com - **Purpose:** Keep policy evaluation out of the application runtime. - **Related:** [The generation command walkthrough](../cmd/genencrypt/README.md), [the plan's policy declarations](../../2026-10-04-plan-builder.md#declarations-from-a-policy) -- **Needs:** Understand the complete `Individuals` policy. The `go:generate` line runs `cmd/genencrypt`. @@ -99,26 +96,3 @@ import ( ``` The application uses the generated functions and never runs these rules. - -## What was checked - -- **Every Go file type-checks:** - Run. - `go vet ./...` passes against a stub of the SDK. - The stub is not in this repository, and it has signatures only. -- **The policy example compiles against real protobuf code:** - Run. - buf v1.50.0 and `protoc-gen-go` wrote `internal/pb/`, and `go vet` passes. -- **A field added to the protobuf message does not stop the build:** - Run. - It builds, as the plan says: CI finds that change. -- **The protobuf source reads the field options, and the rules run:** - Not run. - Neither the source nor the generator exists. -- **The files `stashgen` writes:** - Not run. - `stashgen` does not exist, and the five `_stash.go` files are written by hand. -- **The SDK can be built with these signatures:** - Not run. -- **`stashgen` checks a declaration with the embedded guest:** - Not run. diff --git a/docs/plans/2026-10-04-plan-builder/sqlc/README.md b/docs/plans/2026-10-04-plan-builder/sqlc/README.md index f0fa5968e..7117ab0b6 100644 --- a/docs/plans/2026-10-04-plan-builder/sqlc/README.md +++ b/docs/plans/2026-10-04-plan-builder/sqlc/README.md @@ -18,7 +18,6 @@ The [Go example index](../README.md) lists every example and what was checked. - **Purpose:** Understand the schema files that sqlc reads. - **Related:** [The plan's database design](../../2026-10-04-plan-builder.md#databases), [the generated `userdb` package](../internal/userdb/README.md) -- **Needs:** Nothing `eql-domains.sql` gives sqlc small domain declarations because sqlc cannot parse the EQL installation bundle. The domain spelling must match `schema.sql` and the matching override. @@ -52,7 +51,6 @@ Read the index's [three rules for sqlc with EQL columns](../README.md#use-sqlc-w - **Purpose:** Follow the sqlc inputs and type overrides into the generated package. - **Related:** [The plan's model design](../../2026-10-04-plan-builder.md#models), [the generated `userdb` package](../internal/userdb/README.md) -- **Needs:** Read the EQL domain and table files first. `sqlc.yaml` reads the domain file before the table schema. It writes package `userdb` into `../internal/userdb`. @@ -82,7 +80,6 @@ Run `sqlc generate` in this directory to generate the package again with sqlc v1 - **Purpose:** See how the SQL produces typed create, read, list, and search methods. - **Related:** [The users sqlc store](../users/README.md), [the generated query methods](../internal/userdb/README.md) -- **Needs:** Understand the domain overrides. `query.sql` defines four methods through sqlc's query annotations. The email search casts its parameter directly to `eql_v3.query_text_eq`. @@ -100,15 +97,3 @@ SELECT * FROM users WHERE email = sqlc.arg(email)::eql_v3.query_text_eq; ``` The direct cast lets the override generate an `eql.TextEqQuery` parameter. - -## What was checked - -- **sqlc's row struct converts to the generated type:** - Run. - The conversions in `users/sqlcstore.go` compile against real sqlc output. -- **The code works with a database, GORM or pgx:** - Not run. - Nothing here has connected to a database. -- **The EQL types, and the value the guest returns for them:** - Not run. - `eql-codegen` does not write Go yet, and the examples use a stub of `eql.TextEq`. diff --git a/docs/plans/2026-10-04-plan-builder/users/README.md b/docs/plans/2026-10-04-plan-builder/users/README.md index 8fa0e7a01..c4115fd88 100644 --- a/docs/plans/2026-10-04-plan-builder/users/README.md +++ b/docs/plans/2026-10-04-plan-builder/users/README.md @@ -20,7 +20,6 @@ It shows what `stashgen` is planned to write. - **Purpose:** Connect the tagged struct to its generator command and runtime cipher. - **Related:** [The plan's struct tags](../../2026-10-04-plan-builder.md#struct-tags), [the root example program](../main.go) -- **Needs:** Nothing The `go:generate` line asks `stashgen` to read `User`. The encryption context is `users`, and every exported field has a `stash` tag. @@ -72,7 +71,6 @@ func run(ctx context.Context) error { - **Purpose:** Understand the main parts that `stashgen` is planned to write. - **Related:** [The plan's generated file design](../../2026-10-04-plan-builder.md#what-stashgen-writes), [the plan's call design](../../2026-10-04-plan-builder.md#calls) -- **Needs:** Understand the tagged struct. `EncryptedUser` keeps `ID` and replaces `Email` and `Name` with `eql.TextEq`. Its `String` and `LogValue` methods hide the encrypted fields when code prints the encrypted type. @@ -177,7 +175,6 @@ var Fields = struct { - **Purpose:** Follow encrypted values through a transaction and an EQL equality query. - **Related:** [The plan's database design](../../2026-10-04-plan-builder.md#databases), [the sqlc schema walkthrough](../sqlc/README.md) -- **Needs:** Understand `EncryptedUser` and `Fields.Email.Query`. `SQLStore.Import` encrypts the whole input slice before it starts inserting rows. It passes each `EncryptedUser` field directly to `ExecContext`. @@ -235,7 +232,6 @@ func (s *SQLStore) FindByEmail(ctx context.Context, cipher *encrypt.Cipher, emai - **Purpose:** See why encryption happens before GORM maps the encrypted type. - **Related:** [The plan's database design](../../2026-10-04-plan-builder.md#databases), [the plan's call design](../../2026-10-04-plan-builder.md#calls) -- **Needs:** Understand the generated `Encrypt` function. `EncryptedUser.TableName` sends GORM to `users` instead of `encrypted_users`. A `driver.Valuer` hook cannot batch because it receives one field and no `context.Context`. @@ -274,7 +270,6 @@ func (s *GormStore) Create(ctx context.Context, cipher *encrypt.Cipher, people . - **Purpose:** Connect generated sqlc shapes and field-level updates to the encrypted type. - **Related:** [The generated `userdb` walkthrough](../internal/userdb/README.md), [the sqlc inputs](../sqlc/README.md) -- **Needs:** Understand `EncryptedUser` and `Fields.Email.Encrypt`. `userdb.User` and `userdb.CreateUserParams` have the same fields as `EncryptedUser`. Go therefore converts these structs directly, and a change to either shape stops the build. @@ -325,38 +320,3 @@ func (s *SQLCStore) ChangeEmail(ctx context.Context, cipher *encrypt.Cipher, id return err } ``` - -## What was checked - -- **Every Go file type-checks:** - Run. - `go vet ./...` passes against a stub of the SDK. - The stub is not in this repository, and it has signatures only. -- **A change to a tagged struct, a model or a type in another package stops the build:** - Run. - Each change fails `go build` with "cannot convert". -- **sqlc's row struct converts to the generated type:** - Run. - The conversions in `users/sqlcstore.go` compile against real sqlc output. -- **A generated file from another version does not compile:** - Run. - A file that names an unknown version constant fails `go build`. -- **The files `stashgen` writes:** - Not run. - `stashgen` does not exist, and the five `_stash.go` files are written by hand. -- **The SDK can be built with these signatures:** - Not run. -- **The guest returns an EQL value for a field with `encrypt_into`:** - Not run. - The target form and the dispatch do not exist. -- **`stashgen` checks a declaration with the embedded guest:** - Not run. -- **The code works with a database, GORM or pgx:** - Not run. - Nothing here has connected to a database. -- **The EQL types, and the value the guest returns for them:** - Not run. - `eql-codegen` does not write Go yet, and the examples use a stub of `eql.TextEq`. -- **The steps in "Use the SDK":** - Not run. - Nobody has followed them. From 8bc00c821298802640431be0d442df1cf90996c2 Mon Sep 17 00:00:00 2001 From: Lindsay Holmwood Date: Wed, 7 Oct 2026 09:06:54 +1100 Subject: [PATCH 4/4] docs(plans): write the literal verb where a thing held or carried A field, a column, a file or a cipher does not hold or carry anything. Write the literal verb instead: contains, has, stores or lists. Quoted code comments keep their words, because an excerpt matches its source. Claude-Session: https://claude.ai/code/session_016Usf7cQGY6ULWGyZ4NHQEB --- docs/plans/2026-10-04-plan-builder/README.md | 2 +- .../2026-10-04-plan-builder/cmd/genencrypt/README.md | 2 +- docs/plans/2026-10-04-plan-builder/contacts/README.md | 2 +- docs/plans/2026-10-04-plan-builder/proto/README.md | 2 +- docs/plans/2026-10-04-plan-builder/users/README.md | 8 ++++---- 5 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/plans/2026-10-04-plan-builder/README.md b/docs/plans/2026-10-04-plan-builder/README.md index f8ef3d259..2d5c42fd5 100644 --- a/docs/plans/2026-10-04-plan-builder/README.md +++ b/docs/plans/2026-10-04-plan-builder/README.md @@ -20,7 +20,7 @@ Each directory has a `README.md` that walks through its code. | [`accounts/account.go`](accounts/account.go) | An embedded `gorm.Model`, another library's tags, an unexported field and `-redact` | | [`contacts/contacts.go`](contacts/contacts.go) | [`crm.Contact`](crm/contact.go), a type in another package, in separate columns, with a model | | [`documents/documents.go`](documents/documents.go) | An `opaque` struct, sealed as one value | -| [`proto/`](proto/) | A protobuf message whose fields carry data categories, which generates [`internal/pb/`](internal/pb/) | +| [`proto/`](proto/) | A protobuf message whose fields have data categories, which generates [`internal/pb/`](internal/pb/) | | [`rules/rules.go`](rules/rules.go) | Rules that decide what to encrypt from each field's data categories | | [`cmd/genencrypt/main.go`](cmd/genencrypt/main.go) | The generate program that runs the rules | | [`individuals/`](individuals/) | The file the rules give for the protobuf message, and a store that uses it | diff --git a/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md b/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md index 667df7f70..c19a84236 100644 --- a/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md +++ b/docs/plans/2026-10-04-plan-builder/cmd/genencrypt/README.md @@ -18,7 +18,7 @@ The [Go example index](../../README.md) lists every example and what was checked `protosource.New()` supplies protobuf descriptors and field options. `rules.Individuals` supplies the field decisions. `stashgen.Output` selects the generated file in the `individuals` package. -The output goes in your own `individuals` package, not `internal/pb`, which holds the generated type. +The output goes in your own `individuals` package, not `internal/pb`, which contains the generated type. This separation stops the generate program from importing a file that it writes. [`main.go`, lines 12 to 18](main.go#L12-L18) diff --git a/docs/plans/2026-10-04-plan-builder/contacts/README.md b/docs/plans/2026-10-04-plan-builder/contacts/README.md index 8e0485efe..2acccdbdb 100644 --- a/docs/plans/2026-10-04-plan-builder/contacts/README.md +++ b/docs/plans/2026-10-04-plan-builder/contacts/README.md @@ -69,7 +69,7 @@ type contactShape struct { ## Read the separate encrypted outputs -- **Purpose:** See how each encrypted field holds a ciphertext and its declared search terms. +- **Purpose:** See that each encrypted field contains a ciphertext and its declared search terms. - **Related:** [The plan's column layouts](../../2026-10-04-plan-builder.md#columns), [the plan's generated file design](../../2026-10-04-plan-builder.md#what-stashgen-writes) `EncryptedContact` nests one encrypted type for each encrypted field. diff --git a/docs/plans/2026-10-04-plan-builder/proto/README.md b/docs/plans/2026-10-04-plan-builder/proto/README.md index d84da4107..cc641a8fe 100644 --- a/docs/plans/2026-10-04-plan-builder/proto/README.md +++ b/docs/plans/2026-10-04-plan-builder/proto/README.md @@ -1,7 +1,7 @@ # Annotate protobuf fields for encryption rules This example defines a protobuf field option and applies it to an `Individual` message. -The option holds [data categories](https://ethyca.github.io/fideslang/taxonomy/data_categories/) from [Fideslang](https://ethyca.github.io/fideslang/), a published taxonomy of personal data. +The option lists [data categories](https://ethyca.github.io/fideslang/taxonomy/data_categories/) from [Fideslang](https://ethyca.github.io/fideslang/), a published taxonomy of personal data. `buf generate` works for these protobuf files. The Golang SDK for CipherStash Stack is planned and does not exist yet. diff --git a/docs/plans/2026-10-04-plan-builder/users/README.md b/docs/plans/2026-10-04-plan-builder/users/README.md index c4115fd88..9f6bc6d48 100644 --- a/docs/plans/2026-10-04-plan-builder/users/README.md +++ b/docs/plans/2026-10-04-plan-builder/users/README.md @@ -11,7 +11,7 @@ It shows what `stashgen` is planned to write. ## What you will learn - Declare stored fields and create a tenant cipher. -- Read what the generated file holds. +- Read what the generated file contains. - Insert and search users with `database/sql`. - Batch encryption before GORM receives values. - Convert between sqlc structs and update one EQL column. @@ -43,7 +43,7 @@ type User struct { ``` The root program creates the client and derives one cipher for tenant `tenant-42`. -The cipher holds the tenant's keyset and its extension to each encryption context. +The cipher has the tenant's keyset and its extension to each encryption context. `Extend` adds the tenant segment to each field's declared encryption context. No call takes a keyset or an encryption context, so the write, query, and read cannot use different ones. @@ -67,7 +67,7 @@ func run(ctx context.Context) error { store := users.NewSQLStore(db) ``` -## Read what the generated file holds +## Read what the generated file contains - **Purpose:** Understand the main parts that `stashgen` is planned to write. - **Related:** [The plan's generated file design](../../2026-10-04-plan-builder.md#what-stashgen-writes), [the plan's call design](../../2026-10-04-plan-builder.md#calls) @@ -304,7 +304,7 @@ It converts each sqlc row back with `EncryptedUser(row)` before decryption. ``` `ChangeEmail` encrypts one field and updates the single EQL column. -That column holds the ciphertext and its search term together. +That column stores the ciphertext and its search term together. [`sqlcstore.go`, lines 79 to 88](sqlcstore.go#L79-L88)