Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ a binder that drafts a call or a loader that merges `allOf` is prose.
|---|---|
| **audit** | The credentialed stage that exercises a live API to learn its true behaviour — minimum and maximum valid configuration, field dependencies, value-conditional rules. `tfpfgen audit run`. The only stage that touches a network. |
| **observation** | One recorded finding of an audit: what the live API actually accepted or rejected, with a redacted request/response excerpt as proof. Committed per entity in `audit/observations/<entity>.observations.json`, stamped with the spec hash it was observed against. Deliberately not replayable. |
| **request bodies** | The create request bodies a run got the API to accept, committed per entity in `audit/request_bodies/<entity>.request_bodies.json` with the status each was answered and the response it was answered with. An observation says something about one property; these say what a whole create looked like when it worked, which is the one thing a generated acceptance test cannot derive — the document describes what should be accepted, and only a run knows what was. Acceptance fixtures replay these values rather than deriving them again. Named for the request half deliberately: the response is carried as evidence of what the API echoed, and it is the request that a configuration has to reproduce. |
| **correction** | One committed correction to the imported OpenAPI document: RFC 6902 operations plus a required justification and an optional evidence pointer to an observation. Lives in `spec/corrections/`; proposed ones await a human in `spec/corrections/proposed/`; rejected ones leave a marker in `spec/corrections/rejected/`. Kinds listed in config `audit.auto_accept` skip `proposed/` and land accepted directly, named with an `auto-NNN-` prefix. |
| **revise** | To fold observations into proposed corrections and apply accepted ones — `tfpfgen spec revise`. The spec is revised based on audit observations; the output is the revised spec (`spec/revised.yaml`), the single source of truth for all generation. |
| **import** | To pin the upstream OpenAPI document by hash — `tfpfgen spec import`. The imported document is immutable evidence of what the vendor published. |
Expand Down
14 changes: 7 additions & 7 deletions handoff.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ For `<entity>` under `internal/services/resources/<group>/v1/<entity>/`:
| File | Purpose |
|---|---|
| `resource_acceptance_test.go` | the live lifecycle: **step 1** apply minimal, **step 2** import + `ImportStateVerify`, **step 3** apply maximal |
| `tests/terraform/acceptance/resource_{minimal,maximal}.tf` | what the live steps apply — **built from recorded bodies** |
| `tests/terraform/acceptance/resource_{minimal,maximal}.tf` | what the live steps apply — **built from recorded request bodies** |
| `tests/terraform/unit/resource_{minimal,maximal}.tf` | what the unit tests apply — **still derived from the document** |
| `tests/responses/resource_{minimal,maximal}.json` | wire JSON the mocks answer with |
| `mocks/responders.go` | httpmock responders built from those |
Expand All @@ -94,7 +94,7 @@ Under the provider tree:
| Path | What |
|---|---|
| `audit/observations/<entity>.observations.json` | one fact per property |
| `audit/bodies/<entity>.bodies.json` | **the accepted create bodies** — request, response, status |
| `audit/request_bodies/<entity>.request_bodies.json` | **the accepted create request bodies** — request, response, status |
| `audit/inputs.json` | operator-supplied values the probe cannot invent (authored) |
| `spec/corrections/*.correction.json` | accepted corrections (authored) |
| `spec/corrections/proposed/` | awaiting a human decision |
Expand All @@ -111,11 +111,11 @@ OpenAPI doc ──> specmodel ──> IR ──> sdkbind ──> emit ──> pr
^ |
| v
corrections <── revise <── observations <── audit (live probe)
+ bodies
+ request bodies
```

Presence (`required` / `optional` / `computed`) is corrected into the document
and re-derived. **Values are not.** Values come from `audit/bodies/` and are
and re-derived. **Values are not.** Values come from `audit/request_bodies/` and are
replayed directly into acceptance fixtures. That split is the point: deriving
values again from the document is what produced a year of one-field-at-a-time
failures (`icon`, `match_type`, `filters` were all the same bug).
Expand All @@ -127,8 +127,8 @@ failures (`icon`, `match_type`, `filters` were all the same bug).
| Additive minimal search (add a field until 2xx) | `internal/audit/run/steps_create.go: searchMinimal` |
| Subtractive maximal reduction (drop until 2xx) | `internal/audit/run/steps_create.go: reduceMaximal` |
| Refusal grammar (what a 4xx names) | `internal/audit/run/adjust.go: classifyRefusal` |
| Recorded bodies artifact | `internal/audit/observe/bodies.go` |
| Replaying a body into a fixture | `internal/fixtures/fixtures.go: FromAcceptedBody` |
| Recorded request bodies artifact | `internal/audit/observe/request_bodies.go` |
| Replaying a request body into a fixture | `internal/fixtures/fixtures.go: FromAcceptedRequestBody` |
| Per-run unique names | `internal/fixtures/fixtures.go: WithRunSuffix`, `RunSuffixBlock` |
| Acceptance vs unit split | `internal/emit/render_fixtures.go: resourceFixtures` |
| Probe step ordering | `internal/audit/strategy/program.go: buildProgram` |
Expand Down Expand Up @@ -182,7 +182,7 @@ probe**. Everything else is downstream of that.
(`tests_ftp_server`), a dashboard id (`dashboard_snapshot`), a discriminator
`type` for `connectors_generic` / `operations_webhook`.
3. **The four inconsistent-result failures.** A value sent and echoed back
differently. `FromAcceptedBody` already drops what is never returned; this is
differently. `FromAcceptedRequestBody` already drops what is never returned; this is
the narrower case of a value the API rewrites.
4. **`Invalid id` / `request never completed`** — one each, `templates_sharing_setting`
(a singleton) and `stream` (`lastSuccess`/`lastFailure` declared `int64`, the
Expand Down
8 changes: 4 additions & 4 deletions internal/audit/infer/evidence.go
Original file line number Diff line number Diff line change
Expand Up @@ -96,10 +96,10 @@ type ConditionalValue struct {
type Evidence struct {
// Entity is the classified entity key the evidence is about.
Entity string
// AcceptedBodies is every create body the API accepted, resolved as
// AcceptedRequestBodies is every create body the API accepted, resolved as
// sent. Their gate values and field sets are the positive half of
// variant diffing.
AcceptedBodies []map[string]any
AcceptedRequestBodies []map[string]any
// Adjustments is every body change the executor was forced to make.
Adjustments []RequestAdjustment
// CombinedRefusals lists field pairs a create was refused for carrying
Expand Down Expand Up @@ -164,9 +164,9 @@ func gateOf(compiled *strategy.Strategy) gate {
// value that body pinned, returning value -> set of field names present
// (excluding the gate field itself). A body with no gate value lands under the
// empty string.
func acceptedUnder(bodies []map[string]any, gateField string) map[string]map[string]bool {
func acceptedUnder(requestBodies []map[string]any, gateField string) map[string]map[string]bool {
out := map[string]map[string]bool{}
for _, body := range bodies {
for _, body := range requestBodies {
val := ""
if gateField != "" {
if raw, ok := body[gateField]; ok {
Expand Down
4 changes: 2 additions & 2 deletions internal/audit/infer/infer.go
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ type model struct {

func newModel(ev Evidence, compiled *strategy.Strategy) *model {
g := gateOf(compiled)
accepted := acceptedUnder(ev.AcceptedBodies, g.field)
accepted := acceptedUnder(ev.AcceptedRequestBodies, g.field)
created := make([]string, 0, len(accepted))
for v := range accepted {
if v != "" {
Expand Down Expand Up @@ -466,7 +466,7 @@ func (m *model) removedValues(f string) []string {
// acceptedAlone reports whether field a appeared in an accepted body that did
// not also carry field b.
func (m *model) acceptedAlone(a, b string) bool {
for _, body := range m.ev.AcceptedBodies {
for _, body := range m.ev.AcceptedRequestBodies {
if _, hasA := body[a]; !hasA {
continue
}
Expand Down
38 changes: 19 additions & 19 deletions internal/audit/infer/infer_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ func monitorEvidence() Evidence {
}
return Evidence{
Entity: "monitor",
AcceptedBodies: []map[string]any{
AcceptedRequestBodies: []map[string]any{
{"kind": "ping", "interval": 5.0, "target_host": "h", "name": "n"},
{"kind": "web", "interval": 5.0, "web": map[string]any{"url": "u"}, "name": "n"},
{"kind": "dns", "interval": 5.0, "domain": "d", "dnssec": true, "name": "n"},
Expand Down Expand Up @@ -176,7 +176,7 @@ func TestUnit_Infer_PlantedFalseEdgeIsInconclusive(t *testing.T) {
}
ev := Evidence{
Entity: "monitor",
AcceptedBodies: []map[string]any{
AcceptedRequestBodies: []map[string]any{
{"kind": "ping", "foo": "x"},
{"kind": "dns", "foo": "y"}, // foo accepted under BOTH values: not gated
},
Expand All @@ -203,8 +203,8 @@ func TestUnit_Infer_LoneAmbiguousRemovalAssertsNothing(t *testing.T) {
Variants: []strategy.Variant{{}, {GateField: "kind", GateValue: "ping"}, {GateField: "kind", GateValue: "dns"}},
}
ev := Evidence{
Entity: "monitor",
AcceptedBodies: []map[string]any{{"kind": "ping"}},
Entity: "monitor",
AcceptedRequestBodies: []map[string]any{{"kind": "ping"}},
Adjustments: []RequestAdjustment{
{Entity: "monitor", Action: AdjustRemove, Field: "bar", GateField: "kind", GateValue: "dns"},
},
Expand All @@ -223,8 +223,8 @@ func TestUnit_Infer_ConflictingAcceptanceAndRemovalIsNoEdge(t *testing.T) {
Variants: []strategy.Variant{{}, {GateField: "kind", GateValue: "a"}, {GateField: "kind", GateValue: "b"}},
}
ev := Evidence{
Entity: "e",
AcceptedBodies: []map[string]any{{"kind": "a", "foo": 1.0}},
Entity: "e",
AcceptedRequestBodies: []map[string]any{{"kind": "a", "foo": 1.0}},
Adjustments: []RequestAdjustment{
{Entity: "e", Action: AdjustRemove, Field: "foo", GateField: "kind", GateValue: "a"},
},
Expand All @@ -240,7 +240,7 @@ func TestUnit_Infer_MutuallyExclusive(t *testing.T) {
t.Parallel()
ev := Evidence{
Entity: "widget",
AcceptedBodies: []map[string]any{
AcceptedRequestBodies: []map[string]any{
{"a": 1.0, "name": "n"},
{"b": 2.0, "name": "n"},
},
Expand All @@ -262,9 +262,9 @@ func TestUnit_Infer_MutuallyExclusive(t *testing.T) {
func TestUnit_Infer_MutuallyExclusiveNeedsBothAlone(t *testing.T) {
t.Parallel()
ev := Evidence{
Entity: "widget",
AcceptedBodies: []map[string]any{{"a": 1.0, "b": 2.0}}, // never a without b
CombinedRefusals: []FieldPair{{A: "a", B: "b"}},
Entity: "widget",
AcceptedRequestBodies: []map[string]any{{"a": 1.0, "b": 2.0}}, // never a without b
CombinedRefusals: []FieldPair{{A: "a", B: "b"}},
}
if o := find(Infer(ev, &strategy.Strategy{Entity: "widget"}), "", observe.KindMutuallyExclusive); o != nil {
t.Fatalf("asserted %+v without evidence each is valid alone", o)
Expand All @@ -276,8 +276,8 @@ func TestUnit_Infer_MutuallyExclusiveNeedsBothAlone(t *testing.T) {
func TestUnit_Infer_FlatResourceHasNoVariantEdges(t *testing.T) {
t.Parallel()
ev := Evidence{
Entity: "assignment",
AcceptedBodies: []map[string]any{{"name": "n", "agent_id": "agent-1"}},
Entity: "assignment",
AcceptedRequestBodies: []map[string]any{{"name": "n", "agent_id": "agent-1"}},
Adjustments: []RequestAdjustment{
{Entity: "assignment", Action: AdjustBorrow, Field: "agent_id", GateField: "agent"},
},
Expand Down Expand Up @@ -305,7 +305,7 @@ func TestUnit_Infer_ValidConfigurationNeedsADistinguishingField(t *testing.T) {
}
ev := Evidence{
Entity: "e",
AcceptedBodies: []map[string]any{
AcceptedRequestBodies: []map[string]any{
{"kind": "a", "name": "n"},
{"kind": "b", "name": "n"},
},
Expand All @@ -327,8 +327,8 @@ func TestUnit_Infer_HypothesisProvenanceIsCarried(t *testing.T) {
},
}
ev := Evidence{
Entity: "e",
AcceptedBodies: []map[string]any{{"kind": "a", "foo": 1.0}, {"kind": "b"}},
Entity: "e",
AcceptedRequestBodies: []map[string]any{{"kind": "a", "foo": 1.0}, {"kind": "b"}},
Adjustments: []RequestAdjustment{
{Entity: "e", Action: AdjustRemove, Field: "foo", GateField: "kind", GateValue: "b"},
},
Expand Down Expand Up @@ -454,7 +454,7 @@ func TestUnit_Infer_ValueConditionalConfiguration(t *testing.T) {
}
ev := Evidence{
Entity: "stream",
AcceptedBodies: []map[string]any{
AcceptedRequestBodies: []map[string]any{
{"format": "avro", "mode": "streaming", "name": "n"},
{"format": "json", "mode": "batch", "name": "n"},
},
Expand Down Expand Up @@ -486,7 +486,7 @@ func TestUnit_Infer_ValueConditionalNeedsBothDirections(t *testing.T) {
}
ev := Evidence{
Entity: "stream",
AcceptedBodies: []map[string]any{
AcceptedRequestBodies: []map[string]any{
{"format": "avro", "mode": "streaming", "name": "n"},
{"format": "json", "mode": "streaming", "name": "n"},
},
Expand All @@ -511,8 +511,8 @@ func TestUnit_Infer_ValueConditionalNeedsTwoCreatedValues(t *testing.T) {
Variants: []strategy.Variant{{}, {GateField: "format", GateValue: "avro"}, {GateField: "format", GateValue: "json"}},
}
ev := Evidence{
Entity: "stream",
AcceptedBodies: []map[string]any{{"format": "avro", "mode": "streaming", "name": "n"}},
Entity: "stream",
AcceptedRequestBodies: []map[string]any{{"format": "avro", "mode": "streaming", "name": "n"}},
ConditionalValues: []ConditionalValue{
{GateField: "format", GateValue: "avro", Field: "mode", Value: "batch", Accepted: false},
{GateField: "format", GateValue: "json", Field: "mode", Value: "batch", Accepted: true},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,34 +16,36 @@ import (
"sort"
)

// encodeBodies renders one entity's record deterministically, matching the
// observations beside it: sorted map keys, no HTML escaping, two-space indent.
func encodeBodies(b Bodies) ([]byte, error) {
// encodeRequestBodies renders one entity's record deterministically,
// matching the observations beside it: sorted map keys, no HTML escaping,
// two-space indent.
func encodeRequestBodies(b RequestBodies) ([]byte, error) {
var buf bytes.Buffer
enc := json.NewEncoder(&buf)
enc.SetEscapeHTML(false)
enc.SetIndent("", " ")
if err := enc.Encode(b); err != nil {
return nil, fmt.Errorf("encoding bodies for %s: %w", b.Entity, err)
return nil, fmt.Errorf("encoding request bodies for %s: %w", b.Entity, err)
}
return buf.Bytes(), nil
}

// BodiesSuffix is the committed file naming, one file per entity, matching
// the observations beside it.
const BodiesSuffix = ".bodies.json"
// RequestBodiesSuffix is the committed file naming, one file per entity,
// matching the observations beside it.
const RequestBodiesSuffix = ".request_bodies.json"

// Bodies is what one entity's creates looked like when the API accepted them.
type Bodies struct {
// RequestBodies is what one entity's creates looked like when the API
// accepted them.
type RequestBodies struct {
Entity string `json:"entity"`
// Minimal is the smallest create the run got accepted, and Maximal the
// fullest. Either may be absent when no create of that shape succeeded.
Minimal *AcceptedBody `json:"minimal,omitempty"`
Maximal *AcceptedBody `json:"maximal,omitempty"`
Minimal *AcceptedRequestBody `json:"minimal,omitempty"`
Maximal *AcceptedRequestBody `json:"maximal,omitempty"`
}

// AcceptedBody is one create the API answered 2xx to.
type AcceptedBody struct {
// AcceptedRequestBody is one create the API answered 2xx to.
type AcceptedRequestBody struct {
// Status is the code the API answered, kept so a reader can see that this
// was an acceptance rather than an assumption.
Status int `json:"status"`
Expand All @@ -61,23 +63,23 @@ type AcceptedBody struct {
// A field the API accepts and never echoes cannot appear in a generated
// configuration: terraform compares what it planned against what the provider
// answers, and a value that never comes back reads as the provider losing it.
func (b *AcceptedBody) Echoed(wire string) bool {
func (b *AcceptedRequestBody) Echoed(wire string) bool {
if b == nil || b.Response == nil {
return false
}
_, ok := b.Response[wire]
return ok
}

// WriteBodies commits one <entity>.bodies.json per entity under dir.
// Encoding matches the observations: sorted keys, stable bytes, so a re-run
// that learned nothing new rewrites nothing.
func WriteBodies(dir string, bodies []Bodies) error {
if len(bodies) == 0 {
// WriteRequestBodies commits one <entity>.request_bodies.json per entity
// under dir. Encoding matches the observations: sorted keys, stable bytes,
// so a re-run that learned nothing new rewrites nothing.
func WriteRequestBodies(dir string, requestBodies []RequestBodies) error {
if len(requestBodies) == 0 {
return nil
}
byEntity := map[string]Bodies{}
for _, b := range bodies {
byEntity := map[string]RequestBodies{}
for _, b := range requestBodies {
if b.Entity == "" || (b.Minimal == nil && b.Maximal == nil) {
continue
}
Expand All @@ -91,7 +93,7 @@ func WriteBodies(dir string, bodies []Bodies) error {

encoded := make(map[string][]byte, len(entities))
for _, entity := range entities {
raw, err := encodeBodies(byEntity[entity])
raw, err := encodeRequestBodies(byEntity[entity])
if err != nil {
return err
}
Expand All @@ -101,19 +103,20 @@ func WriteBodies(dir string, bodies []Bodies) error {
return fmt.Errorf("creating %s: %w", dir, err)
}
for _, entity := range entities {
path := filepath.Join(dir, entity+BodiesSuffix)
path := filepath.Join(dir, entity+RequestBodiesSuffix)
if err := os.WriteFile(path, encoded[entity], 0o644); err != nil {
return fmt.Errorf("writing %s: %w", path, err)
}
}
return nil
}

// ReadBodies loads every recorded body under dir, keyed by entity. A missing
// directory is not an error: an entity the probe never cleared has none, and
// generation falls back to deriving values from the document.
func ReadBodies(dir string) (map[string]Bodies, error) {
out := map[string]Bodies{}
// ReadRequestBodies loads every recorded request body under dir, keyed by
// entity. A missing directory is not an error: an entity the probe never
// cleared has none, and generation falls back to deriving values from the
// document.
func ReadRequestBodies(dir string) (map[string]RequestBodies, error) {
out := map[string]RequestBodies{}
entries, err := os.ReadDir(dir)
if err != nil {
if os.IsNotExist(err) {
Expand All @@ -129,7 +132,7 @@ func ReadBodies(dir string) (map[string]Bodies, error) {
if err != nil {
return nil, fmt.Errorf("reading %s: %w", e.Name(), err)
}
var b Bodies
var b RequestBodies
if err := json.Unmarshal(raw, &b); err != nil {
return nil, fmt.Errorf("reading %s: %w", e.Name(), err)
}
Expand Down
Loading