Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,9 @@ Set by the repository owner; not open to per-PR relitigating:
- **Never commit generated pilot output or binaries.** Generated provider trees
live in provider repos; this repo holds only the machinery and its fixtures.
- **Every verb keeps its exit-code contract** documented in `docs/contract.md`.
- **Corpus documents are pinned by hash and fetched, not vendored.**
- **Vendor OpenAPI specs are pinned by hash and fetched, not committed.**
A pin names bytes, so a document that moves under it fails loudly rather than
quietly changing what a test means.
- **No hand-written file over 800 lines.** Decompose by protocol instead.
Generated and fixture code under `testdata/` is exempt — it is produced,
not maintained, and reproducing it is cheap.
Expand Down
2 changes: 1 addition & 1 deletion docs/comment-style.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ These are facts, and they are the point of the comment.
status, a framework type name. They are what makes the surrounding claim
checkable.
- **Indented tabular blocks.** They render as godoc code blocks and usually carry
more than the prose around them. See `internal/corpus/cache.go` (the cache
more than the prose around them. See `internal/vendor_openapi_specs/cache.go` (the cache
layout) and `internal/sdkbind/binder_kiota.go` (document shape → builder
chain). Keep the table, rewrite the prose framing it.
- **Cross-references.** Prefer paths and symbols over URLs: `docs/contract.md`,
Expand Down
2 changes: 1 addition & 1 deletion docs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ sweep, doctor, facts, rehearsal, curate) is retired and may not reappear.
| **authored** | A committed data path generation may never write: tfpfgen.yaml, corrections, inputs. Enforced by the manifest, not by convention. There are no authored *code* files — provider repos are 100% generated code. |
| **manifest** | The ledger of every derived file (path, digest, source, origin) and every authored path. `manifest.json` at the provider-repo root. |
| **quirkserver** | The deliberately-misbehaving stub API that serves as offline ground truth for audit logic and as the fake live API in pipeline rehearsals. |
| **corpus** | Third-party OpenAPI documents pinned by SHA-256 and fetched at test time, never vendored. |
| **vendor OpenAPI specs** | The third-party OpenAPI documents this toolkit's own tests parse and derive — a vendor's document exactly as published, pinned by SHA-256 and fetched at test time, never committed. Distinct from **spec**, which is the one document a provider is generated from: these are only ever read, never imported, corrected or revised. `internal/vendor_openapi_specs`, pinned in `vendor_openapi_specs.lock.json`, cached under `TFPFGEN_VENDOR_OPENAPI_SPECS_DIR`. The earlier name *corpus* is retired: it named a collection without saying whose documents or what they are for. |
| **backend** | An SDK generator behind the common interface: `kiota` or `openapi-generator`. Exactly one per provider repo. |
| **intermediate representation** | The ephemeral, never-committed derivation (`internal/intermediate_representation`) recomputed from the revised spec and config on every generation run; its model vocabulary (Model, Resource, Datasource, ListResource, Action, AttributeTree, ComputedOptionalRequired, Operation, Names) is approved. Every identifier in the package is fully worded — no abbreviated type, field, function, parameter or local. `Operation`/`Operations` replace the earlier `Op`/`Ops`, `AttributeType` replaces `TypeKind`, `Parameter`/`PathParameters` replace `Param`/`PathParams`, and `APIVersionDirectory` replaces `APIVersionDir`. `OneOf` is the one deliberate exception: it is named for the `stringvalidator.OneOf` it generates, and generated things are spelt the way HashiCorp spells them. `Datasource` likewise stays one word here, per the fixed spelling below. |
| **ComputedOptionalRequired** | How an attribute participates in a plan, and the four values it takes: `required`, `optional`, `computed`, `computed_optional`. The name and the values are [terraform-plugin-codegen-spec](https://github.com/hashicorp/terraform-plugin-codegen-spec)'s (`schema.ComputedOptionalRequired`), adopted so the toolkit and the specification that could describe its output call the same fact by the same name. It replaces the earlier `Presence` / `PresenceRequired` / `PresenceOptional` / `PresenceComputed` / `PresenceOptionalComputed`, and the value `optional-computed`; those spellings are retired. |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,17 @@ import (
"strings"
"testing"

"github.com/deploymenttheory/terraform-plugin-framework-codegen/internal/corpus"
"github.com/deploymenttheory/terraform-plugin-framework-codegen/internal/specmodel"
"github.com/deploymenttheory/terraform-plugin-framework-codegen/internal/vendor_openapi_specs"
)

// The inline fixtures prove each rule in isolation; this proves derivation
// against a real vendor document, where the shapes were not chosen to pass.
// Skips when the pinned document is not cached and cannot be fetched,
// unless TFPFGEN_CORPUS_REQUIRED says the machine must be honest — the
// same split every corpus-backed test in this repo follows.
// unless TFPFGEN_VENDOR_OPENAPI_SPECS_REQUIRED says the machine must be honest — the
// same split every pinned-document test in this repo follows.
func TestIntegration_IntermediateRepresentation_DerivesAPinnedVendorDocument(t *testing.T) {
path := corpus.SpecPath(t, "thousandeyes")
path := vendor_openapi_specs.SpecPath(t, "thousandeyes")
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("reading %s: %v", path, err)
Expand Down
2 changes: 1 addition & 1 deletion internal/specmodel/describe.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package specmodel
// Describe reports what a document says about itself: the version its info
// object declares, and how much surface it carries.
//
// It satisfies corpus.Describer, which is where a pin's recorded counts come
// It satisfies vendor_openapi_specs.Describer, which is where a pin's recorded counts come
// from. Those counts are what catch a truncated download that happens to
// parse, so a document this cannot read reports zero rather than a guess.
func Describe(doc []byte) (version string, paths, operations int) {
Expand Down
2 changes: 1 addition & 1 deletion internal/specmodel/describe_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package specmodel
import "testing"

// TestUnit_Specmodel_DescribeMeasuresWhatAPinRecords proves Describe reports
// the three values a corpus pin is written from, and reports nothing rather
// the three values a pin is written from, and reports nothing rather
// than a guess for bytes it cannot read — a truncated download must not pass
// as a document with a plausible shape.
func TestUnit_Specmodel_DescribeMeasuresWhatAPinRecords(t *testing.T) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,16 @@ import (
"os"
"testing"

"github.com/deploymenttheory/terraform-plugin-framework-codegen/internal/corpus"
"github.com/deploymenttheory/terraform-plugin-framework-codegen/internal/vendor_openapi_specs"
)

// The inline fixtures above prove each rule in isolation; this proves the
// loader against a real vendor document, where the shapes were not chosen
// to pass. Skips when the pinned document is not cached and cannot be
// fetched, unless TFPFGEN_CORPUS_REQUIRED says the machine must be honest —
// the same split every corpus-backed test in this repo follows.
// fetched, unless TFPFGEN_VENDOR_OPENAPI_SPECS_REQUIRED says the machine must be honest —
// the same split every pinned-document test in this repo follows.
func TestIntegration_Specmodel_LoadsAPinnedVendorDocument(t *testing.T) {
path := corpus.SpecPath(t, "thousandeyes")
path := vendor_openapi_specs.SpecPath(t, "thousandeyes")
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("reading %s: %v", path, err)
Expand All @@ -24,7 +24,7 @@ func TestIntegration_Specmodel_LoadsAPinnedVendorDocument(t *testing.T) {
t.Fatalf("Load: %v", err)
}

pin := corpus.MustPin(t, "thousandeyes")
pin := vendor_openapi_specs.MustPin(t, "thousandeyes")
if doc.Info.Version != pin.Version {
t.Errorf("info.version = %q, want the pinned %q", doc.Info.Version, pin.Version)
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package corpus
package vendor_openapi_specs

// The cache layout, one directory per pinned document:
//
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package corpus
package vendor_openapi_specs

import (
"errors"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package corpus
package vendor_openapi_specs

import (
"context"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package corpus
package vendor_openapi_specs

import (
"net/http"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package corpus
package vendor_openapi_specs

import (
"crypto/sha256"
Expand All @@ -11,11 +11,11 @@ import (
)

// LockFile is the pin set's name.
const LockFile = "corpus.lock.json"
const LockFile = "vendor_openapi_specs.lock.json"

// EnvLockPath relocates the lock for a pin update invoked from outside the
// repository root.
const EnvLockPath = "TFPFGEN_CORPUS_LOCK"
const EnvLockPath = "TFPFGEN_VENDOR_OPENAPI_SPECS_LOCK"

// LockPath is where RewriteLock writes.
//
Expand All @@ -26,7 +26,7 @@ func LockPath() string {
if p := os.Getenv(EnvLockPath); p != "" {
return p
}
return filepath.Join("internal", "corpus", "testdata", LockFile)
return filepath.Join("internal", "vendor_openapi_specs", "testdata", LockFile)
}

// Upstream is what a source is serving now, measured against a pin.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package corpus
package vendor_openapi_specs

import (
"encoding/json"
Expand All @@ -19,7 +19,7 @@ func TestUnit_Refresh_LockPathHonoursTheEnvironment(t *testing.T) {
}

t.Setenv(EnvLockPath, "")
if got, want := LockPath(), filepath.Join("internal", "corpus", "testdata", LockFile); got != want {
if got, want := LockPath(), filepath.Join("internal", "vendor_openapi_specs", "testdata", LockFile); got != want {
t.Errorf("LockPath() = %q, want %q", got, want)
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package corpus
package vendor_openapi_specs

import (
"errors"
Expand All @@ -17,7 +17,7 @@ type TB interface {

// Document returns the pinned document, or ends the test.
//
// Skipping when offline and failing when TFPFGEN_CORPUS_REQUIRED is set is a
// Skipping when offline and failing when TFPFGEN_VENDOR_OPENAPI_SPECS_REQUIRED is set is a
// deliberate split rather than a hedge. Failing always would turn a
// developer's flaky connection into a red suite, which teaches people to
// ignore red. Skipping always would let CI skip its way to green, which is a
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
package corpus
package vendor_openapi_specs

import (
"fmt"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Package corpus acquires the real third-party API documents the tests run
// Package vendor_openapi_specs acquires the real third-party API documents the tests run
// against, without committing any of them to this repository.
//
// This repository turns OpenAPI documents into providers; it is not the home
Expand All @@ -7,7 +7,7 @@
// other, not by synthetic ones that agree with the code -- so the documents
// are fetched at test time and pinned here rather than vendored.
//
// Pinned means pinned. testdata/corpus.lock.json pins a version, a SHA-256 and
// Pinned means pinned. testdata/vendor_openapi_specs.lock.json pins a version, a SHA-256 and
// the path and operation counts for every document, and a fetch that does not
// match all of them fails loudly. That matters more than it looks: several
// tests assert properties of these documents -- enum members, field shapes,
Expand All @@ -18,7 +18,7 @@
// The cache holds one verified copy of each pinned document, in a directory
// named deterministically from the pin so a pinned document is findable again
// rather than merely present. See cache.go for the layout.
package corpus
package vendor_openapi_specs

import (
"crypto/sha256"
Expand All @@ -38,18 +38,18 @@ import (
// resolves it identically. It is the reviewable file of this whole scheme:
// every change to what the tests read against shows up as a diff here.
//
//go:embed testdata/corpus.lock.json
//go:embed testdata/vendor_openapi_specs.lock.json
var lockBytes []byte

// EnvCacheDir relocates the cache. CI sets it so the cache lands somewhere
// actions/cache can restore.
const EnvCacheDir = "TFPFGEN_CORPUS_DIR"
const EnvCacheDir = "TFPFGEN_VENDOR_OPENAPI_SPECS_DIR"

// EnvRequired makes a missing document a failure rather than a skip. Set in
// CI, unset on a developer's machine: see the Document test helper.
const EnvRequired = "TFPFGEN_CORPUS_REQUIRED"
const EnvRequired = "TFPFGEN_VENDOR_OPENAPI_SPECS_REQUIRED"

// cacheSubdir is the corpus's place within the user cache directory.
// cacheSubdir is these documents' place within the user cache directory.
//
// Deliberately outside the repository, and this is not a preference. A
// relative default is resolved against the working directory, and `go test`
Expand All @@ -59,15 +59,15 @@ const EnvRequired = "TFPFGEN_CORPUS_REQUIRED"
// each.
//
// A cache belongs where the operating system puts caches. CI overrides it with
// TFPFGEN_CORPUS_DIR to somewhere its cache action can restore.
var cacheSubdir = filepath.Join("tfpfgen", "corpus")
// TFPFGEN_VENDOR_OPENAPI_SPECS_DIR to somewhere its cache action can restore.
var cacheSubdir = filepath.Join("tfpfgen", "vendor_openapi_specs")

// ErrOffline reports that a document is not cached and could not be fetched.
// Callers decide whether that is fatal; the Document test helper applies the
// policy.
var ErrOffline = errors.New("the document is not cached and could not be fetched")

// Lock is corpus.lock.json.
// Lock is vendor_openapi_specs.lock.json.
type Lock struct {
FormatVersion string `json:"formatVersion"`
OpenAPI map[string]Pin `json:"openapi"`
Expand Down Expand Up @@ -116,11 +116,11 @@ var (
func LoadLock() (Lock, error) {
lockOnce.Do(func() {
if err := json.Unmarshal(lockBytes, &lock); err != nil {
lockErr = fmt.Errorf("parsing corpus.lock.json: %w", err)
lockErr = fmt.Errorf("parsing vendor_openapi_specs.lock.json: %w", err)
return
}
if len(lock.OpenAPI) == 0 {
lockErr = errors.New("corpus.lock.json pins no documents")
lockErr = errors.New("vendor_openapi_specs.lock.json pins no documents")
}
})
return lock, lockErr
Expand All @@ -138,7 +138,7 @@ func PinFor(id string) (Pin, error) {
for k := range l.OpenAPI {
known = append(known, k)
}
return Pin{}, fmt.Errorf("corpus.lock.json pins no document %q (it pins %v)", id, known)
return Pin{}, fmt.Errorf("vendor_openapi_specs.lock.json pins no document %q (it pins %v)", id, known)
}
return p, nil
}
Expand All @@ -156,7 +156,7 @@ func IDs() ([]string, error) {
return out, nil
}

// CacheDir is the corpus cache root.
// CacheDir is the cache root these documents are materialised under.
func CacheDir() string {
if d := os.Getenv(EnvCacheDir); d != "" {
return d
Expand Down Expand Up @@ -281,7 +281,7 @@ func publish(root, name string, doc []byte, pin Pin, source string) (CachedDocum
return CachedDocument{}, fmt.Errorf("creating %s: %w", filepath.Dir(root), err)
}

staging, err := os.MkdirTemp(filepath.Dir(root), ".corpus-staging-*")
staging, err := os.MkdirTemp(filepath.Dir(root), ".vendor-openapi-specs-staging-*")
if err != nil {
return CachedDocument{}, fmt.Errorf("creating a staging directory: %w", err)
}
Expand Down Expand Up @@ -342,7 +342,7 @@ func verifyAgainstPin(id string, doc []byte, pin Pin, source string) error {
Tests assert properties of the pinned document -- enum members, field shapes,
operation counts -- so this changes what they mean rather than merely failing a
transport check. Review the change deliberately and update the %s pin in
internal/corpus/testdata/corpus.lock.json`,
internal/vendor_openapi_specs/testdata/vendor_openapi_specs.lock.json`,
id, source,
shortSHA(pin.SHA256), pin.Version, pin.PathCount, pin.OperationCount,
shortSHA(digest), version, paths, operations,
Expand Down
Loading