Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions .github/workflows/apply-custom-properties.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,17 +48,20 @@ jobs:
env:
GOTOOLCHAIN: local
run: go -C scripts/custom-properties run . --root ../.. --check
- name: Select validated readback repositories
id: readback-repositories
run: jq -r '"repositories=" + (.repositories | keys | join(","))' custom-properties/catalogue.json >>"$GITHUB_OUTPUT"
- name: Get read-only portfolio property token
id: readback-token
uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0
with:
client-id: ${{ vars.APP_CLIENT_ID }}
private-key: ${{ secrets.APP_PRIVATE_KEY }}
owner: devantler-tech
repositories: ${{ steps.readback-repositories.outputs.repositories }}
# Read only: organization property values and repository metadata. A
# deliberate, scoped exception to zizmor's github-app audit, like the
# repository coverage check's: a token limited to the catalogue is
# refused while a declared repository is not created yet, which stops
# the schema reaching GitHub (maintainer direction 2026-11-13). The
# reconciler fails closed unless the installation covers all
# repositories and its listing is complete.
owner: devantler-tech # zizmor: ignore[github-app]
permission-organization-custom-properties: read
permission-metadata: read
- name: Get narrowly scoped organization token
Expand Down
33 changes: 21 additions & 12 deletions custom-properties/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,11 @@ unexpected property is preserved and stops reconciliation for an ownership revie
## Ownership and reconciliation

Repository actors retain the existing permission to set values; Crossplane restores the reviewed
catalogue. Required values remain explicit rather than relying on defaults. Defaults are only the
GitHub fallback and do not establish compliance.
catalogue. The properties are required but do not demand an explicit value when a repository is
created: neither GitHub's create-from-template API nor the provider's `Repository` resource can pass
property values, so demanding one stops Crossplane creating any repository. A new repository holds
the default until its `RepositoryCustomProperty` resources write the catalogue's explicit values.
Defaults are only that fallback and do not establish compliance.

GitHub rejects independent property creation when other required explicit values are missing.
The reviewed-main workflow therefore writes each repository's complete assignment atomically,
Expand All @@ -37,16 +40,22 @@ retiring the unused definition. Repository names, organization ownership and uni
must agree; missing private entries or a failed later page block retirement. The native apply proved
that the organization writer can initialize every assignment while restricted to this repository,
but its repository selection made the organization readback incomplete. The writer stays restricted.
A separate, short-lived read-only token selects exactly the validated catalogue and supplies only
the organization-value readback. It explicitly requests both organization custom-property read
authority and repository metadata read access for the selected repositories, with no repository
content access. Repository selection alone does not request metadata authority when the token's
permissions are explicitly limited. Neither that token nor its alternate credential is passed to schema or value mutations.
The action revokes both tokens at job completion.

The readback selection is exactly the reviewed catalogue; changing that selection requires
maintainer direction, rather than an automatic response to incomplete results. Native full-portfolio readback,
including private entries, remains required after deployment; offline tests do not establish it.
A separate, short-lived read-only token covers the whole App installation and supplies only the
installation's repository listing and the organization-value readback. It explicitly requests
organization custom-property read authority and repository metadata read access, with no
repository content access. Neither that token nor its alternate credential is passed to schema or
value mutations. The action revokes both tokens at job completion.

The token covers the whole installation, by maintainer direction on 2026-11-13, so that a
repository the catalogue declares before Crossplane creates it cannot stop the run. A token
limited to the catalogue fails outright while one of its repositories is missing, and the schema
then never reaches GitHub. The reconciler lists the installation's repositories first. A
catalogue repository missing from that listing is printed as `PENDING` and gets no value write or
readback; its `RepositoryCustomProperty` resources write its values once Crossplane creates it.
The listing proves absence only when it is complete, covers all repositories and includes this
one; anything else stops the run before the first write. Native full-portfolio readback,
including private entries, remains required for every created repository after deployment;
offline tests do not establish it.
Repeated runs reassert the same complete catalogue; an effective default alone never replaces a
successful explicit write.

Expand Down
6 changes: 3 additions & 3 deletions custom-properties/catalogue.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"property_name": "Type",
"value_type": "single_select",
"required": true,
"require_explicit_values": true,
"require_explicit_values": false,
"default_value": "Application",
"description": "Portfolio purpose for catalogue navigation; use native GitHub metadata for language, visibility and template status.",
"allowed_values": [
Expand All @@ -21,7 +21,7 @@
"property_name": "EnableAutoMerge",
"value_type": "true_false",
"required": true,
"require_explicit_values": true,
"require_explicit_values": false,
"default_value": "true",
"description": "Select the existing automatic-merge workflow policy; never bypasses CI, review, draft or branch-protection gates.",
"values_editable_by": "org_and_repo_actors"
Expand All @@ -30,7 +30,7 @@
"property_name": "ScanGitHubActions",
"value_type": "true_false",
"required": true,
"require_explicit_values": true,
"require_explicit_values": false,
"default_value": "true",
"description": "Select the organization workflow-vulnerability scanning policy for this repository.",
"values_editable_by": "org_and_repo_actors"
Expand Down
76 changes: 72 additions & 4 deletions scripts/custom-properties/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ func validate(c catalogue) error {
}
seen := map[string]bool{}
for _, p := range c.Properties {
if !slices.Contains(names, p.Name) || seen[p.Name] || !p.Required || !p.Explicit || p.Description == "" || p.Editable != "org_and_repo_actors" || p.Source != "" {
if !slices.Contains(names, p.Name) || seen[p.Name] || !p.Required || p.Explicit || p.Description == "" || p.Editable != "org_and_repo_actors" || p.Source != "" {
return errors.New("invalid or duplicate property definition")
}
seen[p.Name] = true
Expand Down Expand Up @@ -116,7 +116,8 @@ func ghAPI(method, path string, body []byte) ([]byte, error) {
args = append(args, "--input", "-")
}
cmd := exec.Command("gh", args...)
readback := method == "GET" && strings.HasPrefix(path, "orgs/devantler-tech/properties/values?")
readback := method == "GET" && (strings.HasPrefix(path, "orgs/devantler-tech/properties/values?") ||
strings.HasPrefix(path, "installation/repositories?"))
readToken := os.Getenv("GH_PROPERTY_READBACK_TOKEN")
if readback && readToken == "" {
return nil, errors.New("portfolio readback requires its dedicated read-only credential")
Expand Down Expand Up @@ -489,6 +490,65 @@ func verifyValues(wanted map[string][]assignment, api apiFunc) error {
return errors.New("property readback exceeded its page bound")
}

// createdRepositories lists every repository the App installation can see. The
// read-only credential covers the whole installation, so a catalogue repository
// missing from a complete all-repositories listing has not been created yet. A
// selected-repositories installation or an incomplete listing proves nothing.
func createdRepositories(api apiFunc) (map[string]bool, error) {
created := map[string]bool{}
for page := 1; page <= 100; page++ {
b, err := api("GET", fmt.Sprintf("installation/repositories?per_page=100&page=%d", page), nil)
if err != nil {
return nil, err
}
var listing struct {
Total *int `json:"total_count"`
Selection string `json:"repository_selection"`
Repositories []struct {
Name string `json:"name"`
FullName string `json:"full_name"`
} `json:"repositories"`
}
if json.Unmarshal(b, &listing) != nil || listing.Total == nil || listing.Repositories == nil {
return nil, errors.New("unreadable installation repository listing")
}
if listing.Selection != "all" {
return nil, errors.New("the App installation is limited to selected repositories, so a missing repository proves nothing")
}
for _, r := range listing.Repositories {
if r.Name == "" || r.FullName != "devantler-tech/"+r.Name || created[r.Name] {
return nil, errors.New("ambiguous repository identity in installation listing")
}
created[r.Name] = true
}
if len(listing.Repositories) < 100 {
if len(created) != *listing.Total || !created[".github"] {
return nil, errors.New("installation repository listing is incomplete")
}
return created, nil
}
}
return nil, errors.New("installation repository listing exceeded its page bound")
}

// splitCreated keeps the catalogue repositories that exist. A repository that is
// declared but not created yet has no values to write or read back; its
// RepositoryCustomProperty resources write them once Crossplane creates it.
func splitCreated(c catalogue, created map[string]bool) (catalogue, []string) {
present := c
present.Repositories = map[string]repository{}
pending := []string{}
for name, r := range c.Repositories {
if created[name] {
present.Repositories[name] = r
} else {
pending = append(pending, name)
}
}
slices.Sort(pending)
return present, pending
}

func reconcile(c catalogue, api apiFunc) error {
if err := validate(c); err != nil {
return err
Expand All @@ -512,6 +572,14 @@ func reconcile(c catalogue, api apiFunc) error {
return errors.New("property type changed; nothing was applied")
}
}
created, err := createdRepositories(api)
if err != nil {
return err
}
present, pending := splitCreated(c, created)
for _, name := range pending {
fmt.Println("PENDING", name+": declared but not created yet; Crossplane writes its values when it creates the repository")
}
retirement := &legacyReviewRetirement{}
if err = noConsumers(api, c.Retire, retirement); err != nil {
return err
Expand Down Expand Up @@ -547,7 +615,7 @@ func reconcile(c catalogue, api apiFunc) error {
if err = verifyDesired(c, got, true); err != nil {
return err
}
if err = initializeValues(c, got, api); err != nil {
if err = initializeValues(present, got, api); err != nil {
return err
}
confirmation := &legacyReviewRetirement{}
Expand Down Expand Up @@ -586,7 +654,7 @@ func reconcile(c catalogue, api apiFunc) error {
return err
}
wanted := map[string][]assignment{}
for name, r := range c.Repositories {
for name, r := range present.Repositories {
wanted[name] = repositoryValues(c, r, got)
}
if err = verifyValues(wanted, api); err != nil {
Expand Down
69 changes: 68 additions & 1 deletion scripts/custom-properties/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ func TestGHAPIUsesReadOnlyCredentialOnlyForValueReadback(t *testing.T) {
t.Setenv("GH_PROPERTY_READBACK_TOKEN", "portfolio-reader")
for _, tc := range []struct{ method, path, want string }{
{"GET", "orgs/devantler-tech/properties/values?per_page=100&page=1", "portfolio-reader:unset"},
{"GET", "installation/repositories?per_page=100&page=1", "portfolio-reader:unset"},
{"GET", "orgs/devantler-tech/properties/schema", "schema-writer:unset"},
{"PATCH", "orgs/devantler-tech/properties/values", "schema-writer:unset"},
{"DELETE", "orgs/devantler-tech/properties/schema/LintDocumentation", "schema-writer:unset"},
Expand Down Expand Up @@ -250,7 +251,7 @@ func TestCatalogueRejectsMissingAndUnknownValues(t *testing.T) {
for _, mutation := range []func(*catalogue){
func(c *catalogue) { r := c.Repositories["ksail"]; r.AutoMerge = nil; c.Repositories["ksail"] = r },
func(c *catalogue) { r := c.Repositories["ksail"]; r.Type = "Unknown"; c.Repositories["ksail"] = r },
func(c *catalogue) { c.Properties[0].Explicit = false },
func(c *catalogue) { c.Properties[0].Explicit = true },
func(c *catalogue) { c.Properties = append(c.Properties, c.Properties[0]) },
func(c *catalogue) { c.Retire = append(c.Retire, "EnableAutoMerge") },
func(c *catalogue) { c.Retire = []string{"LintDocumentation"} },
Expand All @@ -269,13 +270,27 @@ type fakeAPI struct {
failRead, failWrite, corrupt bool
writes []string
values map[string][]map[string]string
missing map[string]bool // declared but not created yet
}

func (f *fakeAPI) call(method, path string, body []byte) ([]byte, error) {
if method == "GET" {
if f.failRead {
return nil, errors.New("unavailable")
}
if path == "installation/repositories?per_page=100&page=1" {
c, err := load("../../custom-properties/catalogue.json")
if err != nil {
return nil, err
}
repos := []map[string]string{}
for name := range c.Repositories {
if !f.missing[name] {
repos = append(repos, map[string]string{"name": name, "full_name": "devantler-tech/" + name})
}
}
return json.Marshal(map[string]any{"total_count": len(repos), "repository_selection": "all", "repositories": repos})
}
if path == "orgs/devantler-tech/properties/values?per_page=100&page=1" {
rows := []map[string]any{}
for name, values := range f.values {
Expand Down Expand Up @@ -313,6 +328,9 @@ func (f *fakeAPI) call(method, path string, body []byte) ([]byte, error) {
if len(request.Repositories) != 1 || len(request.Properties) == 0 {
return nil, errors.New("incomplete value request")
}
if f.missing[request.Repositories[0]] {
return nil, errors.New("repository not found")
}
for _, p := range f.schema {
if !p.Required || !p.Explicit {
continue
Expand Down Expand Up @@ -711,3 +729,52 @@ func TestReviewedMainAdmission(t *testing.T) {
t.Setenv(key, good[key])
}
}

// A repository declared in the catalogue before Crossplane creates it must not
// stop the schema from reaching GitHub; it is pending and gets no value writes.
func TestUncreatedRepositoryIsPendingNotFatal(t *testing.T) {
c := fixture(t)
f := &fakeAPI{schema: []property{}, missing: map[string]bool{"ksail": true}}
if err := reconcile(c, f.call); err != nil {
t.Fatalf("an uncreated repository blocked reconciliation: %v", err)
}
if _, written := f.values["ksail"]; written {
t.Fatal("values were written for a repository that does not exist")
}
if len(f.values) != len(c.Repositories)-1 {
t.Fatalf("every created repository needs its values: got %d of %d", len(f.values), len(c.Repositories)-1)
}
if len(f.schema) != len(c.Properties) {
t.Fatal("the schema did not reach the reviewed state")
}
}

// A missing repository is evidence only in a complete listing of an
// all-repositories installation; anything else stops before the first write.
func TestInstallationListingMustBeComplete(t *testing.T) {
for name, listing := range map[string]string{
"selected installation": `{"total_count":1,"repository_selection":"selected","repositories":[{"name":".github","full_name":"devantler-tech/.github"}]}`,
"short of its total": `{"total_count":2,"repository_selection":"all","repositories":[{"name":".github","full_name":"devantler-tech/.github"}]}`,
"foreign repository": `{"total_count":2,"repository_selection":"all","repositories":[{"name":".github","full_name":"devantler-tech/.github"},{"name":"x","full_name":"other/x"}]}`,
"duplicate repository": `{"total_count":2,"repository_selection":"all","repositories":[{"name":".github","full_name":"devantler-tech/.github"},{"name":".github","full_name":"devantler-tech/.github"}]}`,
"without this repository": `{"total_count":1,"repository_selection":"all","repositories":[{"name":"ksail","full_name":"devantler-tech/ksail"}]}`,
"no total": `{"repository_selection":"all","repositories":[]}`,
"unreadable": `not json`,
} {
t.Run(name, func(t *testing.T) {
f := &fakeAPI{schema: []property{}}
api := func(method, path string, body []byte) ([]byte, error) {
if method == "GET" && strings.HasPrefix(path, "installation/repositories?") {
return []byte(listing), nil
}
return f.call(method, path, body)
}
if err := reconcile(fixture(t), api); err == nil {
t.Fatal("an unproven listing was accepted")
}
if len(f.writes) != 0 {
t.Fatalf("wrote before proving the listing: %v", f.writes)
}
})
}
}
11 changes: 2 additions & 9 deletions tests/custom-properties.sh
Original file line number Diff line number Diff line change
Expand Up @@ -50,22 +50,15 @@ jq -e '
.[0].with."permission-organization-custom-properties" == "admin" and
.[0].with.repositories == ".github") and
([.jobs.apply.steps[] | select(.id? == "readback-token")] | length == 1 and
(.[0].with | keys | sort) == ["client-id","owner","permission-metadata","permission-organization-custom-properties","private-key","repositories"] and
(.[0].with | keys | sort) == ["client-id","owner","permission-metadata","permission-organization-custom-properties","private-key"] and
.[0].with.owner == "devantler-tech" and
.[0].with."permission-organization-custom-properties" == "read" and
.[0].with."permission-metadata" == "read" and
.[0].with.repositories == "${{ steps.readback-repositories.outputs.repositories }}") and
.[0].with."permission-metadata" == "read") and
([.jobs.apply.steps[] | select(.run? | strings | endswith("--apply-schema"))] | length == 1 and
.[0].env.GH_PROPERTY_READBACK_TOKEN == "${{ steps.readback-token.outputs.token }}") and
([.jobs.apply.steps | to_entries[] | select(.value.run? | strings | endswith("--check")) | .key][0] <
[.jobs.apply.steps | to_entries[] | select(.value.id? == "token") | .key][0]) and
([.jobs.apply.steps | to_entries[] | select(.value.run? | strings | endswith("--check")) | .key][0] <
[.jobs.apply.steps | to_entries[] | select(.value.id? == "readback-token") | .key][0])
' "$work/workflow.json" >/dev/null
# Exercise the token's repository selector on a distinct complete catalogue.
mkdir -p "$work/selection/custom-properties"
printf '%s\n' '{"repositories":{"public-app":{},"private-intake":{}}}' >"$work/selection/custom-properties/catalogue.json"
jq -r '.jobs.apply.steps[] | select(.id? == "readback-repositories") | .run' "$work/workflow.json" >"$work/selection/run.sh"
(cd "$work/selection" && GITHUB_OUTPUT="$work/selected-repositories" bash run.sh)
test "$(cat "$work/selected-repositories")" = 'repositories=private-intake,public-app'
echo 'PASS: rendered property coverage, native generator and schema refusal controls'
Loading