Skip to content

fix: name a NetBox job's accounts in the job line and require each in the schema - #222

Merged
sindredg merged 2 commits into
mainfrom
fix/accounts-in-the-job-line
Oct 28, 2026
Merged

sindredg merged 2 commits into
mainfrom
fix/accounts-in-the-job-line

Conversation

@sindredg

Copy link
Copy Markdown
Owner

Change

The schema a NetBox job's recipe carried asked for one verdict per account but required only one item, so Goose accepted an output that left an account out and the verification gate stopped on the missing line (#221). Now the trusted side names the accounts, and the schema requires each (ADR 0051).

  • Dispatch step (platform/dispatch/dispatch.py, platform/argo/workflow-template.yaml): mounts the job's checks' verdicts, the comparison step's optional item, reads the account names through the comparison's own reader, and sends them sorted in the job line as accounts. A NetBox job with no item, another job's, or one out of shape fails here, before the job runs, with the stop the comparison step would have given after it.
  • Job program (worker/agent-runtime/job/agent_job.py): requires accounts for a NetBox job (1 to 20 names in the comparison's pattern, no repeat) and refuses the field for any other job. The schema is built per job: verdicts is an object with a required property per account and no other, and a proposal's account is one of the same names. The lines written for the comparison keep their form.
  • Task text (collectors/netbox/task.txt): the last paragraph asks for the verdict under each account's name as the schema lists them.
  • Docs: ADR 0051, pointer sentences in ADRs 0049 and 0030, the index, the agent job page (schema, the dispatch step's new refusal), W.12's reason, and the evidence collection runbook's note on a deleted config map.

Validation

All local, with the suites' stubs and fakes. Nothing ran on the live system.

Check Command Result
Job program suite /usr/bin/python3 -I -S -m unittest discover -s tests in worker/agent-runtime/job 84 tests, OK
Dispatch suite, the comparison's and the task text's tests included python3.15 -I -m unittest discover -s tests in platform/dispatch 79 tests, OK
Fail side Both suites run on the new tests before the code Job suite failed at import; dispatch suite 3 failures, 2 errors on the new cases
Ruff 0.17.2 ruff check and ruff format --check on both programs Clean
Links The link checker over every tracked Markdown file 2242 links, 0 broken
Page rules python3 -I validation/evidence/render.py --check Exit 0
Template Parsed as YAML; kubeconform runs in the pipeline The dispatch step has the verdicts volume and its mount

Not verified:

  • A job on the live system, and the dispatch pod's mount of the verdicts item under /etc/aisec, where the tool certificate is mounted.
  • Goose 1.56.0 refusing an output that leaves an account out. Reasoned from its source: final_output_tool.rs validates each call with the jsonschema crate over the whole schema.

Risks and follow-up

  • The dispatch image and the job program are installed together. A line from the current image has no accounts, and the new job program refuses a NetBox job on it (ADR 0051, trade-offs). Order after the merge: the pipeline pushes the dispatch image from main; a pull request pins its digest in the template; the template, the image, and the job program go live in one step; then two NetBox jobs and the synthetic job, every check, and the records within 24 hours. That follow-up closes The NetBox answer schema accepts an answer that leaves out an account #221.
  • A scored run with the new schema (ADR 0042) is pending. The schema and the task's last paragraph changed what the model reads.
  • The first prompt grows with the number of accounts. The held-out cases have one or two.

…n the schema

The schema a NetBox job's recipe carried asked for one verdict per
account but required one item, so Goose accepted an output that left an
account out and the verification gate stopped on the missing line
(#221). The dispatch step now mounts the job's checks' verdicts, the
comparison step's item, reads the account names through the
comparison's own reader, and sends them in the job line. The job program
builds the schema for each job: a verdict under each name, no other
name, and proposals limited to the same names. A NetBox job with no
verdicts fails at the dispatch step before it runs, with the stop the
comparison would have given after it. The task text names the shape.
The record states the options and the decision behind #221's fix. ADRs
0049 and 0030 point to it, the agent job page describes the per-job
schema and the dispatch step's new refusal, W.12 names both readers of
the verdicts, and the evidence collection runbook says where a job
without verdicts now fails.
@sindredg
sindredg merged commit fafe39b into main Oct 28, 2026
18 checks passed
sindredg added a commit that referenced this pull request Oct 28, 2026
* feat: run the dispatch image that names the job's accounts

The template's three steps move to the image the pipeline pushed from
main at b9de301, the first push after #222 merged. Its dispatch step
reads a NetBox job's accounts from the checks' verdicts and names them
in the job line, which the job program needs to build the job's schema
(ADR 0051).

* docs: record the live run of the accounts in the job line

Four jobs on 2026-10-28 as a run set: two NetBox jobs that matched
with a verdict under each account's name from one accepted call, a job
with no collection that failed at the dispatch step before it ran, and
a synthetic job. A lessons row, the run set's bar on the milestone 5
chart with a category for a job that failed before it ran, the worklog
section, and the validation README's last run: 425 of 426, U.7 on a
newer mesh agent release upstream (#225).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The NetBox answer schema accepts an answer that leaves out an account

1 participant