Assert on parsed structure in the cf2ansible and port_binding tests

[FAZuH]

Part of the whole-repo test-suite audit (2026-09-30). These are Fragile Fixtures: a formatting change breaks them with no behaviour change.

What to build

Tests that read the data the program produces, not the exact bytes it happened to emit.

Findings

• tests/cf2ansible.rs asserts on exact YAML formatting. domain0 (:74) carries about 20 assertions across 10 unrelated behaviours; domain3 (:138) has 12; assert_common (:34) has 7. assert_type_count (:60) counts the literal string "\n type: A\n", so restyling the YAML to block form breaks all four tests with no behaviour change. Parse the YAML with serde_yaml and assert on the structure, or commit one snapshot per fixture.
• The same problem in a model test. crates/natmap/tests/model.rs:42 add_mapping_request_serialize_defaults asserts json.contains("\"host_ip\":\"0.0.0.0\"") for the same reason. Deserialise and compare fields.
• Three port_binding tests parse a human-formatted table. tests/auto_discover/port_binding.rs:220,261,306 use awk "$6 == id {print $8}" on natmap ls output. Reorder or re-pad a table column and all three break. Use the /rules JSON endpoint, or natmap ls --json if it exists.
• The same three tests are also duplicates. bind_ip_strict_address (:196), bind_interface_resolved_address (:237) and bind_interface_overrides_defaults (:278) are 40 lines each, differing only in the YAML bind_ip: versus bind_interface: line and whether a defaults: block is present. One table-driven test.
• Two tests that do not test what they name. src/cmd/cf2ansible.rs:271 output_produces_yaml never calls print_ansible_tasks_to; it calls parse_zone and checks the rtype list, while the real output is covered by tests/cf2ansible.rs. And ns_records_not_proxied (:301) is a for loop with an assert_eq! inside, so it passes vacuously if the fixture has no NS records.
• Test names break standards.md §3.6. domain0, domain1, domain2, domain3 are named after the fixture file, with no module and no scenario.

Acceptance criteria

• No test asserts on whitespace, column position, or a substring of formatted output
• Restyling the YAML emitter, or reordering a table column, does not break these tests. Demonstrate this.
• ns_records_not_proxied fails when the fixture is changed to have zero NS records, i.e. it is no longer vacuous
• Test names follow <module_or_function>_<scenario> per standards.md §3.6
• One concept per test: the 20-assertion domain0 becomes several named tests
• cargo test -p lab-ops --test cf2ansible passes

Blocked by

None (can start immediately).

[FAZuH]

Landed on v0.2 as 6d703ec, merged f45458a. Closed by hand: a closing keyword only fires from commits reaching the default branch, and this is parked on v0.2.

The two criteria that are falsifiability requirements, and the evidence for them

Restyling the YAML emitter, or reordering a table column, does not break these tests. Demonstrate this.
ns_records_not_proxied fails when the fixture is changed to have zero NS records

Emitter restyled — header comment added, module args moved from 4-space to 6-space indent, tags: ["dns"] turned into a block list:

cargo test -p lab-ops --test cf2ansible      12 passed; 0 failed
cargo test -p lab-ops --lib cmd::cf2ansible   4 passed; 0 failed

Emitter reverted afterwards. Under the old tests this change alone broke all four.

Table column order changed — natmap ls docker table rewritten so HOST ADDR comes first and CONTAINER last:

cargo test -p lab-ops --all-features --test auto_discover \
    port_binding::docker_bind_address_from_config
1 passed; 0 failed

Reverted afterwards. Forced to a wrong expected value the new read fails with got 10.99.99.1:32768, so it genuinely asserts rather than passing vacuously.

Zero NS records — the NS line deleted from SAMPLE_ZONE:

thread panicked at src/cmd/cf2ansible.rs:322:
assertion `left == right` failed: SAMPLE_ZONE must carry NS records
  left: 0
 right: 1
test result: FAILED. 0 passed; 1 failed

Restored, green. SAMPLE_ZONE carries one NS record, so the assert encodes 1.

The rest

• No test asserts on whitespace, column position, or a substring of formatted output. rg -c awk tests/auto_discover/port_binding.rs is now 0.
• Names follow <module_or_function>_<scenario> per standards.md §3.6. The four fixture-named tests (domain0–domain3) became twelve behaviour-named ones. Repo-wide grep across .rs, .md, .sh, .yml, .yaml found no references to any old name.
• One concept per test: the 20-assertion domain0 is now four separate tests among the twelve.

Two tests that did not test what they named

• output_produces_yaml never called print_ansible_tasks_to at all — it called parse_zone and checked the rtype list, so the emitter's actual output was covered only by tests/cf2ansible.rs. It is now print_ansible_tasks_renders_every_record_type and calls the emitter.
• ns_records_not_proxied is the vacuous-loop case above.

Counts

cargo test -p lab-ops --test cf2ansible                      12 passed  (was 4)
cargo test -p lab-ops_natmap --test model                     7 passed
cargo test -p lab-ops --all-features --test auto_discover    44 passed  (was 46)
./dev.sh test                              431 passed, 0 failed, 10 targets

Dependency note, flagged for a decision later

serde_yaml came back as a root-crate dev-dependency. It was already resolved in Cargo.lock and already compiled by auto-discover, so the build cost is zero, and it is dev-only so nothing ships. The alternative was a hand-rolled line scanner for record:/type: pairs — which is the same fragile fixture this ticket exists to close.

Worth knowing for later: serde_yaml is deprecated upstream, unmaintained since 2024. serde_yaml_ng and serde_norway are drop-in and the test code would not change.