You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-**Network QA playbooks** — new Ansible-based health check framework in `ansible/`. Entry point `playbooks/network_qa.yml` discovers test cases from `ansible/test_cases/*.yml`, runs each via `_run_check.yml`, and writes results to `ansible/results/results_<timestamp>.json`.
156
+
-**NETCONF transport** — test checks query device state over NETCONF using the `ietf-routing` YANG model (`urn:ietf:params:xml:ns:yang:ietf-routing`) for VRF routing table reads. Credentials sourced from HashiCorp Vault via `community.hashi_vault` Ansible collection.
157
+
-**Read-only test design** — replaced 3 fault-injection test cases (config push, convergence wait, teardown) with a single read-only routing table assertion. No configuration is pushed to devices.
158
+
-**`route_exists` filter** (`ansible/filter_plugins/ospf_filters.py`) — Jinja2 filter that parses NETCONF XML and checks for a destination prefix in the routing table.
159
+
-**Test case format** — declarative YAML defining device, VRF, assertion, and RFC reference. Example: `route_to_a2a.yml` verifies E1C has a route to A2A's loopback (192.168.42.1) in VRF1.
160
+
-**ncclient device handler fix** — added `vars:` binding for `ansible_netconf_ncclient_device_handler` in the netcommon NETCONF plugin so inventory variables are read correctly (forces `iosxe` handler instead of `default`).
161
+
162
+
### `/qa` Skill
163
+
164
+
-**Generic investigation skill** (`.claude/skills/qa/SKILL.md`) — rewritten to be device- and test-agnostic. Loads latest results JSON, triages pass/fail, presents numbered failure list, user picks a failure to investigate, agent runs full diagnostic workflow (intent → live state → skill decision trees → KB search), reports findings, then re-presents remaining failures for the next pick.
165
+
-**Shared root cause detection** — after investigating a failure, if its root cause likely explains other failures on the list, the agent says so — user can skip those.
166
+
167
+
### Documentation
168
+
169
+
-**`WORKFLOW.md`** — complete rewrite covering both operational modes: interactive troubleshooting (user asks questions) and automated QA (Ansible health checks + `/qa` skill). Includes tool table, SSH pipeline diagram, RAG pipeline explanation, step-by-step walkthroughs for both modes, concrete examples, and ASCII architecture diagram.
170
+
-**`README.md`** — added QA & Ansible section with prerequisites, running instructions, and `/qa` skill usage. Updated project structure to reflect `ansible/` directory layout.
171
+
172
+
### Test Cleanup
173
+
174
+
-**269 → 169 tests** (37% reduction) — removed tests that verify Pydantic builtins, duplicate error paths, or Python language features rather than project logic.
175
+
- Gutted `test_input_models.py` to 2 tests (JSON string parsing only — the custom `@model_validator`).
176
+
- Trimmed `test_security.py` to OspfQuery variants only (same VRF regex shared across all models).
177
+
- Removed trivial tests from `test_status.py` (Path.exists, enum string checks), `test_tools.py` (2-line dict builder, duplicate unknown_device), `test_transport.py` (duplicate SSH error), `test_list_devices.py` (duplicate filter), `test_ingest.py` (duplicate RFC), `test_mcp_server.py` (count implied by name check), `test_inventory.py` (fixture-testing, not production code).
178
+
179
+
### Housekeeping
180
+
181
+
-**`.gitignore`** — added `ansible/collections/ansible_collections/` and `ansible/results/` to prevent installed collections and ephemeral result files from being committed.
182
+
- Old fault-injection test cases moved to `core/legacy/`.
│ └── run_tests.sh # Test runner (--live for lab tests)
223
223
├── ingest.py # RAG ingestion pipeline
@@ -231,7 +231,7 @@ YANA/
231
231
232
232
## 🧪 QA & Ansible
233
233
234
-
Automated OSPF QA tests run via Ansible playbooks that push NETCONF config changes, assert protocol behavior against RFC expectations, then tear down and verify recovery.
234
+
Network QA health checks run via Ansible playbooks that query device state over NETCONF and assert expected conditions (route presence, protocol state) against design intent.
235
235
236
236
**Prerequisites:**
237
237
-`ansible-core` and `ncclient` installed in the venv (both included in `requirements.txt`)
@@ -246,8 +246,8 @@ Automated OSPF QA tests run via Ansible playbooks that push NETCONF config chang
246
246
**Running tests:**
247
247
```bash
248
248
cd ansible
249
-
ansible-playbook playbooks/ospf_qa.yml # all scenarios
250
-
ansible-playbook playbooks/ospf_qa.yml -e scenario_filter=auth_mismatch# single scenario
249
+
ansible-playbook playbooks/network_qa.yml # all checks
250
+
ansible-playbook playbooks/network_qa.yml -e scenario_filter=route_to_a2a# single check
251
251
```
252
252
253
253
Results are written to `ansible/results/results_<timestamp>.json`.
0 commit comments