Skip to content

Align Kali, port, and socket docs and stale references with the shipped runtime #954

Closed
Closed
Align Kali, port, and socket docs and stale references with the shipped runtime#954
Labels
documentationImprovements or additions to documentationuser-facingDirectly changes a user journey or user-visible outcome
Milestone
Quickstart from released artifacts

Description

@Brad-Edwards
Brad-Edwards
opened on Sep 4, 2026

The original report found removed scenario paths, a missing default catalog entry, and docs and tests describing old images, ports, and Docker sockets.

#851 has since fixed the front-door quick-start. The README, docs/getting-started/installation.md, and docs/getting-started/quick-start.md only use commands the CLI provides. aptl lab scenarios lists techvault from the installed pack, and the quick-start documents --scenario techvault. This issue now covers only the remaining mismatches.

Remaining work

Kali, published ports, and Docker sockets

  • docs/components/kali-redteam.md contradicts itself and the runtime:
    • line 27 says no host port is published, but line 51 gives ssh … kali@localhost -p 2023;
    • it states the base image is kalilinux/kali-last-release;
    • it links the removed containers/kali/Dockerfile.
  • docs/containers/victim-template-guide.md:165 uses ssh kali@localhost -p 2023.
  • No user docs describe which services mount the host Docker socket or why.

Stale secondary docs

  • docs/components/kali-redteam.md:58 refers to aptl scenario start, which does not exist.

  • docs/requirements/CLI-005/requirement.md:24 traces to the missing src/aptl/cli/scenario.py.

  • Docs under docs/raes/ cite the missing scenarios/techvault.sdl.yaml and scenarios/techvault-operational.sdl.yaml:

    • dsl-008-realization-preflight.md
    • m1-raes-conformance-truth-up-preflight.md
    • techvault-curated-live-validation-gate.md
    • techvault-live-validation-preflight.md
    • techvault-static-validation-preflight.md

    Fix them, or mark them as dated historical records.

Tests against the Compose stub

  • docker-compose.yml is a stub ("No config on this stub is live"), but 37 test files still parse it, for example tests/test_docker_compose_port_bindings.py and tests/test_kali_capture_compose.py. Those assertions test nothing live.

Done when

  • Kali, published-port, and Docker socket documentation matches the rendered candidate runtime.
  • No user-facing or requirement doc references a removed command, source file, or scenario path, unless it is marked as a dated historical record.
  • Tests that assert runtime properties check rendered or observed behavior, not the Compose stub. Tests that only exercise the stub are removed or retargeted.

Related: #851 (closed), #951, #870, #934, #1218, #1220.

Activity

  1. Brad-Edwards
    added this to the Quickstart from released artifacts milestone on Sep 4, 2026
  2. Brad-Edwards
    added
    documentationImprovements or additions to documentation
    user-facingDirectly changes a user journey or user-visible outcome
    on Sep 4, 2026
  3. Brad-Edwards
    marked this as blocked on Sep 4, 2026
  4. Brad-Edwards
    changed the title [-]Quick-start docs, scenario catalog, and compose tests describe a range that no longer exists[/-] [+]Make quick-start instructions and catalog match the shipped lab[/+] on Sep 20, 2026
  5. Brad-Edwards
    changed the title [-]Make quick-start instructions and catalog match the shipped lab[/-] [+]Align Kali, port, and socket docs and stale references with the shipped runtime[/+] on Oct 5, 2026
  6. doublewhy
    added 2 commits that reference this issue on Oct 10, 2026

    docs(requirements): leave CLI-005's statement for the requirement audit

    ef1c3de

    docs(requirements): leave CLI-005's statement for the requirement audit

    c2ffc5a
  7. Brad-Edwards
    added
    in-progressAn agent is actively working this issue via /implement
    on Oct 12, 2026

  8. Brad-Edwards commented on Oct 12, 2026

    @Brad-Edwards
    Brad-Edwards
    on Oct 12, 2026
    CollaboratorAuthor

    🛠️ Picked up by /implement - driver codex, branch 954-align-kali-port-socket-docs, 2026-10-12T17:46:17.441Z.

  9. Brad-Edwards commented on Oct 12, 2026

    @Brad-Edwards
    Brad-Edwards
    on Oct 12, 2026
    CollaboratorAuthor

    gc workflow phase recorded: preflight (issue #954). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

  10. Brad-Edwards commented on Oct 12, 2026

    @Brad-Edwards
    Brad-Edwards
    on Oct 12, 2026
    CollaboratorAuthor

    gc workflow phase recorded: plan (issue #954). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

    Complete #954 in its existing PR #1263, consolidating all work into one delivery branch.

    1. Preserve all docs(reference): describe the run directory and the web API as shipped #1288 commits by merging its branch into docs: align runtime documentation and tests with the shipped lab #1263, bring in test: check TechVault runtime properties on the rendered pack, not the Compose stub #1261's rendered-pack test changes, and merge current dev with real conflict resolution. Close docs(reference): describe the run directory and the web API as shipped #1288 after its commits are published in docs: align runtime documentation and tests with the shipped lab #1263 (explicit user authorization). Keep Align Kali, port, and socket docs and stale references with the shipped runtime #954 open until delivery merges.
    2. Path D: verify and finish Kali, port and Docker-socket documentation against the installed, admitted TechVault realization and existing operator/capture code; preserve the distinction between declared, rendered and observed behavior.
    3. Path D: audit user-facing and requirement documents for removed commands/source/scenario paths. Correct current guidance and traceability where supported; mark retained historical documents and audited requirement statements with prominent dated scope notices and links to current guidance, without rewriting requirement meaning or status.
    4. Path D: finish test cleanup. Reuse the import-safe TechVault realization helper for actual pack assertions. Remove vacuous/stub-only profile and docker-reference assertions with recorded reasons. Retarget or retire the stale in-process-agent suite against current pack-owned Wazuh configuration. Preserve tests of the live project-tree configuration and admitted backend apparatus.
    5. Path D: update CONTRIBUTING.md, AGENTS.md and plan rules: one issue = one PR, every acceptance criterion covered; oversized issues become smaller linked issues, each with one complete PR. The cross-repository rollout is tracked separately in Ground Control #1776; no changes to other repos.
    6. Verify with the targeted test helper for changed test modules and relevant docs/pack/ownership contracts, then staged pre-commit hygiene. Run the configured pre-push review, publish and monitor all PR checks including Sonar; repair failures and rewrite docs: align runtime documentation and tests with the shipped lab #1263's title/body around complete Align Kali, port, and socket docs and stale references with the shipped runtime #954 coverage.

    Design: this change adds no runtime authority, secret transport, endpoint, renderer or lifecycle controller. Existing RAES admission, deployment realization, capture, operator access and runtime validators remain authoritative. TechVault assertions use the installed pack; generic tests retain owned fixtures. Native observations are never inferred from pure rendering. Keep asset inventories and requirement traceability aligned with file moves/deletions. Local checks remain targeted; full suites, coverage, prose and lab lifecycle qualification remain CI-owned. No new requirement or structural gate is introduced.

  11. doublewhy
    added 3 commits that reference this issue on Oct 12, 2026

    docs: align Kali, published-port and Docker socket docs with the real…

    2eb27fd

    docs: retire references to removed commands, files and scenario paths

    07e9e15

    test: check TechVault runtime properties on the rendered pack, not th…

    7d19f2d

  12. Brad-Edwards commented on Oct 12, 2026

    @Brad-Edwards
    Brad-Edwards
    on Oct 12, 2026
    CollaboratorAuthor

    gc_codex_review — sanitized deferred publication for issue #954, cycle 1 of 1
    Reviewed revision: 211574d58e9585f08a51a9aa444e88eaa6efa6aa24437623dd82a3983fd44e43

    Architectural read

    The complete diff reuses the installed pack planner, interpreter and production renderers for TechVault assertions. Documentation distinguishes current runtime contracts from dated historical records and rendering from native observation. Existing admission, network, capability, capture and socket boundaries remain authoritative; this change adds no runtime authority.

    Verdict: ship

    Notes

    • Clarify the Kali guide: the relay is the declared operator-access path, while host MCP clients can use the native workload ingress. The categorical host-routing claim should be removed.

    Findings

    • (none)

  13. Brad-Edwards commented on Oct 12, 2026

    @Brad-Edwards
    Brad-Edwards
    on Oct 12, 2026
    CollaboratorAuthor

    gc_codex_review pre-push cycle 1 of 1 complete for issue #954 on branch '954-align-kali-port-socket-docs'. Posted by the MCP server to enforce the pre-push hard-cap-1 contract (issues #796, #804, #906). Do not edit or delete — used by the next gc_codex_review (uncommitted) invocation to count cycles.

  14. Brad-Edwards commented on Oct 12, 2026

    @Brad-Edwards
    Brad-Edwards
    on Oct 12, 2026
    CollaboratorAuthor

    Review decision record — codex cycle 1 (issue #954)

    Reviewer: codex
    Cycle: 1
    Verdict: ship

    Architectural read:

    The complete diff reuses the installed pack planner, interpreter and production renderers for TechVault assertions. Documentation distinguishes current runtime contracts from dated historical records and rendering from native observation. Existing admission, network, capability, capture and socket boundaries remain authoritative; this change adds no runtime authority.

    Blocking findings: 0 (clean run)

    Notes (non-blocking, no decisions):

    • Clarify the Kali guide: the relay is the declared operator-access path, while host MCP clients can use the native workload ingress. The categorical host-routing claim should be removed.

  15. Brad-Edwards commented on Oct 12, 2026

    @Brad-Edwards
    Brad-Edwards
    on Oct 12, 2026
    CollaboratorAuthor

    Pre-PR base synchronization

    • Source: refs/remotes/origin/dev at 8ab91f687dde82d7c5de56ccd22c2279a5404b9e
    • Outcome: merged_clean
    • Published feature head: fa7c083f0820062293be30c693142a7c9abfde16
    • Synchronized tree: f0804ae3ce05b3b618d422ecbaf69cb7a0183667

  16. Brad-Edwards commented on Oct 12, 2026

    @Brad-Edwards
    Brad-Edwards
    on Oct 12, 2026
    CollaboratorAuthor

    Ready for review — issue #954

    PR: #1263
    Plan: #954 (comment)

    Outcome

    PR #1263 now covers all acceptance criteria of #954, including both #1288 commits and #1261's test cleanup. Runtime docs and tests describe the shipped lab, and coding standards require one complete PR per issue. #1288 is closed; cross-repository rollout is tracked in the parent coordination issue.

    Files changed

    Added:

    • docs/CODING_STANDARDS.md
    • docs/architecture/issue-954-runtime-docs-and-test-authority-preflight.md
    • tests/techvault_pack.py

    Modified:

    • .gc/plan-rules.md
    • AGENTS.md
    • CONTRIBUTING.md
    • docs/architecture/enterprise-infrastructure.md
    • docs/architecture/exp-008-portable-evidence-bundle-preflight.md
    • docs/architecture/exp-010-capture-admission-evidence-preflight.md
    • docs/architecture/index.md
    • docs/architecture/issue-1000-manual-release-qa-preflight.md
    • docs/architecture/issue-1006-container-base-images-preflight.md
    • docs/architecture/issue-1186-seat-privilege-preflight.md
    • docs/architecture/issue-592-scenario-pack-terminology-preflight.md
    • docs/architecture/issue-790-ssh-module-split-preflight.md
    • docs/architecture/issue-824-kiosk-launcher-reset-recovery-preflight.md
    • docs/architecture/issue-866-techvault-realization-contract-preflight.md
    • docs/architecture/issue-875-scenario-content-declaration-preflight.md
    • docs/architecture/issue-880-techvault-env-pack-consumption-preflight.md
    • docs/architecture/issue-912-misp-redis-post-realization-mutation-preflight.md
    • docs/architecture/issue-913-shuffle-post-realization-mutation-preflight.md
    • docs/architecture/issue-957-wazuh-attestation-evidence-credentials-preflight.md
    • docs/architecture/issue-969-platform-safety-gates-preflight.md
    • docs/architecture/issue-993-boot-realization-regressions-preflight.md
    • docs/architecture/networking.md
    • docs/architecture/obs-002-correlation-identity-clock-preflight.md
    • docs/components/default-defensive-posture.md
    • docs/components/kali-redteam.md
    • docs/components/victim-containers.md
    • docs/components/wazuh-siem.md
    • docs/containers/victim-template-guide.md
    • docs/deployment.md
    • docs/getting-started/quick-start.md
    • docs/raes/dsl-008-realization-preflight.md
    • docs/raes/m1-raes-conformance-truth-up-preflight.md
    • docs/raes/techvault-live-validation-preflight.md
    • docs/raes/techvault-static-validation-preflight.md
    • docs/red-team-taxonomy.md
    • docs/reference/experiment-runs.md
    • docs/reference/participant-profile.md
    • docs/reference/web.md
    • docs/requirements/APP-2/requirement.md
    • docs/requirements/CLI-005/requirement.md
    • docs/requirements/OBS-003/requirement.md
    • docs/requirements/SCN-003/requirement.md
    • docs/requirements/SCN-008/requirement.md
    • docs/requirements/SCN-009/requirement.md
    • docs/requirements/SCN-010/requirement.md
    • docs/requirements/SIEM-001/requirement.md
    • docs/reviews/962-lilrae-readiness/tracked-file-inventory.tsv
    • docs/specs/soc-feature-spec.md
    • docs/specs/web-gui-design-preflight.md
    • mkdocs.yml
    • tests/test_compose_capability_necessity.py
    • tests/test_consistency.py
    • tests/test_continuity.py
    • tests/test_docker_compose_port_bindings.py
    • tests/test_env_pack_realization.py
    • tests/test_in_process_agents.py
    • tests/test_misp_redis_realization_contract.py
    • tests/test_shuffle_realization_contract.py

    Reviews

    • codex: Full-diff pre-push review: ship, no findings. Ingress clarification applied; merge conflicts resolved and verified.

    Traceability reconciliation

    Proposed in this PR — verified against the merged tree in Phase E.

    • IMPLEMENTS / TESTS / DOCUMENTS added: 0
    • Links updated: 4
    • Stale links removed: 0

    Status

    • CI: ✅ green
    • SonarCloud: ✅ passed
    • PR ready for user review and merge. Phase E validates the merged requirement state, posts the final report, and closes the issue automatically once this PR merges.

  17. Brad-Edwards commented on Oct 12, 2026

    @Brad-Edwards
    Brad-Edwards
    on Oct 12, 2026
    CollaboratorAuthor

    Delivery readiness recorded — Phase E finalizes issue #954 automatically when PR #1263 merges.

  18. Brad-Edwards
    added a commit that references this issue on Oct 12, 2026

    docs: align runtime documentation and tests with the shipped lab (#1263)

    41551db

  19. github-actions commented on Oct 12, 2026

    @github-actions
    github-actionsBot
    on Oct 12, 2026 – with GitHub Actions
    Contributor

    Final report — issue #954 complete

    PR: #1263
    Plan: #954 (comment)

    Outcome

    PR #1263 now covers all acceptance criteria of #954, including both #1288 commits and #1261's test cleanup. Runtime docs and tests describe the shipped lab, and coding standards require one complete PR per issue. #1288 is closed; cross-repository rollout is tracked in the parent coordination issue.

    Files changed

    Added:

    • docs/CODING_STANDARDS.md
    • docs/architecture/issue-954-runtime-docs-and-test-authority-preflight.md
    • tests/techvault_pack.py

    Modified:

    • .gc/plan-rules.md
    • AGENTS.md
    • CONTRIBUTING.md
    • docs/architecture/enterprise-infrastructure.md
    • docs/architecture/exp-008-portable-evidence-bundle-preflight.md
    • docs/architecture/exp-010-capture-admission-evidence-preflight.md
    • docs/architecture/index.md
    • docs/architecture/issue-1000-manual-release-qa-preflight.md
    • docs/architecture/issue-1006-container-base-images-preflight.md
    • docs/architecture/issue-1186-seat-privilege-preflight.md
    • docs/architecture/issue-592-scenario-pack-terminology-preflight.md
    • docs/architecture/issue-790-ssh-module-split-preflight.md
    • docs/architecture/issue-824-kiosk-launcher-reset-recovery-preflight.md
    • docs/architecture/issue-866-techvault-realization-contract-preflight.md
    • docs/architecture/issue-875-scenario-content-declaration-preflight.md
    • docs/architecture/issue-880-techvault-env-pack-consumption-preflight.md
    • docs/architecture/issue-912-misp-redis-post-realization-mutation-preflight.md
    • docs/architecture/issue-913-shuffle-post-realization-mutation-preflight.md
    • docs/architecture/issue-957-wazuh-attestation-evidence-credentials-preflight.md
    • docs/architecture/issue-969-platform-safety-gates-preflight.md
    • docs/architecture/issue-993-boot-realization-regressions-preflight.md
    • docs/architecture/networking.md
    • docs/architecture/obs-002-correlation-identity-clock-preflight.md
    • docs/components/default-defensive-posture.md
    • docs/components/kali-redteam.md
    • docs/components/victim-containers.md
    • docs/components/wazuh-siem.md
    • docs/containers/victim-template-guide.md
    • docs/deployment.md
    • docs/getting-started/quick-start.md
    • docs/raes/dsl-008-realization-preflight.md
    • docs/raes/m1-raes-conformance-truth-up-preflight.md
    • docs/raes/techvault-live-validation-preflight.md
    • docs/raes/techvault-static-validation-preflight.md
    • docs/red-team-taxonomy.md
    • docs/reference/experiment-runs.md
    • docs/reference/participant-profile.md
    • docs/reference/web.md
    • docs/requirements/APP-2/requirement.md
    • docs/requirements/CLI-005/requirement.md
    • docs/requirements/OBS-003/requirement.md
    • docs/requirements/SCN-003/requirement.md
    • docs/requirements/SCN-008/requirement.md
    • docs/requirements/SCN-009/requirement.md
    • docs/requirements/SCN-010/requirement.md
    • docs/requirements/SIEM-001/requirement.md
    • docs/reviews/962-lilrae-readiness/tracked-file-inventory.tsv
    • docs/specs/soc-feature-spec.md
    • docs/specs/web-gui-design-preflight.md
    • mkdocs.yml
    • tests/test_compose_capability_necessity.py
    • tests/test_consistency.py
    • tests/test_continuity.py
    • tests/test_docker_compose_port_bindings.py
    • tests/test_env_pack_realization.py
    • tests/test_in_process_agents.py
    • tests/test_misp_redis_realization_contract.py
    • tests/test_shuffle_realization_contract.py

    Reviews

    • codex: Full-diff pre-push review: ship, no findings. Ingress clarification applied; merge conflicts resolved and verified.

    Traceability reconciliation

    • IMPLEMENTS / TESTS / DOCUMENTS added: 0
    • Links updated: 4
    • Stale links removed: 0

    No in-scope requirement UIDs. Current traces were corrected, and concurrent requirement retirement records retained.

    Status

    • CI: ✅ green
    • SonarCloud: ✅ passed
    • PR ready for user review and merge.
  20. github-actions
    closed this as completedon Oct 12, 2026
  21. github-actions
    removed
    in-progressAn agent is actively working this issue via /implement
    on Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationuser-facingDirectly changes a user journey or user-visible outcome

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions