diff --git a/docs/test_env/DEVELOPER_Doc.md b/docs/test_env/DEVELOPER_Doc.md new file mode 100644 index 0000000000000..c27eed7be6c56 --- /dev/null +++ b/docs/test_env/DEVELOPER_Doc.md @@ -0,0 +1,669 @@ +# Developer Documentation — Environment Definitions + +> **Scope:** How to define, validate, and maintain vulnerable environment definitions for the `test_env` plugin. + +--- + +## Table of Contents + +1. [What Is an Environment Definition?](#what-is-an-environment-definition) +2. [Directory Structure](#directory-structure) +3. [Schema Reference](#schema-reference) +4. [The Three-Level Merge Hierarchy](#the-three-level-merge-hierarchy) +5. [Validation Rules](#validation-rules) +6. [How to Write a New Definition](#how-to-write-a-new-definition) +7. [Shared Definitions & Module Referencing](#shared-definitions--module-referencing) +8. [CI Metadata](#ci-metadata) +9. [Best Practices](#best-practices) +10. [Troubleshooting](#troubleshooting) + +--- + +## What Is an Environment Definition? + +An **environment definition** is a YAML file that describes a runnable, vulnerable service as an OCI-compliant container. It is the **single source of truth** for: + +- Which container image to use +- Which ports the service exposes +- How to verify the service is ready (health checks) +- Default credentials and datastore options +- One-time provisioning steps (e.g., driving an install wizard) +- CI automation metadata (payload recommendations, validation expectations) + +Definitions live in `data/vuln_envs/` and are referenced by exploit/auxiliary modules via the `VulnerableEnvironment` metadata key. + +--- + +## Directory Structure + +``` +data/ + vuln_envs/ + README.md # This documentation + jenkins.yml # Example: Jenkins CI + activemq.yml # Example: Apache ActiveMQ + wordpress.yml # Example: WordPress + httpd.yml # Example: Apache HTTP Server + openssh.yml # Example: OpenSSH +``` + +**File naming rule:** The filename (without `.yml`) **must** match the `name` field inside the file. The loader enforces this. + +--- + +## Schema Reference + +### Top-Level Keys + +| Key | Type | Required | Description | +|-----|------|----------|-------------| +| `name` | String | **Yes** | Machine-friendly identifier. Must match the filename. | +| `description` | String | **Yes** | Human-readable summary of what this service is. | +| `variants` | Array | **Yes** | List of runnable software versions / configurations. | +| `shared` | Hash | **Yes** | Base configuration inherited by **all** profiles. | +| `profiles` | Hash | **Yes** | Map of profile names to profile-specific overrides. | + +--- + +### `variants` Section + +Each variant represents a distinct runnable configuration — typically a software version, but may also represent different backends or build options for the same version. + +| Key | Type | Required | Description | +|-----|------|----------|-------------| +| `name` | String | **Yes** | Unique identifier for this variant. Used in `test_env build VARIANT=...`. | +| `version` | String | **Yes** | The actual software version string (for information and validation). | +| `image` | String | **Yes** | OCI image reference (e.g., `docker.io/library/httpd:2.4.57`). | +| `build_args` | Hash | No | Docker/Podman build arguments if the image must be built locally. | +| `default` | Boolean | No | If `true`, this variant is selected when no `VARIANT` is specified. Only one variant may be `default`. | + +**Example:** + +```yaml +variants: + - name: "2.361" + version: "2.361" + image: vulnhub/jenkins:2.361 + default: true + + - name: "2.361-postgres" + version: "2.361" + image: vulnhub/jenkins:2.361-pg + build_args: + DB_BACKEND: "postgresql" + + - name: "2.375" + version: "2.375" + image: vulnhub/jenkins:2.375 +``` + +--- + +### `shared` Section + +Base configuration inherited by every profile. Any field here can be overridden by a profile or by module-level metadata. + +#### `shared.ports` (Required) + +Maps logical port names to container ports. These are the ports the service listens on **inside** the container. + +```yaml +shared: + ports: + http: 8080 + broker: 61616 +``` + +> **Important:** These are container ports, not host ports. The `test_env build` command dynamically allocates free host ports and maps them. + +--- + +#### `shared.health_check` (Required in `shared` or in every profile) + +Defines how `test_env` waits for the service to become ready after the container starts. + +| Key | Type | Required | Description | +|-----|------|----------|-------------| +| `type` | String | **Yes** | `http`, `tcp`, or `command`. | +| `path` | String | If `type=http` | HTTP path to request. | +| `expected_status` | Integer | No | Expected HTTP status code. Default: `200`. | +| `match` | String | No | Substring the response body must contain. Use this to distinguish "server responding" from "app actually ready" (e.g., an install wizard vs. a login page). | +| `command` | String | If `type=command` | Shell command to execute **inside** the container. | +| `expected_output` | String | If `type=command` | Substring the command output must contain. | +| `interval` | Integer | No | Seconds between check attempts. Default: `5`. | +| `timeout` | Integer | No | Seconds to wait for a single check. Default: `2`. | +| `retries` | Integer | No | Maximum number of attempts. Default: `12`. | +| `credentials` | Hash | No | Basic Auth credentials for HTTP checks. Keys: `username`, `password`. | + +**HTTP example:** + +```yaml +health_check: + type: http + path: /api/jolokia/ + expected_status: 200 + interval: 5 + timeout: 2 + retries: 12 + credentials: + username: admin + password: admin +``` + +**TCP example:** + +```yaml +health_check: + type: tcp + interval: 2 + timeout: 2 + retries: 10 +``` + +**Command example:** + +```yaml +health_check: + type: command + command: "mysqladmin ping" + expected_output: "mysqld is alive" + interval: 3 + timeout: 5 + retries: 20 +``` + +--- + +#### `shared.credentials` (Optional) + +Default credentials for the service. These are automatically merged into the module datastore as `USERNAME`, `PASSWORD`, etc. + +```yaml +credentials: + default: + username: admin + password: admin +``` + +--- + +#### `shared.datastore_defaults` (Optional) + +Default datastore options for the module. These are applied automatically when the environment is built. + +```yaml +datastore_defaults: + TARGETURI: /script + RHOSTS: 127.0.0.1 +``` + +> **Note:** `RHOSTS` is automatically set to `127.0.0.1` by `test_env build`. You do not need to define it here unless you want a different value. + +--- + +#### `shared.provision` (Optional) + +Some images boot with the target process running but the application itself not yet usable. For example, a fresh WordPress container serves HTTP but has no database schema or admin account until the install wizard is submitted. + +`provision` describes a one-time setup action that runs after the health check passes and before the environment is registered as ready. + +| Key | Type | Required | Description | +|-----|------|----------|-------------| +| `type` | String | **Yes** | Currently only `http_post` is supported. | +| `path` | String | **Yes** | Request path, sent to the primary mapped port. | +| `body` | Hash | No | Form fields, sent as `application/x-www-form-urlencoded`. Values may reference `{{ credentials.default. }}`, which is resolved against the built datastore. | +| `timeout` | Integer | No | Seconds to wait for the request. Default: `10`. | +| `run_once` | Boolean | No | If `true`, provisioning is skipped if a marker file exists inside the container (e.g., after a `stop`/`start` cycle). Default: `false`. | + +**Example:** + +```yaml +provision: + type: http_post + path: /wp-admin/install.php?step=2 + body: + weblog_title: "Vulnerable WP" + user_name: "{{ credentials.default.username }}" + admin_password: "{{ credentials.default.password }}" + admin_password2: "{{ credentials.default.password }}" + admin_email: "admin@example.com" + blog_public: 0 + Submit: "Install WordPress" + run_once: true +``` + +**Architectural constraints:** +- Single stateless request only — no multi-step flows. +- No session/cookie carryover between requests. +- No non-HTTP provisioning (e.g., no `runtime.exec` for setup commands). +- Extend `type` as new provisioning shapes come up rather than building speculatively. + +**`run_once` behavior:** +When `run_once: true` is set, the provisioner creates a marker file (`/tmp/.msf_test_env_provisioned`) inside the container after the first successful provisioning. On subsequent operations (e.g., after `test_env start` restarts a stopped container), the provisioner checks for this marker and skips provisioning if it exists. This prevents duplicate form submissions or setup actions. + +A failure (non-2xx/3xx response, timeout, or request error) aborts the build and tears down the container. + +--- + +#### `shared.verify` (Optional) + +Re-checks the environment after `provision` runs, confirming the setup action actually took effect. Uses the same shape as `health_check` (including the `match` field). + +If `provision` is not defined, `verify` is ignored. If `provision` is defined but `verify` is not, the environment is registered as ready as soon as `provision` returns a `200`–`399` response. + +**Restart behavior:** When a provisioned environment is stopped and later restarted via `test_env start`, the base `health_check` is skipped in favor of `verify`. This is because the container retains its filesystem state across restarts — the service is in its post-provision state, not its fresh-boot state. For example, a WordPress container that was provisioned during `build` will return `200` at `/` after a restart, not `302` to the install wizard. + +```yaml +verify: + type: http + path: /wp-login.php + expected_status: 200 + match: "user_login" + interval: 3 + timeout: 2 + retries: 10 +``` + +--- + +#### `shared.volumes` (Optional) + +Defines volume mounts for the container. + +```yaml +volumes: + jenkins_home: + container_path: /var/jenkins_home + persist: false +``` + +If `host_path` is omitted, `test_env` creates a temporary directory that is cleaned up when the environment is removed. + +--- + +#### `shared.ci` (Optional) + +Metadata for CI-driven automated exploit verification. This is also read during interactive `test_env exec` to apply recommended payloads and options. + +##### `ci.exploit` + +| Key | Type | Required | Description | +|-----|------|----------|-------------| +| `payload` | String | No | Recommended payload for this environment. Applied if the module's current `PAYLOAD` differs. | +| `options` | Hash | No | Additional datastore keys to set (e.g., `LPORT`). | +| `force_exploit` | Boolean | No | If `true`, sets `ForceExploit true` on the module. | + +> **Critical:** Do **not** set `LHOST` here. The target runs inside a container network namespace; a hardcoded `LHOST` (especially `127.0.0.1`) resolves to the container itself, not the host. Leave `LHOST` unset so Metasploit's outbound-interface auto-detection supplies the host's real reachable address. + +**Example:** + +```yaml +ci: + exploit: + payload: cmd/linux/http/x64/meterpreter/reverse_tcp + options: + LPORT: 4444 + force_exploit: true +``` + +##### `ci.validation` + +Defines what "success" looks like for automated validation. Read by `test_env validate `. + +| Key | Type | Required | Description | +|-----|------|----------|-------------| +| `expected_session` | Boolean | No | Default `true`. If `false`, validation passes without checking for a session (used for auxiliary modules). | +| `session_type` | String | No | `meterpreter` or `shell`. If set, the created session's type must match. | +| `expected_output` | String | No | Substring that running a verification command on the session must contain, e.g., `"uid="`. | +| `timeout` | Integer | No | Seconds to wait for a session to appear. Default: `120`. | + +**Example:** + +```yaml +ci: + validation: + expected_session: true + session_type: meterpreter + expected_output: "uid=" + timeout: 120 +``` + +> **Known limitation:** `validate` looks for sessions in the current msfconsole process's `framework.sessions`. Sessions are process-local — they exist only in the msfconsole that opened them. `exec` and `validate` must run in the **same** msfconsole process. + +--- + +### `profiles` Section + +Each profile is a runtime configuration of the service. Profiles override or extend `shared`. + +| Key | Type | Required | Description | +|-----|------|----------|-------------| +| `description` | String | **Yes** | What this profile represents. | +| `health_check` | Hash | No | Overrides base `shared.health_check`. | +| `datastore_defaults` | Hash | No | Overrides base `shared.datastore_defaults`. | +| `credentials` | Hash | No | Overrides base `shared.credentials`. | +| `volumes` | Hash | No | Overrides base `shared.volumes`. | +| `ci` | Hash | No | Overrides base `shared.ci`. | +| `provision` | Hash | No | Overrides base `shared.provision`. | +| `verify` | Hash | No | Overrides base `shared.verify`. | + +**Profile names** must match `[a-z0-9-]+`. + +**Every definition must contain a `default` profile.** + +**Example:** + +```yaml +profiles: + default: + description: Standard ActiveMQ; web console reachable via HTTP/Jolokia. + + broker-only: + description: Web console not assumed reachable; only the broker port is health-checked. + health_check: + type: tcp + ci: + exploit: + force_exploit: true +``` + +--- + +## The Three-Level Merge Hierarchy + +When `test_env build` resolves an environment, configuration is merged in this order: + +``` +Level 1: shared (base configuration, inherited by all profiles) + ↓ +Level 2: profiles[profile_name] (profile-specific overrides) + ↓ +Level 3: Module-level overrides (VulnerableEnvironment['overrides']) +``` + +### Merge Rules + +- **Hash fields** (e.g., `health_check`, `datastore_defaults`) are **deep-merged**: nested keys are combined, not replaced wholesale. +- **Scalar fields** (e.g., `image`, `build_args`) are **replaced** by the higher level. +- **The `description` key** in a profile is informational only and is excluded from the merge. + +### Resolution Steps + +1. Load the YAML definition file by `name`. +2. Validate that `variants` contains the requested `variant`. +3. Validate that `profiles` contains the requested `profile` (default: `'default'`). +4. Start with a copy of `shared`. +5. Deep-merge the selected profile's configuration into it. +6. Deep-merge the module's `VulnerableEnvironment['overrides']` (if any). +7. Attach the variant-specific `image`, `version`, and `build_args` from the matching variant. + +--- + +## Validation Rules + +The `EnvironmentDefinitionLoader` enforces the following rules at load time: + +1. `name` must match the filename (without `.yml`). +2. `variants` must be a non-empty list. +3. Each variant must have a `name` and an `image`. +4. Variant `name` must be unique across all variants. +5. At most one variant may have `default: true`. +6. `shared.ports` must have at least one entry. +7. `profiles` must have at least one entry. +8. `profiles` must contain a `default` profile. +9. Profile names must match `[a-z0-9-]+`. +10. `health_check` must be defined in `shared` or in **every** profile. +11. Module-level `overrides` are deep-merged into the final profile config. + +**Violation of any rule raises an `ArgumentError` with a descriptive message**, which is surfaced to the user by `test_env build`. + +--- + +## How to Write a New Definition + +### Step 1: Identify the Service + +Determine: +- The vulnerable software and version(s) +- Available container images (Docker Hub, GitHub Container Registry, etc.) +- Exposed ports +- Whether the image is "ready on boot" or requires provisioning + +### Step 2: Create the YAML File + +Create `data/vuln_envs/{servicename}.yml`. The `name` field must match the filename. + +### Step 3: Define Variants + +List every version you want to support. At minimum, define one variant with `default: true`. + +### Step 4: Define `shared.ports` + +Map every port the service listens on inside the container. + +### Step 5: Define Health Checks + +Choose the simplest check that proves the service is **fully ready**, not just "accepting connections." + +- For HTTP services: request a known endpoint and check status + optional `match`. +- For TCP services: `type: tcp` is sufficient. +- For services requiring auth: use the `credentials` sub-key under `health_check`. + +### Step 6: Define Profiles (if needed) + +If the service can run in different configurations (e.g., web console on vs. off), create profiles. + +### Step 7: Add CI Metadata (optional but recommended) + +Add `ci.exploit` and `ci.validation` so the environment can be used in automated verification. + +### Step 8: Validate + +Run: + +``` +load test_env.rb +test_env status +``` + +This validates all definitions and reports any schema errors. + +--- + +## Shared Definitions & Module Referencing + +### Principle: DRY (Don't Repeat Yourself) + +Multiple modules targeting the same vulnerable service/version reference **a single shared environment definition**. + +**Example:** Two ActiveMQ exploit modules share `activemq.yml`: + +- `exploit/multi/http/apache_activemq_jolokia_rce` → uses variant `5.18.6`, profile `default` +- `exploit/multi/misc/apache_activemq_rce_cve_2023_46604` → uses variant `5.18.2`, profile `broker-only` + +Both modules reference the same file but use different variants and profiles. + +### Module Metadata + +Modules declare their environment via `VulnerableEnvironment` inside `update_info()`: + +```ruby +'VulnerableEnvironment' => { + 'definition' => 'activemq', + 'default_variant' => '5.18.6', + 'profile' => 'default', + 'port_mapping' => { 8161 => 'RPORT' } +} +``` + +| Key | Required | Description | +|-----|----------|-------------| +| `definition` | **Yes** | Name of the YAML file (without `.yml`). | +| `default_variant` | **Yes** | Default variant to use if user does not specify `VARIANT=`. | +| `profile` | No | Profile to use. Default: `'default'`. | +| `port_mapping` | **Yes** | Hash mapping `{container_port => 'DATASTORE_OPTION'}`. At minimum, map the primary service port to `RPORT`. | +| `overrides` | No | Module-specific overrides deep-merged into the resolved config. | + +**Example with overrides:** + +```ruby +'VulnerableEnvironment' => { + 'definition' => 'jenkins', + 'default_variant' => '2.361', + 'port_mapping' => { 8080 => 'RPORT' }, + 'overrides' => { + 'health_check' => { + 'path' => '/script', + 'expected_status' => 403 + }, + 'datastore_defaults' => { + 'TARGETURI' => '/script' + } + } +} +``` + +--- + +## CI Metadata + +The `ci` block is designed to make environments self-testing. A CI pipeline (or a human running `test_env validate`) can determine success without hardcoding expectations. + +### Typical CI Flow + +1. `use exploit/multi/http/apache_activemq_jolokia_rce` +2. `test_env build` +3. `test_env exec 1` +4. `test_env validate 1` + +Step 4 checks: +- Was a session created? (`expected_session`) +- Is it the right type? (`session_type`) +- Does `id` output contain `uid=`? (`expected_output`) + +### For Auxiliary Modules + +Set `expected_session: false` and `expected_output` to a substring expected in the scanner output or service response. + +```yaml +ci: + validation: + expected_session: false + expected_output: "Apache" +``` + +--- + +## Best Practices + +### 1. Prefer HTTP Health Checks Over TCP + +A TCP check only proves a port is open. An HTTP check proves the application layer is responding correctly. Use `match` to verify the response body contains expected content. + +### 2. Use `provision` + `verify` for Install Wizards + +If an image boots into an install wizard, define both `provision` (to submit the wizard) and `verify` (to confirm the login page appears afterward). Never register the environment as ready based solely on the install wizard being reachable. + +### 3. Never Hardcode `LHOST` in `ci.exploit` + +Metasploit's auto-detection handles this correctly. Hardcoding `127.0.0.1` causes payloads to call back to the container instead of the host. + +### 4. Document Payload Incompatibilities + +If a module's default payload does not work against your image, document why in a comment and set `ci.exploit.payload` to a working alternative. + +### 5. Keep Variant Names Semantic + +Use actual version numbers (e.g., `5.18.6`) rather than codenames. If you need a different backend for the same version, suffix descriptively (e.g., `5.18.2-postgres`). + +### 6. Profile Names Describe Runtime State + +Use profile names like `default`, `http-stopped`, `broker-only`, `minimal`. The name should tell the user what is different about this configuration. + +### 7. Validate Early + +Run `test_env status` after editing a definition. The loader validates the full schema and reports errors immediately. + +--- + +## Troubleshooting + +| Symptom | Cause | Fix | +|---------|-------|-----| +| `Validation failed: name 'foo' does not match filename 'bar'` | `name` field ≠ filename | Make them identical. | +| `Variant 'X' not defined for 'Y'` | Module requests a variant not in the YAML | Add the variant or correct the module's `default_variant`. | +| `Profile 'X' not defined for 'Y'` | Module requests a profile not in the YAML | Add the profile or correct the module's `profile`. | +| `Port mapping mismatch: module maps port 8080 but environment only exposes ports: 80` | `port_mapping` references a port not in `shared.ports` | Add the port to `shared.ports` or correct the module's `port_mapping`. | +| `Health check timed out` | Service not ready within retry budget | Increase `retries` or `timeout`; verify the health check endpoint/path is correct. | +| `Provisioning failed` | Install wizard submission failed | Check `provision.path` and `provision.body` against the actual install wizard form fields. | +| `Post-provision verification failed` | `verify` check fails after provisioning | Ensure `verify.path` and `verify.match` reflect the post-install state. | + +--- + +## Complete Example: `activemq.yml` + +```yaml +name: activemq +description: Apache ActiveMQ Classic with Jolokia API + +variants: + - name: "5.18.6" + version: "5.18.6" + image: docker.io/apache/activemq-classic:5.18.6 + default: true + + - name: "5.18.2" + version: "5.18.2" + image: docker.io/dinifarb/activemq:5.18.2 + +shared: + ports: + web: 8161 + broker: 61616 + + credentials: + default: + username: admin + password: admin + + datastore_defaults: + TARGETURI: / + + health_check: + type: http + path: /api/jolokia/ + expected_status: 200 + interval: 5 + timeout: 2 + retries: 12 + credentials: + username: admin + password: admin + + ci: + exploit: + payload: cmd/linux/http/x64/meterpreter/reverse_tcp + options: + LPORT: 4444 + validation: + expected_session: true + session_type: meterpreter + expected_output: "uid=" + timeout: 120 + +profiles: + default: + description: Standard ActiveMQ; web console reachable via HTTP/Jolokia. + + broker-only: + description: Web console not assumed reachable; only the broker port is health-checked. + health_check: + type: tcp + ci: + exploit: + force_exploit: true +``` + +--- + +*End of Developer Documentation* diff --git a/docs/test_env/USER_Doc.md b/docs/test_env/USER_Doc.md new file mode 100644 index 0000000000000..483d684b84b55 --- /dev/null +++ b/docs/test_env/USER_Doc.md @@ -0,0 +1,778 @@ +# User Documentation — `test_env` + +> **Scope:** How to use the `test_env` plugin to provision, manage, and exploit vulnerable environments within Metasploit. + +--- + +## Table of Contents + +1. [What Is `test_env`?](#what-is-test_env) +2. [Prerequisites](#prerequisites) +3. [Loading the Plugin](#loading-the-plugin) +4. [Quick Start](#quick-start) +5. [Command Reference](#command-reference) +6. [Typical Workflows](#typical-workflows) +7. [Environment Variables & Options](#environment-variables--options) +8. [Understanding the Registry](#understanding-the-registry) +9. [CI Integration](#ci-integration) +10. [Troubleshooting](#troubleshooting) + +--- + +## What Is `test_env`? + +`test_env` is a Metasploit plugin that automates the provisioning of vulnerable environments for exploit and auxiliary modules. Instead of manually building Docker images, looking up port numbers, and configuring datastore options, `test_env` does it all in a single command: + +``` +msf > use exploit/multi/http/apache_activemq_jolokia_rce +msf exploit(...) > test_env build +[+] Environment ready. +[+] Environment ID: 1 + RHOSTS => 127.0.0.1 + RPORT => 49152 + TARGETURI => / + USERNAME => admin + PASSWORD => admin +Suggested: exploit RHOSTS=127.0.0.1 RPORT=49152 TARGETURI=/ USERNAME=admin PASSWORD=admin +``` + +**Key features:** +- **One-command provisioning**: Builds, launches, health-checks, and configures the module automatically. +- **Dynamic port allocation**: Finds free host ports automatically; supports multiple concurrent environments. +- **Persistent tracking**: Environments are tracked across `msfconsole` restarts via a YAML registry. +- **Runtime abstraction**: Works with Docker (primary) and Podman (including rootless). +- **Automated exploit execution**: `test_env exec ` loads the module, applies the correct datastore, and runs the exploit. +- **Validation**: `test_env validate ` checks whether the exploit produced the expected session and output. + +--- + +## Prerequisites + +### Required + +- **Metasploit Framework** (msfconsole) +- **Docker** or **Podman** installed and available in your `$PATH` + +### Optional + +- **Podman networking backends** (`pasta` or `slirp4netns`) if using rootless Podman + +### Verify Your Setup + +```bash +# Check Docker +docker version + +# Or check Podman +podman version +``` + +--- + +## Loading the Plugin + +### Manual Load + +From within `msfconsole`: + +``` +msf > load test_env +[*] TestEnv plugin loaded. Runtime: docker +[*] Successfully loaded plugin: test_env +``` + +If no runtime is found: + +``` +[-] TestEnv plugin loaded, but no container runtime found. +[-] Install Docker or Podman to use test_env. +``` + +### Automatic Load (Optional) + +Add to your `~/.msf4/msfconsole.rc`: + +``` +load test_env +``` + +--- + +## Quick Start + +### 1. Select a Module + +``` +msf > use exploit/multi/http/apache_activemq_jolokia_rce +``` + +### 2. Build the Environment + +``` +msf exploit(...) > test_env build +``` + +This will: +1. Detect the container runtime (Docker or Podman) +2. Pull the required image +3. Allocate free host ports +4. Launch the container bound to `127.0.0.1` +5. Wait for health checks to pass +6. Apply datastore options (`RHOSTS`, `RPORT`, credentials, etc.) +7. Register the environment and display the suggested exploit command + +### 3. Run the Exploit + +``` +msf exploit(...) > test_env exec 1 +``` + +Or manually: + +``` +msf exploit(...) > exploit +``` + +### 4. Validate (Optional) + +``` +msf exploit(...) > test_env validate 1 +[+] PASS: environment 1 validated successfully against activemq's ci.validation. +``` + +### 5. Clean Up + +``` +msf exploit(...) > test_env remove 1 +``` + +Or remove all: + +``` +msf exploit(...) > test_env remove-all +``` + +--- + +## Command Reference + +### `test_env build [VARIANT=...] [PROFILE=...] [RPORT=...]` + +Build and launch the vulnerable environment for the **active module**. + +**Options:** + +| Option | Description | Example | +|--------|-------------|---------| +| `VARIANT=` | Select a specific software version | `test_env build VARIANT=2.375` | +| `PROFILE=` | Select a runtime profile | `test_env build PROFILE=broker-only` | +| `RPORT=` | Request a specific host port | `test_env build RPORT=8081` | + +**Behavior:** +- If the requested port is unavailable, a dynamic port is allocated instead and you are notified. +- If the module does not define `VulnerableEnvironment`, the command fails with a clear error. +- If health checks fail, the container is automatically stopped and removed. +- If provisioning fails (e.g., WordPress install wizard submission), the container is torn down. + +**Example output:** + +``` +msf exploit(...) > test_env build +[*] Resolving environment for exploit/multi/http/apache_activemq_jolokia_rce... +[*] Definition: activemq | Variant: 5.18.6 | Profile: default +[*] Image: docker.io/apache/activemq-classic:5.18.6 +[*] Pulling image docker.io/apache/activemq-classic:5.18.6... +[+] Image pulled successfully. +[*] Starting container... +[+] Container started: a1b2c3d4e5f6 +[*] Waiting for health check (HTTP)... +[*] Attempt 1/12... +[*] Attempt 2/12... +[+] Health check passed. +[+] Environment ready. +[*] Environment ID: 1 + RHOSTS => 127.0.0.1 + RPORT => 49152 + TARGETURI => / + USERNAME => admin + PASSWORD => admin +[*] Suggested: exploit RHOSTS=127.0.0.1 RPORT=49152 TARGETURI=/ USERNAME=admin PASSWORD=admin +``` + +--- + +### `test_env list` + +Display all tracked environments in a table. + +``` +msf > test_env list + +Test Environments +================= + + ID Container Module RHOST RPORT Status Version + -- --------- ------ ----- ----- ------ ------- + 1 a1b2c3d4e5f6 exploit/multi/http/apache_activemq_jolokia_rce 127.0.0.1 49152 running 5.18.6 + 2 b2c3d4e5f6a7 exploit/unix/webapp/wp_admin_shell_upload 127.0.0.1 49153 running latest + +2 environment(s) tracked. +``` + +--- + +### `test_env modules` + +Scan the framework and list all modules that declare `VulnerableEnvironment` support. + +``` +msf > test_env modules +[*] Scanning framework modules for test_env support... + +Modules with test_env Support +============================= + + Module Definition Variant Profile Ports Image + ------ ---------- ------- ------- ----- ----- + exploit/multi/http/apache_activemq_jolokia_rce activemq 5.18.6 default 8161->RPORT docker.io/apache/activemq-classic:5.18.6 + exploit/multi/misc/apache_activemq_rce_cve_... activemq 5.18.2 broker- 61616->RPORT docker.io/dinifarb/activemq:5.18.2 + exploit/unix/webapp/wp_admin_shell_upload wordpress latest default 80->RPORT docker.io/eystsen/vulnerablewordpress + auxiliary/scanner/http/http_version httpd 2.4.57 default 80->RPORT docker.io/library/httpd:2.4.57 + +Found 4 module(s) with test_env support (scanned 5236 total). +``` + +--- + +### `test_env stop ` + +Stop running container(s) without removing them. Accepts single IDs or ranges. + +``` +msf > test_env stop 1 +[+] Environment 1 stopped. + +msf > test_env stop 1-3,5 +[+] Environment 1 stopped. +[+] Environment 2 stopped. +[+] Environment 3 stopped. +[+] Environment 5 stopped. +``` + +--- + +### `test_env start ` + +Restart a previously stopped environment. Re-runs readiness checks after starting. + +``` +msf > test_env start 1 +[+] Environment 1 started. RPORT=49152 +``` + +> **Note:** `start` accepts only a single ID for safety. + +> **Provisioned environments:** For environments that required one-time provisioning during `build` (e.g., WordPress install wizard), `start` runs the `verify` check instead of the base `health_check`. This is because the container retains its filesystem state across restarts — the service is already configured, not fresh. + +--- + +### `test_env remove ` + +Tear down and remove environment(s). Stops the container if running, removes it, and purges the registry entry. + +``` +msf > test_env remove 1 +[+] Environment 1 removed. + +msf > test_env remove 1-3 +[+] Environment 1 removed. +[+] Environment 2 removed. +[-] Environment 3 not found. +``` + +> **Safety:** If the runtime fails to remove the container, the registry entry is **preserved** so you can retry or clean up manually. + +--- + +### `test_env remove-all` + +Tear down **all** tracked environments and reset the registry. + +``` +msf > test_env remove-all +[*] Tearing down 3 environment(s)... +[+] All environments removed. +``` + +--- + +### `test_env exec [-z|--background]` + +Execute the exploit or auxiliary module against a built environment. + +**What it does:** +1. Loads the module the environment was built for +2. Applies the stored datastore options (`RHOSTS`, `RPORT`, credentials, etc.) +3. Applies the recommended payload from the environment definition (if configured) +4. Allocates fresh local ports for `SRVPORT` / `FETCH_SRVPORT` to avoid bind conflicts on repeated runs +5. Runs the module + +``` +msf > test_env exec 1 +[*] Using exploit/multi/http/apache_activemq_jolokia_rce... +[*] Setting recommended payload for this environment: cmd/linux/http/x64/meterpreter/reverse_tcp +[*] Executing: exploit RHOSTS=127.0.0.1 RPORT=49152 TARGETURI=/ USERNAME=admin PASSWORD=admin +``` + +**Background execution:** + +``` +msf > test_env exec 1 -z +``` + +> **Important:** `exec` works even if you are currently `use`ing a different module. It switches context automatically. + +--- + +### `test_env validate ` + +Check whether an environment's exploit produced the expected result, as defined by the environment's `ci.validation` metadata. + +**For exploit modules:** +- Checks if a session was created +- Verifies session type (if specified) +- Runs a verification command and checks output + +``` +msf > test_env validate 1 +[*] Validating environment 1 (exploit/multi/http/apache_activemq_jolokia_rce) against activemq's ci.validation... +[*] Found 2 session(s) for this module: 1, 2 +[*] Using session 1 (meterpreter) +[+] PASS: environment 1 validated successfully against activemq's ci.validation. +``` + +**For auxiliary modules:** +- Probes the service directly +- Checks response contains expected text +- Re-verifies service health after execution + +``` +msf > test_env validate 4 +[*] Validating environment 4 (auxiliary/scanner/http/http_version) against httpd's ci.validation... +[+] Service response contains expected text: 'Apache' +[+] PASS: environment 4 validated successfully against httpd's ci.validation. +``` + +> **Critical constraint:** Sessions are **process-local**. `validate` only sees sessions created in the **same** `msfconsole` process. If you ran `exec` in a different terminal window, run `validate` there too. + +--- + +### `test_env status` + +Show runtime status, managed container counts, and available environment definitions. + +``` +msf > test_env status +[*] Runtime: docker +[*] Managed containers: 2 total, 2 running +[*] Available definitions: activemq, httpd, openssh, wordpress +``` + +If a definition file has schema errors, they are reported here. + +--- + +### `test_env help` + +Display usage information for all commands. + +``` +msf > test_env help +Usage: test_env + +Commands: + build Build and launch environment for active module + list List tracked environments + modules List all modules with test_env support + stop Stop a running environment + start Restart a stopped environment + remove Tear down an environment + remove-all Tear down all environments + exec Execute exploit against environment + validate Check session/output against ci.validation + status Show runtime status + help Show this help +``` + +--- + +## Typical Workflows + +### Workflow A: Interactive Exploit Development + +``` +msf > use exploit/multi/http/apache_activemq_jolokia_rce +msf exploit(...) > test_env build +[+] Environment ready. Environment ID: 1 +msf exploit(...) > test_env exec 1 +[*] Meterpreter session 1 opened ... +msf exploit(...) > sessions -i 1 +meterpreter > shell +Process 129 created. +Channel 1 created. +id +uid=0(root) gid=0(root) groups=0(root) +exit +meterpreter > exit +msf exploit(...) > test_env remove 1 +[+] Environment 1 removed. +``` + +### Workflow B: Multi-Version Testing + +``` +msf > use exploit/multi/http/apache_activemq_jolokia_rce +msf exploit(...) > test_env build VARIANT=5.18.6 +[+] Environment ID: 1 +msf exploit(...) > test_env build VARIANT=5.18.2 PROFILE=broker-only +[+] Environment ID: 2 +msf exploit(...) > test_env list + ID ... Version Status + -- ... ------- ------ + 1 ... 5.18.6 running + 2 ... 5.18.2 running +msf exploit(...) > test_env exec 1 +msf exploit(...) > test_env exec 2 +msf exploit(...) > test_env remove-all +``` + +### Workflow C: Auxiliary Scanner Testing + +``` +msf > use auxiliary/scanner/http/http_version +msf auxiliary(...) > test_env build +[+] Environment ID: 1 +msf auxiliary(...) > test_env exec 1 +[+] 127.0.0.1:49152 Apache/2.4.57 (Unix) +msf auxiliary(...) > test_env validate 1 +[+] PASS: environment 1 validated successfully. +msf auxiliary(...) > test_env remove 1 +``` + +### Workflow D: Resume After Restart + +``` +msf > load test_env +[*] TestEnv plugin loaded. Runtime: docker +[*] Successfully loaded plugin: test_env +msf > test_env list + ID ... Status + -- ... ------ + 1 ... running + 2 ... running +msf > test_env exec 2 +``` + +--- + +## Environment Variables & Options**************************************** + +### Shell Environment Variables + +| Variable | Description | Example | +|----------|-------------|---------| +| `TEST_ENV_RUNTIME` | Force a specific runtime | `TEST_ENV_RUNTIME=podman msfconsole` | +| `TEST_ENV_PRESERVE` | Preserve containers on msfconsole exit | `TEST_ENV_PRESERVE=true msfconsole` | + +### Metasploit Datastore Options + +| Option | Description | Example | +|--------|-------------|---------| +| `TEST_ENV_RUNTIME` | Same as env var, but set inside msfconsole | `set TEST_ENV_RUNTIME podman` | +| `TEST_ENV_PRESERVE` | Same as env var, but set inside msfconsole | `set TEST_ENV_PRESERVE true` | + +**Preserve behavior:** + +``` +msf > set TEST_ENV_PRESERVE true +TEST_ENV_PRESERVE => true +msf > unload test_env +Unloading plugin test_env...[*] TEST_ENV_PRESERVE is set. Leaving containers running. +[*] Run 'test_env remove-all' manually when done. +unloaded. +msf > load test_env +[*] TestEnv plugin loaded. Runtime: docker +[*] Successfully loaded plugin: test_env +msf > test_env list +Test Environments +================= + + ID Container Module RHOST RPORT Status Version + -- --------- ------ ----- ----- ------ ------- + 1 3883fd8183da exploit/multi/http/ 127.0.0.1 49152 running 5.18.6 + apache_activemq_jol + okia_rce + +[*] 1 environment(s) tracked. +``` +**Do Not Preserve behavior:** + +``` +msf > set TEST_ENV_PRESERVE false +TEST_ENV_PRESERVE => false +msf > unload test_env +Unloading plugin test_env...[*] Auto-cleaning test_env environments... +unloaded. +msf > load test_env +[*] TestEnv plugin loaded. Runtime: docker +[*] Successfully loaded plugin: test_env +msf > load test_env +[*] Auto-cleaning test_env environments... +[*] TestEnv plugin loaded. Runtime: docker +[*] Successfully loaded plugin: test_env +msf > test_env list +[*] No environments currently tracked. +``` + +--- + +## Understanding the Registry + +### Where Is It Stored? + +`~/.msf4/test_env_registry.yml` + +This YAML file persists environment metadata across `msfconsole` restarts. + +### What Is Tracked? + +For each environment: +- **ID**: Monotonic local identifier (never renumbered) +- **Container ID**: Short Docker/Podman container ID +- **Module**: Full module path +- **Version**: Which variant was provisioned +- **RHOST / RPORT**: Connection details +- **Datastore**: All applied options +- **Status**: `running`, `stopped`, or `removed` +- **Allocated ports**: Host-to-container port mappings +- **Temp directories**: Cleanup targets for unnamed volumes + +### Container Labels + +Every container created by `test_env` carries these labels: + +``` +msf.vulnenv.managed_by=test_env +msf.vulnenv.instance_id=msf-hostname-12345 +msf.vulnenv.module=exploit/multi/http/apache_activemq_jolokia_rce +msf.vulnenv.env_id=1 +msf.vulnenv.version=5.18.6 +msf.vulnenv.ports=49152:8161 +``` + +These labels enable **state reconstruction** when `msfconsole` restarts. Even if the YAML registry is lost, `test_env` can discover running containers and rebuild the registry from labels. + +--- + +## CI Integration + +`test_env` is designed to support automated CI pipelines. A typical GitHub Actions workflow: + +```yaml +name: Exploit Verification + +on: [push, pull_request] + +jobs: + verify: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Start Metasploit + test_env + run: | + docker run -d --name msf -v $(pwd):/workspace -e TEST_ENV_PRESERVE=true metasploitframework/metasploit-framework:latest + + - name: Verify ActiveMQ exploit + run: | + docker exec msf msfconsole -q -x " + load /workspace/test_env.rb + use exploit/multi/http/apache_activemq_jolokia_rce + test_env build + test_env exec 1 + test_env validate 1 + test_env remove-all + exit + " +``` + +**Key points:** +- Environment definitions are the **single source of truth** for images, health checks, and validation expectations. +- `ci.exploit` recommends payloads that are known to work against the image. +- `ci.validation` defines what "success" means for each module. +- `exec` and `validate` must run in the **same process** (use a single `msfconsole -x` script). + +--- + +## Troubleshooting + +### Plugin fails to load + +``` +[-] TestEnv plugin loaded, but no container runtime found. +``` + +**Fix:** Install Docker or Podman and ensure the binary is in your `$PATH`. + +```bash +# Docker +sudo systemctl start docker + +# Podman (rootless) +podman version +``` + +--- + +### `test_env build` fails: "No active module" + +``` +[-] No active module. Use 'use ' first. +``` + +**Fix:** Select a module that supports `test_env`: + +``` +msf > use exploit/multi/http/apache_activemq_jolokia_rce +``` + +Or scan for supported modules: + +``` +msf > test_env modules +``` + +--- + +### `test_env build` fails: "Module does not define a vulnerable environment configuration" + +**Fix:** The selected module has no `VulnerableEnvironment` metadata. Either: +- Use a different module +- Contribute a definition and module metadata (see Developer Documentation) + +--- + +### Port allocation fails + +``` +[-] No available ports: No available ports in range 49152-65535 +``` + +**Fix:** You have too many services bound to ephemeral ports. Run `test_env remove-all` to clean up, or manually stop orphaned containers: + +```bash +docker ps -q --filter "label=msf.vulnenv.managed_by=test_env" | xargs docker stop +docker ps -aq --filter "label=msf.vulnenv.managed_by=test_env" | xargs docker rm +``` + +--- + +### Health check times out + +``` +[-] Health check timed out after 60 seconds +``` + +**Common causes:** +- The image is still downloading layers. Wait and retry. +- The service inside the container crashed. Check logs: + ```bash + docker logs + ``` +- The health check endpoint is wrong. Verify the environment definition. + +--- + +### `test_env exec` fails with "No session created" + +**Causes:** +- The exploit failed. Check the module output for errors. +- `LHOST` is misconfigured. Do not hardcode `LHOST` in environment definitions. +- The target is not actually vulnerable (wrong version, patched image). + +--- + +### `test_env validate` fails: "no session was created" + +``` +[-] FAIL: no session was created within 120s +``` + +**Causes:** +- You ran `exec` and `validate` in **different** `msfconsole` processes. Sessions are process-local. +- The exploit genuinely failed. Run `test_env exec ` again and watch the output. +- The timeout is too short for slow payloads. The definition's `ci.validation.timeout` may need adjustment. + +**Fix:** Run both commands in the same msfconsole: + +``` +msf > test_env exec 1 +msf > test_env validate 1 +``` + +--- + +### Podman rootless networking issues + +``` +[-] Rootless Podman detected but no networking backend (pasta or slirp4netns) found. +``` + +**Fix:** Install `pasta` or `slirp4netns`: + +```bash +# Debian/Ubuntu +sudo apt install passt + +# Fedora +sudo dnf install passt + +# Or slirp4netns +sudo apt install slirp4netns +``` + +--- + +### `test_env remove` says "Failed to remove container" but registry entry is gone + +**Actually, this won't happen.** The implementation specifically **preserves** the registry entry if `runtime.remove` fails. You can retry: + +``` +msf > test_env remove 1 +[-] Failed to remove container for environment 1: ... +msf > test_env remove 1 # retry +[+] Environment 1 removed. +``` + +If the container was removed manually (outside `test_env`), run `test_env status` or reload the plugin to trigger pruning. + +--- + +## Summary of Commands + +| Command | Purpose | Args | +|---------|---------|------| +| `test_env build` | Provision environment | `[VARIANT=]` `[PROFILE=]` `[RPORT=]` | +| `test_env list` | Show tracked environments | — | +| `test_env modules` | Discover supported modules | — | +| `test_env stop` | Stop container(s) | `` | +| `test_env start` | Restart container | `` | +| `test_env remove` | Remove environment(s) | `` | +| `test_env remove-all` | Remove everything | — | +| `test_env exec` | Run exploit/module | `` `[-z]` | +| `test_env validate` | Verify exploit success | `` | +| `test_env status` | Show runtime + definitions | — | +| `test_env help` | Show usage | — | + +--- + diff --git a/docs/test_env/reference_modules.md b/docs/test_env/reference_modules.md index 4071878e26f6b..09753f59b5897 100644 --- a/docs/test_env/reference_modules.md +++ b/docs/test_env/reference_modules.md @@ -50,3 +50,60 @@ - **Credentials:** none required — this CVE is unauthenticated - **Exploit Context:** Unauthenticated; sends crafted OpenWire packet that loads attacker-hosted Spring XML config. Requires TARGET => 1 (Linux) override — default target is Windows. +--- + +## Module 4: HTTP Version Scanner (Auxiliary) +- **Path:** `auxiliary/scanner/http/http_version` +- **Type:** Auxiliary scanner (no session produced) +- **Port:** 80 +- **Profile:** `default` +- **Health Check:** HTTP GET `/` expecting 200 +- **Why:** Trivial scanner module suggested by mentor. Demonstrates that `test_env` works for auxiliary modules, not just exploits. Produces `[+]` output (server banner) without requiring a shell. +- **VulnerableEnvironment Definition:** `httpd` (shared with http_header and robots_txt) +- **Docker Image:** `docker.io/library/httpd:2.4.57` +- **Credentials:** none required +- **Scanner Context:** Detects HTTP server version from response headers. No payload, no session. + +--- + +## Module 5: HTTP Header Scanner (Auxiliary) +- **Path:** `auxiliary/scanner/http/http_header` +- **Type:** Auxiliary scanner (no session produced) +- **Port:** 80 +- **Profile:** `default` +- **Health Check:** HTTP GET `/` expecting 200 +- **Why:** Reuses the same `httpd` definition as `http_version`, proving shared definitions work across multiple independent auxiliary modules. +- **VulnerableEnvironment Definition:** `httpd` +- **Docker Image:** `docker.io/library/httpd:2.4.57` +- **Credentials:** none required +- **Scanner Context:** Displays HTTP response headers. Uses `IGN_HEADER`, `HTTP_METHOD`, and `TARGETURI` options. + +--- + +## Module 6: HTTP Robots.txt Scanner (Auxiliary) +- **Path:** `auxiliary/scanner/http/robots_txt` +- **Type:** Auxiliary scanner (no session produced) +- **Port:** 80 +- **Profile:** `default` +- **Health Check:** HTTP GET `/` expecting 200 +- **Why:** Third auxiliary module reusing `httpd`. Demonstrates that shared definitions scale to many modules without duplication. +- **VulnerableEnvironment Definition:** `httpd` +- **Docker Image:** `docker.io/library/httpd:2.4.57` +- **Credentials:** none required +- **Scanner Context:** Detects and analyzes `robots.txt` content. Uses `PATH` option. + +--- + +## Module 7: SSH Version Scanner (Auxiliary) +- **Path:** `auxiliary/scanner/ssh/ssh_version` +- **Type:** Auxiliary scanner (no session produced) +- **Port:** 22 +- **Profile:** `default` +- **Health Check:** TCP connect on port 22 (SSH daemon accepts connections immediately) +- **Why:** Demonstrates non-HTTP auxiliary scanner with a different health check type (`tcp` instead of `http`). Produces `[+]` output with SSH banner and encryption details. +- **VulnerableEnvironment Definition:** `openssh` +- **Docker Image:** `docker.io/rastasheep/ubuntu-sshd:16.04` +- **Credentials:** root / root (defined in YAML for completeness, but scanner does not use them) +- **Scanner Context:** Detects SSH version and supported ciphers. No payload, no session. + + diff --git a/plugins/test_env.rb b/plugins/test_env.rb index 87a3322431342..6fbefca80fbaa 100644 --- a/plugins/test_env.rb +++ b/plugins/test_env.rb @@ -25,7 +25,7 @@ class VulnerableEnvironment attr_reader :definition, :default_variant, :profile, :port_mapping, :overrides # Required keys that must be present - REQUIRED_KEYS = %w[definition default_variant port_mapping].freeze + REQUIRED_KEYS = %w[definition default_variant port_mapping].freeze unless defined?(REQUIRED_KEYS) # Valid types for each key SCHEMA = { @@ -34,7 +34,7 @@ class VulnerableEnvironment 'profile' => String, 'port_mapping' => Hash, 'overrides' => Hash - }.freeze + }.freeze unless defined?(SCHEMA) def initialize(raw_hash) @raw = raw_hash || {} @@ -709,7 +709,7 @@ def self.parse_time(val) class VulnEnvironment include ActiveModel::Validations - DEFAULT_VERSION = '1.0.0' + DEFAULT_VERSION = '1.0.0' unless defined?(DEFAULT_VERSION) attr_accessor :version, :instance_id @@ -795,7 +795,7 @@ def valid_targets # VulnEnvironmentStore — YAML persistence in ~/.msf4/ # ===================================================================== class VulnEnvironmentStore - DEFAULT_PATH = File.join(Dir.home, '.msf4', 'test_env_registry.yml') + DEFAULT_PATH = File.join(Dir.home, '.msf4', 'test_env_registry.yml') unless defined?(DEFAULT_PATH) def initialize(path = DEFAULT_PATH) @path = path @@ -984,36 +984,62 @@ def reconstruct_state(runtime) # RECONSTRUCT: add containers found by labels but missing from registry containers = runtime.list(filters: { 'label' => 'msf.vulnenv.managed_by=test_env' }) + elog("reconstruct_state: found #{containers.length} managed container(s)") containers.each do |container| labels = container.dig('Config', 'Labels') || container['Labels'] || {} + if labels.is_a?(String) + labels = labels.split(',').each_with_object({}) do |pair, hash| + key, value = pair.split('=', 2) + hash[key] = value || '' + end + end container_id = normalize_container_id(container['ID'] || container['Id']) # Identify by container_id, NOT by env_id label (labels are immutable # and become stale after ID compaction) - next if @vuln_env.find_by_container(container_id) + if @vuln_env.find_by_container(container_id) + elog("reconstruct_state: skip #{container_id} — already in registry") + next + end module_fullname = labels['msf.vulnenv.module'] version = labels['msf.vulnenv.version'] ports = decode_port_label(labels['msf.vulnenv.ports']) + unless module_fullname && !module_fullname.empty? + elog("reconstruct_state: skip #{container_id} — missing msf.vulnenv.module label") + next + end + mod = @framework.modules.create(module_fullname) rescue nil - next unless mod + unless mod + elog("reconstruct_state: skip #{container_id} — could not load module '#{module_fullname}'") + next + end vuln_env_meta = mod.send(:module_info)['VulnerableEnvironment'] rescue nil - next unless vuln_env_meta + unless vuln_env_meta + elog("reconstruct_state: skip #{container_id} — module has no VulnerableEnvironment") + next + end definition_name = vuln_env_meta['definition'] profile = vuln_env_meta['profile'] || 'default' overrides = vuln_env_meta['overrides'] || {} loader = EnvironmentDefinitionLoader.new(Msf::Config.data_directory) - config = loader.resolve(definition_name, version, profile, overrides) rescue nil - next unless config + begin + config = loader.resolve(definition_name, version, profile, overrides) + rescue => e + elog("reconstruct_state: skip #{container_id} — resolve failed for #{definition_name}/#{version}: #{e.message}") + next + end datastore = { 'RHOSTS' => '127.0.0.1' } - vuln_env_meta['port_mapping'].each do |container_port, ds_option| - datastore[ds_option] = ports[container_port] if ports[container_port] + (vuln_env_meta['port_mapping'] || {}).each do |container_port, ds_option| + host = ports[container_port.to_i] || ports[container_port.to_s.to_i] + datastore[ds_option] = host if host end if config['datastore_defaults'] @@ -1022,10 +1048,23 @@ def reconstruct_state(runtime) end end - created_time = Time.parse(container['Created']) rescue Time.now - started_time = Time.parse(container.dig('State', 'StartedAt')) rescue Time.now + # Merge default credentials the same way build does + if config['credentials'] && config['credentials']['default'] + config['credentials']['default'].each do |key, value| + ds_key = key.to_s.upcase + datastore[ds_key] = value unless datastore.key?(ds_key) + end + end - assigned_id = @vuln_env.next_id + created_time = Time.parse(container['Created']) rescue Time.now + started_time = Time.parse(container.dig('State', 'StartedAt') || container['StartedAt']) rescue Time.now + + preferred_id = labels['msf.vulnenv.env_id'].to_i + assigned_id = if preferred_id > 0 && @vuln_env.find_by_id(preferred_id).nil? + preferred_id + else + @vuln_env.next_id + end target = VulnTarget.new( local_id: assigned_id, container_id: container_id, @@ -1040,12 +1079,13 @@ def reconstruct_state(runtime) ) @vuln_env.add_target(target) - print_status("Reconstructed environment #{assigned_id} from container labels.") + elog("reconstruct_state: reconstructed environment #{assigned_id} from container #{container_id}") end @store.save(@vuln_env) rescue => e elog("Label reconstruction failed: #{e.message}") + elog(e.backtrace.join("\n")) if e.backtrace end def find_by_container(container_id) @@ -1245,7 +1285,7 @@ def check_command # schema or admin account yet, so nothing is actually exploitable until # this runs. class Provisioner - PROVISION_MARKER = '/tmp/.msf_test_env_provisioned'.freeze + PROVISION_MARKER = '/tmp/.msf_test_env_provisioned'.freeze unless defined?(PROVISION_MARKER) def initialize(runtime, container_id, provision_config, host_port, dispatcher = nil) @runtime = runtime @@ -1632,14 +1672,22 @@ def cmd_test_env_build(args) ) registered = true - # Step 17: apply datastore to the active module + # Step 17: free host ports for stage/fetch payloads (avoids Rex::BindFailed on 8080) + %w[SRVPORT FETCH_SRVPORT].each do |opt| + if mod.options.include?(opt) + free_port = free_local_port + datastore[opt] = free_port + end + end + + # Step 18: apply datastore to the active module datastore.each do |key, value| if mod.options.include?(key) mod.datastore[key] = value end end - # Step 18: display results to user + # Step 19: display results to user build_display_results(env_id, config, datastore, mod) rescue PortAllocator::NoPortsAvailable => e @@ -1915,6 +1963,14 @@ def build_display_results(env_id, config, datastore, mod) print_good("Environment ready.") print_status("Environment ID: #{env_id}") + %w[SRVPORT FETCH_SRVPORT].each do |opt| + if mod.options.include?(opt) + free_port = free_local_port + mod.datastore[opt] = free_port + datastore[opt] = free_port + end + end + applicable = datastore.select { |k, _v| mod.options.include?(k) } applicable.each do |key, value| print_status(" #{key.ljust(12)} => #{value}") @@ -1924,6 +1980,7 @@ def build_display_results(env_id, config, datastore, mod) opts = applicable.map { |k, v| "#{k}=#{v}" }.join(' ') print_status("Suggested: #{action} #{opts}") end + def cmd_test_env_help print_line("Usage: test_env ") print_line @@ -2324,7 +2381,7 @@ def cmd_test_env_validate(args) if config && config['health_check'] primary_port = target.allocated_ports[env_meta.port_mapping.key('RPORT')] health_port = primary_port || target.allocated_ports.values.first - runtime = RuntimeAdapter.detect rescue nil + runtime = self.class.runtime || (RuntimeAdapter.detect(framework.datastore) rescue nil) if runtime && health_port HealthManager.new(runtime, target.container_id, @@ -2786,7 +2843,8 @@ def parse_id_range(range_str) # ===================================================================== def initialize(framework, opts = nil) super(framework, opts) - @runtime = RuntimeAdapter.detect + # Prefer framework.datastore (from 'set TEST_ENV_RUNTIME ...') over ENV + @runtime = RuntimeAdapter.detect(framework.datastore) @registry = BuiltEnvironmentRegistry.new(framework) ConsoleCommandDispatcher.runtime = @runtime @@ -2802,7 +2860,14 @@ def initialize(framework, opts = nil) print_error("TestEnv plugin loaded, but no container runtime found.") print_error("Install Docker or Podman to use test_env.") end - @registry.reconstruct_state(@runtime) if @runtime + + if @runtime + before = @registry.list.length + @registry.reconstruct_state(@runtime) + added = @registry.list.length - before + print_status("Reconstructed #{added} environment(s) from running containers.") if added > 0 + end + add_console_dispatcher(ConsoleCommandDispatcher) end