[Issue #473] Add Consul KV variable source

[DeekshithaTimmareddy]

Description

Description

This PR implements Subtask 2a of Issue #473 (External Variable Sources for
nomad-pack). It adds support for fetching pack variables from Consul KV, a shared
JSON→cty conversion utility, and the --var-source CLI flag with full
integration.

Key behaviors

• URL-based config following standard URL rules:

  • consul:///<prefix> — Consul address from the environment (e.g. CONSUL_HTTP_ADDR)
  • consul://<host>:<port>/<prefix> — explicit Consul address

• Default KV layout: <prefix>/<pack-name>/<variable-name>.
  Pass ?full-path=true to use <prefix> as-is (pack name not appended).

• Type-aware conversion: a Pack variable typed string keeps the raw Consul
  bytes; only non-string types are JSON-decoded — no type-guessing before the
  schema is known.

• Parse/build split: only run, plan, and render read remote sources.
  stop and generate varfile do not accept --var-source, so stopping a
  job never depends on a remote source still being reachable.

• Variable precedence (highest → lowest):

  1. CLI flags (--var)
  2. External sources (--var-source)
  3. Variable files (--var-file)
  4. Environment variable

Testing

# 1. Start Consul
consul agent -dev

# 2. Store variables in Consul KV
#    Default layout is <prefix>/<pack-name>/<variable-name>
consul kv put nomad-pack/my-pack/replicas 3
consul kv put nomad-pack/my-pack/region "us-west-2"

# 3. Render pack with Consul variables  (note the THREE slashes: consul:///)
nomad-pack render my-pack --var-source=consul:///nomad-pack

# 4. Priority: CLI --var overrides Consul
nomad-pack render my-pack --var-source=consul:///nomad-pack --var replicas=5

# 5. Explicit Consul address instead of CONSUL_HTTP_ADDR
nomad-pack render my-pack --var-source=consul://localhost:8500/nomad-pack

# 6. Use the prefix exactly as-is (do NOT append the pack name)
nomad-pack render my-pack --var-source='consul:///my/custom/path?full-path=true'

Test Data

Store variables in Consul:

Priority Override:

Multiple sources:

Priority: CLI > Consul File

Reminders

• Add CHANGELOG.md entry

• If a change needs to be reverted, we will roll out an update to the code within 7 days.

Changes to Security Controls

Are there any changes to security controls (access controls, encryption, logging) in this pull request? If so, explain.

[tgross]

Still no consumer code.

[DeekshithaTimmareddy]

Still no consumer code.

Added! adding integration tests and documentation will make this pr huge so limiting this pr to CLI integration and consumer code only

[wiz-inc-0e7a25329d]

Wiz Scan Summary

Scanner	Findings
Vulnerabilities	-
Sensitive Data	-
Secrets	-
IaC Misconfigurations	-
SAST Findings	1
Software Management Findings	-
Total	1

View scan details in Wiz

To detect these findings earlier in the dev lifecycle, try using Wiz Code VS Code Extension.

[tgross]

Tested end-to-end:

$ nomad-pack generate pack example
...

$ consul kv put pack/count 3
Success! Data written to: pack/count

$ consul kv put pack/datacenters '["dc1"]'
Success! Data written to: pack/datacenters

$ consul kv put pack/message Hi
Success! Data written to: pack/message

$ consul kv put pack/region philly
Success! Data written to: pack/region

$ consul kv put pack/register_service true
Success! Data written to: pack/register_service

$ consul kv put pack/service_name "www"
Success! Data written to: pack/service_name

$ consul kv put pack/service_tags '["whatever"]'
Success! Data written to: pack/service_tags

$ nomad-pack render --var-source consul:///pack example
example/example.nomad:

job "example" {
  region      = "philly"
  datacenters = ["dc1"]
  type        = "service"

  group "app" {
    count = 3

    network {
      port "http" {
        to = 8000
      }
    }


    service {
      name     = "www"
      tags     = ["whatever"]
      provider = "nomad"
      port     = "http"
      check {
        name     = "alive"
        type     = "http"
        path     = "/"
        interval = "10s"
        timeout  = "2s"
      }
    }


    restart {
      attempts = 2
      interval = "30m"
      delay    = "15s"
      mode     = "fail"
    }

    task "server" {
      driver = "docker"

      config {
        image = "mnomitch/hello_world_server"
        ports = ["http"]
      }

      env {
        MESSAGE = "Hi"
      }
    }
  }
}

[tgross]

This is looking pretty good! A few more minor items to resolve and I think this will be ready to merge.

[DeekshithaTimmareddy]

This is looking pretty good! A few more minor items to resolve and I think this will be ready to merge.

After this pr got merged, i will be focusing on integration tests and also documentation of this consul source in the follow up pr, let me know your comments

[tgross]