Skip to content

Refactor snippet imports and parsing logic - #2

Closed
dyay108 wants to merge 5 commits into
mcsdodo:mainfrom
dyay108:feature/complex-snippet-directives-support
Closed

dyay108 wants to merge 5 commits into
mcsdodo:mainfrom
dyay108:feature/complex-snippet-directives-support

Conversation

@dyay108

@dyay108 dyay108 commented Feb 1, 2026 •

Copy link
Copy Markdown
Contributor

Refactor snippet import handling to support import_N and args. Enhance parsing of named matchers, handle blocks, and transport configurations.

Summary by CodeRabbit

  • New Features

    • Snippet import system with parameterized imports and argument substitution for reusable route fragments.
    • Named matcher and handle-block support to generate per-block routes with domain/matcher-aware logic.
    • Per-prefix transport and header configuration, including resolver support and remote-IP range parsing.
    • Static-response and header handler capabilities for route-level responses.
  • Improvements

    • Enhanced logging and warnings for imports, substitutions, and merged configuration parsing.
  • Chores

    • CI workflow added to build and publish container images to the registry.

✏️ Tip: You can customize this high-level summary in your review settings.

Refactor snippet import handling to support import_N and args. Enhance parsing of named matchers, handle blocks, and transport configurations.
@coderabbitai

coderabbitai Bot commented Feb 1, 2026 •

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Adds a snippet import/substitution system and expands route-building: named matcher and handle_N parsing, per-prefix transport/header extraction, handler builders (headers/respond), remote-IP parsing, per-handle route generation, and a GitHub Actions workflow to build/publish the container image.

Changes

Cohort / File(s) Summary
Core CLI / agent
caddy-agent-watch.py
Major additions and refactors: snippet import system, import argument substitution and application; named matcher parsing; handle_N block parsing and per-handle route generation; per-prefix transport/header extraction; handler builders (header, respond); remote-IP parsing; extended logging and warnings.
CI / distribution
.github/workflows/publish-container-gh.yml
New GitHub Actions workflow to build and push Docker image to GHCR on dispatch and main pushes; metadata generation and permission scoping added.
Snippet handling helpers
caddy-agent-watch.py
Added parse_imports(), substitute_import_args(), apply_snippet_imports() to discover, substitute, and merge snippet directives (removes import_* keys).
Routing & matchers
caddy-agent-watch.py
Added parse_named_matchers(), parse_handle_blocks(); parse flow updated to build per-handle routes and integrate matchers; parse_container_labels now invokes snippet application.
Per-prefix transport & headers
caddy-agent-watch.py
Added parse_transport_config_for_prefix() and parse_header_config_for_prefix() to collect transport and header_up/down settings scoped to prefixes (e.g., handle_0.reverse_proxy).
Handler builders & utilities
caddy-agent-watch.py
Added parse_header_handler(), parse_respond_handler(), parse_remote_ip_ranges() and related utilities for constructing handlers and matcher data.
Logging & diagnostics
caddy-agent-watch.py
New debug/info logs for import parsing, substitutions, merged configs, and warnings for missing/unresolved snippets.

Sequence Diagram

sequenceDiagram
    participant Config as Configuration
    participant ImportParser as parse_imports()
    participant SnippetStore as Snippet Registry
    participant Substituter as substitute_import_args()
    participant ImportApplier as apply_snippet_imports()
    participant RouteBuilder as Route Builder (parse_container_labels / parse_handle_blocks)
    participant HandlerBuilder as Handlers (header/respond / transport/header per-prefix)

    Config->>ImportParser: extract import / import_N directives
    ImportParser-->>Config: ordered snippet-name/args pairs
    ImportApplier->>SnippetStore: lookup snippet definitions
    SnippetStore-->>Substituter: snippet values
    Substituter-->>ImportApplier: values with {args[N]} substituted
    ImportApplier->>RouteBuilder: merge substituted directives into route config
    RouteBuilder->>HandlerBuilder: construct per-handle handlers and apply per-prefix transport/headers
    RouteBuilder-->>Config: emit final route entries
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🐇 I nibble through imports, one arg at a time,
Stitching snippets, shaping each route line,
Handles hop in, matchers follow the trail,
Headers and transport set sail,
A tiny rabbit logs the final compile 🥕

🚥 Pre-merge checks | ✅ 2 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 58.82% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Refactor snippet imports and parsing logic' accurately captures the main changes in the PR, which focus on restructuring snippet import handling and enhancing parser functions.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Fix all issues with AI agents
In `@caddy-agent-watch.py`:
- Around line 445-474: The apply_snippet_imports function logs raw snippet args
and merged config which may contain secrets; update logging to avoid printing
sensitive data by (1) changing the logger.info call that currently includes args
to only log non-sensitive context (e.g., snippet_name and route_num) and remove
args from the message, (2) remove or replace the logger.debug that dumps the
full config/merged_config with either a redacted summary (e.g., log only keys,
not values) or a masked/hashed representation, and (3) ensure any warnings
(e.g., snippet not found) do not include args or config values; update
references in apply_snippet_imports to stop emitting sensitive variables
(snippet_name, args, merged_config, config) in logs and, if needed, implement a
small redact_keys helper to mask known secret keys before logging keys-only
summaries.
- Around line 827-829: The current loop sorting handle_blocks by
int(k.split('_')[1]) can raise ValueError for non-numeric suffixes; update the
sort key used in the for loop over handle_blocks to a safe extractor (e.g., a
small helper/inline lambda) that tries to parse the numeric suffix after the
underscore and returns a tuple such as (0, number) when numeric or (1,
original_suffix_or_key) when not numeric so non-numeric keys sort
deterministically instead of crashing; modify the sorting expression where
handle_key is computed (the for handle_key in sorted(handle_blocks.keys(), ... )
line) to use this safe key so block and block_directives processing continues
even with malformed labels.

Comment thread caddy-agent-watch.py
Comment thread caddy-agent-watch.py Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In `@caddy-agent-watch.py`:
- Around line 562-582: In parse_respond_handler, the handler currently treats a
single numeric token as the body; change the parsing so that when parts has one
element and parts[0].isdigit() you set status_code = int(parts[0]) and body = ""
(empty string); otherwise preserve the existing behavior where body = parts[0]
and if parts[1].isdigit() set status_code from parts[1]. This fixes the
status-only case (e.g., "respond 404") while keeping shlex and the fallback
split logic intact.
🧹 Nitpick comments (3)
caddy-agent-watch.py (3)

436-448: Consider logging when arg placeholder index is out of bounds.

When {args[N]} references an index beyond the provided args, the function silently returns an empty string. This could hide configuration errors where a snippet expects more arguments than provided.

💡 Optional: Add debug logging for missing args
     def repl(match):
         idx = int(match.group(1))
-        return args[idx] if idx < len(args) else ""
+        if idx < len(args):
+            return args[idx]
+        logger.debug(f"Arg placeholder {{args[{idx}]}} has no corresponding value (only {len(args)} args provided)")
+        return ""

619-668: Consider extracting shared logic with parse_header_config.

This function duplicates ~50 lines of header parsing logic from parse_header_config (lines 1019-1099). Both handle the same header_up/header_down parsing with identical delete/set logic.

♻️ Suggested approach to reduce duplication

Extract a common helper that both functions can use:

def _parse_header_value(value, target_dict, direction):
    """Helper to parse header_up/header_down value into target dict."""
    if value.startswith('-'):
        header_name = value[1:].strip()
        if 'delete' not in target_dict:
            target_dict['delete'] = []
        target_dict['delete'].append(header_name)
    else:
        clean_value = value[1:].strip() if value.startswith('+') else value
        parts = clean_value.split(None, 1)
        if len(parts) == 2:
            header_name, header_value = parts[0], parts[1].strip('"')
            if 'set' not in target_dict:
                target_dict['set'] = {}
            target_dict['set'][header_name] = [header_value]

def parse_header_config_for_prefix(config, prefix):
    headers_config = {}
    for key, value in config.items():
        if key.startswith(f"{prefix}.header_up"):
            # ... use _parse_header_value for 'request'
        elif key.startswith(f"{prefix}.header_down"):
            # ... use _parse_header_value for 'response'
    return headers_config

Then parse_header_config can delegate: return parse_header_config_for_prefix(config, 'reverse_proxy')


821-901: Nested function captures loop variables - consider refactoring.

The build_routes_for_domains function captures multiple outer variables (https_domains, http_only_domains, handle_blocks, config, named_matchers, route_num, base_route_id, domain) which triggers B023 late-binding warnings. While this works correctly because the function is called immediately within the same loop iteration, it makes the code harder to reason about and fragile if refactored.

Note: The handle_sort_key safe sorting (lines 831-835) correctly addresses the previous review concern about ValueError on malformed handle keys.

♻️ Option 1: Pass captured variables as parameters
-        def build_routes_for_domains(domains, http_only_flag):
+        def build_routes_for_domains(domains, http_only_flag, *,
+                                      handle_blocks, named_matchers, config,
+                                      base_route_id, route_num, domain,
+                                      https_domains, http_only_domains):
             if not domains:
                 return
             # ... rest of function uses parameters instead of closures
         
-        build_routes_for_domains(http_only_domains, True)
-        build_routes_for_domains(https_domains, False)
+        build_routes_for_domains(http_only_domains, True,
+            handle_blocks=handle_blocks, named_matchers=named_matchers,
+            config=config, base_route_id=base_route_id, route_num=route_num,
+            domain=domain, https_domains=https_domains, http_only_domains=http_only_domains)
+        build_routes_for_domains(https_domains, False, ...)
♻️ Option 2: Extract to module-level function

Move build_routes_for_domains outside parse_container_labels as a module-level helper function, passing all required context as parameters. This improves testability and eliminates the closure issue entirely.

Comment thread caddy-agent-watch.py
Comment on lines +562 to +582
def parse_respond_handler(value):
import shlex

try:
parts = shlex.split(value)
except Exception:
parts = value.split()

if not parts:
return None

body = parts[0]
status_code = 200
if len(parts) > 1 and parts[1].isdigit():
status_code = int(parts[1])

return {
"handler": "static_response",
"body": body,
"status_code": status_code
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Bug: Status-only respond directive parsed incorrectly.

When the respond value is a status code only (e.g., respond 404), the current logic incorrectly treats it as body="404" with status=200. In Caddyfile syntax, respond 404 should return status 404 with an empty body.

🐛 Proposed fix to handle status-only respond
 def parse_respond_handler(value):
     import shlex

     try:
         parts = shlex.split(value)
     except Exception:
         parts = value.split()

     if not parts:
         return None

-    body = parts[0]
-    status_code = 200
-    if len(parts) > 1 and parts[1].isdigit():
-        status_code = int(parts[1])
+    # Handle status-only: respond 404
+    if len(parts) == 1 and parts[0].isdigit():
+        return {
+            "handler": "static_response",
+            "status_code": int(parts[0])
+        }
+
+    body = parts[0]
+    status_code = 200
+    if len(parts) > 1 and parts[1].isdigit():
+        status_code = int(parts[1])

     return {
         "handler": "static_response",
         "body": body,
         "status_code": status_code
     }
🧰 Tools
🪛 Ruff (0.14.14)

[warning] 567-567: Do not catch blind exception: Exception

(BLE001)

🤖 Prompt for AI Agents
In `@caddy-agent-watch.py` around lines 562 - 582, In parse_respond_handler, the
handler currently treats a single numeric token as the body; change the parsing
so that when parts has one element and parts[0].isdigit() you set status_code =
int(parts[0]) and body = "" (empty string); otherwise preserve the existing
behavior where body = parts[0] and if parts[1].isdigit() set status_code from
parts[1]. This fixes the status-only case (e.g., "respond 404") while keeping
shlex and the fallback split logic intact.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Fix all issues with AI agents
In @.github/workflows/publish-container-gh.yml:
- Line 19: Update the GitHub Actions step that uses actions/checkout@v3 to the
current stable release by replacing the usage reference with
actions/checkout@v6; locate the line containing "uses: actions/checkout@v3" in
the workflow and change it to "uses: actions/checkout@v6" to ensure the action
runs on supported GitHub Actions runtimes.

packages: write # Required to push to GHCR

steps:
- uses: actions/checkout@v3

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

actions/checkout v3 deprecated GitHub Actions latest version 2025

💡 Result:

  • Yes—actions/checkout@v3 is effectively deprecated/legacy because it was built for the Node.js 16 GitHub Actions runtime, and GitHub completed the platform transition away from Node 16. GitHub began enforcing Node 20 by default on June 3, 2024 [2], and removed Node 16 from the Actions runner on November 12, 2024, so Node 16–based actions can no longer run [3].
  • Latest actions/checkout major version (as of Feb 2026): v6. GitHub’s official repo/Marketplace docs show Checkout v6 as the current major version [1].
  • What to use in 2025+ workflows: switch to a maintained major tag, typically:
    - uses: actions/checkout@v6
    (Pinning to the major tag is the recommended pattern for GitHub-maintained actions.) [1]

Sources: official actions/checkout repo/Marketplace page [1], GitHub Actions changelog (Node20 enforcement + Node16 end-of-life) [2][3].


Update deprecated actions/checkout@v3 to latest version to fix workflow failure.

actions/checkout@v3 is deprecated because it relies on the Node.js 16 runtime, which GitHub removed from Actions runners on November 12, 2024. The workflow will fail at runtime without an update. The latest stable version is v6.

🧰 Tools
🪛 actionlint (1.7.10)

[error] 19-19: the runner of "actions/checkout@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue

(action)

🤖 Prompt for AI Agents
In @.github/workflows/publish-container-gh.yml at line 19, Update the GitHub
Actions step that uses actions/checkout@v3 to the current stable release by
replacing the usage reference with actions/checkout@v6; locate the line
containing "uses: actions/checkout@v3" in the workflow and change it to "uses:
actions/checkout@v6" to ensure the action runs on supported GitHub Actions
runtimes.

@dyay108 dyay108 closed this Feb 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant