Skip to content

docs: clarify LiteLLM Proxy gateway semantics#27

Merged
rlippmann merged 1 commit into
mainfrom
llm-middleware-hardening
Jun 29, 2026
Merged

docs: clarify LiteLLM Proxy gateway semantics#27
rlippmann merged 1 commit into
mainfrom
llm-middleware-hardening

Conversation

@rlippmann

Copy link
Copy Markdown
Owner

Summary

Clarifies the LiteLLM Proxy reference integration by aligning terminology, documentation, and test naming with current Context Compiler behavior.

No runtime behavior changes are included.

Why

The integration already demonstrates gateway enforcement before downstream model calls, but documentation and tests used a mixture of clarify and confirm terminology.

This made the integration harder to compare with the generic gateway middleware examples and less explicit about the boundary between the authority layer and the gateway surface.

Changes

  • Normalized confirm terminology across documentation and tests.
  • Updated README language to make architectural ownership clearer:
    • Context Compiler owns authoritative state.
    • LiteLLM Proxy owns the gateway boundary.
    • Enforcement occurs before downstream model calls.
  • Improved consistency between code behavior, documentation, and test expectations.

Scope

This PR is intentionally limited to terminology and documentation hardening.

It does not:

Relationship to Gateway Middleware Examples

The generic gateway middleware examples teach the enforcement-point pattern:

  • allow
  • block
  • default route
  • confirmation handling

The LiteLLM Proxy reference integration demonstrates how that pattern maps onto a real gateway technology.

Validation

  • uv run --no-sync pytest python/tests/test_litellm_proxy_hooks.py python/tests/test_litellm_proxy_with_directive_drafter.py
  • ./scripts/validate_python.sh

Both passed successfully.

Follow-up

Issue #13 remains open for:

  • LiteLLM Proxy lifecycle validation
  • running-proxy request-routing tests
  • proving blocked requests never reach upstream services

@rlippmann
rlippmann merged commit a4188f4 into main Jun 29, 2026
5 checks passed
@rlippmann
rlippmann deleted the llm-middleware-hardening branch June 29, 2026 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant