feat(filter): add ext_proc state foundation and pipeline pinning

[nerdalert]

Summary

This PR adds the request-scoped state and pipeline-lifecycle foundation needed for the upcoming generic ext_proc full-duplex integration.

It introduces typed per-filter state on HttpFilterContext, keyed by a stable filter invocation ID assigned when the pipeline is built. That lets a filter store state during one lifecycle phase and read it later during request body, response, or response body processing without using globals or shared mutable state.

It also pins the active FilterPipeline for the lifetime of each Pingora request context. This prevents hot reloads from mixing pipeline versions mid-request: a request that starts on pipeline A continues using pipeline A across later hooks, while new requests after reload use pipeline B.

The working llm-d PoC integration demo, PR stack details, code walkthrough and architecture notes here: nerdalert/praxis-research-spikes

What Changed

1. Stable Filter Identity

Each PipelineFilter now has a stable filter_id assigned at pipeline build time.

The ID is:

• unique within a built FilterPipeline
• stable for the lifetime of that pinned pipeline
• assigned across top-level filters and nested branch chains
• independent of filter type, so multiple instances of the same filter get separate state

This is needed because request-scoped state has to belong to a specific filter instance, not just a filter type. A pipeline can contain the same filter type more than once, and branch chains can introduce additional filter instances outside the top-level pipeline loop.

Relying on only the top-level loop index is not enough because nested branch filters do not have a stable top-level index. Relying on filter type is also not enough because two ext_proc filters, two header filters, or two branch filters of the same type must not share state.

Assigning a stable filter_id when the pipeline is built gives the runtime a small, deterministic key for per-request state. That key stays meaningful for the lifetime of the pinned pipeline.

2. Typed Per-Request Filter State

HttpFilterContext now carries:

current_filter_id: Option<usize>
filter_state: HashMap<usize, Box<dyn Any + Send + Sync>>

Filters can use typed accessors:

  insert_filter_state<T>()
  get_filter_state<T>()
  get_filter_state_mut<T>()
  remove_filter_state<T>()

This gives filters a safe request-local memory slot. A filter can store state during one lifecycle callback and retrieve it later during request body, response, or response body processing.

For the upcoming full-duplex ext_proc integration, this is the foundation that allows the filter to open a processor exchange during request/header or request-body handling, then find that same exchange again when later body chunks arrive.

The map is keyed by current_filter_id, which the pipeline sets only while a filter hook is executing. That means filters do not need to manually pass keys around, and they cannot accidentally read another filter instance's state through the public accessor methods.

Box<dyn Any + Send + Sync> is used because different filters may store different concrete state types. Any allows typed downcasting through the accessor methods, while Send + Sync keeps the state compatible with PingoraRequestCtx, which must satisfy Pingora's thread-safety bounds.

Type mismatches return None instead of corrupting or removing the stored entry. remove_filter_state() also preserves the entry on type mismatch.

3. Identity Set/Clear Around Filter Hooks

The HTTP pipeline executor sets current_filter_id before invoking a filter hook and clears it immediately afterward.

This is covered for:

• request filters
• request body filters
• response filters
• response body filters
• branch filter execution
• rejection/error paths
• skipped filters and branch exits

The important detail is cleanup ordering. The executor stores the hook result first, clears current_filter_id, and only then propagates errors or returns rejections. That avoids leaking a stale filter identity into later pipeline work.

When no filter hook is actively running, current_filter_id is None. Calls like get_filter_state<T>() then return None, and insert_filter_state<T>() is a no-op. This keeps state access scoped to the currently executing filter.

4. Pingora Request Context State Persistence

PingoraRequestCtx now owns the durable per-request filter_state map between filter phases.

HttpFilterContext is temporary: the protocol layer builds one for a specific hook execution, runs the filter pipeline, and then drops it. Without a durable owner outside that temporary context, state written during request handling would disappear before request-body or response handling.

The protocol layer now swaps filter_state into HttpFilterContext before filter execution and writes it back afterward, following the existing filter_metadata ownership pattern. This keeps state owned by exactly one context at a time and avoids shared references or locks.

The Pingora handler changes are intentionally narrow. They do not change request routing, body buffering, response handling, retry behavior, or upstream selection. They only extend the existing context move/writeback boundary so typed filter state survives across lifecycle callbacks.

5. Pipeline Pinning Across Hot Reload

PingoraRequestCtx now pins the current Arc<FilterPipeline> during request_filter.

Later hooks use the pinned pipeline instead of loading from ArcSwap again. The helper is idempotent, so a second call does not repin after a reload.

This matters because filter_state is keyed by filter_id, and filter_id is only meaningful within the pipeline that assigned it. If a request started on pipeline A, then a hot reload swapped in pipeline B before the response phase, the same numeric filter_id could refer to a different filter instance.

Pinning prevents that. A request keeps using the pipeline it started with for the full request lifecycle.

This guarantees:

• request A started before reload continues on pipeline A
• request B started after reload uses pipeline B
• old pipeline Arcs stay alive while pinned by active requests
• filter state keyed by filter_id remains meaningful for the pinned pipeline
• hot reload cannot mix filter identity/state from two different pipeline versions in one request

More on Pingora Handler Changes

The Pingora handler changes are intentionally narrow. They do not change request routing, body buffering, response handling, retry behavior, or upstream selection.

They do two things:

1. Carry filter state across lifecycle callbacks.

   Each handler phase already builds a short-lived HttpFilterContext from PingoraRequestCtx, runs the filter pipeline, then writes durable fields back to PingoraRequestCtx.

   This PR adds filter_state to that existing move/writeback pattern, matching how filter_metadata is already handled. The state map is moved into the filter context with std::mem::take() and moved back after the hook returns.

2. Use one pinned pipeline for the full request lifecycle.

   request_filter pins the current Arc<FilterPipeline> from ArcSwap into PingoraRequestCtx.

   Later Pingora callbacks call ctx.pipeline(&self.pipeline) instead of loading from ArcSwap directly. If the request was pinned, this returns the same pipeline. If a callback runs before pinning in an early-failure path, it falls back to loading the current pipeline.

This means hot reload remains safe without changing the behavior of existing filters: active requests keep using the pipeline they started with, and new requests pick up the reloaded pipeline.

PR Stack Context

PR	Title	Status	Scope
1	Request-Scoped State and Pipeline Pinning	This PR	Typed per-filter state on HttpFilterContext, stable filter identity, pipeline pinning across hot reload
2	Duplex Exchange Core	Follow-up	Standalone ExtProcExchange transport core for one persistent bidirectional gRPC ExternalProcessor.Process stream per request
3	Request-Routing Integration	Follow-up	Wires full-duplex ext_proc into the filter lifecycle for the llm-d Go EPP request-routing path using generic ext_proc + endpoint_selector

Why This Is Needed

The upcoming full-duplex ext_proc integration needs to open a processor exchange during one phase and continue using that same exchange in later phases.

Without this PR, filters do not have a safe request-scoped place to store typed state across hooks. Using globals, static maps, or ad hoc protocol-layer fields would make ownership, cleanup, hot reload, and concurrency harder to reason about.

This PR keeps the foundation generic:

• no intended behavior change for existing filters beyond stable hot-reload pinning
• no ext_proc-specific state in the core filter pipeline
• no background tasks
• no locks for state access
• no public protocol behavior change

Validation

• cargo test -p praxis-proxy-filter
• cargo test -p praxis-proxy-protocol
• cargo test -p praxis-proxy-ext-proc
• cargo clippy -p praxis-proxy-filter --all-targets -- -D warnings
• cargo clippy -p praxis-proxy-protocol --all-targets -- -D warnings
• cargo clippy -p praxis-proxy-ext-proc --all-targets -- -D warnings
• cargo +nightly fmt -p praxis-proxy-filter -p praxis-proxy-protocol -p praxis-proxy-ext-proc -- --check
• cargo doc -p praxis-proxy-filter --no-deps --document-private-items
• cargo doc -p praxis-proxy-protocol --no-deps --document-private-items
• cargo doc -p praxis-proxy-ext-proc --no-deps --document-private-items
• git diff --check

[praxis-bot]

test

[praxis-bot]

PR Review

Summary: Adds per-filter typed state (filter_state HashMap) and pipeline pinning to HttpFilterContext, with stable filter_id assignment at build time.

Overall: Well-structured foundation — the current_filter_id lifecycle (set before hook, clear after) is correct, the Any + Send + Sync downcasting API is clean, and branch chain ID uniqueness tests are thorough. One issue: branch filter error handling doesn't respect FailureMode.

Severity	Count
Critical	0
Large	1
Medium	0

[praxis-bot]

PR Review

Summary: Adds typed per-filter state (filter_state HashMap) and stable filter invocation IDs (filter_id) to support state persistence across filter lifecycle phases.

Overall: Well-structured foundation. The filter state API is clean, ID uniqueness is thoroughly tested across branch nesting depths, and the pipeline pinning prevents hot-reload state loss. One behavioral inconsistency in branch error handling needs attention.

Severity	Count
Critical	0
Large	1
Medium	0

[shaneutt]

Some small comments, and then a question:

Had you considered options to resolve this (for MCP) that were more bespoke?

[nerdalert]

Some small comments, and then a question:

Had you considered options to resolve this (for MCP) that were more bespoke?

Good idea! Kept this generic because the lifecycle-state need applies to any filter instance and avoids baking MCP/ext_proc details into the protocol handler. I’ll add MCP helpers or typed wrappers to the MCP workstream if that works. Need to think about it a bit more tbh.

[shaneutt]

Some small comments, and then a question:
Had you considered options to resolve this (for MCP) that were more bespoke?

Good idea! Kept this generic because the lifecycle-state need applies to any filter instance and avoids baking MCP/ext_proc details into the protocol handler. I’ll add MCP helpers or typed wrappers to the MCP workstream if that works. Need to think about it a bit more tbh.

Sounds good, this is OK for a follow-up PR.