feat(agentserver): WIP - responses package durable orchestration (split out of #46997)

sdk/agentserver/azure-ai-agentserver-responses/CHANGELOG.md

@@ -1,31 +1,92 @@
 # Release History
 
-## 1.0.0b7 (2026-05-25)
+## 1.0.0b8 (Unreleased)
 
 ### Features Added
 
-- Added MCP output item builder enhancements for hosted MCP relay scenarios: `ResponseEventStream.add_output_item_mcp_call()` now supports caller-supplied item IDs, and MCP call `emit_done()` supports optional `output` and `error` payloads for canonical `mcp_call` persistence and replay.
-
-## 1.0.0b6 (2026-05-21)
-
-### Features Added
-
-- Error source classification headers: All HTTP error responses now include `x-platform-error-source` with a value of `user`, `platform`, or `upstream` to indicate which component caused the error. Client validation errors (400/404) are classified as `user`, Foundry storage infrastructure errors (transport failures, 5xx) as `platform`, and developer handler exceptions as `upstream`. Platform errors additionally include `x-platform-error-detail` with truncated exception details (max 2048 characters) for diagnostics. Matches the container image specification §8 error source classification.
-
-### Breaking Changes
-
-- Removed the automatic `invoke_agent` server span that was created on each response creation request. Trace context propagation is now handled by the core `TraceContextMiddleware`, and user-created spans inside handlers are correctly parented without framework-generated spans.
-- Removed `_safe_set_attrs`, `_wrap_streaming_response`, and `_classify_error_code` internal helpers (no longer needed without framework-level span management).
-- Removed OTel error tagging attributes (`azure.ai.agentserver.responses.error.code`, `azure.ai.agentserver.responses.error.message`) that were set on the framework span.
+- **Resilient background responses.** `ResponsesServerOptions(resilient_background=True)`
+  makes `store=true`, `background=true` responses survive process crashes:
+  the framework persists handler progress and re-invokes the registered
+  handler on the next process start when a prior attempt did not reach a
+  terminal event. Defaults to `False`.
+
+- **Steerable conversations.** `ResponsesServerOptions(steerable_conversations=True)`
+  lets clients post a new turn on an in-flight conversation; the running
+  handler is woken (via the cancellation signal, distinguished by
+  `context.pending_input_count > 0`), drains the queued input on a fresh
+  invocation, and the turns are linked in a stable conversation chain.
+  Defaults to `False`.
+
+- **`ResponseContext` resilience + steering surface.** Flat fields stamped on
+  each invocation: `context.is_recovery`, `context.is_steered_turn`,
+  `context.pending_input_count`, and `context.conversation_chain_id` (a stable
+  identifier shared by every turn of a conversation chain, usable as a key into
+  application-side session state).
+
+- **Developer checkpoints.** `yield stream.checkpoint()` persists the
+  current response snapshot at a developer-chosen boundary (gated to resilient
+  background responses; a no-op otherwise; backpressured and idempotent). On a
+  recovered entry, `context.persisted_response` exposes the last persisted
+  snapshot so the handler can seed its stream and resume — the basis of the
+  one-`OutputItem`-per-phase recovery pattern.
+
+- **`internal_metadata`.** A single-turn, platform-internal `MutableMapping[str, Any]`
+  on output items (`item.internal_metadata`) and on the response
+  (`stream.internal_metadata`). It is persisted with the response (so it is
+  available on recovery) and is always stripped before any client-facing
+  HTTP/SSE payload, and on ingress. Distinct from the public
+  `ResponseObject.metadata`.
+
+- **`context.conversation_chain_metadata`.** Cross-turn, named-scope,
+  explicit-`flush()` resilient metadata over a conversation chain, typed by the
+  public `ConversationChainMetadataNamespace` Protocol.
+
+- **`await context.exit_for_recovery()`.** A single uniform graceful-shutdown
+  recovery primitive that works in every handler shape (coroutine, async
+  generator, sync) — it raises `ResponseExitForRecovery` internally to leave
+  the response `in_progress` for next-lifetime recovery.
+
+- **Stream recovery.** SSE events are persisted incrementally; clients reconnect
+  with `GET /responses/{id}?stream=true&starting_after=<event_id>` and resume
+  from their last received event.
+
+- **Response acceptor hook.** Register `@app.response_acceptor` to customize the
+  response shape returned when a turn is queued behind an active steerable
+  conversation.
+
+- **Storage.** `FileResponseStore` is exported from
+  `azure.ai.agentserver.responses` and is the default local-development store
+  (under `${AGENTSERVER_STATE_ROOT:-~/.agentserver}/responses/`) when no `store=`
+  is supplied in a non-hosted environment; pass
+  `store=InMemoryResponseProvider()` to opt out. The `AGENTSERVER_STATE_ROOT`
+  environment variable sets the local state storage root. A typed
+  `ResponseAlreadyExistsError` is raised by the response-store providers on a
+  duplicate `create_response` (the idempotent-create signal on recovery).
+
+- **Error source classification headers.** HTTP error responses carry
+  `x-platform-error-source` (`user` / `platform` / `upstream`); platform errors
+  also include `x-platform-error-detail` with truncated diagnostics.
+
+- **Handlers are `async def`.** `@app.response_handler` requires an async
+  handler with the `(request, context, cancellation_signal)` signature so it can
+  observe the `asyncio.Event` cancellation signal.
+
+- Added resilient samples demonstrating real SDK integrations (Claude Agent SDK,
+  Copilot SDK, LangGraph) and resilient streaming / steering / multi-turn
+  patterns.
 
 ### Bugs Fixed
 
-- Removed `ContentDecodePolicy` from the `FoundryStorageProvider` HTTP pipeline.  The policy eagerly decoded every response body as JSON and crashed with `UnicodeDecodeError` when the storage backend (or an intermediary gateway/load-balancer) returned a non-UTF-8 body — for example a gzip-compressed payload, an HTML error page, or a transport-corrupted response.  The crash propagated up before our error-classification code could see the response, masking the underlying status with a generic decode error.  Our serializers and error-extraction helpers already call `http_resp.text()` lazily with defensive error handling, so the eager decode policy was never needed.
-
-### Other Changes
-
-- Platform header name constants (e.g. `x-platform-error-source`, `x-platform-error-detail`) are now imported from `azure-ai-agentserver-core` (`_platform_headers` module). Error source classification helpers remain internal to this package.
-- Simplified request handling: baggage entries (`response_id`, `conversation_id`, `streaming`, `x-request-id`) are still set on each request, but span creation and lifecycle management are left to downstream frameworks.
+- **Resilient background streaming responses now engage resilience even when SSE
+  keep-alive is enabled.** Previously the resilient task was created only on the
+  no-keep-alive streaming path, so when SSE keep-alive was enabled (e.g. the
+  hosted platform sets `SSE_KEEPALIVE_INTERVAL`), a `store=true`,
+  `background=true`, `stream=true` response ran the handler inline on the
+  request connection and never created a resilient task. Such responses could
+  hang `in_progress` on a client/proxy disconnect and were not recoverable.
+  Stored responses now always run via the resilient task; keep-alive comments are
+  interleaved into the wire stream while the resilient body runs independently of
+  the client connection.
 
 ## 1.0.0b5 (2026-04-22)
 
@@ -49,7 +110,7 @@
 
 ### Bugs Fixed
 
-- `DELETE /responses/{id}` no longer returns intermittent 404 when the background task's eager eviction races with the delete handler. Previously, `try_evict` could remove the record from in-memory state between the handler's `get()` and `delete()` calls, causing `delete()` to return `False` and producing a spurious 404. The handler now falls through to the durable provider when the in-memory delete fails due to a concurrent eviction.
+- `DELETE /responses/{id}` no longer returns intermittent 404 when the background task's eager eviction races with the delete handler. Previously, `try_evict` could remove the record from in-memory state between the handler's `get()` and `delete()` calls, causing `delete()` to return `False` and producing a spurious 404. The handler now falls through to the resilient provider when the in-memory delete fails due to a concurrent eviction.
 - `POST /responses` with `background=true, stream=false` now correctly returns `status: "in_progress"` instead of `"completed"`. Handlers that yield events synchronously (no `await` between yields — the normal pattern with `ResponseEventStream`) would cause the background task to run to completion before `run_background` captured the initial snapshot. A cooperative yield after `response_created_signal.set()` now ensures the POST handler resumes promptly.
 - Conversation history IDs (`previous_response_id`, `conversation_id`) are now validated eagerly before the handler is invoked. A nonexistent reference now returns a 404 error to the client immediately, instead of being silently ignored or surfacing as an opaque error deep inside the handler. The prefetched IDs are reused by `ResponseContext.get_history()`, eliminating a redundant provider call.
 

sdk/agentserver/azure-ai-agentserver-responses/README.md

@@ -24,12 +24,20 @@ This automatically installs `azure-ai-agentserver-core` as a dependency.
 
 ```python
 @app.response_handler
-def my_handler(
-    request: CreateResponse, context: ResponseContext, cancellation_signal: asyncio.Event
-):
+async def my_handler(request: CreateResponse, context: ResponseContext, cancellation_signal: asyncio.Event):
     ...
 ```
 
+Handlers MUST be `async def` and take exactly three positional parameters
+(`request`, `context`, `cancellation_signal`). Sync handlers and the 2-arg
+signature `(request, context)` are hard-rejected at decoration time.
+Cancellation is observed via the `cancellation_signal` event (set on
+client cancel, `/cancel` API, or steering pressure). Server shutdown is
+a **distinct** signal observed via `context.shutdown` — shutdown does
+NOT fire the cancellation signal; handlers that care about both must
+inspect each independently. See the handler implementation guide for
+the full surface.
+
 ### Protocol endpoints
 
 | Method | Route | Description |
@@ -90,14 +98,28 @@ The `ResponseContext` provides request-scoped state:
 | Property / Method | Description |
 |---|---|
 | `response_id` | Unique ID for this response |
-| `is_shutdown_requested` | Whether the server is draining |
+| `conversation_id` / `conversation_chain_id` | Conversation identifiers; `conversation_chain_id` is the framework-computed stable id shared by every turn in a chain |
 | `isolation` | `IsolationContext` with `user_key` and `chat_key` for multi-tenant state partitioning |
 | `client_headers` | Dictionary of `x-client-*` headers forwarded from the platform (keys normalized to lowercase) |
 | `query_parameters` | Dictionary of query string parameters |
+| `shutdown` | `asyncio.Event` set on graceful server shutdown — distinct from the per-request cancellation signal |
+| `client_cancelled` | `bool` set when the cancel cause is `/cancel` endpoint or non-bg POST disconnect |
+| `is_recovery` | `bool` set on a crash-recovered re-entry |
+| `is_steered_turn` | `bool` set on the drain re-entry that follows a steering input |
+| `pending_input_count` | `int` count of queued steering inputs |
+| `conversation_chain_metadata` | `ConversationChainMetadataNamespace` for handler-managed checkpoint state |
+| `exit_for_recovery()` | `await` to opt into the graceful-shutdown recovery path |
 | `get_input_items()` | Load resolved input items as `Item` subtypes |
 | `get_input_text()` | Extract all text content from input items as a single string |
 | `get_history()` | Load conversation history items |
 
+The per-request cancellation signal is delivered as the **3rd
+positional handler argument** (`cancellation_signal: asyncio.Event`),
+not via a `ResponseContext` attribute. It fires on client cancel
+(`/cancel` API or non-bg POST disconnect) or steering pressure; it
+does NOT fire on server shutdown — `context.shutdown` is the
+independent surface for that case.
+
 ### Streaming and background modes
 
 The SDK automatically handles all combinations of `stream` and `background` flags:
@@ -113,13 +135,15 @@ The library orchestrates the complete response lifecycle: `created` → `in_prog
 
 For detailed handler implementation guidance, see [docs/handler-implementation-guide.md](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agentserver/azure-ai-agentserver-responses/docs/handler-implementation-guide.md).
 
+### Resilience
+
+Crash recovery is **opt-in** via `ResponsesServerOptions(resilient_background=True)`. When opted in, background responses with `store=True` are crash-recoverable: the handler is re-invoked on restart and the recovered context exposes `context.is_recovery == True`. Stream events are persisted incrementally so clients can reconnect and resume from where they left off. Without the opt-in (the default), a crash mid-handler marks the response `failed` instead of re-invoking the handler. For advanced scenarios (metadata checkpointing, multi-turn steering), see the [Resilient Responses Developer Guide](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agentserver/azure-ai-agentserver-responses/docs/resilient-responses-developer-guide.md).
+
 ## Examples
 
 ### Echo handler
 
 ```python
-import asyncio
-
 from azure.ai.agentserver.responses import (
     CreateResponse,
     ResponseContext,
@@ -188,7 +212,7 @@ app = ResponsesAgentServerHost(options=options)
 ### Common errors
 
 - **400 Bad Request**: The request body failed validation. Check that optional fields such as `model` (when provided) are valid and that `input` items are well-formed.
-- **404 Not Found**: The response ID does not exist or has expired past the configured TTL.
+- **404 Not Found**: The response ID does not exist. In hosted deployments persisted responses live in the Foundry hosted responses store; in local development they live under `${AGENTSERVER_STATE_ROOT:-~/.agentserver}/responses/` by default. A missing record may indicate the response was never persisted or was deleted via `DELETE /responses/{id}`.
 - **400 Bad Request** (cancel): The response was not created with `background=true`, or it has already reached a terminal state.
 
 ### Reporting issues
@@ -214,6 +238,11 @@ Visit the [Samples](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/
 | [File Inputs](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_14_file_inputs.py) | Receive files via base64 data URL, URL, or file ID |
 | [Annotations](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_15_annotations.py) | Attach file_path, file_citation, and url_citation annotations |
 | [Structured Outputs](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_16_structured_outputs.py) | Return structured JSON as a `structured_outputs` item |
+| [Resilient Copilot](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_18_resilient_copilot.py) | GitHub Copilot SDK with `resilient_background=True, steerable_conversations=True` |
+| [Resilient Streaming](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_19_resilient_streaming.py) | Three-phase streaming handler with `resilient_background=True` and `context.conversation_chain_metadata` watermarks |
+| [Resilient Steering](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_20_resilient_steering.py) | `context.is_steered_turn` on the drain re-entry with `resilient_background=True, steerable_conversations=True` |
+| [Resilient LangGraph](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_21_resilient_langgraph.py) | LangGraph integration with `resilient_background=True, steerable_conversations=True` |
+| [Resilient Multi-turn](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/agentserver/azure-ai-agentserver-responses/samples/sample_22_resilient_multiturn.py) | Multi-turn conversation with `resilient_background=True, steerable_conversations=False` |
 
 - [Handler implementation guide](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/agentserver/azure-ai-agentserver-responses/docs/handler-implementation-guide.md) — Detailed reference for building handlers
 

sdk/agentserver/azure-ai-agentserver-responses/azure/ai/agentserver/responses/__init__.py

@@ -8,15 +8,22 @@
 
 from . import _data_url as data_url
 from ._options import ResponsesServerOptions
-from ._response_context import IsolationContext, ResponseContext
+from ._response_context import (
+    ConversationChainMetadataNamespace,
+    ExitForRecoverySignal,
+    IsolationContext,
+    ResponseContext,
+    ResponseExitForRecovery,
+)
 from .hosting._routing import ResponsesAgentServerHost
 from .models import CreateResponse, ResponseObject
 from .models._helpers import (
     get_conversation_id,
     get_input_expanded,
     to_output_item,
 )
-from .store._base import ResponseProviderProtocol, ResponseStreamProviderProtocol
+from .store._base import ResponseProviderProtocol
+from .store._file import FileResponseStore
 from .store._foundry_errors import (
     FoundryApiError,
     FoundryBadRequestError,
@@ -32,13 +39,16 @@
 __all__ = [
     "__version__",
     "data_url",  # pylint: disable=naming-mismatch
+    "ConversationChainMetadataNamespace",
+    "ExitForRecoverySignal",
+    "ResponseExitForRecovery",
     "ResponsesAgentServerHost",
     "ResponseContext",
     "IsolationContext",
     "ResponsesServerOptions",
     "ResponseProviderProtocol",
-    "ResponseStreamProviderProtocol",
     "InMemoryResponseProvider",
+    "FileResponseStore",
     "FoundryStorageProvider",
     "FoundryStorageSettings",
     "FoundryStorageError",