Problem Statement
It seems that OCR currently lacks explicit session affinity control.
Depending on the provider, it may need to provide an explicit header to enable prompt caching.
- OpenAI:
prompt_cache_key in body.
- Anthropic: don't use explicit parameter. (undeterministic)
- Cloudflare/Fireworks/Mistral/others: Commonly
x-session-affinity in header.
Since the "session" must be managed by OCR, users cannot provide the provider themselves.
Proposed Solution
Define the session key in OCR's lifecycle. Maybe related to #59 (comment)
Inject it to the request, or provide a template variable so that the user can configure extra_headers directly. e.g. x-session-affinity: {ocr_session_key}
Alternatives Considered
No response
Affected Area
Configuration
Additional Context
No response
Problem Statement
It seems that OCR currently lacks explicit session affinity control.
Depending on the provider, it may need to provide an explicit header to enable prompt caching.
prompt_cache_keyin body.x-session-affinityin header.Since the "session" must be managed by OCR, users cannot provide the provider themselves.
Proposed Solution
Define the session key in OCR's lifecycle. Maybe related to #59 (comment)
Inject it to the request, or provide a template variable so that the user can configure extra_headers directly. e.g.
x-session-affinity: {ocr_session_key}Alternatives Considered
No response
Affected Area
Configuration
Additional Context
No response