Repository navigation
fix: type the ContextVar BYOK Token (fixes #30) - #50
sgoel2be24-cyber wants to merge 2 commits into
Conversation
set_context_provider/reset_context_provider now use the typed Token[Optional[LLMProvider]] — a Token from a different ContextVar is a type error instead of a silent cross-context reset. Lifecycle docs clarified (pair set/reset in a finally; the provider never outlives its request).
Reviewer's guide (collapsed on small PRs)Reviewer's GuideThe PR updates the context provider helpers to use Token[Optional[LLMProvider]], allowing type checkers to reject tokens from unrelated ContextVars without changing runtime behavior, and clarifies safe request-lifecycle usage in the API documentation strings. Sequence diagram for typed context provider lifecyclesequenceDiagram
participant Request
participant ContextHelpers
participant ContextVar
Request->>ContextHelpers: set_context_provider(provider)
ContextHelpers->>ContextVar: set(provider)
ContextVar-->>ContextHelpers: Token[Optional[LLMProvider]]
ContextHelpers-->>Request: token
Request->>ContextHelpers: reset_context_provider(token)
ContextHelpers->>ContextVar: reset(token)
ContextVar-->>Request: provider context restored
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
|
👋 Thanks for opening this PR! A maintainer will review it soon. In the meantime, please make sure you've read CONTRIBUTING.md and filled out the PR template, including screenshots for any visual changes. |
There was a problem hiding this comment.
Hey - I've found 2 issues
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="core/llm_context.py" line_range="53" />
<code_context>
+ type error, not a silent cross-context reset. Always call this in a
`finally` block paired with the `set_context_provider` that produced
`token`, so a provider never outlives the request that set it."""
_current_llm_provider.reset(token)
</code_context>
<issue_to_address>
**Child tasks retain request credentials**
When a request creates a child task while the provider is set and that task continues after the request ends, `asyncio.create_task` copies the current context, so `reset_context_provider` clears only the request context and the child retains the BYOK provider, allowing it to use that credential after the request ends.
Prevent spawned tasks from retaining or using the request provider after the request ends, for example by preventing provider propagation or cancelling and draining those tasks.
Also at `core/llm_context.py:47`, `core/llm_context.py:51-52`.
</issue_to_address>
### Comment 2
<location path="core/llm_context.py" line_range="47" />
<code_context>
-def reset_context_provider(token: Token) -> None:
- """Undo a prior set_context_provider call. Always call this in a
+def reset_context_provider(token: Token[Optional["LLMProvider"]]) -> None:
+ """Undo a prior set_context_provider call. The Token is typed to this
+ ContextVar's value type, so a Token from a different ContextVar is a
</code_context>
<issue_to_address>
**Cross-context cleanup fails at runtime**
When a caller passes a token from another `ContextVar[Optional[LLMProvider]]`, `Token[Optional[LLMProvider]]` checks only the value type, so `_current_llm_provider.reset(token)` raises `ValueError` instead of rejecting the mix-up statically, and cleanup fails at runtime.
Remove the claim that the annotation catches cross-ContextVar tokens, or check `token.var` against `_current_llm_provider` before resetting.
Also at `core/llm_context.py:48-50`.
</issue_to_address>Sourcery assessment
Approval pending. 2 findings to address first.
Blocking findings: core/llm_context.py:53, core/llm_context.py:47
| type error, not a silent cross-context reset. Always call this in a | ||
| `finally` block paired with the `set_context_provider` that produced | ||
| `token`, so a provider never outlives the request that set it.""" | ||
| _current_llm_provider.reset(token) |
There was a problem hiding this comment.
🟡 Medium · Child tasks retain request credentials
When a request creates a child task while the provider is set and that task continues after the request ends, asyncio.create_task copies the current context, so reset_context_provider clears only the request context and the child retains the BYOK provider, allowing it to use that credential after the request ends.
Prevent spawned tasks from retaining or using the request provider after the request ends, for example by preventing provider propagation or cancelling and draining those tasks.
Also at core/llm_context.py:47, core/llm_context.py:51-52.
Prompt for AI agents
In `core/llm_context.py` at line 53:
**Child tasks retain request credentials**
When a request creates a child task while the provider is set and that task continues after the request ends, `asyncio.create_task` copies the current context, so `reset_context_provider` clears only the request context and the child retains the BYOK provider, allowing it to use that credential after the request ends.
Prevent spawned tasks from retaining or using the request provider after the request ends, for example by preventing provider propagation or cancelling and draining those tasks.
Also at `core/llm_context.py:47`, `core/llm_context.py:51-52`.|
|
||
| def reset_context_provider(token: Token) -> None: | ||
| """Undo a prior set_context_provider call. Always call this in a | ||
| def reset_context_provider(token: Token[Optional["LLMProvider"]]) -> None: |
There was a problem hiding this comment.
🟡 Medium · Cross-context cleanup fails at runtime
When a caller passes a token from another ContextVar[Optional[LLMProvider]], Token[Optional[LLMProvider]] checks only the value type, so _current_llm_provider.reset(token) raises ValueError instead of rejecting the mix-up statically, and cleanup fails at runtime.
Remove the claim that the annotation catches cross-ContextVar tokens, or check token.var against _current_llm_provider before resetting.
Also at core/llm_context.py:48-50.
Prompt for AI agents
In `core/llm_context.py` at line 47:
**Cross-context cleanup fails at runtime**
When a caller passes a token from another `ContextVar[Optional[LLMProvider]]`, `Token[Optional[LLMProvider]]` checks only the value type, so `_current_llm_provider.reset(token)` raises `ValueError` instead of rejecting the mix-up statically, and cleanup fails at runtime.
Remove the claim that the annotation catches cross-ContextVar tokens, or check `token.var` against `_current_llm_provider` before resetting.
Also at `core/llm_context.py:48-50`.
DevSidd2006
left a comment
There was a problem hiding this comment.
Two changes to fix:
-
When a caller passes a token from another ContextVar[Optional[LLMProvider]], Token[Optional[LLMProvider]] checks only the value type, so _current_llm_provider.reset(token) raises ValueError instead of rejecting the mix-up statically, and cleanup fails at runtime. Remove the claim that the annotation catches cross-ContextVar tokens, or check token.var against
_current_llm_providerbefore resetting. -
When a request creates a child task while the provider is set and that task continues after the request ends, asyncio.create_task copies the current context, so reset_context_provider clears only the request context and the child retains the BYOK provider, allowing it to use that credential after the request ends. Prevent spawned tasks from retaining or using the request provider after the request ends, for example by preventing provider propagation or cancelling and draining those tasks. Also at
core/llm_context.py:47, core/llm_context.py:51-52
DevSidd2006
left a comment
There was a problem hiding this comment.
Closing as this is duplicate PR of #31
Fixes #30
What changed
set_context_provider/reset_context_providernow useToken[Optional[LLMProvider]]— a Token from a different ContextVar is a type error instead of a silent cross-context reset. Lifecycle docs clarified (set/reset paired in a finally; the provider never outlives its request).Testing
Round-trip verified (
set→reset→getreturns None); no behavior change — typing + docs only.Summary by Sourcery
Type context-provider tokens to their associated ContextVar and clarify safe request-scoped cleanup.
Bug Fixes:
Enhancements: