Repository navigation
Phase 2: Core product — cache, deadline, SSE, history search, waitress - #10
Conversation
…(TD-03) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implement a configurable total deadline (default 45 seconds) that limits the combined time spent across all provider attempts in the fallback chain. The deadline is checked before the first provider attempt and before each fallback attempt. If exceeded, a RuntimeError is raised to prevent wasted retries. - Add deadline_seconds config (fallback.deadline_seconds) - Check deadline before primary provider attempt - Check deadline before each fallback provider attempt - Add tests for instant deadline (raises) and generous deadline (succeeds)
…ment Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
promptfix/promptfix/rewriter.py
Line 158 in 8c68bc8
Keep cache key stable across fallback provider success
When a fallback provider succeeds, provider_name is reassigned to the fallback name, and the cache is saved under that new provider key. On the next request, lookup still uses the configured primary provider key, so this cached result is never reused. In outage/degradation scenarios for the primary provider, this causes repeated avoidable fallback calls and defeats the new cache’s reliability/performance benefit.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # --- Cache lookup --- | ||
| if use_cache and config.get("cache", {}).get("enabled", True): | ||
| from promptfix.cache import CacheEntry, get_cache_key, load_cached | ||
| _cache_key = get_cache_key(text, mode, provider_name) |
There was a problem hiding this comment.
The cache lookup key in rewrite() is built only from text, mode, and provider, but the generated prompt also depends on preset_hint (used in build_rewrite_prompt(...)). This means two runs with the same text/mode/provider but different presets can return a stale cached output from a different preset, producing incorrect optimization results for promptfix preset use flows.
Useful? React with 👍 / 👎.
| if time.time() >= deadline: | ||
| last_exc = RuntimeError(f"Deadline of {deadline_seconds}s exceeded during fallback") | ||
| break | ||
| try: | ||
| fallback_provider = create_provider(config, fallback_name) | ||
| raw_output = fallback_provider.complete(messages) |
There was a problem hiding this comment.
The deadline is checked only before each provider attempt, but each complete() call can still block for its full timeout even if little or no budget remains. For example, a first attempt near the 45s limit can still allow a second attempt that runs tens of seconds longer, violating the documented total deadline behavior. This should use remaining-budget timeouts (or post-call enforcement) to guarantee the global cap.
Useful? React with 👍 / 👎.
1 participant
Phase 2 — Core Product
use_cache=Falsebypass (TD-03)promptfix history: --grep/--mode/--since/--limit searchAll 336 tests green, ruff + mypy clean.