feat: pass response_format through iaas with fallback - #179
Draft
RohanAwhad wants to merge 2 commits into
Draft
Conversation
its_hub/integration/iaas.py; its_hub/lms.py; its_hub/algorithms/self_consistency.py; its_hub/algorithms/bon.py; its_hub/algorithms/particle_gibbs.py; tests/test_iaas.py; tests/test_lms.py; tests/test_algorithms.py - Add optional ChatCompletionRequest.response_format and forward it to scaling algorithms - Thread response_format through self-consistency and best-of-n generation calls - Ignore response_format for particle-filtering with warning instead of failing - Add OpenAI/LiteLLM fallback that retries once without response_format when unsupported - Add red-green tests for request passthrough, LM fallback, algorithm forwarding, and particle-filter warning Remaining: Update adapter_critic dependency pins; Create and link draft PR
its_hub/lms.py - Add OrderedDict-based response_format support cache (bounded size) - Centralize fallback helper logic and reuse for OpenAI-compatible and LiteLLM paths - Skip repeated fail-then-retry requests once a provider/model is known unsupported its_hub/base.py - Add response_format argument to ainfer/infer interface and forward it in sync wrapper its_hub/algorithms/beam_search.py its_hub/algorithms/planning_wrapper.py its_hub/algorithms/particle_gibbs.py its_hub/algorithms/__init__.py - Thread response_format through algorithm interfaces; warn/ignore for step-wise beam search tests/test_lms.py tests/test_algorithms.py - Add cache behavior coverage and sync infer response_format forwarding checks Remaining: tighten unsupported-error detection for invalid schema vs unsupported capability Signed-off-by: Rohan Awhad <rawhad@redhat.com>
Author
|
Implemented the requested follow-ups (2/3/4) and pushed commit
Validation run:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
response_formatsupport to IaaS chat requests and forward it through scaling algorithms.response_format, retry once without it and continue.response_formatwith a warning instead of failing.Test plan
uv run --extra dev pytest tests/test_lms.py tests/test_iaas.py tests/test_algorithms.pyuv run --extra dev pytest tests/test_lms.py tests/test_iaas.py tests/test_algorithms.py -k "response_format"uv run --extra dev pytest tests/test_lms.py tests/test_iaas.py tests/test_algorithms.pyuv run --extra dev pytestadapter_criticrun using localits_hub+reward_hubbranchUnit tests