feat(provider): crof.ai / kimi-k2.5 compatibility and non-streaming fallback fix#331
Merged
mpfaffenberger merged 2 commits intompfaffenberger:mainfrom May 9, 2026
Merged
Conversation
added 2 commits
May 8, 2026 16:26
…nfig - Introduce _model_allows_streaming() to check per-model streaming config - Add "streaming": false support in models.json for models with flaky SSE (e.g. crof.ai kimi) - Clear _event_stream_handler on pydantic_agent when streaming disabled to force non-streaming request path - Update DBOS plugin to skip fallback render only when streaming is actually active - Add comprehensive tests for streaming override behavior and handler clearing
- Create _CompatChatModel wrapper for providers with OpenAI API incompatibilities - Strip "strict" from tool schemas when strict_tools=false (crof.ai doesn't support it) - Flatten tool calls into assistant content to avoid tool_call/result wiring errors - Degrade tool results into user messages for providers that error on role='tool' - Only send parallel_tool_calls setting when provider advertises support - Add tests for crof.ai tool result mapping and settings behavior
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
crof.ai's kimi-k2.5-lightning endpoint returned 500 errors and silently dropped agent responses when used through code-puppy. Three distinct issues were identified:
1.
strict: truecauses 500 on non-streaming requestscrof.ai's non-streaming endpoint rejects
strict: truein tool function schemas with a500 Failed to generate response. Only one tool (load_image_for_analysis) hadstrict: trueset by pydantic-ai, but that was enough to kill every request.strict=falsestrict=truestream=Falsestream=True2. Flaky SSE streaming
crof.ai's streaming transport intermittently fails with "Streamed response ended without content or tool calls" in the real runtime (DBOS + TUI + event handler). Raw HTTP tests pass, but the full CLI stack hits timing issues. Disabling streaming via
streaming: falsein models.json provides a stable single-shot path.3. Silent agent output when streaming is disabled
When
streaming: falseis set for a model, the runtime correctly disables the stream handler — but DBOS'sskip_fallback_rendercallback still returnedTrue, preventing the non-streaming fallback renderer from running. Nobody printed the output.Changes
code_puppy/model_factory.py_CompatChatModel(subclass ofOpenAIChatModel) for providers needing compatibility shims. Activated whenstrict_tools: falseorprovider: "crof"is set in models.json._map_tool_definition: stripsstrictkey from tool schemas whenstrict_tools: false._map_model_response: flattens tool calls into assistant text for providers that can't handletool_callsin assistant messages._map_user_message: degradesrole=toolresults intorole=usermessages for providers that reject tool-role messages.parallel_tool_callssetting when the provider/model advertises support (prevents 500 on unknown fields).code_puppy/agents/_runtime.py_model_allows_streaming(): readsstreamingfrom models.json config. Models with"streaming": falseskip the SSE path entirely._event_stream_handleron the pydantic agent so pydantic-ai usesmodel.request()instead ofrequest_stream(). Restore after the run.📡 Streaming disabledinfo emission (noisy, not actionable).code_puppy/plugins/dbos_durable_exec/runtime.pyskip_fallback_render()now checks whether streaming is actually active for the current model. When streaming is disabled, returnsFalseso the core non-streaming fallback renderer can produce output. Previously it returnedTruewhenever DBOS was launched, causing silent agent responses.code_puppy/models.jsonstreaming: falseandstrict_tools: falseforcrof-kimi-k2.5-lightning.Tests
tests/agents/test_streaming_retry.py: new test classTestModelAllowsStreamingcovering_model_allows_streaming()for None, crof-kimi, and normal models. New classTestNonStreamingHandlerClearverifying_event_stream_handleris nuked on DBOSAgent-like objects.tests/test_crof_tool_result_compat.py: tests for_CompatChatModelbehavior (strict stripping, tool result mapping, tool call flattening).tests/test_model_factory_coverage.py: updated to verifyparallel_tool_callsis conditionally included based on provider support.Testing
All changes verified with 1700+ passing tests and live e2e requests against crof.ai:
strict_tools=false→ no 500s across 10+ consecutive requestsstreaming=false+ DBOS → fallback render fires, agent output visibleparallel_tool_callsnot sent for crof models