Skip to content

feat(provider): crof.ai / kimi-k2.5 compatibility and non-streaming fallback fix#331

Merged
mpfaffenberger merged 2 commits intompfaffenberger:mainfrom
thomwebb:fix/crof-kimi-compat
May 9, 2026
Merged

feat(provider): crof.ai / kimi-k2.5 compatibility and non-streaming fallback fix#331
mpfaffenberger merged 2 commits intompfaffenberger:mainfrom
thomwebb:fix/crof-kimi-compat

Conversation

@thomwebb
Copy link
Copy Markdown
Contributor

@thomwebb thomwebb commented May 8, 2026

Problem

crof.ai's kimi-k2.5-lightning endpoint returned 500 errors and silently dropped agent responses when used through code-puppy. Three distinct issues were identified:

1. strict: true causes 500 on non-streaming requests

crof.ai's non-streaming endpoint rejects strict: true in tool function schemas with a 500 Failed to generate response. Only one tool (load_image_for_analysis) had strict: true set by pydantic-ai, but that was enough to kill every request.

strict=false strict=true
stream=False ❌ 500
stream=True

2. Flaky SSE streaming

crof.ai's streaming transport intermittently fails with "Streamed response ended without content or tool calls" in the real runtime (DBOS + TUI + event handler). Raw HTTP tests pass, but the full CLI stack hits timing issues. Disabling streaming via streaming: false in models.json provides a stable single-shot path.

3. Silent agent output when streaming is disabled

When streaming: false is set for a model, the runtime correctly disables the stream handler — but DBOS's skip_fallback_render callback still returned True, preventing the non-streaming fallback renderer from running. Nobody printed the output.

Changes

code_puppy/model_factory.py

  • Add _CompatChatModel (subclass of OpenAIChatModel) for providers needing compatibility shims. Activated when strict_tools: false or provider: "crof" is set in models.json.
  • _map_tool_definition: strips strict key from tool schemas when strict_tools: false.
  • _map_model_response: flattens tool calls into assistant text for providers that can't handle tool_calls in assistant messages.
  • _map_user_message: degrades role=tool results into role=user messages for providers that reject tool-role messages.
  • Only send parallel_tool_calls setting when the provider/model advertises support (prevents 500 on unknown fields).

code_puppy/agents/_runtime.py

  • Add _model_allows_streaming(): reads streaming from models.json config. Models with "streaming": false skip the SSE path entirely.
  • When streaming is disabled, clear _event_stream_handler on the pydantic agent so pydantic-ai uses model.request() instead of request_stream(). Restore after the run.
  • Remove the 📡 Streaming disabled info emission (noisy, not actionable).

code_puppy/plugins/dbos_durable_exec/runtime.py

  • skip_fallback_render() now checks whether streaming is actually active for the current model. When streaming is disabled, returns False so the core non-streaming fallback renderer can produce output. Previously it returned True whenever DBOS was launched, causing silent agent responses.

code_puppy/models.json

  • Add streaming: false and strict_tools: false for crof-kimi-k2.5-lightning.

Tests

  • tests/agents/test_streaming_retry.py: new test class TestModelAllowsStreaming covering _model_allows_streaming() for None, crof-kimi, and normal models. New class TestNonStreamingHandlerClear verifying _event_stream_handler is nuked on DBOSAgent-like objects.
  • tests/test_crof_tool_result_compat.py: tests for _CompatChatModel behavior (strict stripping, tool result mapping, tool call flattening).
  • tests/test_model_factory_coverage.py: updated to verify parallel_tool_calls is conditionally included based on provider support.

Testing

All changes verified with 1700+ passing tests and live e2e requests against crof.ai:

  • strict_tools=false → no 500s across 10+ consecutive requests
  • streaming=false + DBOS → fallback render fires, agent output visible
  • Tool calls (create_file, list_files) complete and produce text responses
  • parallel_tool_calls not sent for crof models

Cell added 2 commits May 8, 2026 16:26
…nfig

- Introduce _model_allows_streaming() to check per-model streaming config
- Add "streaming": false support in models.json for models with flaky SSE (e.g. crof.ai kimi)
- Clear _event_stream_handler on pydantic_agent when streaming disabled to force non-streaming request path
- Update DBOS plugin to skip fallback render only when streaming is actually active
- Add comprehensive tests for streaming override behavior and handler clearing
- Create _CompatChatModel wrapper for providers with OpenAI API incompatibilities
- Strip "strict" from tool schemas when strict_tools=false (crof.ai doesn't support it)
- Flatten tool calls into assistant content to avoid tool_call/result wiring errors
- Degrade tool results into user messages for providers that error on role='tool'
- Only send parallel_tool_calls setting when provider advertises support
- Add tests for crof.ai tool result mapping and settings behavior
@mpfaffenberger mpfaffenberger merged commit a83ffb0 into mpfaffenberger:main May 9, 2026
2 checks passed
mpfaffenberger added a commit that referenced this pull request May 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants