Skip to content

honor model-aware reasoning defaults and controls - #37

Closed
laceyp99 wants to merge 2 commits into
mainfrom
fix/metadata-aware-reasoning
Closed

laceyp99 wants to merge 2 commits into
mainfrom
fix/metadata-aware-reasoning

Conversation

@laceyp99

Copy link
Copy Markdown
Owner

Summary

  • preserve OpenRouter's modern reasoning metadata and distinguish declared, null, and omitted effort capabilities
  • apply model defaults, safe medium fallback behavior, mandatory-model restrictions, and payload omission from one shared capability result
  • align selector copy, request guards, regression coverage, and README guidance around quality, latency, token usage, and cost

Testing

  • .\venv\Scripts\python.exe -m ruff format --check . - passed
  • .\venv\Scripts\python.exe -m ruff check . - passed
  • .\venv\Scripts\python.exe -m pytest - passed (122 tests)
  • live-catalog browser check - passed for provider-default omission, mandatory GPT-5 choices/default, optional GPT-5.4 Mini disabled default, and model-switch reset behavior

Notes

  • intended as the metadata-aware replacement for Make reasoning effort opt-in by default #36; that pull request was not modified
  • token-budget controls remain outside this change; supports_max_tokens is preserved for capability reporting

Closes #34

@laceyp99
laceyp99 marked this pull request as ready for review July 22, 2026 04:26
@github-actions

Copy link
Copy Markdown

Agent PR review

Validation

Validation passed: python -m ruff format --check ., python -m ruff check ., and python -m pytest all completed successfully.

Summary

PR #37 updates reasoning-control behavior to use OpenRouter’s modern reasoning metadata, preserve that metadata during catalog normalization, derive safer defaults, block stale/unsupported effort payloads, and document the resulting latency/token/cost behavior.

Issues and risks

No blocking correctness, security, performance, or maintainability issues found in the reviewed diff.

Test coverage

The PR adds focused unit and integration coverage for modern reasoning.supported_efforts, null vs omitted effort metadata, mandatory models, disabled defaults, stale/hidden effort rejection, request payload shaping, and UI control updates.

I did not rerun validation because this review sandbox is read-only and the project checks may write cache artifacts. The supplied review context reports that python -m ruff format --check ., python -m ruff check ., and python -m pytest all passed, with 122 tests.

Recommendation

Looks safe to continue review.


This is an automated review. Please treat it as advisory; human review is still required.

@laceyp99

Copy link
Copy Markdown
Owner Author

This doesn't solve the issue properly either. I remember facing this issue before, but Openrouter's API is not a single source of truth. This solution's coverage holes are best represented by switching between Anthropic's Claude Sonnet 5 (all reasoning effort levels available; even a default value given) vs. Claude Opus 4.5 (just shows it supports reasoning; nothing else).

For now, I am happy with leaving the reasoning app default to medium as it seems to leave my approach with the best coverage. At the end of the day the users can still control the reasoning effort level and move it lower if they don't want that much reasoning.

This overall issue with real-time updated model registries with complete metadata is inspiring me to create a long term solution as its own repo to have all of my AI apps utilize.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Design: reasoning effort silently defaults to 'medium', opting every capable model into paid reasoning

1 participant