Skip to content

feat(models): expand provider-native capability readers#5576

Open
RaresKeY wants to merge 13 commits into
odysseus-dev:devfrom
RaresKeY:feature/model-capability-catalog
Open

feat(models): expand provider-native capability readers#5576
RaresKeY wants to merge 13 commits into
odysseus-dev:devfrom
RaresKeY:feature/model-capability-catalog

Conversation

@RaresKeY

@RaresKeY RaresKeY commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Expand the canonical model-capability bootstrap from #2739 with provider-native catalog detection and readers while making the general fallback strictly inventory-only. The serialized record is reduced to one versioned shape, provider identity remains separate from per-model capability, and unrecognized configured providers can still list stable model identities without inferring tasks, modalities, features, controls, or limits.

This also adds bounded normalization diagnostics to Odysseus's existing LOG_LEVEL=DEBUG toggle. The event reports canonical version, provider source, detected catalog shape, fallback status, record count, and normalized capability names; it omits model IDs and raw payload values.

Stack / Review Boundary

This PR is stacked on #2739 at d8627fcd. Because #2739's head is on a fork, this upstream PR targets dev and temporarily includes the parent commits in its comparison.

The follow-up-only review range is d8627fcd..9d4d182b, containing three commits. After #2739 lands, this branch will be rebased onto current dev so the PR comparison contains only the follow-up.

Target branch

  • This PR targets dev, not main. All PRs land in dev; main is curated by the maintainer at each release.

Linked Issue

Fixes #5575

Part of #2737.

Depends on #4145 / #2739.

Type of Change

  • Bug fix (non-breaking — fixes a confirmed issue)
  • New feature (non-breaking — adds new behaviour)
  • Refactor / cleanup (behaviour unchanged)
  • Documentation only
  • CI / tooling / configuration

Checklist

  • I searched open issues and open PRs — this is not a duplicate.
  • This PR targets dev.
  • My changes are limited to provider/model catalog normalization, diagnostics, and focused regressions.
  • I actually ran the app (docker compose up or uvicorn app:app) and verified the change end-to-end. The catalog normalization layer is not yet wired into runtime model discovery, so this PR is validated through native-shape and fallback fixtures rather than a user-facing runtime path.

How to Test

  1. Run:

    python -m pytest -q \
      tests/test_model_capabilities.py \
      tests/test_model_capability_readers.py \
      tests/test_provider_capability_schemas.py \
      tests/test_model_capability_diagnostics.py

    Expected result: 50 tests pass.

  2. Inspect the fallback regression containing type, modalities, capabilities, supported_parameters, and max_model_len; verify its canonical result remains unknown with empty features, limits, and controls.

  3. Set LOG_LEVEL=DEBUG, normalize a native and a fallback fixture, and verify the log includes canonical/shape/fallback metadata but not the model ID or raw payload values.

  4. Run git diff --check and Python compilation for the changed modules.

Exact stacked-head adjacent validation also passes:

  • 388 provider detection, endpoint resolution, model context, Copilot, Anthropic, Ollama, Mistral, Moonshot, and reasoning tests;
  • 285 model route, endpoint probe, model selection, streaming, Kimi, Cerebras, vision, and local-provider tests;
  • Python compilation for the changed application, capability, reader, schema, and diagnostic-test modules.

A current-dev integration replay completed with 4,659 passing tests and 12 unrelated failures. The same 12 failures reproduced on the unchanged current-dev baseline; focused capability/diagnostic tests are green on this exact stacked branch.

Visual / UI changes

N/A — backend normalization, logging configuration, and tests only; no UI rendering changed.

Screenshots / clips

N/A — no visual changes.

@github-actions github-actions Bot added the ready for review Description complete — ready for maintainer review label Jul 17, 2026
@RaresKeY
RaresKeY marked this pull request as ready for review July 17, 2026 13:40
@github-actions github-actions Bot added merge conflict Conflicts with the base branch; needs a rebase before review. ready for review Description complete — ready for maintainer review and removed ready for review Description complete — ready for maintainer review labels Jul 18, 2026
@RaresKeY
RaresKeY marked this pull request as draft July 18, 2026 11:13
@RaresKeY
RaresKeY force-pushed the feature/model-capability-catalog branch from 9d4d182 to 0f19339 Compare July 18, 2026 14:11
@RaresKeY
RaresKeY marked this pull request as ready for review July 19, 2026 11:03
@github-actions github-actions Bot removed the merge conflict Conflicts with the base branch; needs a rebase before review. label Jul 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for review Description complete — ready for maintainer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

models: expand native capability readers with conservative fallback

1 participant