Skip to content

🥼 fix: Separate Agent Inference Settings From SDK Configuration - #17220

Open
lia-by-librechat[bot] wants to merge 6 commits into
devfrom
lia/agent-inference-boundary
Open

lia-by-librechat[bot] wants to merge 6 commits into
devfrom
lia/agent-inference-boundary

Conversation

@lia-by-librechat

@lia-by-librechat lia-by-librechat Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Agent model parameters previously accepted arbitrary records that reached SDK client construction after model authorization. User-controlled nested kwargs could replace the authorized model, and fallback client options could introduce unchecked providers or destinations.

Separate user inference input from server-owned SDK configuration. New create/update requests reject unsupported or invalid fields with value-free validation errors. Execution independently projects saved settings and request overrides, so legacy records cannot keep passing SDK configuration. The selected model remains separately authorized and pinned.

Started from current dev as a new PR at the maintainer's request. Related to #16800; that PR is unchanged.

How it works

create/update → provider-specific typed inference input → persisted settings
saved settings + request overrides → inference projection
  → trusted endpoint configuration + authorized model → SDK adapter

Custom inference extensions require explicit administrator-defined types and constraints. Normal YAML loading validates those declarations, and the existing agent editor exposes and preserves their controls. Adapters construct request extras and preserve default < explicit value < admin addition < final drop ordering, including whole-envelope drops. Credentials, headers, client construction, routes and fallbacks are not user inference settings.

Discovery, strict writes, runtime projection and actual custom initialization share trusted effective parameter-profile resolution, including administrator environment-resolved URLs. Explicit and native profiles remain authoritative. Disabling or dropping reasoning does not remove the provider’s independent caching controls; reasoning visibility, catalogue access and payload shaping honor their own capability decisions. Disabled-reasoning cleanup applies after all generic and hinted OpenAI adapter branches, not only to OpenRouter. Runtime agents retain canonical validated input separately for memory reinitialization. Recognized legacy Bedrock thinking/effort preferences are recovered into typed flat settings before opaque envelopes are dropped. Generated thinking bindings are normalized and regenerated for the current model. Partial updates preserve omission. Existing records are not rewritten; no database reads are added.

Type of change

  • Bug fix
  • Tests / tooling / CI

Testing

Head 5ffa5cb299f5e1dc2b5c381486b04452dde0336f:

  • 1,310 focused API/loader/profile/inference/provider/SDK/reasoning tests passed.
  • 42 agent-editor/reasoning-hook tests passed, including exposing typed custom controls, retaining saved values, and keeping cache preferences independent of disabled reasoning.
  • API, data-schemas and client npx tsc --noEmit passed.
  • Exact-head PR-scoped static checks passed; full config/i18n/dependency scans were not run.
  • Exact-head API and client production builds passed.
  • 221 MongoDB controller tests passed at this head.
  • Fresh exact-head CI and independent review are running. Earlier-head CI and review results are not reused.

Local Lighthouse production preparation passed on the initial head but Chrome was absent; browser screenshots could not be captured. This head’s Lighthouse is tracked in CI. No live provider inference or credential-exfiltration test is claimed.

Risk / compatibility

Raw SDK options and undeclared extras are no longer user-editable. Invalid new writes return 400 rather than persisting partial settings; old records are filtered during execution without a migration. Built-in inference controls and explicitly declared flat custom parameters are preserved. Fallback disclosure requires activation, SDK credential resolution and network reachability; this PR does not claim demonstrated exfiltration.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Head 5b70226b1c46f435c4d4eb9ae373a94f8b88406f: a fresh dev-based inference-input boundary. Typed writes and independent runtime projection replace arbitrary user SDK configuration; trusted custom extension ordering and repeated initialization are covered. Fresh-baseline verification and independent exact-head review are in progress. No workflow or documentation changes.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Head ec9164de63f437bc94d2bfbd02acba0666867051: admit safe explicitly typed YAML extensions through the real loader and existing agent settings UI, and recover only validated legacy Bedrock thinking/effort preferences before rejecting SDK envelopes. API/loader/provider tests: 1,055 passed; client control tests: 20 passed. API and client typechecks plus touched-file lint passed. Builds/persistence/static verification and independent review are in progress for this head.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Head 20dc6b1a4857449ef943f35f2236b129db09e8d4: share the trusted custom parameter-profile resolver between endpoint discovery, write validation, and runtime projection. The invariant matrix covers resolved administrator URLs, explicit/native profile precedence, unrelated/user-provided URLs, and real fake-fetch OpenRouter requests preserving caching off. API/profile matrix: 1,131 tests passed; API typecheck and touched-file lint passed. Exact-head build/persistence/static checks and independent review are in progress.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Head 02ed11f341175856fdd636718fdc37484ae6c80b: preserve the trusted OpenRouter/cache profile independently of reasoning drops or disabling, suppress reasoning advertisement/catalogue use and wire fields separately, and retain cache controls in the agent editor. A 36-case loader/write/runtime/SDK matrix and catalogue/editor checks cover the invariant. Focused verification: 588 API and 40 client tests passed; typechecks/lint passed. Full regression, build, static and independent exact-head review are pending for this head.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Head 54183824474e76a01acd3d471532134b28af2cb7: remove both top_p/topP aliases at the final sampling-drop boundary and remove legacy include_reasoning from both SDK envelopes when reasoning is disabled or dropped. New regressions capture the real SDK request body and preserve independent caching preferences. Local checks: 1,297 API tests, 42 client tests, API/client typechecks and scoped lint passed. Full static/build/persistence checks and independent exact-head review are in progress.

@lia-by-librechat

Copy link
Copy Markdown
Contributor Author

Head 5ffa5cb299f5e1dc2b5c381486b04452dde0336f: carry the trusted effective profile through real custom initialization and enforce disabled reasoning independently of provider identity and after hinted-adapter transforms. Production initializer-to-SDK tests cover resolved administrator URLs, reasoning drops, generic/OpenRouter/Google/Anthropic hints, enabled controls and cache opt-outs. Local checks: 1,310 API tests, API typecheck/lint, API build and 221 MongoDB controller tests passed. Full static checks and fresh exact-head CI/review are pending.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant