Skip to content

Expose ORT placement controls for user-defined text models - #280

Merged
anush008 merged 2 commits into
anush008:mainfrom
corbet-labs:cfetch-session-controls
Aug 23, 2026
Merged

anush008 merged 2 commits into
anush008:mainfrom
corbet-labs:cfetch-session-controls

Conversation

@julian-corbet

Copy link
Copy Markdown
Contributor

Adds opt-in controls to InitOptionsUserDefined for two ORT session settings needed by accelerator applications:

  • disable CPU EP fallback, so successful session creation proves complete accelerator placement;
  • override named free dimensions, so static-shape EPs can compile a user-defined graph for an admitted shape.

Both controls default off, preserving existing behavior. BGE-M3 callers ignore the new text-only controls.

Verified with:

  • cargo check --no-default-features
  • cargo test --no-default-features --features ort/load-dynamic user_defined_session_controls_are_opt_in_and_composable --lib

@anush008 anush008 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking the time to contribute @julian-corbet.

LGTM!

@anush008
anush008 merged commit 54bb66e into anush008:main Aug 23, 2026
9 checks passed
anush008 pushed a commit that referenced this pull request Aug 23, 2026
* feat: expose ORT controls for user-defined text models

* feat: expose reranker session controls
github-actions Bot pushed a commit that referenced this pull request Aug 23, 2026
## [6.0.1](v6.0.0...v6.0.1) (2026-08-23)

### 🧹 Chores

* Expose ORT placement controls for user-defined text models ([#280](#280)) ([8aeca82](8aeca82))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants