Repository navigation
perf(relay): keep the models.dev bodies off the relay - #3144
Merged
Merged
Conversation
Every cross-machine model catalog read carried the full models.dev projections. Opening Model Settings, and creating a session, therefore waited on a multi-MiB body that the receiver already owns: models.dev is public data and every host refreshes its own snapshot. The catalog that crosses a machine boundary now carries only the configured models, the defaults and the session selection. That covers the peer host answer, the remote session poll, the mobile and bot controllers, and the CLI peer host. The slim build still reports the built-in provider catalog revision, so `RemoteModelCatalog::version` does not move and a controller's change detection is unaffected. A controller that renders Model Settings while a peer is selected composes the surface from the host facts plus its own snapshot, through the new controller-local `get_local_models_dev_catalogs` command. Session creation reads three small config keys and asks for one model's reasoning projection instead of pulling the whole catalog. Co-authored-by: bitfun-ai <bitfun-ai@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Every cross-machine model catalog read carried the full models.dev projections. The catalog that crosses a machine boundary now carries only the configured models, the defaults and the session selection. A controller that renders Model Settings while a peer is selected composes that surface from the host facts plus its own models.dev snapshot.
Why
models.devis public data and every host refreshes its own snapshot, so the projections are identical by construction on both ends of a connection. Shipping them meant:get_ai_model_catalog, the dominant part of the "new session takes a long time" report).Changes
RemoteModelCatalogis now built without the models.dev bodies on every cross-machine path: the peer host answer, the remote session poll, the mobile and bot controllers, and the CLI peer host.RemoteModelCatalog::versiondoes not move. A controller compares that version, so a slim build that dropped the revision would have made every poll look like a catalog change —provider_catalog.rsnow exposesbuiltin_provider_catalog_identity(), and a test pins slim/full version equality plus revision sensitivity.get_local_models_dev_catalogscommand serves the controller's own snapshot;get_ai_model_catalogkeeps returning the full catalog only for in-process readers (TUI/app-server projections, plugin host) throughget_local_model_catalog/get_remote_model_catalog.project_ai_model_reasoning_catalog) instead of pulling the whole catalog.Remote scenarios
Exercised in code and by tests for remote control (session poll path, bot router) and peer device mode (registry row, controller-local settings read). The remote-workspace path only consumes the same host trait, so it inherits the slim build.
Verification
cargo test -p openbitfun-services-integrations --no-default-features --features remote-connect(including the newslim_model_catalog_keeps_the_version_of_the_full_catalog, which fails if the slim build drops the revision)provider_catalog, reasoning catalog, bot routing)cargo test -p openbitfun-product-domainsflow_chat+infrastructure/config+infrastructure/apisuites,type-check:web,type-check:mobile-web, eslint on the changed filestest:device-compatibility,test:account-loginpnpm run check:core-boundaries,pnpm run capabilities:check,pnpm run fmt:rsNotes
get_ai_model_catalog, so a browser-hosted Model Settings page does not receive provider templates.Co-authored-by: bitfun-ai bitfun-ai@users.noreply.github.com