Skip to content

fix(server): warm BGE-M3 through patchable loaders - #545

Merged
chris-dare-dev merged 1 commit into
mainfrom
claude/test-query-encoder
Nov 15, 2026
Merged

chris-dare-dev merged 1 commit into
mainfrom
claude/test-query-encoder

Conversation

@chris-dare-dev

@chris-dare-dev chris-dare-dev commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

What

Resources.startup warmed BGE-M3 through _get_tokenizer and _get_model, imported by name from server.query_encoder. That binds the original functions when server/resources.py is first imported. Tests that mock the model (mocked_bge_m3) patch server.query_encoder, so the startup warm-up never saw the mock. Every test that boots the server loaded the real 2.3 GB model wherever it was cached, and failed wherever it was not.

mocked_bge_m3's docstring promises the suite "runs without GPU and finishes in milliseconds, not minutes". On a machine with the model cached, every startup test has been loading the real model instead.

Change. Startup now calls query_encoder._get_tokenizer / query_encoder._get_model through the module, so one patch covers both the warm-up and request-time encoding.

The trap was known and routed around rather than fixed:

  • tests/_corpus_helpers.patch_bge_m3_model patched both modules.
  • tests/test_bootstrap_mode.py and one test_server_startup.py test patched server.resources directly.
  • Three docstrings called dual-module patching load-bearing.

All of those now patch query_encoder only, and the docstrings say so. test_a_cacheless_boot_says_cacheless_and_not_warm patched neither module, and now mocks the model too.

Verification

Run on Ubuntu (Linux) at 60331e6, built on main at e2357bd, with no BGE-M3 files available. The failure baseline below was measured on main at d9ddaa8, before #539, #540 and #541 merged. No test those PRs added shows up as a failure here.

  • New guard: TestModelWarmupIsPatchable boots the app with recording stand-ins for both loaders and expects ["tokenizer", "model"]. On the old binding it fails here with the model-load error. With the model cached, it would see no calls at all.
  • Full pytest, this branch alone: 123 failures and errors, against the baseline's 214, with none new. The 91 fixed are every test that boots the server: test_security.py, test_tools_all.py, test_snippet_contract.py, test_server_startup.py, test_server_metrics.py, the Resources integration tests in tests/retrieval/, and test_corpus_integrity_probes.py.
  • Full pytest with test: run chunker tests on a stand-in tokenizer #544 merged in as well: 17 left:
    • 16 are proxy 403s in test_formal_release_pin.py. Those tests need the mfc contract package, which Remove release pins and the formal-contract tooling #547 removes along with the file.
    • 1 is test_symlink_loop_is_rejected, which fails in this sandbox for a reason unrelated to the model.
  • ruff check . is clean.
  • Before merge: a workstation make test. Startup tests there should also get noticeably faster, since they stop loading the real model. I could not measure that here.

Notes

Resources.startup warmed BGE-M3 through _get_tokenizer and _get_model
imported by name from server.query_encoder, which binds the original
functions when resources.py is first imported. Tests that mock the
model patch server.query_encoder, so the startup warm-up never saw
the mock. Every test that boots the server loaded the real 2.3 GB
model wherever it was cached, and failed wherever it was not.

Startup now calls the loaders through the module, so one patch covers
both the warm-up and request-time encoding.

The trap was known and routed around rather than fixed:
patch_bge_m3_model and one test patched server.resources as well,
and three test docstrings described dual-module patching as a rule.
Those now patch query_encoder only, and the docstrings say so. The
cacheless-boot probe, which patched neither, now mocks the model too.

A new test boots the app with recording stand-ins for the two loaders
and expects both called. It fails on the old binding, where the real
loaders run instead; with the model cached it would see no calls.

Measured on Linux with no model available: the full suite goes from
214 failures to 123, all 91 fixed being tests that boot the server,
and none new.
@chris-dare-dev
chris-dare-dev merged commit 128059b into main Nov 15, 2026
1 check passed
@chris-dare-dev
chris-dare-dev deleted the claude/test-query-encoder branch November 15, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants