Skip to content

Vendor BufferingWrapper with flush() from bluesky - #77

Merged
genematx merged 6 commits into
bluesky:mainfrom
genematx:blocking-buffer
Nov 2, 2026
Merged

genematx merged 6 commits into
bluesky:mainfrom
genematx:blocking-buffer

Conversation

@genematx

@genematx genematx commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Vendor bluesky.callbacks.buffer.BufferingWrapper (following the _dispatcher precedent) with the addition of the flush() method, contributed upstream on the bluesky add-buffer-flush branch but not yet released. This lets beamline profiles adopt the read-after-write synchronization barrier via a bluesky-tiled-plugins release: documents are processed in a background thread, so code that reads a run back immediately after RE(plan) returns (e.g. db[uid]) must call flush() first or it may observe a missing or incomplete run, or silently pick up the previous one.

The vendored module is self-contained (standard library only) and exported from the package root, so profiles can import TiledInserter/TiledWriter and BufferingWrapper from one place. It can be replaced by a re-export once a bluesky release includes flush().

@genematx
genematx requested a review from thopkins32 November 2, 2026 20:31
@danielballan

Copy link
Copy Markdown
Member

Merge at will.

tiled 0.2.21 migrated its client from httpx to httpx2, changing the
exception family raised through it: ClientError now subclasses
httpx2.HTTPStatusError, so our `except httpx.HTTPStatusError` clauses
silently stopped matching. Concretely, the remote-validation 404 fallbacks
in BlueskyRun.validate and TiledWriter raised instead of falling back to
local validation.

Pin tiled >=0.2.21 (runtime and test group) and switch both sites to
httpx2, declared as a direct dependency since we import it. A small test
guards the contract (ClientError is catchable via httpx2.HTTPStatusError)
so the next migration fails loudly instead of silently.
tiled 0.2.21 (the httpx2 generation) dropped Python 3.10, which made the
dependency resolution unsatisfiable for the 3.10 split of the supported
range and failed CI on every job, including 3.13 and the docs build.

- Bump `requires-python` to >=3.11 and drop the 3.10 classifier
- Update mypy `python_version` and pylint `py-version` to 3.11
- Test on 3.11 (new floor) and 3.13 in CI
- Update the Python version requirement in the installation tutorials
- Use `itertools.pairwise` in StreamDatum batching, resolving an old TODO
SQLAlchemy 2.1 deprecates the `noload` loader strategy, which tiled's
catalog ORM uses for the `Webhook.node` relationship. With warnings
raised as errors, the deprecation fails `configure_mappers()` and with
it every test that starts the in-process server. This is a third-party
issue, to be fixed in tiled.
Vendor bluesky.callbacks.buffer.BufferingWrapper (following the _dispatcher
precedent) with the addition of the flush() method, contributed upstream on
the bluesky add-buffer-flush branch but not yet released. This lets beamline
profiles adopt the read-after-write synchronization barrier via a
bluesky-tiled-plugins release: documents are processed in a background
thread, so code that reads a run back immediately after RE(plan) returns
(e.g. db[uid]) must call flush() first or it may observe a missing or
incomplete run, or silently pick up the previous one.

The vendored module is self-contained (standard library only) and exported
from the package root, so profiles can import TiledInserter/TiledWriter and
BufferingWrapper from one place. It can be replaced by a re-export once a
bluesky release includes flush().
@codecov

codecov Bot commented Nov 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.72222% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.98%. Comparing base (366b68e) to head (7c95b46).

Files with missing lines Patch % Lines
src/bluesky_tiled_plugins/writing/_buffer.py 83.07% 11 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #77      +/-   ##
==========================================
+ Coverage   78.78%   78.98%   +0.20%     
==========================================
  Files          16       17       +1     
  Lines        2432     2498      +66     
==========================================
+ Hits         1916     1973      +57     
- Misses        516      525       +9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@genematx
genematx merged commit c463781 into bluesky:main Nov 2, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants