Repository navigation
Make dependencies smaller: use tiled[minimal-client] - #69
Merged
Merged
Conversation
The plugin imports xarray, pandas, pyarrow and numpy at module level, and tiled.ndslice needs ndindex, which tiled's xarray extra omits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #69 +/- ##
==========================================
+ Coverage 78.38% 78.76% +0.38%
==========================================
Files 16 16
Lines 2387 2430 +43
==========================================
+ Hits 1871 1914 +43
Misses 516 516 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…d deps With tiled[minimal-client], the awkward and ragged packages are no longer installed, so writing ragged (variable-length) array data fails at the lazy import inside TiledWriter. Declare the optional dependencies as published extras -- bluesky-tiled-plugins[ragged] (pins matching the tiled[client] extra) and an [all] aggregate -- and wrap the lazy import so the mid-scan failure tells the user which extra to install. Also bump the minimum tiled to 0.2.19 (runtime and test group).
The test dependency group includes tiled[server], which transitively provides awkward, ragged, and more -- so the regular test jobs can not catch a missing runtime dependency. Add a minimal-install job that installs the package with no extras and no dependency groups, then runs tests/minimal_smoke.py: it import-walks every submodule (except the server-only routers.validator), exercises the document-processing machinery without a server, and asserts the optional ragged dependencies are genuinely absent so the slimmed dependency footprint can not silently regress. The smoke script is not collected by pytest (no test_ prefix); it fails by design in a full dev environment.
Contributor
|
Thank you, Zoheb. I've added ragged and awkward as optional dependencies in |
The name-tests-test pre-commit hook requires files under tests/ to match test_*.py, and the smoke script must not be collected by pytest (it fails by design in a full dev environment). Place it under .github/scripts/, next to the workflow that runs it.
The CI jobs only check out the repository and run tests; the codecov upload authenticates with its own CODECOV_TOKEN secret. Setting an explicit minimal permissions block follows the practice already used in cd.yml and resolves the GitHub Actions security warning.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I'm trying to make the dependencies of
bluesky-tiled-pluginssmaller so that downstream services such as blueapi don't have to install the whole oftiled[client]just to useTiledWriter.This swaps
tiled[client]fortiled[minimal-client]and lists the packages the plugin actually imports at module level:xarray:clients/bluesky_event_stream.pypandas:streaming.pypyarrow:writing/tiled_writer.pynumpyndindex: imported bytiled.ndslice. None of tiled's smaller extras include it.Size
Measured as a runtime install of blueapi (no dev dependencies):
tiled[client]Most of the saving comes from llvmlite and numba (about 190 MB together), plus blosc2, zstandard, awkward, sparse and numexpr.
Known limitation
Ragged arrays still need
awkwardandragged, which onlytiled[client]provides._write_internal_dataimportstiled.structures.raggedlazily, so a scan with a variable-length array signal fails during the scan, not at import time. Happy to add an optionalraggedextra if you'd prefer that.Testing
countwithTiledWriteragainst a local tiled server writes and reads back correctly, including arrays large enough to be written as zarr arrays.Downstream PR: DiamondLightSource/blueapi#1665
🤖 Generated with Claude Code