Publish areaDetector statistics and ROI statistics as a bounded data logic

CHANGELOG.md

@@ -34,6 +34,12 @@
 - `SimMotor.stage()` now calls `SimMotorLogic.ensure_ready()`, which calls `ensure_stopped()` and so stops the simulated motion, waits for it to finish, and sets the setpoint to the current readback. `unstage()` cancels a fly in progress, then calls `ensure_stopped()`. The simulated motion runs in the task of the verb that moves, so `stop()` during a `prepare()` or fly, or `stage()` during a `set()`, cancels that verb, which raises `CancelledError`.
 - The progress of `StandardFlyable.prepare()` now comes from the `WatcherUpdate`s the logic's `prepare_to_fly` yields, replacing the heuristic that watched setpoint writes (#1448).
 - `WatchableAsyncStatus` closes the async generator it drives as soon as it is cancelled or fails, rather than when the generator is collected, and a watcher that raises is logged and ignored rather than failing the status.
+- Each statistic of an `NDStatsIO` has one name, that of its scalar signal (e.g. `det-stats-total`), as a readable, as a stats time series and as an HDF NDAttribute, so the highest of these serving a scan suppresses the others, and the hint of the scalar stays. The plugin's `EnableCallbacks` and `compute_statistics`/`compute_centroid` PVs at the first `prepare()` after `stage()` decide which statistics are used: the scalars of those switched off are left out of `read()`, `describe()` and `hints`, `AreaDetector.prepare()` raises, naming what changed, if any of these switches has changed since the first `prepare()` after `stage()`, and a stats time series publishes every statistic switched on that has a read format on the plugin.
+- `AreaDetector` registers every plugin as a `CHILD` readable, so the signals a plugin declares as read, configuration or hinted appear in the detector's `read()`, `read_configuration()` and `hints`. Opt a plugin out with `set_readable_format(plugin, None)`. This covers the plugins in `plugins`, which are all the detector's plugins including those it creates (the circular buffer and process plugin), and the plugin each writer factory creates. Every factory's `build` is given all of them. A logic writes only the switches of the plugins it was built to own; the switches of every other plugin it only reads.
+- `setup_ndstats_sum` is removed: list the stats plugin in `ADDataFactory.hdf(ndattribute_sources=[stats])` and the HDF writer writes its NDAttributes, named after their scalars, so `det-sum` is now `det-stats-total`. A listed plugin's `nd_attributes_file` is overwritten when the HDF writer serves a scan; `setup_ndattributes` on another plugin or the driver works as before.
+- `AreaDetector` hides the read signals of a disabled plugin: at the first `prepare()` after `stage()` it walks the readable tree from the detector and asks each `NDPluginBaseIO` it finds for its `get_enabled_params()`, through a `DetectorSuppressLogic`, and keeps the answer until the next `stage()`. It leaves out of `read()`, `describe()` and `hints` each read signal of a plugin that is not in the answer, and `prepare()` raises if a plugin now answers differently. A plugin that is not a `StandardReadable`, such as a trigger logic's process plugin or an acquire logic's circular buffer, is not asked.
+- `NDROIStatIO.stage()` names each channel `<plugin name>-<Name PV>`, with characters other than letters, digits and underscores replaced by underscores and the channel number used for an empty name, and raises if two would be the same. This changes the datakey of every ROI stat signal. A channel that is not used leaves `read()`, `describe()` and `hints`, and a `Name` changed after `stage()` takes effect at the next one.
+- `adcore.prepare_filter` sets `enable_filter` to False for a single exposure, rather than filtering one, and switches the process plugin on as before.
 - A `StandardReadable` that a detector suppresses with its data keeps the hints of the signals it drops from `read()` and `describe()`, as the data has the same name. A data logic that hints such a name now makes `hints` raise.
 - `StandardReadable` now keeps one format per registered child, keyed by the child, so registering a child again replaces its format instead of adding a second contribution, and the order of `hints` is the order children were first registered. The private `_describe_config_funcs`, `_read_config_funcs` and `_has_hints` tuples are gone: a subclass that added to them now adds a hook to `_describe_configuration_funcs`, `_read_configuration_funcs` or `_hints_funcs` (see ADR 0023).
 - The deprecated `ConfigSignal` and `HintedSignal` aliases are removed from `ophyd_async.core`; use `StandardReadableFormat.CONFIG_SIGNAL`, `HINTED_SIGNAL` or `HINTED_UNCACHED_SIGNAL`. `StandardReadable.add_readables()` is deprecated in favour of `set_readable_format()` on each device.
@@ -48,14 +54,18 @@
 - `StandardDetector` exposes `collect_asset_docs()` or `collect_pages()`, whichever the serving providers produce, as an instance attribute bound by `prepare()`, so the bluesky run bundler asks for the right one.
 - A registered signal with the same name as a datakey a serving provider makes is left out of `read()`, `describe()` and `hints`, so the provider's data replaces it; `describe()` raises `ValueError` if the two differ in dtype or shape beyond one leading dimension.
 - `TriggerInfo.number_of_events` must now be positive.
-- `ADHDFDataLogic` sizes its HDF chunks from `TriggerInfo.flush_period`: when it and the collection period are both positive, `NumFramesChunks` is set to the frames collected in a flush period, `round(flush_period / collection_period)`, at least 1 (#1309). Otherwise the value on the IOC is used, 0 taken as 1. `NumFramesChunks` is now written back at every `prepare()` that starts the HDF writer, where it used to be written only to change 0 to 1.
-- `adcore.ADWriterFactory` is renamed `ADDataFactory`; the old name still works with a `DeprecationWarning`. Its factories take `always_enable`, default `False`: the plugin's `EnableCallbacks` PV is followed and the plugin sits out the scans of a stage if it is off at the first `prepare()`, so another data logic can serve them. The HDF writer used to be switched on whatever the PV said, so pass `always_enable=True` to keep that.
+- `ADHDFDataLogic` sizes its HDF chunks from `TriggerInfo.flush_period`: when it and the collection period are both positive, `NumFramesChunks` is set to the frames collected in a flush period, `round(flush_period / collection_period)`, at least 1 (#1309), and at most what fits in a 4 MiB chunk. Otherwise the value on the IOC is used, 0 taken as 1. `NumFramesChunks` is now written back at every `prepare()` that starts the HDF writer, where it used to be written only to change 0 to 1.
+- `adcore.ADWriterFactory` is renamed `ADDataFactory`; the old name still works with a `DeprecationWarning`. `ADDataFactory` is built with keywords only: `build`, a callable `(prefix, driver, plugins) -> (named plugin or None, data logic)`, in place of `writer_cls`, `writer_suffix` and `writer_name`, which `hdf`, `jpeg` and `tiff` still take. Its factories take `always_enable`, default `False`: the plugin's `EnableCallbacks` PV is followed and the plugin sits out the scans of a stage if it is off at the first `prepare()`, so another data logic can serve them. The HDF writer used to be switched on whatever the PV said, so pass `always_enable=True` to keep that. `ADHDFDataLogic` and `ADMultipartDataLogic` with it switch their plugin on at `stage()`, and no longer write `EnableCallbacks` at each `prepare()`.
 - `StreamResourceDataProvider` takes `collections_written` (main's keyword was `collections_written_signal`), the signal counting the collections written, and `collections_per_event`.
 - `StreamableDataProvider.collections_written_signal` is now `DataProvider.collections_written`, an attribute every provider has, the signal counting the collections written.
 
 ### Added
 
+- `NDParam`, a scalar a plugin publishes with its asyn parameter, address and time series array, and the `ts_*` array signals of every statistic on `NDStatsIO`.
+- `ADDataFactory.hdf(ndattribute_sources=)`, the plugins whose NDAttributes the HDF writer writes. A source must be in the detector's `plugins`, as one outside it is never connected. The NDAttributes of other plugins and the driver are parsed from the XML the user set, with the names in it.
+- `NDPluginBaseIO.get_enabled_params()`, the `NDParam`s whose switches are on for the coming scan; those with a read format on the plugin are published, overridden by `NDStatsIO` and `NDROIStatIO`.
 - `DetectorSuppressLogic`, set as `DetectorLogic.suppress`, returns from `hidden_datakeys()` the names of the readables the detector's data hides. `DetectorLogic.prepare_to_fly` asks it at the first `prepare()` after `stage()`, keeps the answer in the context and raises at a later `prepare()` if it differs.
+- `NDROIStatIO` stamps the `min_value`, `max_value`, `mean_value`, `total` and `net` of each channel that is used and has a read format, named after the scalar. `NDROIStatIO` is a `StandardReadable` whose channels are registered as `CHILD` readables, so there is no need to register each channel.
 - `KICKED_OFF`, the sentinel `FlyableLogic.fly` yields once the scan is up to speed.
 - `watch_readback()` reports the progress of a move by watching a `MovableLogic` readback, for use from `FlyableLogic` hooks.
 - `SimMotor.prepare()` and `SimMotor.complete()` report progress to watchers, as `Motor` does.
@@ -67,9 +77,13 @@
 - `StandardReadable.set_readable_format()`, `reset_readable_formats()`, `get_readable_formats()` and `get_readable_format_changes()` (which yield each child with its format), to change how a child contributes to `read()`, `read_configuration()` and `hints` at runtime, the equivalent of `Kind` in ophyd v1 (see "How to change what a Device reports at runtime").
 - `ReadableEntry`, `ReadableFormats`, `walk_readable_formats()`, `apply_readable_formats()` and `READABLE_FORMATS_KEY` in `ophyd_async.core`, and a `readable_formats` field on `Settings`, so formats are stored and retrieved with the signal values.
 - pyright now runs with `reportMatchNotExhaustive`.
-- `BoundedDataLogic`, `UnboundedDataLogic`, `DetectorLogic`, `DataProvider`, `PageableDataProvider` and `EventPageDataProvider` in `ophyd_async.core`, and `ADDataFactory` in `ophyd_async.epics.adcore`.
-- `EventPageDataProvider(refresh=)`, an awaitable callable awaited before each page's arrays are read.
 - `TriggerInfo.flush_period`, how many seconds between flushes of the data being written, default 0 for whatever the hardware is set to. Each data logic's `start_provider()` is given it.
+- `BoundedDataLogic`, `UnboundedDataLogic`, `DetectorLogic`, `DataProvider`, `PageableDataProvider` and `EventPageDataProvider` in `ophyd_async.core`, and `ADDataFactory` and `NDStatsTSAcquireMode` in `ophyd_async.epics.adcore`.
+- `adcore.TimeSeriesDataLogic` and the `ADDataFactory.time_series(plugin)` factory, a `BoundedDataLogic` that publishes the time series of an `NDPluginStats` or `NDPluginROIStat` as event pages, for an areaDetector that writes no file. It needs a fixed number of collections, as the buffer is sized before acquisition, and sits out when the plugin is switched off at the first `prepare()` after `stage()`, so a detector can switch between its file writer and its time series from one stage to the next. It never switches the plugin on itself; for an `NDStatsIO` it does switch on the embedded `TS:` time series plugin, which the IOC starts disabled. For an `NDROIStatIO` it publishes one array per used channel per scalar with a read format, named after the scalar (e.g. `det-roistat-Blue-total`), which it suppresses.
+- `NDROIStatTSControl`, the `ts_control`, `ts_num_points`, `ts_current_point` and `ts_acquiring` signals of `NDROIStatIO`, and the `ts_*` arrays of `NDROIStatNIO`, which `TimeSeriesDataLogic` uses.
+- `adcore.NDStatsTimeSeriesIO`, the `TS:` sub-plugin of `NDStatsIO`, available as `NDStatsIO.ts_plugin`; `TimeSeriesDataLogic` switches it on.
+- `NDROIStatNIO.net` and `NDROIStatNIO.ts_net`: `net` has no read format, as on `NDStatsIO`, so give it one to read, stamp or publish the `TSNet` time series of a channel.
+- `EventPageDataProvider(refresh=)`, an awaitable callable awaited before each page's arrays are read. The stats and ROI stat time series use it to ask the plugin to post its arrays (`NDStatsIO.ts_read`, or `NDROIStatTSControl.READ`), which it otherwise does only about once a second while acquiring, so a page collected mid-scan no longer reads arrays shorter than the count. Both time series raise `ValueError` at `prepare()` if the plugin's arrays (`ts_timestamp_nelm`, the waveforms' `NELM`) hold fewer points than the scan needs.
 
 ### Fixed
 

docs/explanations/decisions/0024-standard-detector-is-a-standard-readable.md

@@ -6,6 +6,9 @@ Date: 2026-10-30
 
 Accepted
 
+Partly superseded by [ADR 0027](0027-areadetector-statistic-names.md), under which
+`AreaDetector` registers every plugin.
+
 ## Context
 
 [ADR 0012](0012-detector-rewrite.md) gave `StandardDetector` its own ways to put a signal