diff --git a/CLAUDE.md b/CLAUDE.md index e1032b8b40..88927ff8f2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -39,6 +39,7 @@ tox -p # all envs in parallel (CI equivalent) - **No private-attribute access in tests** (`det._trigger_logic`, …) — call public methods and assert on output (e.g. `trigger()` then `describe()`), unless no public equivalent exists. - **Happy path in one public-interface test; unhappy paths small.** For a multi-step lifecycle (e.g. `prepare` → `kickoff` → `complete`), write a *single* "happy path" test that drives the whole sequence through the public interface and asserts the observable end state — don't split one test per step. Then add small "unhappy path" tests (ordering errors, limit violations, injected failures); these stay majority-public-interface but may use mocks to shorten sequences or inject errors. The smell this avoids is a step-scoped test that has to call the *other* steps to set itself up (e.g. a `kickoff` test that also calls `prepare`+`complete`) — fold that into the happy path instead. - **Parametrize normal + edge cases together** in one `@pytest.mark.parametrize`, not two functions. +- **Keep CI test stages to about a minute.** Each "Run tests" stage for unit tests and for system tests should take about a minute in CI. Take care to trim tests down in size, scale and sleeping to stay in that range. - `set_mock_value(signal, value)` injects state; `init_devices(mock=True)` (async CM) builds devices; `assert_has_calls(device, [...])` from `ophyd_async.testing` checks PV writes in order. ## Docs & docstring conventions diff --git a/docs/explanations/devices-signals-backends.md b/docs/explanations/devices-signals-backends.md index 3dbeb4c643..4dc2b96079 100644 --- a/docs/explanations/devices-signals-backends.md +++ b/docs/explanations/devices-signals-backends.md @@ -26,12 +26,13 @@ The `Device` class is the base of all ophyd-async objects that are published to - a [](#Device.set_name) method to set its name and also set the names of its children using the parent name as a prefix, called at init and also when a new child is attached to an already named Device - a [](#Device.connect) method that connects it and its children -All the above methods are concrete, but `connect()` calls out to a [](#DeviceConnector) to actually do the connection, only handling caching itself. This enables plug-in behavior on connect (like the introspection of child Attributes in Tango or PVI, or the special case for `Signal` we will see later). +All the above methods are concrete, but `connect()` calls out to a [](#DeviceConnector) to actually do the connection, only handling caching itself. This enables plug-in behavior on connect (like the introspection of child Attributes in Tango or PVI, or the special case for `Signal` we will see later). A connect is cached: if the last connect succeeded, calling `connect()` again with the same mock choice (the `DeviceMock` class, `mocked` and `mock_types`) returns without connecting again, in mock mode as well as real. Concurrent calls are not merged: each one that is not cached connects. A connect that is not cached, because the mock choice changed, the last connect failed or has not finished, or `force_reconnect=True` was passed, connects everything below the Device again, so the tree under a re-connected Device is always rebuilt as a whole. A `DeviceConnector` provides the ability to: - [](#DeviceConnector.create_children_from_annotations) that is called during `__init__` to turn annotations into concrete child Devices -- [](#DeviceConnector.connect_mock) that is called if `connect(mock=True)` is called, and should connect the child Devices in mock mode for testing without a control system -- [](#DeviceConnector.connect_real) that is called if `connect(mock=False)` is called, and should connect the child Devices to the control system in parallel +- [](#DeviceConnector.connect) that is called on every connect that is not cached, and should connect the Device to the control system or in mock mode as its `DeviceMock` says, then connect its children with the same timeout and `force_reconnect`: in parallel, or serially when the `DeviceMock` leaves nothing real below the Device. Each child is given a `DeviceMock` that says whether it is mocked, made from the `mock_types` of its parent's, which is how a mapping passed to `init_devices` reaches the whole Device tree + +A connect that is not cached, in either mode, may drop subscriptions made before it, so subscribe again afterwards. In mock mode it also resets mock values and the `Mock` call record, and drops staging, so `get_value(cached=True)` raises until the Signal is staged again. Calling `child.connect(mock=True)` directly on a child with a different mock choice from its last connect rebuilds the child's mock as a root, detached from the parent's mock tree and without the parent's `mock_types`, so re-connect from the parent instead. The base `DeviceConnector` provides suitable methods for use with non-introspected Devices, but there are various control system specific connectors that handle filling annotations in [declarative Devices](./declarative-vs-procedural.md). diff --git a/docs/tutorials/writing-tests-for-devices.md b/docs/tutorials/writing-tests-for-devices.md index 876333aa8a..2b2d7c0f9a 100644 --- a/docs/tutorials/writing-tests-for-devices.md +++ b/docs/tutorials/writing-tests-for-devices.md @@ -25,7 +25,7 @@ We will be writing a test using the pytest framework which encourages fixtures t :pyobject: mock_motor ``` -This fixture opts out of the [automatic mock behaviour](../explanations/when-to-extend-movable.md) by connecting with a plain [](#LazyMock), giving the tests control over when the readback updates mid-move. [](#set_mock_units) and [](#set_mock_precision) inject units and precision metadata directly on the readback signal, without needing dedicated child signals on the device. +This fixture opts out of the [automatic mock behaviour](../explanations/when-to-extend-movable.md) by connecting with a plain [](#DeviceMock), giving the tests control over when the readback updates mid-move. [](#set_mock_units) and [](#set_mock_precision) inject units and precision metadata directly on the readback signal, without needing dedicated child signals on the device. If we had any cleanup to do, we would do that after the yield statement. @@ -53,9 +53,47 @@ used when connected in mock mode: ``` Now whenever a `Motor` is connected using [](#init_devices)`(mock=True)`, it will -automatically use `InstantMotorMock` without any fixture setup. You can still override -the automatic mock for specific tests by passing an explicit [](#DeviceMock) instance -or a plain [](#LazyMock) directly to `connect()`, as the `mock_motor` fixture above does. +automatically use `InstantMotorMock` without any fixture setup. + +A default mock class should make the Device's verbs work in the most minimal way +possible, e.g. a move completes instantly. Mocks that model real behavior, like velocity +or timing, should be opt-in rather than registered as the default. + +You can still override the automatic mock for specific tests by passing a +[](#DeviceMock) instance to `connect()`, as the `mock_motor` fixture above does. + +:::{note} +A second `connect(mock=True)` with the same mock choice is cached, so it keeps the mock values and the call record. `connect(mock=True, force_reconnect=True)`, or a different mock choice, makes a new mock for the Device and all of its children, resetting mock values and the call record and dropping subscriptions and staging made before it. Subscribe or stage again afterwards. +::: + +(override-mock-by-type)= +### Mocking by Device type + +To mock only some Devices, or to override their mocks, including Devices deep in a composite Device, pass a mapping of Device type to `True`, `False` or a `DeviceMock` class to [](#init_devices) or [](#ensure_connected): + +```python +# Real, except Motors and everything under them, which are mocked +with init_devices(mock={Motor: True}): + stage = Stage("PREFIX:") + +# Mocked, except Motors, which are real all the way down, signals included +with init_devices(mock={Device: True, Motor: False}): + stage = Stage("PREFIX:") + +# Real, except Motors, which use VeloAndAcclRespectingMotorMock +with init_devices(mock={Motor: VeloAndAcclRespectingMotorMock}): + stage = Stage("PREFIX:") + +# The whole detector is mocked, plugins included +with init_devices(mock={AreaDetector: True}): + det = AreaDetector(...) +``` + +Each Device looks itself up by the nearest class in its MRO, and a `DeviceMock` class means mocked with that class. A Device with no entry follows its parent, so mocking a Device mocks its whole subtree. An entry under `Device` is the default for the first level only, the top level Devices, or the direct children of a `DeviceMock` passed to `connect()`. Below that a Device with no entry follows its parent, so `{Device: True}` mocks the whole tree, and `{Device: XMock}` gives `XMock` to the top level Devices and each child its own default mock class. `Signal` and `Command` types cannot be keys, passing one raises `TypeError`: they always follow their parent Device. + +Every connected Device holds the [](#DeviceMock) it was connected with, real ones included, and the mocks of mocked children are nested under the mock of their parent, real or not. So in the examples above [](#get_mock) on `stage` shows the calls made on the mocked motors, as `get_mock(stage).mtr.user_setpoint.put`. Real Signals record nothing, so a fully real tree has no calls to see. + +`Device.connect()` takes a `bool` or a `DeviceMock` instance instead. The instance is mocked unless made with `mocked=False`, and its children follow it unless its `mock_types` say otherwise, with a `Device` entry applying to its direct children only. Only `init_devices` and `ensure_connected` check a mapping, so pass one through them. ## Simulating a signal that stops updating @@ -96,7 +134,9 @@ timing — the [](#Motor) module also ships `VeloAndAcclRespectingMotorMock`, wh drives the readback towards the setpoint over time while respecting the motor's velocity and acceleration. -To use it, pass an instance directly to `connect()` instead of relying on the default: +To use it for every Device of a type, rather than one Device, pass `init_devices(mock={DeviceType: MockType})`, as explained in [](#override-mock-by-type). + +To use it for a single Device, pass an instance directly to `connect()` instead of relying on the default: ```{literalinclude} ../../tests/unit_tests/epics/test_motor.py :language: python @@ -146,7 +186,7 @@ Now let's call some verbs and check that they do the right thing. We want to che :pyobject: test_motor_stopped ``` -This time we use [](#get_mock_execute) to get an [](#unittest.mock.AsyncMock) that will be called every time `stop_.trigger()` is called. We check it hasn't been called, then call our method, then check it has been called with no arguments. We also show that we can call [](#get_mock) on the parent to see all of the mock calls that have been made on all its children, useful to check ordering. +This time we use [](#get_mock_execute) to get an [](#unittest.mock.AsyncMock) that will be called every time `stop_.trigger()` is called. We check it hasn't been called, then call our method, then check it has been called with no arguments. We also show that we can call [](#get_mock) on the parent to see all of the mock calls that have been made on all its children, useful to check ordering. [](#get_mock) works on any connected Device, real or mocked. For [](#Signal)s, the equivalent is [](#get_mock_put), which returns an `AsyncMock` that records every `Signal.set()` / `put()` call. diff --git a/src/ophyd_async/core/__init__.py b/src/ophyd_async/core/__init__.py index 1dde570c3d..b10e2ea56a 100644 --- a/src/ophyd_async/core/__init__.py +++ b/src/ophyd_async/core/__init__.py @@ -36,12 +36,8 @@ DeviceConnector, DeviceMap, DeviceMock, - DeviceProcessor, DeviceVector, - LazyMock, default_mock_class, - init_devices, - print_connection_status, ) from ._device_filler import DeviceAnnotation, DeviceFiller from ._enums import ( @@ -52,6 +48,12 @@ YesNo, ) from ._flyable import FlyableLogic, FlyMotorInfo, StandardFlyable +from ._init_devices import ( + DeviceProcessor, + connect_devices, + init_devices, + print_connection_status, +) from ._log import config_ophyd_async_logging, logger from ._mock_signal_backend import MockSignalBackend from ._mock_signal_utils import ( @@ -172,6 +174,7 @@ def __getattr__(name): renames = { "NotConnected": NotConnectedError, + "LazyMock": DeviceMock, } rename = renames.get(name) if rename is not None: @@ -194,6 +197,7 @@ def __getattr__(name): "DeviceMap", "DeviceVector", "DeviceProcessor", + "connect_devices", "init_devices", "print_connection_status", # Movable @@ -240,7 +244,6 @@ def __getattr__(name): "soft_signal_rw", # Mock signal "DeviceMock", - "LazyMock", "MockSignalBackend", "default_mock_class", # Mocking utilities diff --git a/src/ophyd_async/core/_command.py b/src/ophyd_async/core/_command.py index 297b7f9db2..af947e20cd 100644 --- a/src/ophyd_async/core/_command.py +++ b/src/ophyd_async/core/_command.py @@ -8,7 +8,7 @@ from typing import Generic, cast from unittest.mock import AsyncMock -from ._device import Device, DeviceConnector, LazyMock +from ._device import Device, DeviceConnector, DeviceMock from ._soft_signal_backend import SoftConverter, make_converter from ._status import AsyncStatus from ._utils import ( @@ -61,15 +61,20 @@ class CommandConnector(DeviceConnector, Generic[P, T]): def __init__(self, backend: CommandBackend[P, T]): self.backend = self._init_backend = backend - async def connect_mock(self, device: Device, mock: LazyMock): - """Connect the backend in mock mode.""" - self.backend = MockCommandBackend(self._init_backend, mock) - - async def connect_real(self, device: Device, timeout: float, force_reconnect: bool): - """Connect the backend to real hardware.""" - self.backend = self._init_backend - device.log.debug(f"Connecting to {self.backend.source(device.name)}") - await self.backend.connect(timeout) + async def connect( + self, + device: Device, + mock: DeviceMock, + timeout: float, + force_reconnect: bool, + ): + """Connect the backend in mock mode or to real hardware.""" + if mock.mocked: + self.backend = MockCommandBackend(self._init_backend, mock) + else: + self.backend = self._init_backend + device.log.debug(f"Connecting to {self.backend.source(device.name)}") + await self.backend.connect(timeout) class Command(Device, Generic[P, T]): @@ -265,7 +270,7 @@ async def execute(self, *args: P.args, **kwargs: P.kwargs) -> T: class MockCommandBackend(CommandBackend[P, T]): """A backend for a Command that uses a mock for testing.""" - def __init__(self, initial_backend: CommandBackend[P, T], mock: LazyMock): + def __init__(self, initial_backend: CommandBackend[P, T], mock: DeviceMock): self._initial_backend = initial_backend self._mock = mock self._mock_execute_callback: MockExecuteCallback[P, T] | None = None diff --git a/src/ophyd_async/core/_device.py b/src/ophyd_async/core/_device.py index ac844259ff..10a48811cf 100644 --- a/src/ophyd_async/core/_device.py +++ b/src/ophyd_async/core/_device.py @@ -1,8 +1,6 @@ from __future__ import annotations -import asyncio import os -import sys from collections.abc import Awaitable, Callable, Iterator, Mapping, MutableMapping from functools import cached_property from logging import LoggerAdapter, getLogger @@ -10,7 +8,6 @@ from unittest.mock import Mock from bluesky.protocols import HasName -from bluesky.run_engine import call_in_bluesky_event_loop, in_bluesky_event_loop from ._utils import ( DEFAULT_TIMEOUT, @@ -73,7 +70,7 @@ class DeviceMock(Generic[DeviceT]): ```python >>> parent = DeviceMock() - >>> child = DeviceMock("child", parent) + >>> child = DeviceMock(name="child", parent=parent) >>> child_mock = child() >>> child_mock() # doctest: +ELLIPSIS @@ -84,9 +81,30 @@ class DeviceMock(Generic[DeviceT]): ``` """ - def __init__(self, name: str = "", parent: DeviceMock | None = None) -> None: + def __init__( + self, + name: str = "", + parent: DeviceMock | None = None, + mocked: bool = True, + *, + mock_types: Mapping[type[Device], bool | type[DeviceMock]] | None = None, + ) -> None: + """Create a DeviceMock. + + :param name: The name of the Device within its parent. + :param parent: The DeviceMock of the parent Device. + :param mocked: + Whether the Device is mocked. A real Device still holds a DeviceMock, + so the mocks of its mocked children nest under it. + :param mock_types: + The mock choice for each Device type in the tree below this one, as + passed to [](#init_devices). Children not matched by any entry + follow `mocked`. + """ + self.mocked = mocked self.name = name self.parent = parent + self.mock_types = mock_types or {} self._mock: Mock | None = None def __call__(self) -> Mock: @@ -104,9 +122,54 @@ async def connect(self, device: DeviceT) -> None: """ -# Keep LazyMock as an alias for backwards compatibility -# Remove for ophyd-async 1.0 -LazyMock = DeviceMock +def _mocked_and_class( + device: Device, + mocked: bool, + mock_types: Mapping[type[Device], bool | type[DeviceMock]], +) -> tuple[bool, type[DeviceMock]]: + # Default mock class, and whether we are mocked is inherited from parent + mock_class = device._mock_class # noqa: SLF001 + # The nearest class in the MRO decides if we override + for cls in type(device).__mro__: + if cls in mock_types: + choice = mock_types[cls] + if isinstance(choice, bool): + # Presence tells us if we should be mocked + return choice, mock_class + else: + # Presence in dict says we should be mocked with this cls + return True, choice + # Fallthrough means inherit from parent + return mocked, mock_class + + +def _without_device_entry( + mock_types: Mapping[type[Device], bool | type[DeviceMock]], +) -> dict[type[Device], bool | type[DeviceMock]]: + # The entry under Device is only a default for the first level of lookup + return {cls: choice for cls, choice in mock_types.items() if cls is not Device} + + +def _make_mock( + device: Device, + name: str, + parent: DeviceMock, +) -> DeviceMock: + mocked, mock_class = _mocked_and_class(device, parent.mocked, parent.mock_types) + return mock_class( + name, parent, mocked, mock_types=_without_device_entry(parent.mock_types) + ) + + +async def _connect_serially(coros: Mapping[str, Awaitable[None]]) -> None: + exceptions: dict[str, Exception] = {} + for name, coro in coros.items(): + try: + await coro + except Exception as exc: + exceptions[name] = exc + if exceptions: + raise NotConnectedError.with_other_exceptions_logged(exceptions) class DeviceConnector: @@ -130,38 +193,41 @@ def create_children_from_annotations(self, device: Device): during `__init__`. """ - async def connect_mock(self, device: Device, mock: DeviceMock): - """Use during [](#Device.connect) with `mock=True`. - - This is called when there is no cached connect done in `mock=True` - mode. It connects the Device and all its children in mock mode. - """ - # Connect serially, no errors to gather up as in mock mode - exceptions: dict[str, Exception] = {} - for name, child_device in device.children(): - try: - child_mock_class = child_device._mock_class # noqa: SLF001 - await child_device.connect(mock=child_mock_class(name, mock)) - except Exception as exc: - exceptions[name] = exc - if exceptions: - raise NotConnectedError.with_other_exceptions_logged(exceptions) - - # Call the DeviceMock's connect method to inject custom logic - await mock.connect(device) - - async def connect_real(self, device: Device, timeout: float, force_reconnect: bool): - """Use during [](#Device.connect) with `mock=False`. - - This is called when there is no cached connect done in `mock=False` - mode. It connects the Device and all its children in real mode in parallel. + async def connect( + self, + device: Device, + mock: DeviceMock, + timeout: float, + force_reconnect: bool, + ): + """Use during [](#Device.connect) to connect the Device and its children. + + It is called on every connect that is not cached. It connects the children, + each with the DeviceMock made for it from `mock`, and with `timeout` and + `force_reconnect`: in parallel, or serially when `mock` leaves nothing real + below `device`. Overrides connect the Device itself as `mock.mocked` says, + then call this. + + :param device: The Device being connected. + :param mock: The DeviceMock for `device`. + :param timeout: Time to wait before failing with a TimeoutError. + :param force_reconnect: If True, connect the children even if their last + connect is cached. [](#Device.connect) always passes True. """ - # Connect in parallel, gathering up NotConnectedErrors coros = { - name: child_device.connect(timeout=timeout, force_reconnect=force_reconnect) - for name, child_device in device.children() + name: child.connect( + mock=_make_mock(child, name, mock), + timeout=timeout, + force_reconnect=force_reconnect, + ) + for name, child in device.children() } - await wait_for_connection(**coros) + nothing_real_below = mock.mocked and False not in mock.mock_types.values() + if nothing_real_below: + # A Task per child is slow, and there is nothing real to wait for + await _connect_serially(coros) + else: + await wait_for_connection(**coros) def _fail_if_overwriting_parent(self: Device, name: str, value: Any): @@ -197,11 +263,9 @@ class Device(HasName): parent: Device | None = None """The parent Device if it exists""" _name: str = "" - # None if connect hasn't started, a Task if it has - _connect_task: asyncio.Task | None = None # The mock class to be used if we connect in mock mode _mock_class: type[DeviceMock] = DeviceMock - # The mock if we have connected in mock mode + # The DeviceMock of the last successful connect, mocked or not _mock: DeviceMock | None = None # The separator to use when making child names _child_name_separator: str = "-" @@ -305,49 +369,58 @@ async def connect( ) -> None: """Connect the device and all child devices. - Successful connects will be cached so subsequent calls will return - immediately. Contains a timeout that gets propagated to child.connect - methods. + A connect is cached: if the last connect succeeded, a later call with the + same mock choice (the [](#DeviceMock) class, `mocked` and `mock_types`) + returns without touching the children. A connect that is not cached + connects everything below this Device again. Contains a timeout that gets + propagated to child.connect methods. :param mock: - If True then use [](#MockSignalBackend) for all Signals. If passed a - [](#DeviceMock) then pass this down for use within the Signals, - otherwise create one using the registered default mock for this device - type, or a plain [](#DeviceMock) if no default is registered. + `False` connects for real, `True` mocks this Device and everything + below it with their default [](#DeviceMock). A [](#DeviceMock) + instance connects this Device as its `mocked` says, and its + `mock_types` choose for the children, see [](#init_devices). An entry + under `Device` is the default for its direct children only. Every + connected Device holds its DeviceMock, and the mocks of mocked + children nest under their parent's, see [](#get_mock). + Calling this directly on a child with a different mock choice from its + last connect makes its mock a root, so re-connect from the parent + instead. :param timeout: Time to wait before failing with a TimeoutError. :param force_reconnect: - If True, force a reconnect even if the last connect succeeded. + If True, connect this Device and everything below it even if the + connect is cached. + + A connect that is not cached, in either mode, may drop subscriptions made + before it, so subscribe again afterwards. In mock mode it also resets mock + values and the Mock's call record. """ connector: DeviceConnector = error_if_none( getattr(self, "_connector", None), f"{self}: doesn't have attribute `_connector`," f" did you call `super().__init__` in your `__init__` method?", ) - if mock: - # Always connect in mock mode serially - if isinstance(mock, DeviceMock): - # Use the user supplied mock - self._mock = mock - elif not self._mock: - # Make a new mock of the registered type - self._mock = self._mock_class() - await connector.connect_mock(self, self._mock) + if mock is False or mock is True: + # Use the default mock class for this device and children + new_mock = self._mock_class(mocked=mock) else: - # Try to cache the connect in real mode - can_use_previous_connect = ( - self._mock is None - and self._connect_task - and not (self._connect_task.done() and self._connect_task.exception()) - ) - if force_reconnect or not can_use_previous_connect: - self._mock = None - coro = connector.connect_real(self, timeout, force_reconnect) - self._connect_task = asyncio.create_task(coro) - connect_task = error_if_none( - self._connect_task, "Connect task not created, this shouldn't happen" - ) - # Wait for it to complete - await connect_task + new_mock = mock + use_previous_connect = ( + not force_reconnect + and self._mock is not None + and type(self._mock) is type(new_mock) + and self._mock.mocked == new_mock.mocked + and self._mock.mock_types == new_mock.mock_types + ) + if not use_previous_connect: + # Clear last connect, we'll make a new one + self._mock = None + await connector.connect(self, new_mock, timeout, force_reconnect=True) + if new_mock.mocked: + # Install mock behaviour once the connector has finished + await new_mock.connect(self) + # Connect succeeded, store the mock that encodes what it did + self._mock = new_mock def __repr__(self) -> str: if self.name == "": @@ -364,7 +437,6 @@ def __str__(self) -> str: "_connector", "_timeout", "_mock", - "_connect_task", "_child_name_separator", "_attempts", } @@ -481,160 +553,6 @@ def __hash__(self): # to allow DeviceMap to be used as dict keys and in sets return hash(id(self)) -class DeviceProcessor: - """Sync/Async Context Manager that finds all the Devices declared within it. - - Used in `init_devices` - """ - - def __init__(self, process_devices: Callable[[dict[str, Device]], Awaitable[None]]): - self._process_devices = process_devices - self._locals_on_enter: dict[str, Any] = {} - self._locals_on_exit: dict[str, Any] = {} - - def _caller_locals(self) -> dict[str, Any]: - """Walk up until we find a stack frame that doesn't have us as self.""" - try: - raise ValueError - except ValueError: - _, _, tb = sys.exc_info() - tb = error_if_none(tb, "Can't get traceback, this shouldn't happen") - - caller_frame = tb.tb_frame - while caller_frame.f_locals.get("self", None) is self: - caller_frame = caller_frame.f_back - if not caller_frame: - msg = ( - "No previous frame to the one with self in it, " - "this shouldn't happen" - ) - raise RuntimeError( # noqa: B904 - msg - ) - return caller_frame.f_locals.copy() - - def __enter__(self) -> DeviceProcessor: - # Stash the names that were defined before we were called - self._locals_on_enter = self._caller_locals() - return self - - async def __aenter__(self) -> DeviceProcessor: - return self.__enter__() - - async def __aexit__(self, type, value, traceback): - self._locals_on_exit = self._caller_locals() - await self._on_exit() - - def __exit__(self, type_, value, traceback): - if in_bluesky_event_loop(): - raise RuntimeError( - "Cannot use DeviceConnector inside a plan, instead use " - "`yield from ophyd_async.plan_stubs.ensure_connected(device)`" - ) - self._locals_on_exit = self._caller_locals() - try: - fut = call_in_bluesky_event_loop(self._on_exit()) - except RuntimeError as exc: - raise NotConnectedError( - "Could not connect devices. Is the bluesky event loop running? See " - "https://blueskyproject.io/ophyd-async/main/" - "user/explanations/event-loop-choice.html for more info." - ) from exc - return fut - - async def _on_exit(self) -> None: - # Find all the devices - devices = { - name: obj - for name, obj in self._locals_on_exit.items() - if isinstance(obj, Device) and self._locals_on_enter.get(name) is not obj - } - # Call the provided process function on them - await self._process_devices(devices) - - -def init_devices( - set_name: bool = True, - child_name_separator: str = "-", - connect: bool = True, - mock: bool = False, - timeout: float = 10.0, - on_connect_hook: Callable[[str, Awaitable[None]], Awaitable[None]] | None = None, -): - """Auto initialize top level Device instances: to be used as a context manager. - - :param set_name: - If True, call `device.set_name(variable_name)` on all Devices created - within the context manager that have an empty `name`. - :param child_name_separator: Separator for child names if `set_name` is True. - :param connect: - If True, call `device.connect(mock, timeout)` in parallel on all Devices - created within the context manager. - :param mock: If True, connect Signals in mock mode. - :param timeout: How long to wait for connect before logging an exception. - :param on_connect_hook: - Async function called with `(variable_name, connect_coro)` for each - Device, which must await `connect_coro`. Any exception it lets escape - is raised from `init_devices` as a `NotConnectedError`, so it can - catch exceptions to suppress them. See [](#print_connection_status) - for an example. - :raises RuntimeError: If used inside a plan, use [](#ensure_connected) instead. - :raises NotConnectedError: If devices could not be connected. - - For example, to connect and name 2 motors in parallel: - ```python - [async] with init_devices(): - t1x = motor.Motor("BLxxI-MO-TABLE-01:X") - t1y = motor.Motor("pva://BLxxI-MO-TABLE-01:Y") - # Names and connects devices here - assert t1x.name == "t1x" - ``` - """ - - async def process_devices(devices: dict[str, Device]): - if set_name: - for name, device in devices.items(): - if not device.name: - device.set_name(name, child_name_separator=child_name_separator) - if connect: - coros: dict[str, Awaitable[None]] = { - name: device.connect(mock, timeout) for name, device in devices.items() - } - if on_connect_hook: - coros = { - name: on_connect_hook(name, coro) for name, coro in coros.items() - } - await wait_for_connection(**coros) - - return DeviceProcessor(process_devices) - - -async def print_connection_status(name: str, coro: Awaitable[None]): - """Print a coloured connection status line, for use as `on_connect_hook`. - - Exceptions are caught and reported rather than raised. - - :param name: The variable name of the Device. - :param coro: The Device's connect coroutine. - - For example: - ```python - [async] with init_devices(on_connect_hook=print_connection_status): - t1x = motor.Motor("BLxxI-MO-TABLE-01:X") - ``` - """ - try: - await coro - except NotConnectedError: - status = "\033[1;31m DISCONNECTED \033[0m" - except Exception: - status = "\033[1;31m FAILED \033[0m" - else: - status = "\033[1;32m OK \033[0m" - dots = "." * (40 - len(name)) - print(f" {name} {dots} [{status}]") - - def default_mock_class( mock_cls: type[DeviceMock], ) -> Callable[[type[DeviceT]], type[DeviceT]]: diff --git a/src/ophyd_async/core/_init_devices.py b/src/ophyd_async/core/_init_devices.py new file mode 100644 index 0000000000..2b58bf7760 --- /dev/null +++ b/src/ophyd_async/core/_init_devices.py @@ -0,0 +1,252 @@ +from __future__ import annotations + +import sys +from collections.abc import Awaitable, Callable, Mapping +from typing import Any + +from bluesky.run_engine import call_in_bluesky_event_loop, in_bluesky_event_loop + +from ._command import Command +from ._device import Device, DeviceMock, _mocked_and_class, _without_device_entry +from ._signal import Signal +from ._utils import ( + DEFAULT_TIMEOUT, + NotConnectedError, + error_if_none, + wait_for_connection, +) + + +def _check_mock_types(mock_types: Mapping[Any, Any]) -> None: + for typ, choice in mock_types.items(): + if not (isinstance(typ, type) and issubclass(typ, Device)): + raise TypeError(f"mock mapping key {typ!r} is not a Device subclass") + if issubclass(typ, Signal | Command): + raise TypeError( + f"mock mapping key {typ.__name__} is a Signal or Command type: " + "these always follow their parent Device, so key on the Device" + ) + if not ( + isinstance(choice, bool) + or (isinstance(choice, type) and issubclass(choice, DeviceMock)) + ): + raise TypeError( + f"mock mapping value {choice!r} for {typ.__name__} is not a bool " + "or a DeviceMock subclass" + ) + + +async def connect_devices( + devices: Mapping[str, Device], + mock: bool | Mapping[type[Device], bool | type[DeviceMock]] = False, + timeout: float = DEFAULT_TIMEOUT, + force_reconnect: bool = False, + on_connect_hook: Callable[[str, Awaitable[None]], Awaitable[None]] | None = None, +) -> None: + """Connect Devices in parallel, raising one error that lists any failures. + + :param devices: The Devices to connect, keyed by the name to report them as. + :param mock: + `False` connects for real, `True` mocks everything. A `Mapping` chooses + per Device type, see [](#init_devices). + :param timeout: Time to wait before failing with a TimeoutError. + :param force_reconnect: If True, connect each Device and everything below it + even if the connect is cached, see [](#Device.connect). + :param on_connect_hook: + Async function called with `(name, connect_coro)` for each Device, which + must await `connect_coro`. See [](#init_devices). + :raises TypeError: + If `mock` is not a bool or a `Mapping`, or a `Mapping` key is not a Device + subclass (or is a Signal or Command subclass) or a value is not a bool or + DeviceMock subclass. + :raises NotConnectedError: If devices could not be connected. + """ + if isinstance(mock, Mapping): + _check_mock_types(mock) + elif not isinstance(mock, bool): + raise TypeError(f"mock must be a bool or a Mapping, not {type(mock).__name__}") + coros: dict[str, Awaitable[None]] = {} + for name, device in devices.items(): + root_mock: bool | DeviceMock + if isinstance(mock, Mapping): + mocked, mock_class = _mocked_and_class(device, False, mock) + root_mock = mock_class( + mocked=mocked, mock_types=_without_device_entry(mock) + ) + else: + root_mock = mock + coros[name] = device.connect( + mock=root_mock, timeout=timeout, force_reconnect=force_reconnect + ) + if on_connect_hook: + coros = {name: on_connect_hook(name, coro) for name, coro in coros.items()} + await wait_for_connection(**coros) + + +class DeviceProcessor: + """Sync/Async Context Manager that finds all the Devices declared within it. + + Used in `init_devices` + """ + + def __init__(self, process_devices: Callable[[dict[str, Device]], Awaitable[None]]): + self._process_devices = process_devices + self._locals_on_enter: dict[str, Any] = {} + self._locals_on_exit: dict[str, Any] = {} + + def _caller_locals(self) -> dict[str, Any]: + """Walk up until we find a stack frame that doesn't have us as self.""" + try: + raise ValueError + except ValueError: + _, _, tb = sys.exc_info() + tb = error_if_none(tb, "Can't get traceback, this shouldn't happen") + + caller_frame = tb.tb_frame + while caller_frame.f_locals.get("self", None) is self: + caller_frame = caller_frame.f_back + if not caller_frame: + msg = ( + "No previous frame to the one with self in it, " + "this shouldn't happen" + ) + raise RuntimeError( # noqa: B904 + msg + ) + return caller_frame.f_locals.copy() + + def __enter__(self) -> DeviceProcessor: + # Stash the names that were defined before we were called + self._locals_on_enter = self._caller_locals() + return self + + async def __aenter__(self) -> DeviceProcessor: + return self.__enter__() + + async def __aexit__(self, type, value, traceback): + self._locals_on_exit = self._caller_locals() + await self._on_exit() + + def __exit__(self, type_, value, traceback): + if in_bluesky_event_loop(): + raise RuntimeError( + "Cannot use DeviceConnector inside a plan, instead use " + "`yield from ophyd_async.plan_stubs.ensure_connected(device)`" + ) + self._locals_on_exit = self._caller_locals() + try: + fut = call_in_bluesky_event_loop(self._on_exit()) + except RuntimeError as exc: + raise NotConnectedError( + "Could not connect devices. Is the bluesky event loop running? See " + "https://blueskyproject.io/ophyd-async/main/" + "user/explanations/event-loop-choice.html for more info." + ) from exc + return fut + + async def _on_exit(self) -> None: + # Find all the devices + devices = { + name: obj + for name, obj in self._locals_on_exit.items() + if isinstance(obj, Device) and self._locals_on_enter.get(name) is not obj + } + # Call the provided process function on them + await self._process_devices(devices) + + +def init_devices( + set_name: bool = True, + child_name_separator: str = "-", + connect: bool = True, + mock: bool | Mapping[type[Device], bool | type[DeviceMock]] = False, + timeout: float = 10.0, + on_connect_hook: Callable[[str, Awaitable[None]], Awaitable[None]] | None = None, +): + """Auto initialize top level Device instances: to be used as a context manager. + + :param set_name: + If True, call `device.set_name(variable_name)` on all Devices created + within the context manager that have an empty `name`. + :param child_name_separator: Separator for child names if `set_name` is True. + :param connect: + If True, call `device.connect(mock, timeout)` in parallel on all Devices + created within the context manager. + :param mock: + `False` connects everything for real, `True` mocks everything. A `Mapping` + of Device type to `bool` or [](#DeviceMock) subclass chooses per type: each + Device looks itself up by the nearest class in its MRO, and a DeviceMock + subclass means mocked with that class. A Device with no entry follows its + parent, so "mock this Device" means its whole subtree. An entry under + `Device` is the default for the first level only, the top level Devices, + real if there is none. `Signal` and `Command` types cannot be keys, as they + always follow their parent Device. + :param timeout: How long to wait for connect before logging an exception. + :param on_connect_hook: + Async function called with `(variable_name, connect_coro)` for each + Device, which must await `connect_coro`. Any exception it lets escape + is raised from `init_devices` as a `NotConnectedError`, so it can + catch exceptions to suppress them. See [](#print_connection_status) + for an example. + :raises TypeError: + If `mock` is not a bool or a `Mapping`, or a `Mapping` key is not a Device + subclass (or is a Signal or Command subclass) or a value is not a bool or + DeviceMock subclass. + :raises RuntimeError: If used inside a plan, use [](#ensure_connected) instead. + :raises NotConnectedError: If devices could not be connected. + + For example, to connect and name 2 motors in parallel: + ```python + [async] with init_devices(): + t1x = motor.Motor("BLxxI-MO-TABLE-01:X") + t1y = motor.Motor("pva://BLxxI-MO-TABLE-01:Y") + # Names and connects devices here + assert t1x.name == "t1x" + ``` + + To mock only some types of Device, or the opposite: + ```python + [async] with init_devices(mock={Motor: True}): + ... # Motors, and everything below them, are mocked, the rest is real + [async] with init_devices(mock={Device: True, Motor: False}): + ... # Everything is mocked, except Motors which are real all the way down + ``` + """ + + async def process_devices(devices: dict[str, Device]): + if set_name: + for name, device in devices.items(): + if not device.name: + device.set_name(name, child_name_separator=child_name_separator) + if connect: + await connect_devices( + devices, mock, timeout, on_connect_hook=on_connect_hook + ) + + return DeviceProcessor(process_devices) + + +async def print_connection_status(name: str, coro: Awaitable[None]): + """Print a coloured connection status line, for use as `on_connect_hook`. + + Exceptions are caught and reported rather than raised. + + :param name: The variable name of the Device. + :param coro: The Device's connect coroutine. + + For example: + ```python + [async] with init_devices(on_connect_hook=print_connection_status): + t1x = motor.Motor("BLxxI-MO-TABLE-01:X") + ``` + """ + try: + await coro + except NotConnectedError: + status = "\033[1;31m DISCONNECTED \033[0m" + except Exception: + status = "\033[1;31m FAILED \033[0m" + else: + status = "\033[1;32m OK \033[0m" + dots = "." * (40 - len(name)) + print(f" {name} {dots} [{status}]") diff --git a/src/ophyd_async/core/_mock_signal_backend.py b/src/ophyd_async/core/_mock_signal_backend.py index dd5d276995..610ff28463 100644 --- a/src/ophyd_async/core/_mock_signal_backend.py +++ b/src/ophyd_async/core/_mock_signal_backend.py @@ -15,7 +15,7 @@ from ._utils import Callback if TYPE_CHECKING: - from ._device import LazyMock + from ._device import DeviceMock MockPutCallback = ( Callable[[SignalDatatypeT], SignalDatatypeT | None] @@ -30,7 +30,7 @@ class MockSignalBackend(SignalBackend[SignalDatatypeT]): def __init__( self, initial_backend: SignalBackend[SignalDatatypeT], - mock: LazyMock, + mock: DeviceMock, ) -> None: if isinstance(initial_backend, MockSignalBackend): raise ValueError("Cannot make a MockSignalBackend for a MockSignalBackend") diff --git a/src/ophyd_async/core/_mock_signal_utils.py b/src/ophyd_async/core/_mock_signal_utils.py index e09c3f8cba..627bb8197c 100644 --- a/src/ophyd_async/core/_mock_signal_utils.py +++ b/src/ophyd_async/core/_mock_signal_utils.py @@ -6,10 +6,11 @@ from unittest.mock import AsyncMock, Mock from ._command import Command, CommandConnector, MockCommandBackend, MockExecuteCallback -from ._device import Device, DeviceMock +from ._device import Device from ._mock_signal_backend import MockCallbackFilter, MockPutCallback, MockSignalBackend from ._signal import Signal, SignalConnector, SignalR from ._signal_backend import SignalDatatypeT +from ._utils import error_if_none MockT = TypeVar("MockT") @@ -43,13 +44,16 @@ def set_mock_attr(device: Device, attr: str, mock: MockT) -> MockT: def get_mock(device: Device | Signal) -> Mock: """Return the mock (which may have child mocks attached) for a Device. - The device must have been connected in mock mode. + This works for any connected Device, real or mocked. The mocks of its mocked + children are nested under it, so a call on a child is in `mock_calls` here. + + :raises RuntimeError: If the last connect of the Device has not succeeded. """ - mock = device._mock # noqa: SLF001 - if not isinstance(mock, DeviceMock): - msg = f"Device {device} not connected in mock mode" - raise RuntimeError(msg) - return mock() + device_mock = error_if_none( + device._mock, # noqa: SLF001 + f"Device {device} not connected", + ) + return device_mock() def _get_mock_signal_backend(signal: Signal) -> MockSignalBackend: diff --git a/src/ophyd_async/core/_signal.py b/src/ophyd_async/core/_signal.py index 99ab1ce406..09e2b8138e 100644 --- a/src/ophyd_async/core/_signal.py +++ b/src/ophyd_async/core/_signal.py @@ -21,7 +21,7 @@ from event_model import DataKey from stamina import retry_context -from ._device import Device, DeviceConnector, LazyMock +from ._device import Device, DeviceConnector, DeviceMock from ._mock_signal_backend import MockSignalBackend from ._protocol import AsyncReadable, AsyncStageable from ._signal_backend import SignalBackend, SignalDatatypeT, SignalDatatypeV @@ -51,13 +51,20 @@ class SignalConnector(DeviceConnector): def __init__(self, backend: SignalBackend): self.backend = self._init_backend = backend - async def connect_mock(self, device: Device, mock: LazyMock): - self.backend = MockSignalBackend(self._init_backend, mock) - - async def connect_real(self, device: Device, timeout: float, force_reconnect: bool): - self.backend = self._init_backend - device.log.debug(f"Connecting to {self.backend.source(device.name, read=True)}") - await self.backend.connect(timeout) + async def connect( + self, + device: Device, + mock: DeviceMock, + timeout: float, + force_reconnect: bool, + ): + if mock.mocked: + self.backend = MockSignalBackend(self._init_backend, mock) + else: + self.backend = self._init_backend + source = self.backend.source(device.name, read=True) + device.log.debug(f"Connecting to {source}") + await self.backend.connect(timeout) class _ChildrenNotAllowed(dict[str, Device]): @@ -193,22 +200,33 @@ def _backend_or_cache( self, cached: bool | None = None ) -> _SignalCache | SignalBackend: # If cached is None then calculate it based on whether we already have a cache + cache = self._live_cache() if cached is None: - cached = self._cache is not None + cached = cache is not None if cached: - cache = error_if_none(self._cache, f"{self.source} not being monitored") + cache = error_if_none(cache, f"{self.source} not being monitored") return cache else: return self._connector.backend - def _get_cache(self) -> _SignalCache: - if not self._cache: - self._cache = _SignalCache(self._connector.backend, self) + def _live_cache(self) -> _SignalCache | None: + # A connect that swapped the backend leaves the cache subscribed to the + # old one, which never updates again, so drop it and its listeners + if self._cache and self._cache.backend is not self._connector.backend: + self._cache.close() + self._cache = None return self._cache + def _get_cache(self) -> _SignalCache: + cache = self._live_cache() + if cache is None: + cache = self._cache = _SignalCache(self._connector.backend, self) + return cache + def _del_cache(self, needed: bool): - if self._cache and not needed: - self._cache.close() + cache = self._live_cache() + if cache and not needed: + cache.close() self._cache = None @_add_timeout diff --git a/src/ophyd_async/epics/core/_pvi_connector.py b/src/ophyd_async/epics/core/_pvi_connector.py index 07ec021c32..ff0224107d 100644 --- a/src/ophyd_async/epics/core/_pvi_connector.py +++ b/src/ophyd_async/epics/core/_pvi_connector.py @@ -14,8 +14,8 @@ Device, DeviceFiller, DeviceMap, + DeviceMock, DeviceVector, - LazyMock, Signal, SignalR, SignalRW, @@ -87,19 +87,27 @@ def command_backend_factory( fill_children_with_prefix(self.prefix, self.filler, filled=False) self.filler.check_created() - async def connect_mock(self, device: Device, mock: LazyMock) -> None: - # A DeviceVector gets fabricated mock children; a DeviceMap does not (its - # entries come from the served PVI tree, so in mock mode it stays empty). - if isinstance(device, DeviceVector): - self.filler.create_device_dict_entries_to_mock( - self.mock_device_vector_children - ) + async def connect( + self, + device: Device, + mock: DeviceMock, + timeout: float, + force_reconnect: bool, + ) -> None: + if mock.mocked: + # A DeviceVector gets fabricated mock children; a DeviceMap does not + # (its entries come from the served PVI tree, so in mock mode it stays + # empty). + if isinstance(device, DeviceVector): + self.filler.create_device_dict_entries_to_mock( + self.mock_device_vector_children + ) + else: + await self._fill_from_pvi_tree(device, timeout) device.set_name(device.name) - return await super().connect_mock(device, mock) + await super().connect(device, mock, timeout, force_reconnect) - async def connect_real( - self, device: Device, timeout: float, force_reconnect: bool - ) -> None: + async def _fill_from_pvi_tree(self, device: Device, timeout: float) -> None: if self.pvi_tree is None: # Top-level device: discover the served PVI tree. self.pvi_tree = await PviTree.build_device_tree( @@ -123,8 +131,6 @@ async def connect_real( suffix = f"\n{self.error_hint}" if self.error_hint else "" self.filler.check_filled(f"{self.pvi_pv}: {tree}{suffix}") - device.set_name(device.name) - return await super().connect_real(device, timeout, force_reconnect) def _fill_named_entries(self, tree: PviTree, *, as_map: bool) -> None: """Fill a node's normal (non-"__N") entries. diff --git a/src/ophyd_async/plan_stubs/_ensure_connected.py b/src/ophyd_async/plan_stubs/_ensure_connected.py index 08ac41473b..12d79c95f0 100644 --- a/src/ophyd_async/plan_stubs/_ensure_connected.py +++ b/src/ophyd_async/plan_stubs/_ensure_connected.py @@ -1,6 +1,13 @@ +from collections.abc import Mapping + from bluesky.utils import plan -from ophyd_async.core import DEFAULT_TIMEOUT, Device, LazyMock, wait_for_connection +from ophyd_async.core import ( + DEFAULT_TIMEOUT, + Device, + DeviceMock, + connect_devices, +) from ._wait_for_awaitable import wait_for_awaitable @@ -8,21 +15,30 @@ @plan def ensure_connected( *devices: Device, - mock: bool | LazyMock = False, + mock: bool | Mapping[type[Device], bool | type[DeviceMock]] = False, timeout: float = DEFAULT_TIMEOUT, force_reconnect=False, ): - """Plan stub to ensure devices are connected with a given timeout.""" + """Plan stub to ensure devices are connected with a given timeout. + + :param devices: The Devices to connect, which must have unique names. + :param mock: As for [](#init_devices): a `bool` or a `Mapping` per Device type. + :param timeout: Time to wait before failing with a TimeoutError. + :param force_reconnect: If True, force a reconnect even if already connected. + :raises TypeError: If `mock` is not valid, as for [](#init_devices). + """ device_names = [device.name for device in devices] non_unique = { device: device.name for device in devices if device_names.count(device.name) > 1 } if non_unique: raise ValueError(f"Devices do not have unique names {non_unique}") - coros = { - device.name: device.connect( - mock=mock, timeout=timeout, force_reconnect=force_reconnect + + yield from wait_for_awaitable( + connect_devices( + {device.name: device for device in devices}, + mock=mock, + timeout=timeout, + force_reconnect=force_reconnect, ) - for device in devices - } - yield from wait_for_awaitable(wait_for_connection(**coros)) + ) diff --git a/src/ophyd_async/tango/core/_base_device.py b/src/ophyd_async/tango/core/_base_device.py index f31075120c..daf4bc9829 100644 --- a/src/ophyd_async/tango/core/_base_device.py +++ b/src/ophyd_async/tango/core/_base_device.py @@ -12,8 +12,8 @@ Device, DeviceConnector, DeviceFiller, + DeviceMock, DeviceVector, - LazyMock, NotConnectedError, Signal, ) @@ -132,15 +132,24 @@ def create_children_from_annotations(self, device: Device): self.filler.check_created() - async def connect_mock(self, device: Device, mock: LazyMock): - if isinstance(device, DeviceVector): - # Make 2 entries for this DeviceVector - self.filler.create_device_dict_entries_to_mock([1, 2]) + async def connect( + self, + device: Device, + mock: DeviceMock, + timeout: float, + force_reconnect: bool, + ): + if mock.mocked: + if isinstance(device, DeviceVector): + # Make 2 entries for this DeviceVector + self.filler.create_device_dict_entries_to_mock([1, 2]) + else: + await self._fill_from_proxy(device) # Set the name of the device to name all children device.set_name(device.name) - return await super().connect_mock(device, mock) + await super().connect(device, mock, timeout, force_reconnect) - async def connect_real(self, device: Device, timeout: float, force_reconnect: bool): + async def _fill_from_proxy(self, device: Device) -> None: if not self.trl: raise RuntimeError(f"Could not created Device Proxy for TRL {self.trl}") try: @@ -192,7 +201,3 @@ async def connect_real(self, device: Device, timeout: float, force_reconnect: bo # Check that all the requested children have been filled self.filler.check_filled(f"{self.trl}: {children}") - - # Set the name of the device to name all children - device.set_name(device.name) - return await super().connect_real(device, timeout, force_reconnect) diff --git a/tests/system_tests/tango/core/test_tango_signal_lifecycle.py b/tests/system_tests/tango/core/test_tango_signal_lifecycle.py index f2b6d867a3..f4911069ec 100644 --- a/tests/system_tests/tango/core/test_tango_signal_lifecycle.py +++ b/tests/system_tests/tango/core/test_tango_signal_lifecycle.py @@ -45,7 +45,7 @@ (44 seconds of pure sleep alone), and a from-scratch `TangoDevice(...). connect()` per case, which for the exhaustive tier's `auto_fill_signals=True` means rediscovering all 37 attributes/commands on -every one of the 37 cases (see `TangoDeviceConnector.connect_real`). +every one of the 37 cases (see `TangoDeviceConnector.connect`). `reset_everything_device` (below) is fixed by a plain per-process guard flag: every case's own `assert_signal_lifecycle` already restores its @@ -60,14 +60,11 @@ `ioc_devices` fixture + `assert_monitor_then_put`'s `await signal.connect (timeout=1)`): construct the (unconnected) device *once*, in a plain *synchronous* `scope="module"` fixture, then `await device.connect()` -inside each test as before. `Device.connect()` caches its result in an -`asyncio.Task` (`self._connect_task`, `src/ophyd_async/core/_device.py`) - -once that task is done, every later `.connect()` call just re-awaits the -same completed task, which is safe from a different test's event loop: -awaiting an already-done `asyncio.Task`/`Future` never touches the loop it -was created on (`Task.__await__` returns `self.result()` immediately -without yielding). So only the *first* parametrized case actually pays -the connect/discovery cost; the rest are cache hits. This avoids +inside each test as before. `Device.connect()` caches a successful +connect, so every later `.connect()` with the same mock choice returns +without connecting again, which is safe from a different test's event +loop as it awaits nothing. So only the *first* parametrized case actually +pays the connect/discovery cost; the rest are cache hits. This avoids pytest-asyncio's `loop_scope="module"` entirely (an earlier attempt using it broke CI - see this PR's commit history for why), so it doesn't need any change to the shared, function-scoped `event_loop` fixture in this @@ -163,11 +160,10 @@ def lifecycle_device(everything_device_trl: str) -> TangoTestDevice: Deliberately a plain *synchronous* fixture that only constructs the Device - it does not `connect()` it. Each test still calls `await device.connect()` itself (same as `exhaustive_device` below), which is - what makes sharing this safe: `Device.connect()` caches its work in an - `asyncio.Task` (`src/ophyd_async/core/_device.py`), so only the first - test's call actually connects - every later call, even from a - different test's event loop, just re-awaits that already-done task, - which never touches the loop it was created on. This is the same + what makes sharing this safe: `Device.connect()` caches a successful + connect, so only the first test's call actually connects - every later + call, even from a different test's event loop, returns without + awaiting anything. This is the same pattern EPICS's own system tests already use for this exact scenario (`tests/system_tests/epics/signal/test_signals.py`'s `ioc_devices` fixture, constructed once and `.connect()`-ed per test/signal). @@ -184,7 +180,7 @@ def exhaustive_device(everything_device_trl: str) -> TangoDevice: See `lifecycle_device` above for why this is safe despite being shared across tests/event loops. `auto_fill_signals=True` discovers and connects *every* attribute/command on the device on first `connect()` - (see `TangoDeviceConnector.connect_real`) - without this fixture, each + (see `TangoDeviceConnector.connect`) - without this fixture, each of the 37 parametrized cases reconnecting its own fresh `TangoDevice` paid that whole-device discovery cost again, ~37x more network round trips than the single connect this module actually needs. @@ -304,7 +300,7 @@ async def test_signal_mock_parity( """A mock-connected device agrees on shape/dtype with a real-connected one, and never touches the network for get/set.""" real = TangoTestDevice(everything_device_trl, name="real") - # Never dialled: mock mode skips connect_real entirely. + # Never dialled: a mocked connect never makes the proxy. mock = TangoTestDevice("does/not/matter#dbase=no", name="mock") await real.connect() await mock.connect(mock=True) @@ -339,7 +335,7 @@ class _DeviceVectorTestDevice(TangoDevice, StandardReadable): Neither `TangoTestDevice` nor `OneOfEverythingTangoDevice` (the real server it pairs with) declares a `DeviceVector` field, so mock-connecting the real curated device never exercises - `TangoDeviceConnector.connect_mock`'s `isinstance(device, DeviceVector)` + `TangoDeviceConnector.connect`'s `isinstance(device, DeviceVector)` branch (added on this same PR to fix a bug where it unconditionally called `create_device_vector_entries_to_mock` for *every* device, vector or not). This one-off Device exists purely to mock-connect and exercise diff --git a/tests/unit_tests/core/test_device.py b/tests/unit_tests/core/test_device.py index fad45e194f..67190b14af 100644 --- a/tests/unit_tests/core/test_device.py +++ b/tests/unit_tests/core/test_device.py @@ -2,22 +2,31 @@ import os import time import traceback -from unittest.mock import AsyncMock, MagicMock, Mock +from unittest.mock import AsyncMock, MagicMock, Mock, call import pytest from ophyd_async.core import ( DEFAULT_TIMEOUT, + Command, Device, + DeviceConnector, DeviceFiller, DeviceMap, + DeviceMock, DeviceProcessor, DeviceVector, NotConnectedError, Reference, + Signal, SignalRW, + connect_devices, + default_mock_class, + get_mock, + get_mock_put, init_devices, set_mock_attr, + soft_command, soft_signal_rw, wait_for_connection, ) @@ -240,6 +249,463 @@ async def test_device_with_init_devices(): assert parent.dict_with_children[123].connected +class RecordingMock(DeviceMock): + async def connect(self, device: Device) -> None: + device.connected_with = self # type: ignore + + +class RootMock(RecordingMock): + pass + + +class MiddleMock(RecordingMock): + pass + + +class LeafMock(RecordingMock): + pass + + +class VectorMock(RecordingMock): + pass + + +class XMock(RecordingMock): + pass + + +class YMock(RecordingMock): + pass + + +@default_mock_class(VectorMock) +class Leaves(DeviceVector): + pass + + +class Sensor(Device): + pass + + +@default_mock_class(LeafMock) +class Leaf(Sensor): + def __init__(self) -> None: + self.sig = soft_signal_rw(float) + super().__init__() + + +@default_mock_class(MiddleMock) +class Middle(Device): + def __init__(self) -> None: + self.leaf = Leaf() + super().__init__() + + +@default_mock_class(RootMock) +class Root(Device): + def __init__(self, name: str = "root") -> None: + self.middle = Middle() + self.leaves = Leaves({1: Leaf()}) + super().__init__(name) + + +def is_mocked(device: Device) -> bool: + if isinstance(device, Signal): + try: + get_mock_put(device) + except RuntimeError: + return False + return True + # The DeviceMock of the last mock connect is the one get_mock reads from + recorder = getattr(device, "connected_with", None) + return recorder is not None and recorder() is get_mock(device) + + +def connected_with(device: Device) -> dict[str, type[DeviceMock] | None]: + """Map each non-Signal Device name to its DeviceMock class, None if real.""" + mocked = is_mocked(device) + result = {device.name: type(device.connected_with) if mocked else None} # type: ignore + for _, child in device.children(): + if isinstance(child, Signal): + # Signals always follow their parent + assert is_mocked(child) == mocked + else: + result.update(connected_with(child)) + return result + + +def tree(root, middle, middle_leaf, leaves, leaves_1) -> dict[str, type | None]: + return { + "root": root, + "root-middle": middle, + "root-middle-leaf": middle_leaf, + "root-leaves": leaves, + "root-leaves-1": leaves_1, + } + + +REAL_TREE = tree(None, None, None, None, None) +DEFAULT_TREE = tree(RootMock, MiddleMock, LeafMock, VectorMock, LeafMock) + +MOCK_CHOICES = [ + (False, REAL_TREE), + ({}, REAL_TREE), + (True, DEFAULT_TREE), + ({Device: True}, DEFAULT_TREE), + # Mock by type, with everything below it + ({Middle: True}, tree(None, MiddleMock, LeafMock, None, None)), + ({Leaf: True}, tree(None, None, LeafMock, None, LeafMock)), + # Mock everything except by type, which is real all the way down + ( + {Device: True, Middle: False}, + tree(RootMock, None, None, VectorMock, LeafMock), + ), + # The entry under Device is dropped below the first level, so Top real is all real + ({Device: True, Root: False}, REAL_TREE), + # A mocked Device under a real parent + ( + {Device: True, Middle: False, Leaf: True}, + tree(RootMock, None, LeafMock, VectorMock, LeafMock), + ), + # Mock by type with a DeviceMock class + ({Leaf: XMock}, tree(None, None, XMock, None, XMock)), + ({Root: XMock}, tree(XMock, MiddleMock, LeafMock, VectorMock, LeafMock)), + ({Middle: XMock, Root: YMock}, tree(YMock, XMock, LeafMock, VectorMock, LeafMock)), + ({Device: XMock}, tree(XMock, MiddleMock, LeafMock, VectorMock, LeafMock)), + ({Device: XMock, Leaf: YMock}, tree(XMock, MiddleMock, YMock, VectorMock, YMock)), + # A class under Device is for the first level only, True gets the default class + ( + {Device: XMock, Middle: True}, + tree(XMock, MiddleMock, LeafMock, VectorMock, LeafMock), + ), + # The nearest class in the MRO wins + ({Sensor: XMock}, tree(None, None, XMock, None, XMock)), + ({Sensor: XMock, Leaf: YMock}, tree(None, None, YMock, None, YMock)), + ({Sensor: True, Leaf: False}, REAL_TREE), +] +MOCK_CHOICE_IDS = [ + "False", + "empty", + "True", + "Device True", + "Middle True", + "Leaf True", + "Device True, Middle False", + "Device True, Root False", + "mocked under real", + "Leaf class", + "Root class", + "Middle class, Root class", + "Device class", + "Device class, Leaf class", + "Device class, Middle True", + "base class", + "nearest class", + "nearest False", +] + + +@pytest.mark.parametrize("mock, expected", MOCK_CHOICES, ids=MOCK_CHOICE_IDS) +async def test_init_devices_and_ensure_connected_choose_per_type(mock, expected, RE): + async with init_devices(mock=mock): + root = Root() + assert connected_with(root) == expected + # Mocked Signals are mocked all the way down, real ones are real + if expected["root-middle-leaf"]: + await root.middle.leaf.sig.set(1.0) + assert is_mocked(root.middle.leaf.sig) + # Mocks nest under their parent's, real Devices included + get_mock(root).middle.leaf.sig.put.assert_called_once_with(1.0) + else: + assert not is_mocked(root.middle.leaf.sig) + + other = Root("other") + RE(ensure_connected(other, mock=mock)) + assert connected_with(other) == { + k.replace("root", "other"): v for k, v in expected.items() + } + + +async def test_connect_mock_instance_is_used_and_children_default_to_mocked(): + root = Root() + instance = XMock() + + await root.connect(mock=instance) + + assert get_mock(root) is instance() + assert connected_with(root) == tree( + XMock, MiddleMock, LeafMock, VectorMock, LeafMock + ) + await root.middle.leaf.sig.set(1.0) + instance().middle.leaf.sig.put.assert_called_once_with(1.0) + + +@pytest.mark.parametrize( + "mock, expected", + [ + ( + XMock(mock_types={Middle: False, Leaf: YMock}), + tree(XMock, None, YMock, VectorMock, YMock), + ), + # A Device entry is the default for the direct children only + ( + DeviceMock(mocked=False, mock_types={Device: True, Middle: False}), + tree(None, None, None, VectorMock, LeafMock), + ), + ], + ids=["children by type", "Device entry for direct children"], +) +async def test_connect_mock_instance_mock_types_choose_for_children(mock, expected): + root = Root() + + await root.connect(mock=mock) + + assert connected_with(root) == expected + + +@pytest.mark.parametrize( + "parent_mocked", [False, True], ids=["real parent", "mocked parent"] +) +async def test_real_children_get_the_callers_args_and_connect_in_parallel( + parent_mocked, +): + class Slow(Device): + started = 0 + args_seen = [] + + async def connect( + self, mock=False, timeout=DEFAULT_TIMEOUT, force_reconnect=False + ): + Slow.args_seen.append((timeout, force_reconnect)) + Slow.started += 1 + while Slow.started < 2: + await asyncio.sleep(0) + await super().connect(mock, timeout, force_reconnect) + + class Holder(Device): + def __init__(self): + self.a, self.b, self.c = Slow(), Slow(), Leaf() + super().__init__("holder") + + holder = Holder() + + # Deadlocks if the real children a and b were connected serially + mock = DeviceMock(mock_types={Slow: False, Leaf: True}, mocked=parent_mocked) + await asyncio.wait_for( + holder.connect(mock=mock, timeout=2.5, force_reconnect=True), 1 + ) + + assert Slow.args_seen == [(2.5, True), (2.5, True)] + assert is_mocked(holder.c) and not is_mocked(holder.a) + + +class CountingConnector(DeviceConnector): + real_connects = 0 + + async def connect(self, device, mock, timeout, force_reconnect): + if not mock.mocked: + CountingConnector.real_connects += 1 + await super().connect(device, mock, timeout, force_reconnect) + + +async def test_real_connect_is_cached_until_the_mock_choice_changes(): + CountingConnector.real_connects = 0 + root = Device(connector=CountingConnector()) + root.leaf = Leaf() + + await root.connect() + await root.connect() + assert CountingConnector.real_connects == 1 + assert not is_mocked(root.leaf) + + # A different choice for the children reconnects, the same one does not + await root.connect(mock=DeviceMock(mocked=False, mock_types={Leaf: True})) + await root.connect(mock=DeviceMock(mocked=False, mock_types={Leaf: True})) + assert CountingConnector.real_connects == 2 + assert is_mocked(root.leaf) + + await root.connect() + assert CountingConnector.real_connects == 3 + assert not is_mocked(root.leaf) + + +class RealOne(Device): + def __init__(self) -> None: + self.leaf = Leaf() + super().__init__(connector=CountingConnector()) + + +class Top(Device): + def __init__(self) -> None: + self.a = RealOne() + super().__init__("top") + + +@pytest.mark.parametrize("force_reconnect", [False, True]) +async def test_mocks_under_a_real_child_stay_nested_when_connected_again( + force_reconnect, +): + top = Top() + mock = {Device: True, RealOne: False, Leaf: True} + await connect_devices({"top": top}, mock=mock) + + await connect_devices({"top": top}, mock=mock, force_reconnect=force_reconnect) + + await top.a.leaf.sig.set(2.0) + get_mock(top).a.leaf.sig.put.assert_called_once_with(2.0) + + +async def test_connect_that_is_not_cached_connects_everything_below_again(): + CountingConnector.real_connects = 0 + top = Top() + real_top = DeviceMock(mocked=False, mock_types={RealOne: False, Leaf: True}) + mocked_top = DeviceMock(mocked=True, mock_types={RealOne: False, Leaf: True}) + await top.connect(mock=real_top) + await top.connect(mock=real_top) + assert CountingConnector.real_connects == 1 + + # The real child's own mock choice is unchanged, but its parent connects again + await top.connect(mock=mocked_top) + + assert CountingConnector.real_connects == 2 + await top.a.leaf.sig.set(3.0) + get_mock(top).a.leaf.sig.put.assert_called_once_with(3.0) + + +@pytest.mark.parametrize( + "bad_mock, match", + [ + ("yes", "mock must be a bool"), + (1, "mock must be a bool"), + (None, "mock must be a bool"), + (DeviceMock, "mock must be a bool"), + ({Leaf: "X"}, "value 'X' for Leaf is not a bool or a DeviceMock subclass"), + ({Leaf: 1}, "is not a bool or a DeviceMock subclass"), + ({Leaf: Leaf}, "is not a bool or a DeviceMock subclass"), + ({Leaf: XMock()}, "is not a bool or a DeviceMock subclass"), + ({"Leaf": XMock}, "key 'Leaf' is not a Device subclass"), + ({XMock: XMock}, "is not a Device subclass"), + ({Signal: True}, "key Signal is a Signal or Command type"), + ({SignalRW: True}, "key SignalRW is a Signal or Command type"), + ({Command: XMock}, "key Command is a Signal or Command type"), + ({Device: True, Command: False}, "is a Signal or Command type"), + ], +) +async def test_bad_mock_raises_type_error(bad_mock, match, RE): + with pytest.raises(TypeError, match=match): + await connect_devices({"root": Root("root")}, mock=bad_mock) + with pytest.raises(TypeError, match=match): + async with init_devices(mock=bad_mock): + Root("root") + with pytest.raises(TypeError, match=match): + RE(ensure_connected(Root("root"), mock=bad_mock)) + + +async def test_init_devices_and_ensure_connected_reject_device_mock_instance(RE): + # Only Device.connect takes a DeviceMock instance + with pytest.raises(TypeError, match="mock must be a bool or a Mapping"): + async with init_devices(mock=XMock()): # type: ignore + Root("root") + with pytest.raises(TypeError, match="mock must be a bool or a Mapping"): + RE(ensure_connected(Root("root"), mock=XMock())) # type: ignore + + +async def test_signal_follows_its_parent_whatever_the_mapping(): + leaf = Leaf() + + await connect_devices({"leaf": leaf}, mock={Leaf: True}) + assert is_mocked(leaf.sig) + await connect_devices({"leaf": leaf}, mock={Device: True, Leaf: False}) + assert not is_mocked(leaf.sig) + # Connected on its own, a Signal follows the entry under Device + await connect_devices({"sig": leaf.sig}, mock={Device: True}) + assert is_mocked(leaf.sig) + + +async def test_connect_mock_again_is_cached_until_forced(): + root = Root() + await root.connect(mock=True) + root_mock, leaf_mock = get_mock(root), get_mock(root.middle.leaf) + await root.middle.leaf.sig.set(1.0) + + await root.connect(mock=True) + + assert get_mock(root) is root_mock + assert get_mock(root.middle.leaf) is leaf_mock + root_mock.middle.leaf.sig.put.assert_called_once_with(1.0) + + await root.connect(mock=True, force_reconnect=True) + + assert get_mock(root) is not root_mock + assert get_mock(root.middle.leaf) is not leaf_mock + await root.middle.leaf.sig.set(2.0) + get_mock(root).middle.leaf.sig.put.assert_called_once_with(2.0) + + +async def test_connect_mock_child_then_parent_attaches_child_under_parent(): + root = Root() + await root.middle.connect(mock=True) + alone = get_mock(root.middle) + + await root.connect(mock=True) + + assert get_mock(root.middle) is not alone + await root.middle.leaf.sig.set(1.0) + get_mock(root).middle.leaf.sig.put.assert_called_once_with(1.0) + + +async def test_connect_mock_again_keeps_soft_signal_value(): + root = Root() + await root.connect(mock=True) + await root.middle.leaf.sig.set(1.5) + + await root.connect(mock=True) + + assert await root.middle.leaf.sig.get_value() == 1.5 + + +async def test_connect_mock_same_instance_again_is_cached(): + root = Root() + instance = XMock() + await root.connect(mock=instance) + leaf_mock = get_mock(root.middle.leaf) + + await root.connect(mock=instance) + + assert get_mock(root) is instance() + assert get_mock(root.middle.leaf) is leaf_mock + + +@pytest.mark.parametrize( + "first, second", + [ + (True, DeviceMock(mocked=False, mock_types={Leaf: XMock})), + (DeviceMock(mocked=False, mock_types={Leaf: XMock}), True), + ( + DeviceMock(mocked=False, mock_types={Leaf: XMock}), + DeviceMock(mocked=False, mock_types={Leaf: YMock}), + ), + (True, XMock()), + (XMock(), YMock()), + (True, False), + (False, True), + ( + DeviceMock(mocked=False, mock_types={Leaf: True}), + DeviceMock(mocked=False, mock_types={Middle: True}), + ), + ], +) +async def test_connect_again_applies_new_mock_choice(first, second): + fresh = Root() + await fresh.connect(mock=second) + root = Root() + await root.connect(mock=first) + + await root.connect(mock=second) + + assert connected_with(root) == connected_with(fresh) + + async def test_wait_for_connection(): class DummyDeviceWithSleep(DummyBaseDevice): def __init__(self, name) -> None: @@ -491,3 +957,89 @@ def test_device_repr_and_str_without_name(): unnamed_device = Device() expected = object.__repr__(unnamed_device) assert repr(unnamed_device) == str(unnamed_device) == expected + + +class MockedByType(Device): + def __init__(self, name: str = "") -> None: + self.leaf = Leaf() + super().__init__(name) + + +async def test_init_devices_mapping_applies_to_top_level_and_nested_devices(): + async with init_devices(mock={Leaf: XMock}): + leaf = Leaf() + holder = MockedByType() + + assert connected_with(leaf)["leaf"] is XMock + assert connected_with(holder.leaf)["holder-leaf"] is XMock + assert connected_with(holder)["holder"] is None + + +class SeenMock(DeviceMock): + seen: list[str] = [] + + async def connect(self, device: Device) -> None: + self.seen.append(device.name) + + +def _noop() -> None: + pass + + +class HookInner(Device): + def __init__(self, name: str = "") -> None: + self.sig = soft_signal_rw(float) + self.cmd = soft_command(_noop) + super().__init__(name) + + +class HookOuter(Device): + def __init__(self, name: str = "") -> None: + self.inner = HookInner() + super().__init__(name) + + +async def test_hooks_run_children_first_for_every_device_signal_and_command(): + SeenMock.seen.clear() + async with init_devices(mock={HookOuter: SeenMock, HookInner: SeenMock}): + outer = HookOuter() + assert SeenMock.seen == ["outer-inner", "outer"] + + # Reconnecting runs the hooks again + SeenMock.seen.clear() + await outer.connect(mock=SeenMock(mock_types={HookInner: SeenMock})) + assert SeenMock.seen == ["outer-inner", "outer"] + + # A Device entry reaches top level Signals and Commands + SeenMock.seen.clear() + await connect_devices( + {"sig": outer.inner.sig, "cmd": outer.inner.cmd}, mock={Device: SeenMock} + ) + assert SeenMock.seen == ["outer-inner-sig", "outer-inner-cmd"] + + +def test_lazy_mock_alias_is_device_mock(): + with pytest.warns(DeprecationWarning, match="'LazyMock' is deprecated") as record: + from ophyd_async.core import LazyMock + + assert LazyMock is DeviceMock + assert record[0].filename == __file__ + + +async def test_get_mock_works_for_every_connected_device(): + root = Root() + + await root.connect(mock=DeviceMock(mocked=False, mock_types={Leaf: True})) + await root.middle.leaf.sig.set(1.0) + + # The real parents hold the mock of the mocked Leaf below them + expected = [call.put(1.0)] + assert get_mock(root).middle.leaf.sig.mock_calls == expected + assert get_mock(root.middle).leaf.sig.mock_calls == expected + assert get_mock(root.middle.leaf).sig.mock_calls == expected + # Real Devices record nothing themselves + assert get_mock(root.leaves).mock_calls == [] + assert get_mock(root).mock_calls == [call.middle.leaf.sig.put(1.0)] + # A Device that was never connected has no DeviceMock + with pytest.raises(RuntimeError, match="not connected"): + get_mock(Root()) diff --git a/tests/unit_tests/core/test_signal.py b/tests/unit_tests/core/test_signal.py index e9a7b7a973..e4a35f95f0 100644 --- a/tests/unit_tests/core/test_signal.py +++ b/tests/unit_tests/core/test_signal.py @@ -17,12 +17,14 @@ Array1D, AsyncReadable, Device, + DeviceMock, SignalR, SignalRW, SoftSignalBackend, StandardReadable, StrictEnum, callback_on_mock_put, + default_mock_class, init_devices, set_and_wait_for_other_value, set_and_wait_for_value, @@ -1344,3 +1346,49 @@ async def test_soft_signal_r_and_setter_with_poll_period(): async def test_soft_signal_r_and_setter_poll_period_without_getter_raises(): with pytest.raises(ValueError, match="poll_period requires a getter"): soft_signal_r_and_setter(float, poll_period=0.1) + + +@pytest.mark.parametrize( + "make_signal", + [ + lambda: epics_signal_rw(float, "PV:X"), + lambda: soft_signal_rw(float), + ], + ids=["epics", "soft"], +) +async def test_mock_reconnect_allows_new_subscriptions_and_cached_reads(make_signal): + sig = make_signal() + await sig.connect(mock=True) + sig.subscribe_reading(lambda r: None) + await sig.connect(mock=True, force_reconnect=True) + seen: list[float] = [] + sig.subscribe_reading(lambda r: seen.append(r[sig.name]["value"])) + set_mock_value(sig, 42.0) + await asyncio.sleep(0) + assert seen[-1] == 42.0 + assert await sig.get_value(cached=True) == 42.0 + + +class _SubscribingMock(DeviceMock["_SubscribingDevice"]): + async def connect(self, device: "_SubscribingDevice") -> None: + device.sig.subscribe_reading( + lambda r: device.seen.append(r[device.sig.name]["value"]) + ) + + +@default_mock_class(_SubscribingMock) +class _SubscribingDevice(Device): + def __init__(self, name=""): + self.seen: list[float] = [] + self.sig = epics_signal_rw(float, "PV:Y") + super().__init__(name) + + +async def test_repeated_mock_connects_leave_one_live_subscription(): + dev = _SubscribingDevice() + for _ in range(3): + await dev.connect(mock=True, force_reconnect=True) + dev.seen.clear() + set_mock_value(dev.sig, 5.0) + await asyncio.sleep(0) + assert dev.seen == [5.0] diff --git a/tests/unit_tests/epics/adcore/test_detector.py b/tests/unit_tests/epics/adcore/test_detector.py index 6b9edb2633..229fbba339 100644 --- a/tests/unit_tests/epics/adcore/test_detector.py +++ b/tests/unit_tests/epics/adcore/test_detector.py @@ -1,10 +1,13 @@ import pytest from ophyd_async.core import ( + Device, StaticPathProvider, TriggerInfo, + get_mock, init_devices, set_mock_value, + soft_signal_rw, ) from ophyd_async.epics import adcore @@ -55,6 +58,35 @@ def test_writer_image_hints_enabled_by_default( assert det.hints == {"fields": ["det"]} +class DetectorRig(Device): + def __init__(self, det: adcore.AreaDetector) -> None: + self.det = det + self.flag = soft_signal_rw(int) + super().__init__("rig") + + +async def test_init_devices_mock_area_detector_mocks_plugins_too( + static_path_provider: StaticPathProvider, +): + async with init_devices(mock={adcore.AreaDetector: True}): + rig = DetectorRig( + adcore.AreaDetector( + adcore.ADBaseIO("PREFIX:DRV:"), + "PREFIX:", + adcore.ADWriterFactory.hdf(static_path_provider), + plugins={"stats": adcore.NDStatsIO("PREFIX:STAT:")}, + ) + ) + + # The plugins are mocked with the detector, the rest of the rig is real + set_mock_value(rig.det.get_plugin("stats", adcore.NDStatsIO).total, 3.0) + with pytest.raises(RuntimeError, match="not connected in mock mode"): + set_mock_value(rig.flag, 1) + # Calls on the mocks are nested under the real parent's mock + await rig.det.driver.acquire_time.set(0.5) + get_mock(rig).det.driver.acquire_time.put.assert_called_once_with(0.5) + + # --------------------------------------------------------------------------- # AreaDetector.__init__ guards # --------------------------------------------------------------------------- diff --git a/tests/unit_tests/epics/demo/test_epics_demo.py b/tests/unit_tests/epics/demo/test_epics_demo.py index 753630bf71..db8dbd8cbc 100644 --- a/tests/unit_tests/epics/demo/test_epics_demo.py +++ b/tests/unit_tests/epics/demo/test_epics_demo.py @@ -10,7 +10,7 @@ from bluesky.run_engine import RunEngine from ophyd_async.core import ( - LazyMock, + DeviceMock, get_mock, get_mock_execute, get_mock_put, @@ -37,11 +37,11 @@ @pytest.fixture async def mock_motor(): - # Connect with a plain LazyMock, rather than mock=True that will use + # Connect with a plain DeviceMock, rather than mock=True that will use # a InstantMovableMock, so we can have full control of how the readback is set # in the tests mock_motor = demo.DemoMotor("BLxxI-MO-TABLE-01:X:", name="mock_motor") - await mock_motor.connect(mock=LazyMock()) + await mock_motor.connect(mock=DeviceMock()) set_mock_units(mock_motor.readback, "mm") set_mock_precision(mock_motor.readback, 3) set_mock_value(mock_motor.velocity, 1) @@ -137,7 +137,7 @@ async def test_retrieve_mock_and_assert(mock_motor: demo.DemoMotor): async def test_mocks_in_device_share_parent(): - lm = LazyMock() + lm = DeviceMock() mock_motor = demo.DemoMotor("BLxxI-MO-TABLE-01:Y:") await mock_motor.connect(mock=lm) mock = lm() diff --git a/tests/unit_tests/epics/pvi/test_pvi.py b/tests/unit_tests/epics/pvi/test_pvi.py index 7dcafdd3e4..4538fb66fc 100644 --- a/tests/unit_tests/epics/pvi/test_pvi.py +++ b/tests/unit_tests/epics/pvi/test_pvi.py @@ -7,6 +7,7 @@ from ophyd_async.core import ( Device, DeviceMap, + DeviceMock, DeviceVector, SignalR, SignalRW, @@ -68,6 +69,28 @@ def with_pvi_connector( return device +async def test_init_devices_mock_mapping_overrides_pvi_filled_children(): + class RecordingMock(DeviceMock): + connected: list[Device] = [] + + async def connect(self, device: Device) -> None: + RecordingMock.connected.append(device) + + async with init_devices(mock={Device: True, Block1: RecordingMock}): + test_device = with_pvi_connector(Block3, "PREFIX:") + + # Block1s at every depth were connected with RecordingMock, and nothing else + connected = RecordingMock.connected + assert all(isinstance(device, Block1) for device in connected) + for block in [ + test_device.signal_device, + test_device.device.device, + test_device.device_vector[1].device, + test_device.device_vector[1].device_vector[1], + ]: + assert block in connected + + async def test_fill_pvi_entries_mock_mode(): async with init_devices(mock=True): test_device = with_pvi_connector(Block3, "PREFIX:") diff --git a/tests/unit_tests/epics/test_motor.py b/tests/unit_tests/epics/test_motor.py index eeed03faf0..6331fe73b9 100644 --- a/tests/unit_tests/epics/test_motor.py +++ b/tests/unit_tests/epics/test_motor.py @@ -11,12 +11,14 @@ FlyMotorInfo, callback_on_mock_put, default_mock_class, + get_mock, get_mock_put, init_devices, set_mock_precision, set_mock_put_proceeds, set_mock_units, set_mock_value, + soft_signal_rw, wait_for_value, ) from ophyd_async.epics.motor import ( @@ -644,3 +646,33 @@ async def test_velo_and_accl_respecting_motor_mock_interrupted_move( # the target means the move ended early rather than finishing assert await motor.motor_done_move.get_value() == 1 assert await motor.user_readback.get_value() < 10.0 + + +class Stage(Device): + def __init__(self, name: str = "") -> None: + self.mtr = Motor("PREFIX:MTR") + self.flag = soft_signal_rw(int) + super().__init__(name) + + +@pytest.mark.parametrize( + "mock, expected_velocity", + [ + # Real except motors, which are mocked with their default + ({Motor: True}, 1000), + # Real except motors, which are mocked with a given class + ({Motor: VeloAndAcclRespectingMotorMock}, 10), + ], +) +async def test_init_devices_mock_by_motor_type(mock, expected_velocity): + async with init_devices(mock=mock): + stage = Stage() + + assert await stage.mtr.velocity.get_value() == expected_velocity + await stage.mtr.set(1.0) + assert await stage.mtr.user_readback.get_value() == pytest.approx(1.0) + # The Stage and its flag were connected for real + with pytest.raises(RuntimeError, match="not connected in mock mode"): + set_mock_value(stage.flag, 1) + # The mock of the motor is nested under the mock of its real parent + get_mock(stage).mtr.user_setpoint.put.assert_called_with(1.0) diff --git a/tests/unit_tests/fastcs/panda/test_panda_connect_mock.py b/tests/unit_tests/fastcs/panda/test_panda_connect_mock.py index bb59837941..0578b7e1f3 100644 --- a/tests/unit_tests/fastcs/panda/test_panda_connect_mock.py +++ b/tests/unit_tests/fastcs/panda/test_panda_connect_mock.py @@ -138,7 +138,7 @@ async def test_panda_gets_types_from_common_class(panda_t): ``SeqBlock``/``PulseBlock``'s own class annotations, needing at most ``connect(mock=True)`` and no live PVI tree at all (see ``PviDeviceConnector.create_children_from_annotations``, which runs at - ``Device.__init__`` time from the annotations alone, vs. ``connect_real``, + ``Device.__init__`` time from the annotations alone, vs. a real ``connect``, which is the only thing that ever talks to a network): - sub-block type resolution (``panda.pcap`` is a ``PcapBlock``, @@ -162,7 +162,7 @@ async def test_panda_gets_types_from_common_class(panda_t): - ``panda.pcap.newsignal`` and its ``NO_ARG_VOID_SIGNATURE`` signature check -- ``newsignal`` is not declared anywhere in ``PcapBlock``; it only ever appears by walking a live PVI tree in - ``PviDeviceConnector.connect_real`` (gated behind the + ``PviDeviceConnector.connect`` (gated behind the ``INCLUDE_EXTRA_SIGNAL`` macro in the old system test's IOC database). - ``panda.extra`` and its ``isinstance(..., DeviceVector)`` check -- same story, ``extra`` is a block gated behind ``INCLUDE_EXTRA_BLOCK`` diff --git a/tests/unit_tests/plan_stubs/test_ensure_connected.py b/tests/unit_tests/plan_stubs/test_ensure_connected.py index 6d34c6dbe6..018ea08473 100644 --- a/tests/unit_tests/plan_stubs/test_ensure_connected.py +++ b/tests/unit_tests/plan_stubs/test_ensure_connected.py @@ -2,7 +2,7 @@ import pytest -from ophyd_async.core import Device, NotConnectedError, soft_signal_rw +from ophyd_async.core import Device, NotConnectedError, get_mock, soft_signal_rw from ophyd_async.epics.core import epics_signal_rw from ophyd_async.plan_stubs import ensure_connected @@ -24,20 +24,18 @@ def connect(): ): RE(connect()) - assert isinstance(device1.signal._connect_task.exception(), NotConnectedError) + with pytest.raises(RuntimeError, match="not connected"): + get_mock(device1.signal) device1.signal = soft_signal_rw(str) RE(connect()) - assert device1.signal._connect_task.exception() is None + get_mock(device1.signal) device2 = MyDevice("PREFIX2", name="device2") - - def connect_with_mocking(): - assert device2.signal._mock is None - yield from ensure_connected(device2, mock=True, timeout=0.1) - assert device2.signal._mock is not None - - RE(connect_with_mocking()) + with pytest.raises(RuntimeError, match="not connected"): + get_mock(device2.signal) + RE(ensure_connected(device2, mock=True, timeout=0.1)) + get_mock(device2.signal) def test_ensure_connected_fails_for_non_unique_device_names(RE):