Some devices produce bluesky documents in unpredictable order #1499

This is related to this bluesky-tiled-plugins issue, but I'm not entirely sure which package is even causing this problem.

My question here is what is different about these devices that could cause this behavior.

I have three devices:

1. A StandardWritable for our counter that implements DetectorDataLogic.prepare_single
2. A StandardReadable for our ion chamber
3. As a simple test, a StandardReadable with just two signals and a async def trigger() method.

I have three tests where I run a scan with these devices like RE(bp.scan([device], sim_motor_2, -10, 10, sim_motor_3, -5, 5, num=3)) and check the event documents.

I would expect that in all three cases, the keys of the resulting event document data will be in the same order.

Instead, the tests for device 1 and device 2 produce one event doc with the motors at the beginning and then additional event docs with the motors at the end. This causes an exception when writing to Tiled because Tiled tries to merge two tables with mis-matched keys.

I'm not convinced that this is an ophyd-async problem because device.read() produces the same key order. However I'm struggling to produce a MWE to even work on because the test for device 3 does not produce this behavior and the test passes.

Any ideas?