Establish Single Point of Truth (SPOT) read/write rules between NeXus, DataCats, and Tiled. #389

Activity

1. Relm-Arrowny

   added

   core dependencyMore needed to know before we can fully scope this work.More needed to know before we can fully scope this work.

   on Sep 9, 2026
2. Relm-Arrowny

   added a parent issue on Sep 9, 2026

   • P99 Acquisition & Visualisation Interface Overhaul #373
3. Relm-Arrowny

   added

   Decision neededDecision is required before this is ready to be scopedDecision is required before this is ready to be scoped

   on Sep 10, 2026
4. DanPorter commented on Sep 16, 2026

   

   DanPorter

   on Sep 16, 2026

   Collaborator

   More actions

   My current understanding is:

   • Get experiment data from Tiled (ideally via supergraph or websocket to detector data on disk)
   • Get processed data from Workflows (ideally via supergraph to data in S3 or websocket to processed data on disk)
   • save interface metadata to ???

   I have raised this in various forums multiple times and most recently with the architect, so will add more details as I find them out.
5. NKatti2011 commented on Sep 16, 2026

   

   NKatti2011

   on Sep 16, 2026

   Collaborator

   More actions

   Below include notes from meeting from @EmsArnold.

   Decision

   • Tiled is the single point of truth.

   Mechanism

   • A NeXus writer is being developed that sits behind Tiled: you ask Tiled to write the NeXus file you want, rather than writing it directly (being developed by core).
   • Working assumption is that workflow and processing pipeline outputs also go through this NeXus writer. Until that exists, processing outputs are written to the dataset's HDF5 file. The same applies to data tagging.

   Open questions

   • Coordinate frame and offset metadata: see the persistence note in Reference Frame Coordinate Transformations #379. What needs storing is a correction model plus its inputs at a point in time, not a single matrix, and we need to confirm whether Tiled holds that (likely).
6. NKatti2011 commented on Sep 16, 2026

   

   NKatti2011

   on Sep 16, 2026

   Last edited by NKatti2011

   Collaborator

   More actions

   Recommended can you confirm @EmsArnold?:

   1. Device readback for devices come from PVWS as in VISR
   2. Data and images come from Tiled (data displayed via Davidia client, maybe images displayed by some other display, not yet evaluated).
   3. If data transform needs to happen then there needs to be a layer that sits below or bolted onto the Davidia server which applies transform to data coming from Tiled - unknown needs to be evaluated.
   4. Live data potentially coming from Tiled websocket through Davidia server/client? - unknown.
   5. Processing status to come from Workflows websocket (not currently available on supergraph websocket).
   6. Writing metadata (inc data tagging through nexus writer - currently being developed by Core).
7. EmsArnold commented on Sep 16, 2026

   

   EmsArnold

   on Sep 16, 2026

   More actions

   1. Device readback for devices come from PVWS as in VISR
   

   Not ViSR, but currently I15-1. We're using the package in the atlas monorepo for this, and getting readback for various things (synchrotron current, sample loaded on robot, hutch interlock status, etc.). This should be very easy to adopt into p99 by following the instructions & example in the README.

   2. Data and images come from Tiled (data displayed via Davidia client, maybe images displayed by some other display, not yet evaluated).
   

   I don't know if this has been accomplished yet, but I believe this is the intended behaviour for Tiled.

   3. If data transform needs to happen then there needs to be a layer that sits below or bolted onto the Davidia server which applies transform to data coming from Tiled - unknown needs to be evaluated.
   

   Yes, this should likely be the same "maths as a service" idea which serves all acquisition which needs it.

   4. Live data potentially coming from Tiled websocket through Davidia server/client? - unknown.
   

   Potentially similar to 2? Reliant on ensuring Tiled + Davidia is performant enough to handle serving live data quickly enough.

   5. Processing status to come from Workflows websocket (not currently available on supergraph websocket).
   

   This is correct, though is liable to change in the near future.

   6. Writing metadata (inc data tagging through nexus writer - currently being developed by Core).
   

   Yes, though I'm not sure of timescales for this. There has recently been active discussion on this in the athena-components slack channel.

   👍React with 👍1Reacted by NKatti2011
8. NKatti2011 commented on Sep 16, 2026

   

   NKatti2011

   on Sep 16, 2026

   Collaborator

   More actions

   Thanks @EmsArnold for the clarity and info :)