I15 1 pre release

docs/explanations/beamlines.md

@@ -0,0 +1,8 @@
+# Beamlines
+
+```{toctree}
+:maxdepth: 1
+:glob:
+
+i15-1 <beamlines/i15-1/i15-1>
+```

docs/explanations/beamlines/i15-1/auxiliary_scans.md

@@ -0,0 +1,25 @@
+# i15-1 Auxiliary Scans
+
+I15-1 PDF experiments require some scans to be run before hand, for use during analysis. These are added to the queue automatically by the i15-1 queue plugin. For each experiment, we need:
+
+- an air scan, at room temperature
+- an empty capillary scan, matching the capillary type and temperature range to the experiment
+- a standard sample scan, at room temperature, matching the capillary type to the experiment
+
+These must happen in this order, as the analysis process is layered - i.e, the empty capillary scan needs the air scan to be processed, the standard sample needs the empty capillary, and therefore the air, and the actual experiment needs all three. Additionally, to save time, we can merge these auxiliary scans together, to avoid queuing up three additional scans for every experiment. For example, experiments can always share an air scan (as it's always the same); empty capillary scans can be shared as long as the temperature range spans the range of the experiment, and the capillary type is the same; and standard sample scans can be shared as long as the capillary type is the same.
+
+The above requirements lead to the following logic:
+
+- An experiment will require the three auxiliary scans outlined above, the details of which will be a function of the experiment parameters. See `I151Converter._get_required_auxiliary_scans`
+- An `AuxiliaryScan` has a `requires` field, which details the other required auxiliary scans that must proceed it. The `AuxiliaryScan` objects produced by `I151Converter._get_required_auxiliary_scans` have each other as requirements. The empty capillary scan will be required by the standard sample scan, and the air scan will be required by both the other two.
+
+To add auxiliary scans, we build a new queue that includes the tasks along with their auxiliary scans. For each task in the queue, we will:
+
+- Gather the required auxiliary scans
+- For each required auxiliary scan:
+- Check whether a matching auxiliary scan already exists in the queue or tiled. For an existing scan to be considered suitable, all of the required auxiliary scan's own required auxiliary scans must also be required by the existing scan.
+- If a suitable existing does not exist, we will see if we can combine the required auxiliary scan with an existing scan. This, for example, could involve expanding the temperature range of a queued auxiliary scan. Combining these auxiliary scans also combines the requirements of both scans, so no requirement information is lost, but we also don't duplicate similar requirements. We then go to the beginning of this checklist, and repeat for the required auxiliary scans of the combined auxiliary scan, inserting the results before the combined scan in the queue.
+- If we cannot combine the required auxiliary scan, we will add the auxiliary scan to the queue. However, right before we do this, we will go through the required auxiliary scans of the auxiliary scan we added, and repeat this checklist for them. This ensures the auxiliary scan is added behind its requirements.
+- Once all the recursive adding of required auxiliary scans is complete, we add the task itself to the new queue.
+
+![Auxiliary scan logic flowchart](./i15-1_aux_scans.drawio.png)

docs/explanations/beamlines/i15-1/i15-1.md

@@ -0,0 +1,8 @@
+# i15-1
+
+```{toctree}
+:maxdepth: 1
+:glob:
+
+Auxiliary Scans <auxiliary_scans>
+```

docs/reference/rest_api.json

@@ -1072,29 +1072,9 @@
       "OIDCConfig": {
         "properties": {
           "well_known_url": {
-            "anyOf": [
-              {
-                "type": "string"
-              },
-              {
-                "type": "null"
-              }
-            ],
+            "type": "string",
             "title": "Well Known Url",
-            "description": "URL to fetch OIDC config from the provider",
-            "deprecated": true
-          },
-          "issuer": {
-            "anyOf": [
-              {
-                "type": "string"
-              },
-              {
-                "type": "null"
-              }
-            ],
-            "title": "Issuer",
-            "description": "URL of OIDC provider"
+            "description": "URL to fetch OIDC config from the provider"
           },
           "client_id": {
             "type": "string",
@@ -1117,6 +1097,7 @@
         "additionalProperties": false,
         "type": "object",
         "required": [
+          "well_known_url",
           "client_id"
         ],
         "title": "OIDCConfig"

src/daq_queuing_service/external_interaction/tiled/tiled.py

@@ -55,27 +55,30 @@ def get_tiled_client(
 
 
 def get_metadata_from_tiled(
-    tiled_client: Container, instrument_session: str, blueapi_task_id: str
+    tiled_client: Container,
+    instrument: str,
+    instrument_session: str,
+    blueapi_task_id: str,
 ) -> list[tuple[str, dict[str, Any]]]:
 
     @cached(cache)
     def _query_tiled(
-        instrument_session: str, blueapi_task_id: str
+        instrument: str, instrument_session: str, blueapi_task_id: str
     ) -> list[tuple[str, dict[str, Any]]]:
-        result: Container = tiled_client.search(
-            Eq("start.instrument_session", instrument_session)
-        ).search(Eq("start.blueapi_task_id", blueapi_task_id))
+        result: Container = tiled_client[
+            f"{instrument}/{instrument_session.split('-')[0]}/{instrument_session}"
+        ].search(Eq("start.blueapi_task_id", blueapi_task_id))
 
         items = sorted(
             ((key, dict(value.metadata)) for key, value in result.items()),
             key=lambda item: item[1]["start"]["time"],
         )
         return items
 
-    return _query_tiled(instrument_session, blueapi_task_id)
+    return _query_tiled(instrument, instrument_session, blueapi_task_id)
 
 
-def get_tiled_and_scan_ids(tiled_client: Container, call: BlueapiCall):
+def get_tiled_and_scan_ids(tiled_client: Container, instrument: str, call: BlueapiCall):
     tiled_ids: list[str] = []
     scan_ids: list[int] = []
 
@@ -84,7 +87,7 @@ def get_tiled_and_scan_ids(tiled_client: Container, call: BlueapiCall):
         return tiled_ids, scan_ids
 
     scan_metadatas = get_metadata_from_tiled(
-        tiled_client, call.task_request.instrument_session, call.blueapi_id
+        tiled_client, instrument, call.task_request.instrument_session, call.blueapi_id
     )
 
     for item in scan_metadatas:

src/daq_queuing_service/plugins/i15_1/auxiliary.py

@@ -1,8 +1,10 @@
+from collections.abc import Sequence
 from enum import StrEnum
 from pathlib import Path
+from typing import Literal, Self
 
 from daq_config_server.models.i15_1.standards_puck import StandardsPin
-from pydantic import BaseModel, ConfigDict, computed_field
+from pydantic import BaseModel, ConfigDict, Field, computed_field
 
 from daq_queuing_service.task_queue.task import Experiment
 
@@ -32,6 +34,28 @@ def get_background_temperatures(
     )
 
 
+def combine_auxiliary_requirements(
+    required_auxiliaries_1: Sequence["AuxiliaryScan"],
+    required_auxiliaries_2: list["AuxiliaryScan"],
+) -> list["AuxiliaryScan"]:
+    result: list[AuxiliaryScan] = []
+    remaining = required_auxiliaries_2.copy()
+    for aux1 in required_auxiliaries_1:
+        for aux2 in remaining:
+            if combined := aux1.attempt_to_combine_with(aux2):
+                result.append(combined)
+                remaining.remove(aux2)
+
+                break
+        else:
+            result.append(aux1)
+
+    return result + remaining
+
+
+Q_MAX_OPTIONS = Literal[25, 30, 35, 40]
+
+
 class AuxiliaryScan(BaseModel):
     """Auxiliary scans are scans added automatically by the queue, such as background
     scans and standard sample calibration scans. They are needed for analysis of user
@@ -42,7 +66,9 @@ class AuxiliaryScan(BaseModel):
     instrument_session: str
     pin: StandardsPin | None
     time_per_pdf: float
+    q_max: Q_MAX_OPTIONS
     list_of_temperatures: list[int] | None = None
+    requires: list[Self] = Field(default_factory=lambda: [], exclude=True)
 
     @computed_field
     @property
@@ -59,7 +85,9 @@ def is_suitable(
         temperature_step: int = DEFAULT_TEMPERATURE_STEP,
     ) -> bool:
         """Determine if this background is suitable compared to an experiment's required
-        background.
+        background. This includes checking that scans in required_background.requires
+        have a suitable pairing in self.requires, to ensure nested requirements are
+        taken into account.
 
         Args:
             required_background (AuxiliaryScan): The required background
@@ -75,10 +103,15 @@ def is_suitable(
         elif self.list_of_temperatures:
             return False
 
+        for required in required_background.requires:
+            if not any(r.is_suitable(required) for r in self.requires):
+                return False
+
         return (
             self.instrument_session == required_background.instrument_session
             and self.pin == required_background.pin
             and self.time_per_pdf >= required_background.time_per_pdf
+            and self.q_max >= required_background.q_max
         )
 
     def _temperatures_match(
@@ -109,7 +142,8 @@ def attempt_to_combine_with(
         temperature_step: int = DEFAULT_TEMPERATURE_STEP,
     ) -> "AuxiliaryScan | None":
         """Creates a background that combines the requirements of this background object
-        and a provided required background, if possible.
+        and a provided required background, if possible. Anything that previously deemed
+        this scan as suitable should also deem the combined scan suitable.
 
         Args:
             required_background (AuxiliaryScan): The required background
@@ -142,6 +176,10 @@ def attempt_to_combine_with(
             pin=self.pin,
             time_per_pdf=max(self.time_per_pdf, required_background.time_per_pdf),
             list_of_temperatures=list_of_temperatures,
+            requires=combine_auxiliary_requirements(
+                self.requires, required_background.requires
+            ),
+            q_max=max(self.q_max, required_background.q_max),
         )
         assert result.is_suitable(required_background)
         return result
@@ -164,9 +202,13 @@ def from_experiment(cls, experiment: Experiment) -> "AuxiliaryScan":
             instrument_session=experiment.instrument_session,
             pin=pin,
             time_per_pdf=experiment.experiment_definition.data["time_per_pdf"],
+            q_max=experiment.experiment_definition.data["q_max"],
             list_of_temperatures=experiment.experiment_definition.data.get(
                 "list_of_temperatures"
             ),
+            requires=experiment.experiment_definition.data.get(
+                "required_auxiliary_scans", []
+            ),
         )