Repository navigation
Embedded Python scripting terminal #387
Description
Part of #373 Queue Management & Experiment Control:
Integrate an embedded terminal leverage the BlueAPI Python client for manual scripting and custom acquisition execution.
Goals
Acceptance criteria : What journeys should the user be able to complete to consider your feature done
Be as specific as you can, check them off once they work. Add more if you find more requirements during development. Issue should only be considered done after all of them are checked
- Power users can write custom plans and command devices manually from an embedded command-line interface.
- Interactive Web Terminal Workspace: Users can open a dockable/collapsible command-line panel inside the UI with full keyboard navigation, copy/paste support.
- Pressing
Tabinside the embedded terminal auto-completes blueapi plan names, device names (stage_x,stage_y), and detector and PV. - Pre-Authenticated Session: Users can run plans and command devices via blueapi.client Python session without requiring manual connection setup or boilerplate imports
- Streaming Output: Execution logs, stdout, stderr, print emitted data, tracebacks, and plan status updates render in inside the terminal.
- Error Message: Provide useful error reporting including device level error with a stack trace.
- State Synchronization: Plans executed from the terminal update scan numbers, register on the main acquisition progress bar, and stream live scan data to the plot canvas.
- Persistent Command History: Command history (up/down arrow navigation and search) is stored per visit and automatically restored across browser reloads or workspace sessions.
- Useful help function: Ensure all functions and devices have useful doc strings accessible via the help command (e.g. scan -h).
- Execution Safety: The terminal restricts issuing conflicting manual hardware movement commands while a BlueAPI queue plan is actively executing.
Tasks
base on the goals, deduce the tasks required. check them off once they're done
Caution
Should this interact with Blueapi or the queue?
- BlueAPI/Queue Client integration: Terminal inputs and outputs directly through the BlueAPI python client or via block/queue.
- Embedded Terminal Widget: Integrate a terminal widget within the main UI workspace.
- Does it really have to be two in one?
- Does it has to be a web terminal?
Related Resources
Current CLI from BlueAPI: It is worth mentioning that the current CLI met 80% of our requirement and we have wrapped into a ipython terminal together with our cli #406
Bluesky magic: Bluesky kind of has some magics for plans already https://blueskyproject.io/bluesky/v1.14.2/magics.html
Bluesky also offer persistent dictionary: https://blueskyproject.io/bluesky/v1.14.2/generated/bluesky.utils.PersistentDict.html
Current thinking from the UI team Server side Jupyter Enterprise Gateway
Screen shoot of Emily slide
Activity
[-]Integrate an embedded Python terminal with BlueAPI client support to drive experiments.[/-][+]Embedded Python scripting terminal[/+]NKatti2011 commentedon Sep 16, 2026
Include notes from meetings with @EmsArnold.
Architecture is not yet settled. Current expectation is that the terminal sits behind
the queue and submits to it, rather than talking to BlueAPI directly, that keeps a
single ordering of work and avoids the RunEngine contention problem described in #381.
Needs confirmation from the DAQ team.
Relm-Arrowny commentedon Sep 16, 2026
Just noting some question I may have running it off queue:
Running it off the queue raise a different question, how scripting going to work with queue?
This also change the user experience, you send a command in cli and nothing happen until the queue is empty?
Do you send a message back cli will only run if queue is empty or we going to put it at the black of the queue?
How are we going handle the python logic in queue?
Does scripting just mean program block generation for queue?
NKatti2011 commentedon Sep 16, 2026
From @jacob-williamson I gleaned that there was no current or future solutions planned for how scripting will work with the queue.
My own note - scripting does not mean a program block generation for the queue. Scripting and the program block are separate entities. This will likely be hashed out in whatever scripting vertical slice will happen.
oliwenmandiamond commentedon Sep 21, 2026
The current behaviour when scripting with blueapi client if something else is already running, it returns instantly saying worker is busy. So if a bunch of things are queued up via queue service and running, then scripting wouldn't work anyway. I think we should probably leave this as is, queue takes priority as it will auto submit a job as soon as it becomes free? There would be a tiny edge case where if you try to run something via scripting the moment something becomes free, it could beat the queue service and probably break that.
Maybe a baton should be added to blueapi? So you have to request to hold the baton before you are allowed to submit anything to blueapi and different services aren't competing to submit to blueapi?
Relm-Arrowny commentedon Sep 22, 2026
Client side solution: Python within WebAssembly(Pyodide)
Pro
Deploy and go nothing to worry about as it is running on the client side(no DMZ), but need to config our services to have Cross-Origin Resource Sharing and have a valid ssl certificate.
May or may not work with blueAPI client as not all library is supported.
Con
Sharing anything is difficult, no session, no different from running a local python and make use of blueapi cli.
Relm-Arrowny commentedon Sep 22, 2026
Server side JupyterHub
Pro
Con
Relm-Arrowny commentedon Sep 22, 2026
Server side ttyd
Pro
Cons
Relm-Arrowny commentedon Sep 22, 2026
Server side xterm.js + Gateway
Pro
Cons
Relm-Arrowny commentedon Sep 23, 2026
Server side Jupyter Enterprise Gateway
Pro
Cons
EmsArnold commentedon Sep 23, 2026
For some context, the intended UI implementation will be an app within atlas, which will provide both UI, and integrated data visualisation.
Relm-Arrowny commentedon Sep 25, 2026
We must be doing something wrong, the more we think about terminal and script the more it seem, we reintroducing the original bluesky ux, surely the design is not quite what we want somewhere.
Relm-Arrowny commentedon Sep 25, 2026
I wonder if we should separate these two thing:
"Terminal" is just a quick command bar for single plan . These can either go through the queue service as "run-next" tasks or use an explicit baton lock. Because it only needs to send one-off commands, this can easily live client-side without we doing any work.
Scripting: Instead of running an interactive shell, we treat scripts as batch jobs. Users construct them via a block builder or a script editor, which compiles the logic into task and submits it directly to the queue.
Relm-Arrowny commentedon Oct 9, 2026
If JEG is the core team design, when it is ready we will need #411 to covert what we have to run on JEG. And it will effectively complete this issue.