Skip to content

Improve embodied assistant interface - #19

Open
laojiahuo2003 wants to merge 31 commits into
syswonder:mainfrom
laojiahuo2003:feat/embodied-assistant-ui
Open

laojiahuo2003 wants to merge 31 commits into
syswonder:mainfrom
laojiahuo2003:feat/embodied-assistant-ui

Conversation

@laojiahuo2003

Copy link
Copy Markdown
Contributor

Summary

  • Improve the embodied assistant conversation UI, session controls, responsive layout, and localized strings.
  • Correct active task-node selection and recover Vitals polling after service timestamp resets.
  • Improve session-dialog keyboard focus handling and add regression coverage for task, telemetry, snapshot, protobuf compatibility, and TTS streaming behavior.

Verification

  • python3 -m unittest discover -s tests -v (79 passed)
  • npm run test:visual (79 passed, 1 skipped)
  • npm run build

…ed by n8n assistant

- Introduce modern dark obsidian palette with signature AI highlight gradients
- Add floating composer dock with auto-growing textarea and voice-first F2 controls
- Restructure user bubble into sleek right-aligned rounded pill (18px corner)
- Elevate agent messages with avatar badge header, markdown styling, and copy buttons
- Implement code snippets with language tag and one-click copy button
- Add empty-chat hero with embodied robot prompt suggestions (Webots demo, joints/vitals, RTDL plan, spatial perception)
- Enhance activity run widget with shimmering status animations
…io and fix settings popover upward alignment
Copilot AI balanced review requested due to automatic review settings November 16, 2026 04:37

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@enkerewpo enkerewpo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the big redesign. I checked out 39cf3cf locally and ran it at 1440×900 and 390×844 in both en and zh. The unit tests pass (79), the rebuilt vitals.bundle.js matches the committed one, and Playwright reports 79 passed and 1 skipped. The new backend calls (session_end, voiceprint list/delete, tts_stream, RTDL if/repeat/while, ListContracts flags) match robonix dev.

Blocking: Stop is broken. Every abort raises TypeError (see transport.py:850).

Visual consistency: The main problem is that two color themes are layered on top of each other. The new navy surfaces and purple accent sit over the old teal background, and the old cyan accent is still used for icons and active states. Sampled from screenshots of the same chat screen: the chat area is #070e10 (teal-black) next to the composer at #14171f (navy), and the user bubble is #1e2e31 (teal) next to the assistant card at #101218 (navy). Top-bar icons are cyan, while the active nav item, the panel toggle and Send are purple. styles.css defines 24 variables but uses 271 distinct hex colors (main has 188) and 231 !important. There are three greens (#10b981, #35e0a0, #22c55e) and three reds (#f43f5e, #ef4444, #f2726f). Inline comments show where to start. The aim is one base surface, one accent, and semantic success/warning/danger colors defined once as tokens. Cyan should be kept only for live or running status.

Other UI points (no single line to anchor):

  • Audio and Settings were not restyled. They have no page header, and their buttons are bold, left-aligned grey blocks that look like inputs. No sidebar item shows as selected while either page is open.
  • Rename and delete appear both in the chat header and on every session row. "Clear sessions" sits right next to Rename in the header, which puts a destructive action in the most reachable spot.
  • The empty Display Panel shows a dashed "waiting for task" box above a large blank area.
  • zh still mixes in untranslated labels such as "输入 Primitive" and "Executor".

Seen while testing, but these predate this PR (fine as follow-ups):

  • The en locale shows raw task states. app.js:4723 calls t(task.status), and English has no table, so the UI shows in_progress / done.
  • The hands-free chip turns red through .error when unavailable, but its label still says "Hands-free off".
  • In Executor details, missing values ("Not provided") are rendered bold white.
  • Studio's empty state uses the 📡 emoji while everything else uses Lucide icons.
  • The Vitals footer "Made by Zhenyu Zhang" is shown in the product UI.

Housekeeping: Please squash-merge with a conventional title, e.g. feat(client): redesign the assistant interface. The branch has 31 commits, including a working-tree snapshot and a merge commit.



def build_abort_task(settings: ClientSettings, expected_turn_id: str = "") -> Any:
def build_abort_task(settings: ClientSettings) -> Any:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This now takes only settings, but abort_turn (line 943) still calls build_abort_task(settings, expected_turn_id). Every Stop therefore raises

TypeError: build_abort_task() takes 1 positional argument but 2 were given

and Pilot never receives the abort. I reproduced it by calling abort_turn with resolve_liaison stubbed. The existing test calls build_abort_task directly, so it misses this.

Pilot's abort branch (system/pilot/src/service.rs, task_is_abort_turn) only reads session_id, so the simplest fix is to drop expected_turn_id from the whole abort path: abort_turn, /ws/abort in app.py, and the expectedTurnId fields that app.js sends with the abort. Please also add a test that goes through abort_turn.

Comment on lines 178 to 180
background:
radial-gradient(circle at 70% 12%, rgba(44, 126, 132, 0.14), transparent 28%),
linear-gradient(180deg, #081014, #071012 45%, #080d0f);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the old teal theme, and it shows through everywhere a page has no background of its own: the chat area, the Audio and Settings backgrounds, and the area behind the user bubble. The sidebar, top bar, composer and cards meanwhile use the navy --bg / --panel, so one screen has two base colors. For example, the chat area is #070e10 (hue 193) and the composer right below it is #14171f (hue 224). Vitals stays teal throughout, while Perception and Studio are navy.

If navy is the intended base:

Suggested change
background:
radial-gradient(circle at 70% 12%, rgba(44, 126, 132, 0.14), transparent 28%),
linear-gradient(180deg, #081014, #071012 45%, #080d0f);
background: var(--bg);

If teal-black is intended instead, move --bg and --panel* over to it. Either way, there should be a single base surface.

@@ -20,14 +20,85 @@
--shadow: 0 18px 50px rgba(0, 0, 0, 0.45);
--voice-level: 0;
--voice-opacity: 0;

/* Embodied AI Assistant Design System Tokens (Inspired by n8n Assistant) */
--assistant-highlight-1: #5b60e8;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These work as the app's accent now (nav selection, panel toggle, Send, logo), not just for the assistant. Could they become --accent / --accent-strong / --accent-gradient, and replace the hard-coded cyan accents? About 25 rules still color icons and text with --cyan, #5ad1e6 or --blue, for example the Atlas and user icons in the top bar. Cyan could then be kept only for live or running status. It would also help to add --success / --warning / --danger and fold in the duplicates (#10b981, #35e0a0, #22c55e; #f43f5e, #ef4444, #f2726f).

Comment on lines 8367 to 8368
border-color: rgba(56, 189, 248, 0.45);
background: rgba(56, 189, 248, 0.14);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Display Panel tabs select in cyan here, while the sidebar's .nav-item.active (line 622) selects in purple. Both are selection states and should use the same accent:

Suggested change
border-color: rgba(56, 189, 248, 0.45);
background: rgba(56, 189, 248, 0.14);
border-color: rgba(170, 123, 236, 0.32);
background: linear-gradient(135deg, rgba(91, 96, 232, 0.2), rgba(170, 123, 236, 0.16));

Comment on lines +1497 to +1499
border: 1px solid rgba(239, 68, 68, 0.45);
background: rgba(239, 68, 68, 0.12);
color: #fca5a5;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Connect is the primary action when disconnected, but it is styled as a danger chip. Together with the red "offline" chip and the red hands-free chip, the whole top bar turns red and nothing stands out as the thing to click. Suggest giving it the same primary style as Send:

Suggested change
border: 1px solid rgba(239, 68, 68, 0.45);
background: rgba(239, 68, 68, 0.12);
color: #fca5a5;
border: 1px solid var(--assistant-focus-ring);
background: var(--assistant-gradient);
color: #fff;

gap: 6px;
padding: calc(6px + env(safe-area-inset-top, 0px)) calc(10px + env(safe-area-inset-right, 0px)) 6px calc(10px + env(safe-area-inset-left, 0px));
min-height: calc(44px + env(safe-area-inset-top, 0px));
overflow-x: auto;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At 390 px the bar is 426 px wide, so "Hands-free off" is cut off and Connect is hidden until the bar is scrolled sideways. The Vitals page already hides #userBadge and #handsfreeToggle on mobile (line 6677), so the top bar also changes layout between pages. Suggest applying the same hiding on every page at this breakpoint (or collapsing those chips into the menu) so that Connect always fits without scrolling.

@@ -0,0 +1,610 @@
// SPDX-License-Identifier: MulanPSL-2.0

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing in the repo references the ~50 generated .proto files under proto/upstream/ (about 4.5k lines). There is no regeneration script, and they are not used by the tests. Either drop them, or add the script that regenerates *_pb2.py from them so they have a purpose and stay in sync.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants