Spec: Multi-Device Routing — device registry, capability negotiation, cross-device responses

[rolpro]

Motivation

A user may have multiple Yodel devices connected simultaneously (walkie-talkie, TV, monitor, smartwatch, Telegram bot). When a request arrives from one device, the gateway needs to know:

1. What other devices are available?
2. Which device is best suited for the response?
3. What if the requesting device can't handle the response format?

Example: User speaks into walkie-talkie "show me the chart". Walkie has no display. Gateway should ask "On your TV or monitor?" and route the image to the chosen device.

Proposal: Multi-Device Routing

1. Device Registry

The gateway maintains a registry of connected devices with their device_id, type, purpose, and capabilities. Each SSE connection registers a device; disconnect removes it.

2. Capability Negotiation

Before generating a response, the gateway checks whether the requesting device has the necessary capabilities. If not, it can:

• Ask the user which target device to use (via text or voice)
• Auto-select the best device based on capabilities + user preferences
• Fall back to text description on the original device

3. Cross-Device References

Users need to refer to their devices in natural language. Possible approaches:

• Named devices: X-Yodel-Device-Name: "Living Room TV" (client sets it)
• Purpose-based: "send to my big-screen"
• Gateway learns: "my Fernseher" → last used display-capable device

4. Protocol Additions

{
  "yodel": {
    "purpose": "walkie-talkie",
    "target_device": "device-uuid-living-room-tv",   // optional: explicit routing
    "device": {
      "type": "ios",
      "name": "My Walkie",                            // optional: human-readable
      "capabilities": ["audio_out", "audio_in"]
    }
  }
}

New fields:

• device.name — human-readable device label
• target_device — route response to a different device
• routing_preferences — auto | ask | specific:<device_id>

5. Multi-Device Sessions

A session may span multiple devices. When the gateway routes a follow-up to a different device, the conversation context is preserved.

Open Questions

• How does a device discover other devices? Gateway-push or client-poll?
• Should target_device be device_id, name, or purpose?
• How to handle devices going offline mid-conversation?
• Privacy: should devices see each other's presence?

[rolpro]

Implementation: openyodel/plugin-openyodel-hermes#23