VNC Activity Tracking - Update last_activity during active VNC connections

[JoshuaAFerguson]

Overview

Implement proper VNC activity tracking to accurately show session idle/active status in the UI.

Problem

Currently, last_activity is only set once when a session starts running, but is never updated during active VNC usage. This causes:

• Inaccurate idle detection (sessions appear idle even when actively being used)
• Sessions may be auto-hibernated while users are actively connected
• Activity status doesn't reflect actual VNC usage

Current Behavior:

• last_activity set when session transitions to "running"
• Never updated during VNC connection lifetime
• Users actively using VNC still show as idle after threshold

Expected Behavior:

• last_activity updated whenever user interacts with VNC session
• Accurate idle detection based on actual mouse/keyboard activity
• Sessions remain active while VNC client is connected

Solution

1. VNC Proxy Activity Tracking

Update VNC proxy to track user activity:

// In vnc_proxy.go relayVNCData()
func (h *VNCProxyHandler) relayVNCData(...) {
    // When receiving data from UI (user input)
    case data := <-uiConn.ReadMessage():
        // Update last_activity for this session
        h.updateSessionActivity(sessionID)
        
        // Forward to agent...
}

func (h *VNCProxyHandler) updateSessionActivity(sessionID string) error {
    _, err := h.db.DB().Exec(`
        UPDATE sessions
        SET last_activity = NOW()
        WHERE id = $1
    `, sessionID)
    return err
}

2. Periodic Activity Heartbeat

Option A: Update on every VNC message (high DB load)

• Pros: Most accurate
• Cons: Many DB writes (could be 60+ per second during active use)

Option B: Throttled updates (every 30 seconds during activity)

• Pros: Balanced accuracy with performance
• Cons: Slightly less precise

Option C: In-memory tracking with periodic DB sync

• Pros: Best performance
• Cons: Most complex, activity lost on API restart

Recommended: Option B - Throttled updates every 30 seconds

3. Connection-based Tracking

Alternative: Track VNC connection state instead of per-message

// Update last_activity when VNC connection established
func (h *VNCProxyHandler) HandleVNCConnection(...) {
    // After successful connection
    h.updateSessionActivity(sessionID)
    
    // Start heartbeat goroutine
    go h.activityHeartbeat(sessionID, stopChan)
}

func (h *VNCProxyHandler) activityHeartbeat(sessionID string, stop chan struct{}) {
    ticker := time.NewTicker(30 * time.Second)
    defer ticker.Stop()
    
    for {
        select {
        case <-ticker.C:
            h.updateSessionActivity(sessionID)
        case <-stop:
            return
        }
    }
}

Implementation Plan

Phase 1: Basic Tracking (v2.0-beta.2)

• Quick fix: Set last_activity when session starts (commit ce6ad26)
• Update last_activity when VNC connection established
• Update last_activity when VNC connection closes

Phase 2: Heartbeat (v2.1.0)

• Implement 30-second heartbeat during active VNC connections
• Stop heartbeat when VNC disconnects
• Add metric: vnc_activity_updates_total

Phase 3: User Input Tracking (v2.2.0 - Optional)

• Track actual mouse/keyboard events from VNC stream
• Distinguish between "connected but idle" vs "actively using"
• More granular idle detection

Acceptance Criteria

• last_activity updated when VNC connection established
• last_activity updated every 30s while VNC connected
• last_activity stops updating when VNC disconnects
• UI shows "Active" status for sessions with recent VNC activity
• Idle timeout correctly detects inactive sessions
• No performance impact (DB writes throttled)
• Tests verify activity tracking

Related

• Fixed by: Commit ce6ad26 (partial - sets initial timestamp)
• Blocks: Auto-hibernation feature (needs accurate idle detection)
• Related: Issue #XXX (Idle session cleanup)

Timeline

• Milestone: v2.1.0
• Priority: P2 (Important for hibernation feature)
• Effort: ~1 day
  • 0.5 day: VNC connection activity tracking
  • 0.5 day: Heartbeat implementation + testing

---

🤖 Generated with Claude Code

[JoshuaAFerguson]

Closing as obsolete in scope. The VNC code path was removed in #261 (Selkies-only streaming, April 2026). The underlying concern — last_activity not being updated during active streaming — is still valid for Selkies sessions; if you want it tracked, please open a fresh issue framed around the Selkies HTTP/WebRTC proxy path so it points at code that exists.