feat: Enhance session management with persistence and user preferences

[doughayden]

Feature Request

Implement comprehensive session management with persistence, history, and user preferences to improve user experience and personalization.

Level of Effort: 🔥 Large (4-6 days)

• Backend session storage: 2-3 days for session persistence and management
• Frontend integration: 1-2 days for UI components and state management
• User preferences: 1 day for preference storage and application
• Testing: 1 day for session management testing

Current Session Management

What we have:

• Basic session ID generation and tracking
• Temporary session state in Streamlit
• Session cleanup via delete operations
• User identification through OAuth

Limitations:

• No session persistence across browser sessions
• No conversation history retrieval
• Limited user preference management
• No session analytics or insights

Proposed Enhancements

1. Session Persistence

Backend session storage:

# src/answer_app/session_manager.py
class SessionManager:
    def __init__(self, bq_client, project_id: str):
        self.bq_client = bq_client
        self.project_id = project_id
    
    async def create_session(self, user_email: str, preferences: dict = None) -> str:
        """Create new user session with preferences."""
        
    async def get_user_sessions(self, user_email: str, limit: int = 20) -> List[Session]:
        """Retrieve user's recent sessions."""
        
    async def restore_session(self, session_id: str, user_email: str) -> Session:
        """Restore previous session state."""
        
    async def update_session_preferences(self, session_id: str, preferences: dict):
        """Update session-specific preferences."""

Session data structure:

@dataclass
class Session:
    session_id: str
    user_email: str
    created_at: datetime
    last_active: datetime
    conversation_count: int
    preferences: dict
    metadata: dict
    status: str  # active, archived, deleted

2. Conversation History

Conversation storage and retrieval:

class ConversationManager:
    async def get_session_conversations(self, session_id: str) -> List[Conversation]:
        """Get all conversations for a session."""
        
    async def search_conversations(self, user_email: str, query: str) -> List[Conversation]:
        """Search user's conversation history."""
        
    async def export_conversations(self, session_id: str, format: str = "json") -> str:
        """Export session conversations."""

3. User Preferences

Preference categories:

user_preferences:
  response_style:
    - concise
    - detailed
    - technical
  
  language_preference: "en"
  
  ui_settings:
    theme: "dark"
    font_size: "medium"
    show_citations: true
  
  privacy_settings:
    save_conversations: true
    analytics_opt_in: false
    data_retention_days: 90
  
  notification_settings:
    email_summaries: false
    session_reminders: true

4. Frontend Session Management

Streamlit session interface:

# src/client/session_ui.py
def render_session_sidebar():
    """Render session management in sidebar."""
    st.sidebar.subheader("Sessions")
    
    # Recent sessions
    sessions = get_user_sessions(st.session_state.user_email)
    for session in sessions:
        if st.sidebar.button(f"📝 {session.created_at.strftime('%m/%d %H:%M')}"):
            load_session(session.session_id)
    
    # New session button
    if st.sidebar.button("➕ New Session"):
        create_new_session()

def render_preferences_panel():
    """Render user preferences panel."""
    with st.expander("⚙️ Preferences"):
        response_style = st.selectbox("Response Style", ["concise", "detailed", "technical"])
        show_citations = st.checkbox("Show Citations", value=True)
        save_conversations = st.checkbox("Save Conversations", value=True)

Implementation Areas

Backend Components:

• src/answer_app/session_manager.py: Session CRUD operations
• src/answer_app/conversation_manager.py: Conversation history management
• src/answer_app/preferences.py: User preference handling
• src/answer_app/main.py: Session endpoints and middleware

Frontend Components:

• src/client/session_ui.py: Session management UI components
• src/client/preferences_ui.py: User preferences interface
• src/client/streamlit_app.py: Integration with session management

Database Schema:

• user_sessions: Session metadata and preferences
• conversation_history: Detailed conversation logs
• user_preferences: Global user preferences

Database Schema Design

BigQuery tables:

-- user_sessions table
CREATE TABLE `project.dataset.user_sessions` (
  session_id STRING NOT NULL,
  user_email STRING NOT NULL,
  created_at TIMESTAMP NOT NULL,
  last_active TIMESTAMP NOT NULL,
  conversation_count INT64 DEFAULT 0,
  preferences JSON,
  metadata JSON,
  status STRING DEFAULT 'active'
);

-- conversation_history table
CREATE TABLE `project.dataset.conversation_history` (
  conversation_id STRING NOT NULL,
  session_id STRING NOT NULL,
  user_email STRING NOT NULL,
  query TEXT NOT NULL,
  response TEXT,
  timestamp TIMESTAMP NOT NULL,
  feedback_score INT64,
  response_time_ms INT64
);

Configuration Options

Add to config.yaml:

session_management:
  max_sessions_per_user: 50
  session_timeout_days: 30
  conversation_retention_days: 90
  auto_archive_inactive_days: 7
  
user_preferences:
  default_response_style: "detailed"
  allow_data_export: true
  max_export_conversations: 1000

Privacy and Data Management

Data Retention:

• Configurable conversation retention periods
• Automatic archival of old sessions
• User-controlled data deletion
• GDPR compliance considerations

Privacy Controls:

• User opt-out from conversation saving
• Anonymous session modes
• Data export capabilities
• Clear data deletion workflows

Testing Strategy

Session Management Tests:

• Session creation and retrieval
• Conversation history accuracy
• Preference persistence and application
• Session cleanup and archival

UI/UX Tests:

• Session switching workflows
• Preference change propagation
• History search functionality
• Export/import capabilities

Acceptance Criteria

• Session persistence across browser sessions
• Conversation history retrieval and search
• User preference storage and application
• Session management UI in Streamlit app
• Data retention and privacy controls
• Session analytics and insights
• Export/import functionality
• Performance optimization for session queries

Priority

Low - User experience enhancement that adds value but isn't critical for core functionality.

When to Implement

This becomes more valuable when:

• Users have longer, multi-session conversations
• User feedback indicates desire for conversation history
• Analytics show users returning to previous topics
• Personalization becomes important for user engagement
• Competition offers similar session management features