diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml new file mode 100644 index 00000000..b5de3915 --- /dev/null +++ b/.github/FUNDING.yml @@ -0,0 +1 @@ +github: spaquet diff --git a/.ruby-version b/.ruby-version index 1a7a02b8..e1c52e3a 100644 --- a/.ruby-version +++ b/.ruby-version @@ -1 +1 @@ -ruby-3.4.4 \ No newline at end of file +ruby-4.0.3 \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..734a4fdc --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,212 @@ +# Listopia Codebase Guide for AI Coding Agents + +This is the code base of the Listopia Ruby on Rails application. + +## Architecture Overview + +This is a Ruby on Rails application with the following structure: + +- **app/models** - Active Record models +- **app/views** - View templates (ERB, etc.) +- **app/controllers** - Controllers +- **app/services** - Service objects for business logic +- **app/queries** - Query objects for complex queries +- **app/jobs** - Background jobs +- **app/policies** - Pundit authorization policies +- **config/** - Application configuration +- **docs/** - Detailed documentation + +## Tech Stack + +- **Database**: PostgreSQL only (no Redis) +- **CSS**: Tailwind CSS 4 with Bun +- **Pagination**: Pagy v43+ (uses `series_nav`, not `pagy_nav`) +- **Components**: ViewComponent for reusable view components +- **JavaScript**: Hotwire (Turbo Streams) preferred over Stimulus +- **Stimulus Controllers**: Plain JavaScript (no TypeScript) + +### UI Guidelines + +- Prefer Turbo Stream responses for reactive UI over custom JavaScript +- Use Stimulus only when Turbo can't solve the problem +- All Stimulus controllers are plain JavaScript in `app/javascript/controllers/` + +## Testing Commands + +This project uses **RSpec** for testing with Capybara. + +### Running Tests + +```bash +bundle exec rspec # Run all specs +bundle exec rspec spec/ # Run all specs +bundle exec rspec spec/models/ # Run model specs +bundle exec rspec spec/models/user_spec.rb +bundle exec rspec -e "test name" # Filter by spec name +``` + +### Running Specific Specs + +```bash +bundle exec rspec spec/models/user_spec.rb:10 # Run specific spec at line 10 +``` + +## Code Conventions + +### Service Objects + +Favor service objects when implementing CRUD and related operations so that the methods can be used by controllers, APIs, background jobs, and other service objects. Services inherit from `ApplicationService` and live in `app/services/`. Name with the operation (e.g., `CreateList`, `UpdateUser`, `Finder`). + +```ruby +class MyService < ApplicationService + def call + # Return success(data: ...) or failure(errors: ...) + end +end +``` + +### Query Objects + +Use query objects in `app/queries/` for complex database queries. + +### Code Style + +- Run RuboCop: `bundle exec rubocop` (there's a project-wide `.rubocop.yml`) +- Run Brakeman: `bundle exec brakeman` (security) +- Use `# frozen_string_literal: true` at top of all files + +## Finding Related Code + +- **Ruby LSP**: This project supports Ruby LSP for code navigation. Use your editor's LSP features for go-to-definition, find references, hover for type info, document symbols, and workspace symbols. +- **Models**: Look in `app/models/` +- **Services**: Check `app/services/` for business logic +- **Policies**: Check `app/policies/` for authorization +- **Queries**: Check `app/queries/` for complex queries +- **Configuration**: Check `config/` for application configuration + +## Key Patterns + +### Organization Scoping +```ruby +policy_scope(List) # Returns only user's org lists +current_organization.lists # Access through org +``` + +### Authorization +```ruby +authorize @list # Use Pundit on every action +``` + +### Turbo Stream Response +```erb +<%= turbo_stream.replace(@list) { render "list", list: @list } %> +``` + +## File Organization Principles + +- `app/models` - Active Record models (UUID primary keys) +- `app/views` - View templates (ERB, etc.) +- `app/controllers` - Controllers +- `app/services` - Service objects (inherit ApplicationService) +- `app/queries` - Query objects for complex queries +- `app/jobs` - Background jobs +- `app/policies` - Pundit authorization policies +- `config/` - Application configuration +- `spec/` - Test files using RSpec with Factory Bot, Faker +- `docs/` - Detailed documentation +- `bin/` - Executable scripts (e.g., `bin/dev`) + +## Documentation + +- Detailed documentation is in `docs/` folder +- See `docs/README.md` for documentation index +- Key docs: ARCHITECTURE_GENERIC_DESIGN.md, CHAT_CONTEXT.md, ORGANIZATIONS_TEAMS.md, AUTHENTICATION.md, REAL_TIME.md +- API docs use standard Rails conventions + +## Quick Setup + +```bash +rails db:create db:migrate +bundle exec rspec +bun install && bun run build:css +``` + +## Common Issues + +| Issue | Solution | +|-------|----------| +| N+1 Queries | Use `includes`, `preload`, or `joins` | +| Auth Failed | Call `authorize @resource` after load | +| Turbo not working | Respond with `format.turbo_stream` | +| Test DB issues | `RAILS_ENV=test rails db:reset` | +| No org selected | Select in nav bar; redirect if `Current.organization` nil | +| Cross-org data leak | Always scope queries with `organization_id`; use `Current.organization` | + +## Behavioral Guidelines for Coding + +Reduce common mistakes. Bias toward clarity over speed. For trivial tasks, use judgment. + +### 1. Think Before Coding + +Don't assume. Surface confusion and tradeoffs explicitly. + +Before implementing: +- State assumptions clearly. If uncertain, ask. +- If multiple interpretations exist, present them—don't pick silently. +- If a simpler approach exists, say so. Push back when warranted. +- If something is unclear, stop. Name what's confusing and ask. + +### 2. Simplicity First + +Minimum code that solves the problem. Nothing speculative. + +- No features beyond what was asked. +- No abstractions for single-use code. +- No "flexibility" or "configurability" that wasn't requested. +- No error handling for impossible scenarios. +- If you write 200 lines and it could be 50, rewrite it. +- **Test:** Would a senior engineer say this is overcomplicated? If yes, simplify. + +### 3. Surgical Changes + +Touch only what you must. Clean up only your own mess. + +When editing existing code: +- Don't "improve" adjacent code, comments, or formatting. +- Don't refactor things that aren't broken. +- Match existing style, even if different approach preferred. +- If you notice unrelated dead code, mention it—don't delete it. + +When your changes create orphans: +- Remove imports/variables/functions that **your** changes made unused. +- Don't remove pre-existing dead code unless asked. + +**Test:** Every changed line traces directly to the user's request. + +### 4. Goal-Driven Execution + +Define success criteria. Loop until verified. + +Transform tasks into verifiable goals: +- "Add validation" → Write tests for invalid inputs, then make them pass +- "Fix the bug" → Write a test that reproduces it, then make it pass +- "Refactor X" → Ensure tests pass before and after + +For multi-step tasks, state a brief plan: +``` +1. [Step] → verify: [check] +2. [Step] → verify: [check] +3. [Step] → verify: [check] +``` + +Strong success criteria enable independent looping. Weak criteria ("make it work") require constant clarification. + +## Testing Strategy + +- **RSpec** for unit and integration tests +- **Factory Bot** for test data +- **WebMock + VCR** for stubbing provider APIs +- **Pundit matchers** for authorization testing +- **Shoulda matchers** for model validations + +Every service should have corresponding test exercising Result pattern (success and failure paths). \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 00000000..1c58ebd7 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,76 @@ +# Changelog + +All notable changes to Listopia are documented in this file. + +## [0.9.0] - 2026-04-29 + +### Highlights + +- Introduced the new Listopia design system across the application, with semantic design tokens, reusable component classes, and full light/dark theme support. +- Added a persistent theme toggle backed by the new `theme` Stimulus controller and CSS variable-based Editorial/Console themes. +- Migrated major UI surfaces to the new design system, including navigation, dashboard, lists, list items, kanban, search/filtering, auth, settings, admin, notifications, invitations, AI agents, chat, connectors, modals, mailers, loading states, and shared components. +- Added modal, toast, spinner, toggle, tab, alert, card, button, form, pill, pagination, and theme-toggle primitives for reuse across views. +- Added fiber-capable Solid Queue support by using `solid_queue` from `https://github.com/crmne/solid_queue.git` on the `async-worker-execution-mode` branch. + +### Added + +- Complete AI Agents system with four-scope access control, agent resources, team memberships, runs, run steps, feedback, event triggers, scheduling hooks, token budgeting, and tool execution. +- Agent UI for browsing, creating, editing, running, and reviewing agents, including run progress, run result, status, and interaction partials. +- AI agent documentation and visual guides covering architecture, lifecycle, HITL flow, triggers, chat integration, and UI usage. +- Event system, audit trail, compliance report, and admin audit views. +- Calendar connector event sync, webhook subscriptions, conflict detection, attendee contact enrichment, people pages, and conflict UI. +- Recurring tasks with recurrence models, jobs, services, specs, and recurrence form fields. +- Generic clarifying-question and follow-up-question UI flows for chat and list creation. +- Context reuse flows after list creation, including buttons to keep or clear planning context. +- AI-generated follow-up options after list creation. +- Shared `AGENTS.md` guidance for coding agents and updated dependency documentation. + +### Changed + +- Upgraded to Rails 8.1.3 and Ruby 4.0.3. +- Upgraded Puma to 8.0.1. +- Upgraded Pagy to 43.4.2 and updated pagination integration. +- Updated current gem and JavaScript dependency documentation. +- Refactored chat orchestration toward a unified AI agent-based flow. +- Renamed and reorganized chat context concepts around `ChatUiContext`. +- Migrated mailer layouts and notification mailers to the design system. +- Updated Docker and CI/runtime dependency alignment. +- Removed unused JavaScript dependencies, including lodash and unused ProseMirror packages. + +### Fixed + +- Fixed profile edit route generation by posting the form to `profile_path` and sending Cancel back to `profile_path`. +- Fixed AI agent form tabs so fields render under the design-system `.tab-content.active` convention. +- Fixed invitations index ERB syntax and form rendering errors. +- Fixed list filter ERB syntax issues. +- Fixed duplicate stylesheet asset loading. +- Fixed duplicate `htmlElement` declaration in theme scripts. +- Fixed Stimulus theme controller registration and console theme toggle helpers. +- Fixed primary button and New List button text contrast on hover. +- Fixed AI agent active scope usage in `AgentEventDispatchJob`. +- Fixed AI agent run enum method names by using status-prefixed predicates. +- Fixed AgentContextBuilder field usage after schema changes. +- Fixed AI agent resource delete buttons to use the correct Turbo confirmation behavior. +- Fixed AI agent resource edit/delete authorization gates. +- Fixed AI agent parameter handling for both string and hash formats. +- Fixed RubyLLM tool integration so tool objects are recognized correctly. +- Fixed AiAgentRunStep token recording. +- Fixed HITL `ask_user` pausing in agent chat flow. +- Fixed List Creator agent behavior so it can create lists and items correctly. +- Fixed clarifying-question rendering, answer submission, Turbo Stream broadcasts, and parameter conversion. +- Fixed structured JSON follow-up question detection and prevented JSON payloads from leaking as user-visible messages. +- Fixed context reuse buttons and completed-context handling in chat. +- Fixed planning context references and hierarchy generation after chat-context refactors. +- Fixed generic subdivision generation so sublists can be created from arbitrary array data. +- Fixed redundant complexity analysis by reusing the first intent/complexity result. +- Fixed template variable errors in planning state and item generation progress views. +- Fixed unsupported or mismatched item-generation parameters. +- Fixed test failures from ChatContext removal. +- Fixed pre-creation planning controller import errors. +- Fixed recurring item background safety-net behavior. + +### Removed + +- Removed legacy list refinement services and specs replaced by the unified chat/list generation flow. +- Removed the old pre-creation planning JavaScript controller and obsolete planning partials. +- Removed outdated documentation files after migration into the refreshed docs structure. diff --git a/CLAUDE.md b/CLAUDE.md index 09d3bdcb..5f901190 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,34 +1,34 @@ # Claude.md - Listopia Development Quick Reference -Rails 8.1 collaborative list management with Hotwire, AI-powered chat, and real-time collaboration. +Rails 8.1: collaborative list mgmt, Hotwire, AI chat, real-time collab. ## Stack -- **Rails 8.1** with Solid Queue, Cache, Cable -- **Ruby 3.4.7** with UUID primary keys -- **PostgreSQL 15+**, RubyLLM 1.11+ (GPT-5) -- **Hotwire** (Turbo Streams + Stimulus) -- **Tailwind CSS 4.1**, Bun package manager -- **RSpec + Capybara** for testing +- Rails 8.1 w/ Solid Queue, Cache, Cable +- Ruby 3.4.7 w/ UUID PKs +- PostgreSQL 15+, RubyLLM 1.11+ (GPT-5) +- Hotwire (Turbo Streams + Stimulus) +- Tailwind CSS 4.1, Bun package mgr +- RSpec + Capybara testing ## Architecture -**Multi-Tenant with Organizations** +**Multi-Tenant w/ Organizations** - User → Organization → Team (optional) → Lists -- All queries scoped to organization (use `policy_scope`) +- All queries scoped to org (use `policy_scope`) - See: [ORGANIZATIONS_TEAMS.md](docs/ORGANIZATIONS_TEAMS.md) **Organization Context & Current Organization** -- **Critical Requirement**: Every user MUST belong to at least one organization -- Organization selection happens in the navigation bar (single place) -- Use `Current.organization` to access the current organization in any controller/service -- **Never** implement local organization selectors in views (use the nav bar selector) -- If no organization is selected, redirect to dashboard/root with alert +- **Critical**: Every user MUST belong to ≥1 org +- Org selection in nav bar (single place) +- Use `Current.organization` in any controller/service +- **Never** add local org selectors in views (nav bar only) +- No org selected? Redirect to root w/ alert - Pattern: `redirect_to root_path, alert: "Please select an organization first" unless Current.organization` - See: [ORGANIZATIONS_TEAMS.md](docs/ORGANIZATIONS_TEAMS.md) **Real-Time Collaboration** -- Prefer Turbo Streams for all reactive UI -- Use Stimulus only when Turbo can't solve it +- Prefer Turbo Streams for reactive UI +- Use Stimulus only when Turbo can't solve - See: [REAL_TIME.md](docs/REAL_TIME.md) **Authorization** @@ -36,15 +36,19 @@ Rails 8.1 collaborative list management with Hotwire, AI-powered chat, and real- - Pundit policies: always `authorize @resource` - See: [AUTHENTICATION.md](docs/AUTHENTICATION.md) -**AI Chat & List Creation** -- Unified chat interface for natural language list creation and management -- **Chat context system** for semantic state persistence across chat messages -- LLM-powered intent detection, complexity analysis, and pre-creation planning -- Domain-aware item generation: Automatically detects planning type (event, project, travel, learning, personal) and generates appropriate parent items and subdivision items -- Generic `ItemGenerationService` replaces hardcoded methods: Generates items for ANY list type and ANY subdivision strategy -- Real-time UI feedback: State indicator, progress tracking, list preview, success confirmation -- Built-in security: Prompt injection detection, content moderation -- **Documentation:** [CHAT_CONTEXT.md](docs/CHAT_CONTEXT.md) (consolidated reference), [CHAT_FLOW.md](docs/CHAT_FLOW.md), [CHAT_REQUEST_TYPES.md](docs/CHAT_REQUEST_TYPES.md), [ITEM_GENERATION.md](docs/ITEM_GENERATION.md) +**AI Chat & List Creation** (Domain-Agnostic) +- Unified chat for natural language list creation/mgmt +- Chat context for semantic state persistence across messages +- LLM intent detection, complexity analysis, pre-creation planning +- Works w/ ANY list type (events, projects, reading, courses, recipes, travel, learning, personal, etc.) + - No hardcoded domains; works equally for any domain + - `ParameterMapperService` detects subdivision strategies via LLM + - `HierarchicalItemGenerator` creates subdivisions (locations, books, modules, topics, phases, etc.) + - Parent items generated dynamically per planning domain & context + - `ItemGenerationService` generates context-appropriate items for any subdivision +- Real-time UI feedback: state indicator, progress, list preview, confirm +- Built-in security: prompt injection detection, content moderation +- Docs: [CHAT_CONTEXT.md](docs/CHAT_CONTEXT.md), [CHAT_FLOW.md](docs/CHAT_FLOW.md), [CHAT_REQUEST_TYPES.md](docs/CHAT_REQUEST_TYPES.md), [ITEM_GENERATION.md](docs/ITEM_GENERATION.md) ## Common Patterns @@ -88,29 +92,29 @@ end ``` ## Key Files -- Models with UUID: `app/models/`, foreign keys as UUID -- Authorization policies: `app/policies/` +- Models w/ UUID: `app/models/`, FKs as UUID +- Auth policies: `app/policies/` - Complex logic: `app/services/` (inherit ApplicationService) -- Tests: RSpec with Factory Bot, Faker -- Database: PostgreSQL with pgcrypto, plpgsql +- Tests: RSpec w/ Factory Bot, Faker +- Database: PostgreSQL w/ pgcrypto, plpgsql - Uses `db/structure.sql` (not schema.rb) — enforced by user change tracking service - - All models annotated with `annotate` gem: see Schema Information at top of each model + - All models annotated w/ `annotate` gem: see Schema Info at top of each model ## Pagination (Pagy v43+) **CRITICAL: Pagy v43+ has major breaking changes from previous versions** -This project uses **Pagy v43.3.2**, which has completely restructured its API. Do NOT rely on old Pagy documentation or examples from pre-v43 versions. +Project uses Pagy v43.3.2 w/ restructured API. Don't rely on old Pagy docs or pre-v43 examples. **Key Differences:** -- ❌ No `pagy_nav` method (was common in older versions) -- ✅ Use `series_nav` for numeric pagination (requires `include Pagy::NumericHelpers` in helpers) -- ❌ Old helper methods are renamed/removed +- ❌ No `pagy_nav` method (old) +- ✅ Use `series_nav` for numeric pagination (need `include Pagy::NumericHelpers` in helpers) +- ❌ Old helper methods renamed/removed - ✅ View helpers must be explicitly included: `include Pagy::NumericHelpers` in ApplicationHelper **Available Pagy v43+ View Helpers** (from `Pagy::NumericHelpers`): -- `series_nav(@pagy)` - Numeric pagination with previous/next links -- `series_nav_js(@pagy)` - JavaScript-powered pagination +- `series_nav(@pagy)` - Numeric pagination w/ prev/next links +- `series_nav_js(@pagy)` - JS-powered pagination - `info_tag(@pagy)` - Shows "Displaying X of Y" - `previous_tag(@pagy)` - Previous page link - `input_nav_js(@pagy)` - Jump to page input @@ -135,24 +139,24 @@ include Pagy::NumericHelpers # Adds series_nav, info_tag, etc. for views <%= series_nav(@pagy) %> ``` -**Before implementing any Pagy features:** -1. Check [Pagy v43 official docs](https://ddnexus.github.io/pagy/): Method names and APIs are NOT compatible with older tutorials +**Before implementing Pagy features:** +1. Check [Pagy v43 official docs](https://ddnexus.github.io/pagy/): Method names & APIs NOT compatible w/ older tutorials 2. Look for existing usage in `app/views/` to match patterns -3. If unsure about a method name, check `lib/pagy/toolbox/helpers/loaders.rb` for available methods +3. Unsure about method name? Check `lib/pagy/toolbox/helpers/loaders.rb` for available methods ## Development **Ruby LSP Integration** -- Ruby LSP plugin is installed in Claude Code -- Use LSP tools for code navigation and analysis: - - `goToDefinition` - Find where a symbol is defined - - `findReferences` - Find all usages of a symbol - - `hover` - Get type info and documentation - - `documentSymbol` - List all symbols in a file +- Ruby LSP plugin installed in Claude Code +- Use LSP tools for code navigation & analysis: + - `goToDefinition` - Find symbol definition + - `findReferences` - Find all usages + - `hover` - Get type info & docs + - `documentSymbol` - List all symbols in file - `workspaceSymbol` - Search symbols across codebase - - `goToImplementation` - Find implementations of interfaces/methods + - `goToImplementation` - Find implementations - `prepareCallHierarchy` / `incomingCalls` / `outgoingCalls` - Analyze call chains -- Useful for understanding service dependencies, model relationships, and controller flows +- Useful for understanding service dependencies, model relationships, controller flows **Quick Setup** ```bash @@ -172,19 +176,22 @@ bundle exec brakeman # Security |-------|----------| | N+1 Queries | Use `includes`, `preload`, or `joins` | | Auth Failed | Call `authorize @resource` after load | -| Turbo not working | Respond with `format.turbo_stream` | +| Turbo not working | Respond w/ `format.turbo_stream` | | Test DB issues | `RAILS_ENV=test rails db:reset` | -| No organization selected | User must select one in nav bar; redirect if `Current.organization` is nil | -| Cross-org data leak | Always scope queries with `organization_id`; use `Current.organization` | -| Organization selector on view | Don't add local selectors; use nav bar only | +| No org selected | Select in nav bar; redirect if `Current.organization` nil | +| Cross-org data leak | Always scope queries w/ `organization_id`; use `Current.organization` | +| Org selector on view | Don't add local selectors; nav bar only | ## Detailed Docs -**Chat Context System** (Consolidated reference) -- [CHAT_CONTEXT.md](docs/CHAT_CONTEXT.md) - Complete system overview: architecture, services, integration, UI components, testing & migration +**Architecture & Design Principles** (START HERE) +- [ARCHITECTURE_GENERIC_DESIGN.md](docs/ARCHITECTURE_GENERIC_DESIGN.md) - **Critical**: Domain-agnostic design for ANY list type. Required reading. -**Chat System** (Integration with chat flow) -- [CHAT_FLOW.md](docs/CHAT_FLOW.md) - Complete message flow & state machine +**Chat Context System** (Consolidated Reference) +- [CHAT_CONTEXT.md](docs/CHAT_CONTEXT.md) - System overview: architecture, services, integration, UI components, testing & migration + +**Chat System** (Integration w/ chat flow) +- [CHAT_FLOW.md](docs/CHAT_FLOW.md) - Message flow & state machine - [CHAT_REQUEST_TYPES.md](docs/CHAT_REQUEST_TYPES.md) - Simple/complex/nested list handling - [CHAT_MODEL_SELECTION.md](docs/CHAT_MODEL_SELECTION.md) - Model selection strategy - [CHAT_FEATURES.md](docs/CHAT_FEATURES.md) - How to add features @@ -208,14 +215,12 @@ bundle exec brakeman # Security ## Useful Commands ```bash rails db:create db:migrate # Setup dev DB -RAILS_ENV=test rails db:reset # Reset test DB bundle exec rspec # Run tests rails g stimulus ControllerName # Create Stimulus controller -kamal deploy # Deploy changes ``` ## External Resources - [Rails 8 Guides](https://guides.rubyonrails.org/) - [Hotwire](https://hotwired.dev/) - [Pundit](https://github.com/varvet/pundit) -- [RSpec Rails](https://github.com/rspec/rspec-rails) +- [RSpec Rails](https://github.com/rspec/rspec-rails) \ No newline at end of file diff --git a/CLAUDE.original.md b/CLAUDE.original.md new file mode 100644 index 00000000..eb3464d7 --- /dev/null +++ b/CLAUDE.original.md @@ -0,0 +1,228 @@ +# Claude.md - Listopia Development Quick Reference + +Rails 8.1 collaborative list management with Hotwire, AI-powered chat, and real-time collaboration. + +## Stack +- **Rails 8.1** with Solid Queue, Cache, Cable +- **Ruby 3.4.7** with UUID primary keys +- **PostgreSQL 15+**, RubyLLM 1.11+ (GPT-5) +- **Hotwire** (Turbo Streams + Stimulus) +- **Tailwind CSS 4.1**, Bun package manager +- **RSpec + Capybara** for testing + +## Architecture + +**Multi-Tenant with Organizations** +- User → Organization → Team (optional) → Lists +- All queries scoped to organization (use `policy_scope`) +- See: [ORGANIZATIONS_TEAMS.md](docs/ORGANIZATIONS_TEAMS.md) + +**Organization Context & Current Organization** +- **Critical Requirement**: Every user MUST belong to at least one organization +- Organization selection happens in the navigation bar (single place) +- Use `Current.organization` to access the current organization in any controller/service +- **Never** implement local organization selectors in views (use the nav bar selector) +- If no organization is selected, redirect to dashboard/root with alert +- Pattern: `redirect_to root_path, alert: "Please select an organization first" unless Current.organization` +- See: [ORGANIZATIONS_TEAMS.md](docs/ORGANIZATIONS_TEAMS.md) + +**Real-Time Collaboration** +- Prefer Turbo Streams for all reactive UI +- Use Stimulus only when Turbo can't solve it +- See: [REAL_TIME.md](docs/REAL_TIME.md) + +**Authorization** +- Rails 8 `has_secure_password`, magic link tokens +- Pundit policies: always `authorize @resource` +- See: [AUTHENTICATION.md](docs/AUTHENTICATION.md) + +**AI Chat & List Creation** (Fully Domain-Agnostic) +- Unified chat interface for natural language list creation and management +- **Chat context system** for semantic state persistence across chat messages +- LLM-powered intent detection, complexity analysis, and pre-creation planning +- **Truly Generic & Domain-Agnostic**: Works with ANY list type (events, projects, reading lists, courses, recipes, travel, learning, personal, etc.) + - Does NOT hardcode specific domains: tests may use events/vacations repeatedly, but system works equally well for any domain + - `ParameterMapperService` uses LLM to intelligently detect subdivision strategies from user input + - `HierarchicalItemGenerator` creates subdivisions based on detected strategy (locations, books, modules, topics, phases, etc.) + - Parent items generated dynamically based on planning domain and request context + - `ItemGenerationService` generates context-appropriate items for any subdivision type +- Real-time UI feedback: State indicator, progress tracking, list preview, success confirmation +- Built-in security: Prompt injection detection, content moderation +- **Documentation:** [CHAT_CONTEXT.md](docs/CHAT_CONTEXT.md) (consolidated reference), [CHAT_FLOW.md](docs/CHAT_FLOW.md), [CHAT_REQUEST_TYPES.md](docs/CHAT_REQUEST_TYPES.md), [ITEM_GENERATION.md](docs/ITEM_GENERATION.md) + +## Common Patterns + +**Query Scoping (Critical)** +```ruby +policy_scope(List) # Returns only user's org lists +current_organization.lists # Access through org +.where(organization_id: current_user.organizations.select(:id)) # Explicit filter +``` + +**Organization Context (Critical)** +```ruby +# In controllers - ensure organization is selected +@organization = Current.organization +redirect_to root_path, alert: "Select organization" unless @organization + +# In models/services - access the current context +Event.where(organization_id: Current.organization.id) + +# For admin/audit features - verify org is set +redirect_to admin_root_path, alert: "Please select an organization first" unless @organization +``` + +**Authorization** +```ruby +authorize @list # Use Pundit on every action +``` + +**Turbo Stream Response** +```erb +<%= turbo_stream.replace(@list) { render "list", list: @list } %> +``` + +**Service Pattern** +```ruby +class MyService < ApplicationService + def call + # Return success(data: ...) or failure(errors: ...) + end +end +``` + +## Key Files +- Models with UUID: `app/models/`, foreign keys as UUID +- Authorization policies: `app/policies/` +- Complex logic: `app/services/` (inherit ApplicationService) +- Tests: RSpec with Factory Bot, Faker +- Database: PostgreSQL with pgcrypto, plpgsql + - Uses `db/structure.sql` (not schema.rb) — enforced by user change tracking service + - All models annotated with `annotate` gem: see Schema Information at top of each model + +## Pagination (Pagy v43+) + +**CRITICAL: Pagy v43+ has major breaking changes from previous versions** + +This project uses **Pagy v43.3.2**, which has completely restructured its API. Do NOT rely on old Pagy documentation or examples from pre-v43 versions. + +**Key Differences:** +- ❌ No `pagy_nav` method (was common in older versions) +- ✅ Use `series_nav` for numeric pagination (requires `include Pagy::NumericHelpers` in helpers) +- ❌ Old helper methods are renamed/removed +- ✅ View helpers must be explicitly included: `include Pagy::NumericHelpers` in ApplicationHelper + +**Available Pagy v43+ View Helpers** (from `Pagy::NumericHelpers`): +- `series_nav(@pagy)` - Numeric pagination with previous/next links +- `series_nav_js(@pagy)` - JavaScript-powered pagination +- `info_tag(@pagy)` - Shows "Displaying X of Y" +- `previous_tag(@pagy)` - Previous page link +- `input_nav_js(@pagy)` - Jump to page input + +**How to Use:** +```erb + + +<%= series_nav(@pagy) %> +``` + +**Common Patterns:** +```ruby +# In controller: +include Pagy::Method # Adds pagy method for backend + +# In helper: +include Pagy::NumericHelpers # Adds series_nav, info_tag, etc. for views + +# In view: +@pagy, @items = pagy(collection) +<%= series_nav(@pagy) %> +``` + +**Before implementing any Pagy features:** +1. Check [Pagy v43 official docs](https://ddnexus.github.io/pagy/): Method names and APIs are NOT compatible with older tutorials +2. Look for existing usage in `app/views/` to match patterns +3. If unsure about a method name, check `lib/pagy/toolbox/helpers/loaders.rb` for available methods + +## Development + +**Ruby LSP Integration** +- Ruby LSP plugin is installed in Claude Code +- Use LSP tools for code navigation and analysis: + - `goToDefinition` - Find where a symbol is defined + - `findReferences` - Find all usages of a symbol + - `hover` - Get type info and documentation + - `documentSymbol` - List all symbols in a file + - `workspaceSymbol` - Search symbols across codebase + - `goToImplementation` - Find implementations of interfaces/methods + - `prepareCallHierarchy` / `incomingCalls` / `outgoingCalls` - Analyze call chains +- Useful for understanding service dependencies, model relationships, and controller flows + +**Quick Setup** +```bash +rails db:create db:migrate +bundle exec rspec +bun install && bun run build:css +``` + +**Code Quality** +```bash +bundle exec rubocop --fix # Style +bundle exec brakeman # Security +``` + +**Common Issues** +| Issue | Solution | +|-------|----------| +| N+1 Queries | Use `includes`, `preload`, or `joins` | +| Auth Failed | Call `authorize @resource` after load | +| Turbo not working | Respond with `format.turbo_stream` | +| Test DB issues | `RAILS_ENV=test rails db:reset` | +| No organization selected | User must select one in nav bar; redirect if `Current.organization` is nil | +| Cross-org data leak | Always scope queries with `organization_id`; use `Current.organization` | +| Organization selector on view | Don't add local selectors; use nav bar only | + +## Detailed Docs + +**Architecture & Design Principles** (START HERE) +- [ARCHITECTURE_GENERIC_DESIGN.md](docs/ARCHITECTURE_GENERIC_DESIGN.md) - **Critical**: Explains how the system is domain-agnostic and works for ANY list type. Required reading for contributors. + +**Chat Context System** (Consolidated reference) +- [CHAT_CONTEXT.md](docs/CHAT_CONTEXT.md) - Complete system overview: architecture, services, integration, UI components, testing & migration + +**Chat System** (Integration with chat flow) +- [CHAT_FLOW.md](docs/CHAT_FLOW.md) - Complete message flow & state machine +- [CHAT_REQUEST_TYPES.md](docs/CHAT_REQUEST_TYPES.md) - Simple/complex/nested list handling (domain-agnostic) +- [CHAT_MODEL_SELECTION.md](docs/CHAT_MODEL_SELECTION.md) - Model selection strategy +- [CHAT_FEATURES.md](docs/CHAT_FEATURES.md) - How to add features + +**Core Features** +- [Database & Queries](docs/DATABASE.md) +- [Testing](docs/TESTING.md) +- [Organizations & Teams](docs/ORGANIZATIONS_TEAMS.md) +- [Real-Time Collaboration](docs/REAL_TIME.md) +- [Authentication](docs/AUTHENTICATION.md) +- [Notifications](docs/NOTIFICATION.md) + +**Performance** +- [Performance Gems Setup](docs/PERFORMANCE_GEMS_SETUP.md) +- [N+1 Query Fixes](docs/n_plus_one_fixes.md) + +**Other** +- [Search & RAG](docs/RAG_SEMANTIC_SEARCH.md) +- [Documentation Index](docs/README.md) + +## Useful Commands +```bash +rails db:create db:migrate # Setup dev DB +RAILS_ENV=test rails db:reset # Reset test DB +bundle exec rspec # Run tests +rails g stimulus ControllerName # Create Stimulus controller +kamal deploy # Deploy changes +``` + +## External Resources +- [Rails 8 Guides](https://guides.rubyonrails.org/) +- [Hotwire](https://hotwired.dev/) +- [Pundit](https://github.com/varvet/pundit) +- [RSpec Rails](https://github.com/rspec/rspec-rails) diff --git a/DEPENDENCIES.md b/DEPENDENCIES.md index 611215af..fe12e88b 100644 --- a/DEPENDENCIES.md +++ b/DEPENDENCIES.md @@ -41,12 +41,15 @@ Overview of all dependencies used in the Listopia. ### AI & Search Features - **ruby_llm** (~> 1.8+) - LLM integration for AI-powered chat, intent detection, and embeddings +- **ruby_llm-schema** - Structured output support with schemas - **neighbor** - Vector similarity search for semantic embeddings (pgvector integration) - **multi_json** - JSON processing for MCP and API responses +- **jwt** - JWT token handling for OAuth ### Markdown & Content Rendering - **redcarpet** - Markdown rendering for rich text - **rouge** - Code syntax highlighting in markdown +- **csv** - CSV processing (required for Ruby 3.4+) ### Development Tools - **debug** - Debugger @@ -56,6 +59,10 @@ Overview of all dependencies used in the Listopia. - **annotaterb** - Model annotations - **bullet** - N+1 query detection - **dotenv-rails** - Environment variables +- **rack-mini-profiler** - Performance profiling and insights +- **stackprof** - Sampling CPU profiler for bottleneck identification +- **prosopite** - Object allocation and memory tracking +- **memory_profiler** - Detailed memory usage reports ### Testing - **rspec-rails** - Testing framework @@ -66,9 +73,11 @@ Overview of all dependencies used in the Listopia. - **cuprite** - Headless browser testing - **vcr** (~> 6.2) - HTTP request recording and playback - **webmock** (~> 3.18) - HTTP mocking for tests +- **simplecov** (~> 0.22.0) - Code coverage reporting - **database_cleaner-active_record** - Test database cleanup - **rails-controller-testing** - Controller testing helpers - **shoulda-matchers** (~> 7.0) - RSpec matchers for model testing +- **pundit-matchers** (~> 4.0.0) - Pundit authorization testing - **rspec-retry** - Retry flaky tests - **timecop** - Time-based testing utilities @@ -84,34 +93,23 @@ Overview of all dependencies used in the Listopia. ### Core Libraries - **@hotwired/stimulus** (^3.2.2) - Stimulus framework -- **@hotwired/turbo-rails** (^8.0.20) - Turbo framework +- **@hotwired/turbo-rails** (^8.0.23) - Turbo framework ### CSS -- **tailwindcss** (^4.1.17) - Utility-first CSS framework (upgraded from 4.1.16) -- **@tailwindcss/cli** (^4.1.17) - Tailwind CLI (upgraded from 4.1.16) +- **tailwindcss** (^4.2.4) - Utility-first CSS framework +- **@tailwindcss/cli** (^4.2.4) - Tailwind CLI ### UI Components & Interactions -- **sortablejs** (^1.15.6) - Drag and drop library +- **sortablejs** (^1.15.7) - Drag and drop library - **@stimulus-components/character-counter** (^5.1.0) - Character counter - **@stimulus-components/notification** (^3.0.0) - Notifications - **@stimulus-components/reveal** (^5.0.0) - Reveal/hide toggle - **@stimulus-components/scroll-to** (^5.0.1) - Scroll behavior - **stimulus-textarea-autogrow** (^4.1.0) - Auto-growing textarea -### Rich Text Editor -- **prosemirror-model** (^1.25.4) - ProseMirror document model -- **prosemirror-state** (^1.4.4) - Editor state management -- **prosemirror-view** (^1.41.4) - Editor view and DOM binding -- **prosemirror-transform** (^1.10.5) - Document transformations -- **prosemirror-commands** (^1.7.1) - Editor commands -- **prosemirror-keymap** (^1.2.3) - Keyboard handling -- **prosemirror-schema-list** (^1.5.1) - List schema support -- **prosemirror-markdown** (^1.13.2) - Markdown serialization - ### Utilities -- **marked** (^17.0.1) - Markdown parser (upgraded from 16.4.1) +- **marked** (^17.0.6) - Markdown parser - **highlight.js** (^11.11.1) - Code syntax highlighting -- **lodash** (^4.17.21) - Utility library --- @@ -120,56 +118,67 @@ Overview of all dependencies used in the Listopia. ### New Gems (Recently Added) **AI & Search:** -- `neighbor` - Vector similarity search for pgvector embeddings -- `redcarpet` - Markdown rendering for rich text content -- `rouge` - Code syntax highlighting in markdown +- `ruby_llm-schema` - Structured output with schemas for LLM responses +- `jwt` - JWT token handling for OAuth -**Database & Content:** -- `friendly_id` - Human-readable URL slugs -- `image_processing` - Image variants and transformations +**Development Tools:** +- `rack-mini-profiler` - HTTP request profiling and caching analysis +- `stackprof` - Sampling-based CPU profiler for bottleneck identification +- `prosopite` - Object allocation and memory leak detection +- `memory_profiler` - Detailed memory usage analysis **Testing:** -- `cuprite` - Headless browser testing -- `vcr` - HTTP request recording/playback for tests -- `webmock` - HTTP mocking for tests -- `timecop` - Time-based testing utilities +- `simplecov` - Code coverage reporting +- `pundit-matchers` - Pundit authorization policy matchers ### Upgraded Dependencies **Ruby Gems:** - `ruby_llm` → 1.8+ (Enhanced LLM integration with embeddings) - `shoulda-matchers` → 7.0 (Better model testing) +- `solid_queue` → async-worker-execution-mode branch (Fiber-based async execution) **JavaScript:** -- `tailwindcss` → 4.1.17 (from 4.1.16) -- `@tailwindcss/cli` → 4.1.17 (from 4.1.16) -- `marked` → 17.0.1 (from 16.4.1) +- `@hotwired/turbo-rails` → 8.0.23 (from 8.0.20) +- `tailwindcss` → 4.2.4 (from 4.1.17) +- `@tailwindcss/cli` → 4.2.4 (from 4.1.17) +- `marked` → 17.0.6 (from 17.0.1) +- `sortablejs` → 1.15.7 (from 1.15.6) ### New Feature Support **AI-Powered Chat:** -- Integrated `ruby_llm` 1.8+ for intent detection, embeddings, and LLM calls -- Added `neighbor` for vector similarity in semantic search +- Integrated `ruby_llm` 1.8+ with `ruby_llm-schema` for structured output +- LLM intent detection, embeddings, and schema validation +- `neighbor` for vector similarity in semantic search - Full RAG (Retrieval-Augmented Generation) support **Rich Text & Content:** -- `ProseMirror` libraries for advanced rich text editing -- `redcarpet` + `rouge` for beautiful markdown rendering +- `redcarpet` + `rouge` for beautiful markdown rendering with syntax highlighting - `image_processing` for image variants and optimization +- `friendly_id` for human-readable URL slugs + +**Performance & Profiling:** +- `rack-mini-profiler` for HTTP request analysis +- `stackprof`, `prosopite`, `memory_profiler` for bottleneck identification +- `bullet` for N+1 query detection and prevention **Testing Infrastructure:** - `cuprite` for headless browser automation - `vcr` + `webmock` for API testing (especially for LLM mocks) - `timecop` for time-dependent feature testing +- `simplecov` for code coverage reporting +- `pundit-matchers` for authorization testing --- ## Dependency Statistics -- **Total Ruby Gems:** 50+ -- **Total JavaScript Dependencies:** 20+ -- **Testing Libraries:** 13 (comprehensive test suite support) -- **AI/ML Libraries:** 2 (RubyLLM, Neighbor) +- **Total Ruby Gems:** 55+ +- **Total JavaScript Dependencies:** 11 +- **Testing Libraries:** 15 (comprehensive test suite support) +- **Performance Profiling:** 4 (rack-mini-profiler, stackprof, prosopite, memory_profiler) +- **AI/ML Libraries:** 3 (RubyLLM, RubyLLM-Schema, Neighbor) - **UI Framework Libraries:** 3 (Stimulus, Turbo, Tailwind) ## Notes @@ -178,3 +187,6 @@ Overview of all dependencies used in the Listopia. - Development/test gems are properly grouped - pgvector PostgreSQL extension required for semantic search (via `neighbor` gem) - Bun is the preferred package manager (faster than npm/yarn) +- `solid_queue` uses async-worker-execution-mode branch for Fiber-based async execution +- Performance profiling tools (rack-mini-profiler, stackprof, prosopite) enabled for development +- `ruby_llm-schema` provides structured output validation for LLM responses diff --git a/DESIGN_SYSTEM_MIGRATION.md b/DESIGN_SYSTEM_MIGRATION.md new file mode 100644 index 00000000..53c18195 --- /dev/null +++ b/DESIGN_SYSTEM_MIGRATION.md @@ -0,0 +1,433 @@ +# Listopia → Secure Mail Design System Migration + +**Design System**: Editorial (light) + Console (dark) themes from Secure Mail +**Timeline**: Phased approach (core → features → polish) +**Key Tech**: Tailwind CSS v4 with CSS variables, Turbo Streams, Stimulus + +--- + +## Progress Summary + +**✅ Phases Completed:** +- Phase 1.1 ✅ Design tokens setup (commit a1e18d6) +- Phase 1.2 ✅ Base styles & resets (commit 6972044) +- Phase 1.3 ✅ Utility classes (colors, typography, spacing) +- Phase 1.4 ✅ Typography system +- Phase 2 ✅ Navigation & headers (commit 01c5c5d) +- Phase 2.2-2.4 ✅ Cards, forms, buttons (commit a2efcca) +- Phase 4.1 ✅ Lists index & grid (commit 8defa86) +- Phase 4.2 ✅ List cards with design system +- Phase 4.3 ✅ List show page & item rows +- Phase 4.4 ✅ List item editor (modal form) +- Phase 4.5 ✅ Quick add form & recurrence + +**Current Status:** +- Design tokens & theme system fully operational +- All components (cards, buttons, forms, alerts, pills) available +- Navigation redesigned with Editorial/Console themes +- Lists index/grid views updated to design system +- List show/edit views completely redesigned +- Quick add form with custom selects styled +- Theme toggle working with localStorage persistence + +**Completed Templates:** +- app/views/shared/_navigation.html.erb ✅ +- app/views/lists/index.html.erb ✅ +- app/views/lists/_list_card.html.erb ✅ +- app/views/lists/show.html.erb ✅ +- app/views/lists/_header.html.erb ✅ +- app/views/list_items/_item.html.erb ✅ +- app/views/list_items/edit.html.erb ✅ +- app/views/list_items/_quick_add_form.html.erb ✅ +- app/views/shared/_item_type_select.html.erb ✅ +- app/views/list_items/_recurrence_fields.html.erb ✅ + +**Remaining Priority:** +- Phase 5: Search & filtering +- Phase 8-12: Chat, email, admin, testing & polish + +--- + +## PHASE 1: Foundation & Theme Infrastructure + +### 1.1 Setup Design Tokens ✅ COMPLETE +- [x] Copy `design-system/tokens.css` to `app/assets/stylesheets/design-system/` +- [x] Update `application.tailwind.css` to import new tokens +- [x] Create `app/views/layouts/application.html.erb` wrapper with `data-theme="editorial"` attribute +- [x] Add theme toggle controller in `app/javascript/controllers/theme_controller.js` +- [x] Create `app/assets/stylesheets/design-system/utilities.css` with semantic patterns +- [x] Create `app/views/shared/_theme_toggle.html.erb` component +- [x] Add theme toggle to navigation +- [ ] Test in dev browser (theme toggle + localStorage) + +### 1.2 Update Base Styles & Resets ✅ COMPLETE +- [x] Update `application.tailwind.css` to use design tokens +- [x] Replace hardcoded colors with CSS variables (--color-*, --font-*) +- [x] Update custom-select-dropdown styling with design tokens +- [x] Add global html + body styles with smooth transitions +- [x] Style scrollbars with design colors +- [x] Update link colors and ::selection with design tokens + +### 1.3 Create Utility Classes for Common Patterns ✅ COMPLETE +- [x] Add `.eyebrow`, `.kbd`, `.status-dot`, `.mark` utility classes +- [x] Add `.section-divider` for labeled horizontal rules +- [x] Add `.text-ink*` variants (ink, ink-muted, ink-subtle, ink-faint, ink-inverse) +- [x] Add `.bg-surface*` variants (surface, surface-raised, surface-sunken) +- [x] Create semantic color utilities (.text-success, .text-warning, .text-danger) +- [x] Add typography presets (.t-display-l, .t-body, .t-meta, .t-eyebrow) + +### 1.4 Typography System Integration ✅ COMPLETE +- [x] Map design system font sizes (--text-2xs through --text-3xl) +- [x] Configure font families as CSS variables (--font-display, --font-body, --font-mono, etc.) +- [x] Create size scale helpers: `.text-display-l`, `.text-display-m`, `.text-display-s`, `.text-body-l`, `.text-body`, `.text-body-s`, `.text-meta` +- [x] Font weights mapped (--font-weight-regular through --font-weight-bold) +- [x] Leading/line-height tokens available (--leading-tight through --leading-relaxed) + +--- + +## PHASE 2: Core Components & Layouts ✅ COMPLETE + +### 2.1 Navigation & Header ✅ COMPLETE +- [x] Update navigation with new surface/rule colors +- [x] Redesign user menu dropdown using new surface-raised styles +- [x] Update theme toggle button styling +- [x] Replace gray/blue hardcoded colors with design tokens +- [x] Update all navigation links with hover:text-accent +- [x] Ensure Turbo Stream updates to nav work with new styles + +### 2.2 Cards & Surfaces ✅ COMPLETE +- [x] Create `.card` component using surface-raised with proper rule borders +- [x] Style `.card-header` with typography sizing +- [x] Design `.card-body` with proper padding (using spacing tokens) +- [x] Create `.card-footer` with rules +- [x] Style `.card:hover` state with shadow-pop +- [x] Component ready for template integration + +### 2.3 Forms & Inputs ✅ COMPLETE +- [x] Update `` base styles (surface-sunken bg, ink text, rule borders) +- [x] Style `:focus` states with accent color + box-shadow +- [x] Create `.form-group` wrapper with proper spacing +- [x] Style `