diff --git a/README.md b/README.md index 71ce50d..ad8ef50 100644 --- a/README.md +++ b/README.md @@ -33,6 +33,7 @@ Master (server) environment variables | `MASTER_API_KEY` | Secret key for API authentication (optional) | (disabled if empty) | | `MASTER_LOG_LEVEL`| Logging verbosity (`debug`, `info`, `warn`, `error`) | `info` | | `MASTER_SHUTDOWN_TIMEOUT` | Graceful shutdown timeout (duration string) | `30s` | +| `DASHBOARD_PASSWORD` | Optional password for dashboard access | (unprotected if empty) | | `MASTER_STALE_JOB_THRESHOLD` | Stale threshold (seconds) after which a processing job is considered abandoned by the background cleanup | `604800` (7 days) | | `MASTER_CLEANUP_INTERVAL` | How often (seconds) the master runs the stale-job cleanup background task | `21600` (6 hours) | @@ -96,6 +97,16 @@ Endpoints (except `/health`) require an `X-API-KEY` header if `MASTER_API_KEY` i **Dashboard Security:** The web-based dashboard (Phase 10) will be protected by a simple password authentication mechanism controlled via an environment variable (`DASHBOARD_PASSWORD`), without requiring a database for session management. +### Dashboards & Monitoring +The project includes a built-in dashboard for real-time fleet monitoring and historical analytics. + +- **Access:** Visit `http://localhost:8080/dashboard` in your browser. +- **Security:** Set the `DASHBOARD_PASSWORD` environment variable to protect access. Session management uses signed cookies. +- **Real-time Updates:** Powered by WebSockets (HTMX + `github.com/coder/websocket`) for live throughput and worker status updates. +- **Tiers:** Aggregates statistics into daily, monthly, and lifetime snapshots for long-term tracking. + +See [Dashboard Development Guide](docs/api/ui-development.md) for more technical details. + ## Repository Layout ``` eth-scanner/ diff --git a/docs/api/ui-development.md b/docs/api/ui-development.md new file mode 100644 index 0000000..5a45c6e --- /dev/null +++ b/docs/api/ui-development.md @@ -0,0 +1,76 @@ +# Dashboard & UI Development Guide + +This document outlines the architecture and development patterns for the EthScanner monitoring dashboard (Phase 10). + +## Tech Stack +- **Go `html/template`**: Server-side rendering for type-safety and performance. +- **Tailwind CSS**: Utility-first CSS via a standalone CLI or CDN. +- **HTMX**: High-performance AJAX interaction with zero-JavaScript boilerplate. +- **WebSockets via `github.com/coder/websocket`**: Used for real-time updates of throughput and worker stats. + +## Directory Structure +- `go/internal/server/ui/templates/`: HTML templates. +- `go/internal/server/ui/static/`: Static assets (images, CSS, frontend libraries). +- `go/internal/server/ui_handlers.go`: Backend Go handlers that serve the dashboard pages. +- `go/internal/server/hub.go`: WebSocket message hub for broadcasting events. + +## Key HTMX Patterns + +### 1. Polling & Partial Swaps +The dashboard uses HTMX to refresh specific fragments of the page (like a worker's last seen time) without a full page reload. + +```html +
+ +
+``` + +### 2. Real-time Updates via WebSockets (OOB Swaps) +For truly real-time updates (e.g., "live throughput"), we use HTMX Out-of-Band (OOB) swaps triggered by WebSocket messages. +When a worker submits a checkpoint, the `hub.go` broadcasts a fragment for the "Cumulative Throughput" widget. + +```html + +
+ 1,234,567 keys/sec +
+``` + +## How to Add a New Statistic or Component + +### Step 1: Add the SQL Query +Add the necessary aggregation logic in `go/internal/database/sql/queries.sql`. +```sql +-- name: GetTopWorkers :many +SELECT worker_id, total_keys FROM worker_stats_lifetime ORDER BY total_keys DESC LIMIT 5; +``` +Run `make sqlc` in the `go/` directory to generate the Go code. + +### Step 2: Fetch Data in the Handler +Update `go/internal/server/ui_handlers.go` to fetch the data and include it in the template context. + +### Step 3: Create a Template Component +Add a new template fragment in `go/internal/server/ui/templates/fragments.html` or a new file. + +### Step 4: Add to the Dashboard +Include the component in `go/internal/server/ui/templates/index.html` (the main dashboard page). + +## Troubleshooting + +### WebSocket/CORS Issues +- Ensure the `MASTER_PORT` environment variable is correctly set; the frontend calculates the `ws://` URL based on `location.host`. +- If running behind a reverse proxy (like Nginx), ensure `Upgrade` and `Connection` headers are forwarded correctly. + +### Template Formatting +- Changes to templates require the Go server to be restarted for the changes to take effect (unless using a development auto-reloader). +- Ensure all Go variables being passed to the templates are capitalized (exported). + +## CSS Development (Tailwind) +The dashboard uses Tailwind CSS. For rapid development, the project includes a script in `go/Makefile` to monitor and rebuild the CSS bundle. + +```bash +cd go +make tailwind-watch +``` + +This watches the `go/internal/server/ui/templates/` folder and generates a minified `go/internal/server/ui/static/css/output.css`. diff --git a/docs/tasks/backlog/P10-T120.md b/docs/tasks/done/P10-T120.md similarity index 87% rename from docs/tasks/backlog/P10-T120.md rename to docs/tasks/done/P10-T120.md index cfd32f4..e611194 100644 --- a/docs/tasks/backlog/P10-T120.md +++ b/docs/tasks/done/P10-T120.md @@ -1,8 +1,10 @@ # P10-T120: Implement "Error Log" and alert notifications **Phase:** P10 - Dashboard & Monitoring UI -**Status:** Not Started -**Priority:** Medium +**Status:** Deferred +**Deferred on:** 2026-02-23 +**Deferred reason:** Not doing this now; revisit later when monitoring priorities increase. +**Priority:** Medium **Dependencies:** [P10-T050] **Estimated Effort:** Medium diff --git a/docs/tasks/backlog/P10-T130.md b/docs/tasks/done/P10-T130.md similarity index 89% rename from docs/tasks/backlog/P10-T130.md rename to docs/tasks/done/P10-T130.md index 52302ff..1161256 100644 --- a/docs/tasks/backlog/P10-T130.md +++ b/docs/tasks/done/P10-T130.md @@ -1,7 +1,9 @@ # P10-T130: Add export functionality (CSV/JSON download) **Phase:** P10 - Dashboard & Monitoring UI -**Status:** Not Started +**Status:** Deferred +**Deferred on:** 2026-02-23 +**Deferred reason:** Not doing this now; revisit later when export needs arise. **Priority:** Low **Dependencies:** [P10-T080] **Estimated Effort:** Small diff --git a/docs/tasks/backlog/P10-T140.md b/docs/tasks/done/P10-T140.md similarity index 66% rename from docs/tasks/backlog/P10-T140.md rename to docs/tasks/done/P10-T140.md index b260cd8..ffcb5d0 100644 --- a/docs/tasks/backlog/P10-T140.md +++ b/docs/tasks/done/P10-T140.md @@ -1,7 +1,7 @@ # P10-T140: Write documentation for dashboard setup and local development **Phase:** P10 - Dashboard & Monitoring UI -**Status:** Not Started +**Status:** Completed **Priority:** Medium **Dependencies:** All P10 tasks **Estimated Effort:** Small @@ -14,10 +14,10 @@ Finalize the UI phase by documenting how to configure, run, and develop for the ## Acceptance Criteria -- [ ] Update `README.md` with instructions for `DASHBOARD_PASSWORD`. -- [ ] Create `docs/api/ui-development.md` for developers (template locations, HTMX patterns). -- [ ] Add troubleshooting section for common WebSocket/CORS issues. -- [ ] Document how to add a new statistic or component to the dashboard. +- [x] Update `README.md` with instructions for `DASHBOARD_PASSWORD`. +- [x] Create `docs/api/ui-development.md` for developers (template locations, HTMX patterns). +- [x] Add troubleshooting section for common WebSocket/CORS issues. +- [x] Document how to add a new statistic or component to the dashboard. ## Implementation Notes