Skip to content

DEVELOPMENT.md

Latest commit

 

History

History
281 lines (231 loc) · 21.5 KB

DEVELOPMENT.md

File metadata and controls

281 lines (231 loc) · 21.5 KB

js8web — Development Documentation

Project Overview

js8web is a web-based monitor and control interface for JS8Call, an amateur radio digital communication application built on top of the FT8 protocol. js8web connects to a running JS8Call instance via its TCP API, captures all incoming events (received packets, spots, rig status changes, etc.), persists them to a local SQLite database, and exposes them through a real-time web UI and REST/WebSocket API.

The goal is to provide a remote, browser-accessible dashboard for monitoring and eventually controlling a JS8Call station — useful for headless operation, multi-operator setups, and logging/archival.

Technology Stack

Layer Technology
Backend language Go 1.18
Web framework net/http standard library + labstack/echo (imported but unused currently; net/http ServeMux is the active router)
WebSocket gorilla/websocket
Database SQLite 3 (mattn/go-sqlite3)
Logging go.uber.org/zap
Frontend framework Vue.js 3 (ESM via CDN, no build step)
CSS framework Bootstrap 5.2 + Bootstrap Icons
HTTP client (frontend) axios (CDN import map)
Static file serving Go embed (webapp directory is embedded into the binary)

Architecture

┌─────────────┐       TCP JSON        ┌──────────────────────────────────────────┐
│   JS8Call    │ ◄──────────────────── │              js8web (Go binary)          │
│  Application │ ─────────────────────►│                                          │
└─────────────┘   localhost:2442       │  ┌──────────────────────────────────┐    │
                                       │  │  js8call.go                      │    │
                                       │  │  TCP client with auto-reconnect  │    │
                                       │  └────────┬───────────┬─────────────┘    │
                                       │           │ incoming  │ outgoing         │
                                       │           ▼           │                  │
                                       │  ┌────────────────┐   │                  │
                                       │  │ dispatcher.go   │   │                  │
                                       │  │ Event routing   │   │                  │
                                       │  └───┬────────┬───┘   │                  │
                                       │      │        │        │                  │
                                       │      ▼        ▼        │                  │
                                       │  ┌──────┐ ┌───────┐   │                  │
                                       │  │SQLite│ │  WS   │   │                  │
                                       │  │  DB  │ │Events │   │                  │
                                       │  └──────┘ └───┬───┘   │                  │
                                       │               │        │                  │
                                       │  ┌────────────▼────────▼─────────────┐   │
                                       │  │     webappServer.go               │   │
                                       │  │  HTTP server (:8080)              │   │
                                       │  │  ┌─────────┐ ┌──────┐ ┌───────┐  │   │
                                       │  │  │REST API  │ │  WS  │ │Static │  │   │
                                       │  │  │/api/*    │ │/ws/* │ │webapp │  │   │
                                       │  │  └─────────┘ └──────┘ └───────┘  │   │
                                       │  └──────────────────────────────────┘   │
                                       └──────────────────────────────────────────┘
                                                         │
                                                         ▼
                                               ┌─────────────────┐
                                               │  Browser (Vue3) │
                                               │  Chat-style UI  │
                                               └─────────────────┘

Source File Reference

Backend (Go, root package main)

File Responsibility
main.go Entry point. Initializes logger, config, DB, channels, JS8Call connection, dispatcher, WebSocket session container, and HTTP server. Handles graceful shutdown via OS signals (SIGINT/SIGTERM).
const.go Configuration: CLI flags, environment variable parsing, defaults. Embeds res/initDb.sql via //go:embed. Supports -log-level flag for configurable logging.
js8call.go Manages the persistent TCP connection to JS8Call. Auto-reconnects on failure. Reads newline-delimited JSON events from JS8Call and writes outgoing events.
dispatcher.go Central event router. Applies a fix for the ambiguous STATION.STATUS event. Dispatches each event type to its specific notifier function, producing WebsocketEvent or DbObj items.
rxActivity.go Notifier for RX.ACTIVITY, RX.DIRECTED, RX.DIRECTED.ME events → creates RxPacketObj for DB. Also handles RX.SPOT → creates RxSpotObj.
rigStatus.go Notifier for RIG.STATUS (synthesized type) → updates in-memory rigStatusCache and emits WS event on change. Also handles RIG.PTT.
stationInfo.go Notifier for STATION.CALLSIGN, STATION.GRID, STATION.INFO, STATION.STATUS → updates in-memory stationInfoCache, emits WS event and persists to DB. Initializes cache from last DB record on startup.
txActivity.go Notifier for TX.FRAME → creates TxFrameObj, applies current rig status, saves to DB.
db.go SQLite database initialization. Creates file and runs initDb.sql if DB does not exist. Creates default admin user.
webappServer.go HTTP server setup. Registers REST API routes (including auth and TX message endpoints), WebSocket endpoint, and static file handler for the embedded webapp. Implements method routing and auth middleware integration.
api.go REST API handler functions: GET /api/station-info, GET /api/rig-status, GET /api/rx-packets, GET /api/chat-messages (combined RX+TX history), POST /api/tx-message (sends message to JS8Call).
auth.go Authentication system: cookie-based session management, login/logout/check API handlers, authRequired and roleRequired middleware. Sessions are stored in-memory with 24-hour expiry.
userApi.go User management API: list, create, update, delete users, change passwords. Admin-only endpoints.
websocket.go WebSocket upgrade handler and session management. Each connected browser gets a session; all sessions receive broadcast messages.
Dockerfile Multi-stage Docker build: Go 1.18 builder compiles the binary, then copies it into a minimal debian:bullseye-slim runtime image. Exposes port 8080, uses /data volume for the database.
.dockerignore Excludes build artifacts, database files, docs, and IDE files from Docker build context.

Model Package (model/)

File Responsibility
js8callEvent.go Defines Js8callEvent and Js8callEventParams structs (JSON mapping of JS8Call TCP API). Declares all event type constants and WS type constants. Helper functions for channel calculation and speed naming.
db.go Defines DbObj interface: objects that can be Save()d to DB and have a WsType().
websocketEvent.go Defines WebsocketEvent interface (anything with WsType()).
websocketMessage.go Defines WebsocketMessage struct sent over WebSocket to browsers.
rxPacket.go RxPacketObj — model for received packets. Insert, Scan, query logic. Supports filtered listing with pagination by timestamp (before/after).
rxSpot.go RxSpotObj — model for RX spot reports. Insert logic. Stub for listing by days.
txFrame.go TxFrameObj — model for transmitted frames. Stores tone data as JSON. Applies rig status before saving. Supports filtered listing with pagination by timestamp.
rigStatus.go RigStatusWsEvent — in-memory rig status (dial freq, offset, speed, selected callsign). Not persisted.
rigPtt.go RigPttWsEvent — PTT on/off event for WebSocket broadcast.
stationInfo.go StationInfoObj / StationInfoWsEvent — station callsign, grid, info, status. Persisted with a "latest" flag pattern.
user.go User model with SHA-256 password hashing. Default admin/admin user. Roles: admin, monitor, operator. FetchUserByName and FetchUserById for lookups. FetchAllUsers, UpdateUser, UpdateUserPassword, DeleteUser for management. UserPublic for safe serialization.
utils.go Time conversion helpers: JS8Call millisecond timestamps ↔ time.Time ↔ SQLite RFC3339 strings.
chatMessage.go ChatMessage — unified wrapper for RX packets and TX frames. FetchChatMessages merges both types, sorted by timestamp, for the chat API.

Database Schema (res/initDb.sql)

Table Purpose
USERS User accounts (name, password hash, role, bio)
RX_PACKET Every received packet: timestamp, type, frequency info, SNR, speed, grid, from/to callsigns, text content, command/extra fields
RX_SPOT Spot reports: call, grid, SNR, frequency info
TX_FRAME Transmitted frames: frequency info, mode, speed, selected callsign, tone data
STATION_INFO Station metadata snapshots: callsign, grid, info, status. Uses LATEST=1 flag for current.

All timestamp-bearing tables have indexes on TIMESTAMP.

Frontend (webapp/)

File Responsibility
index.html Single-page app shell. Loads Bootstrap, Vue 3, axios via CDN. Mounts Vue app.
app.mjs Root Vue component. Manages authentication state (login/logout). Fetches initial station info and rig status via REST API. Opens WebSocket with auto-reconnect logic (3s interval). Updates local state for station info, rig status, and PTT from WebSocket events. Dispatches events to child components via browser CustomEvents.
login-page.mjs Login form component. Submits credentials to POST /api/auth/login. Emits login event on success with username and role.
toast-container.mjs Toast notification system. Displays success/error/warning/info messages with auto-dismiss (3s for success, 6s for errors).
status-bar.mjs Status bar component showing connection state (wi-fi icon), station callsign, grid, dial frequency, offset, speed mode, selected callsign, station info, logged-in user, and logout button.
chat-window.mjs Tab management component. Default "All messages" tab + dynamic filter tabs (by callsign or frequency). Settings tab with "show raw packets" toggle. Admin tab with user management (admin role only).
chat.mjs Core chat/message list component. Infinite scroll (loads older/newer pages). Listens for RX.PACKET and TX.FRAME WebSocket events and appends new messages in real-time. Uses /api/chat-messages for combined RX+TX history. Applies client-side filtering. Includes message input field for sending messages to JS8Call (visible to operator and admin roles).
chat-message.mjs Router component: renders ChatRxPacket for raw RX.ACTIVITY, ChatRxMessage for RX.DIRECTED/RX.DIRECTED.ME messages, and ChatTxFrame for transmitted frames.
chat-rx-message.mjs Renders a directed message with sender callsign, recipient, grid, timestamp, SNR/speed/drift gauges, and message text. Messages directed to own station are visually highlighted.
chat-rx-packet.mjs Renders a raw activity packet with timestamp and gauges.
chat-tx-frame.mjs Renders a transmitted frame indicator with timestamp, frequency, speed, and selected callsign.
chat-rx-header-icons.mjs Reusable gauge icons: frequency (clickable to filter), SNR (color-coded blue→yellow→red), speed indicator, time drift.
admin-users.mjs Admin panel for user management. List all users, create new users, change roles, reset passwords, delete users. Only accessible to admin role.
style.css Chat-style layout. Flex-based full-height UI. Message bubbles, gauge styling, speed-color classes. Mobile-responsive media queries.

Data Flow

Incoming (JS8Call → Browser)

  1. TCP Read — js8call.go reads newline-delimited JSON from JS8Call TCP socket.
  2. Parse — JSON is unmarshalled into model.Js8callEvent.
  3. Fix — dispatcher.go renames ambiguous STATION.STATUS events that carry frequency info to RIG.STATUS.
  4. Dispatch — Event is routed to the appropriate notifier based on type.
  5. Process — Notifier creates either:
    • A DbObj (saved to SQLite, then broadcast to WS as "object" type), or
    • A WebsocketEvent (broadcast to WS as "event" type, not persisted), or both.
  6. Broadcast — mainDispatcher sends WebsocketMessage to all connected WS sessions.
  7. Display — Browser receives JSON via WebSocket, fires a CustomEvent, and Vue components update reactively.

Outgoing (Browser → JS8Call)

  1. User input — authenticated user types a message in the chat input field.
  2. API call — POST /api/tx-message with {"text": "..."} body.
  3. Auth check — authRequired middleware validates the session cookie.
  4. Queue — handler creates a Js8callEvent with type TX.SEND_MESSAGE and sends it to the outgoingEvents channel.
  5. TCP Write — js8call.go writes the JSON event to the JS8Call TCP socket.
  6. JS8Call — JS8Call processes the message and queues it for transmission.

REST API

Endpoint Method Description
GET /api/station-info GET Returns current station info (callsign, grid, info, status) from in-memory cache.
GET /api/rig-status GET Returns current rig status (dial, freq, offset, channel, speed, selected) from in-memory cache.
GET /api/rx-packets GET Returns up to 100 RX packets. Params: startTime (RFC3339), direction (before/after), optional filter (JSON with Callsign and/or Freq.From/Freq.To).
GET /api/chat-messages GET Returns up to 100 combined RX packets and TX frames, sorted by timestamp. Same params as /api/rx-packets.
POST /api/tx-message POST Sends a text message to JS8Call. Requires operator or admin role. Body: {"text": "..."}.
POST /api/auth/login POST Authenticates user. Body: {"username": "...", "password": "..."}. Returns session cookie.
POST /api/auth/logout POST Clears session cookie and invalidates server-side session.
GET /api/auth/check GET Checks if current session is valid. Returns {"ok": true/false, "username": "...", "role": "..."}.
GET /api/users GET Lists all users (admin only). Returns {"ok": true, "users": [...]}.
POST /api/users POST Creates a new user (admin only). Body: {"username": "...", "password": "...", "role": "...", "bio": "..."}.
GET /api/users/{id} GET Gets a single user (admin only).
PUT /api/users/{id} PUT Updates user role and bio (admin only). Body: {"role": "...", "bio": "..."}.
DELETE /api/users/{id} DELETE Deletes a user (admin only). Cannot delete yourself.
PUT /api/users/{id}/password PUT Changes a user's password (admin only). Body: {"password": "..."}.

WebSocket

Endpoint Direction Description
ws://host:8080/ws/events Server → Client Broadcasts all state changes. Message format: { EventType, WsType, Event }. EventType is "object" (persisted) or "event" (transient).

Configuration

Configuration is handled via CLI flags and environment variables (defined in const.go). CLI flags take precedence over environment variables, which take precedence over defaults.

CLI Flag Environment Variable Default Description
-js8call-addr JS8WEB_JS8CALL_ADDR localhost:2442 JS8Call TCP API address
-reconnect-interval JS8WEB_RECONNECT_SEC 5 Seconds between reconnection attempts
-db JS8WEB_DB_PATH ./js8web.db SQLite database file path
-port JS8WEB_PORT 8080 HTTP server listen port
-log-level JS8WEB_LOG_LEVEL info Log level: debug, info, warn, error

Run ./js8web -help to see all options.

Build & Run

# Prerequisites: Go 1.18+, GCC (for CGo/SQLite)
go build -o js8web .
./js8web

The webapp is embedded in the binary — no separate deployment needed.

Current Implementation Status

✅ Working

  • TCP connection to JS8Call with auto-reconnect
  • Parsing of all major JS8Call event types
  • SQLite persistence for RX packets, RX spots, TX frames, station info
  • In-memory caching of rig status and station info
  • REST API for station info, rig status, and paginated RX packet listing with filters
  • WebSocket broadcast of real-time events to all connected browsers
  • Vue 3 SPA with chat-style message display
  • Status bar showing callsign, grid, dial frequency, offset, speed, selected callsign, and station info
  • Connection status indicator (green wifi icon when connected, blinking red when disconnected)
  • WebSocket auto-reconnect (3-second interval) with automatic data refresh on reconnect
  • PTT indicator — red banner when transmitting
  • TX frame display — transmitted frames shown in real-time in the chat
  • RX.DIRECTED.ME highlighting — messages directed to own station visually distinguished (green background)
  • Tab system with dynamic filter tabs (by callsign or frequency)
  • Infinite scroll for message history
  • Color-coded SNR, speed indicators, time drift gauges
  • Raw packet toggle in settings
  • Embedded static files (single binary deployment)
  • Configuration via CLI flags and environment variables (-port, -db, -js8call-addr, -reconnect-interval)
  • Configurable log level via -log-level flag / JS8WEB_LOG_LEVEL environment variable (debug/info/warn/error)
  • Graceful shutdown via SIGINT/SIGTERM signals
  • Thread-safe WebSocket session management with sync.RWMutex
  • Non-blocking WebSocket broadcast (slow clients don't block others)
  • Proper JSON Content-Type headers on all API responses
  • Proper <!DOCTYPE html> and viewport meta for mobile support
  • Cookie-based authentication — login/logout with session cookies, authRequired middleware, 24-hour session expiry
  • Send messages to JS8Call from the web UI via POST /api/tx-message (requires authentication)
  • Login page — dedicated login form shown to unauthenticated users
  • Toast notifications — success/error feedback for user actions (message sent, login failures, etc.)
  • User display — logged-in username shown in status bar with logout button
  • Role-based access control — admin, operator, monitor roles enforced via roleRequired middleware. Monitors are read-only; operators can send messages; admins have full access including user management.
  • TX frame historical loading — transmitted frames appear alongside RX packets when scrolling through message history via combined /api/chat-messages endpoint
  • User management — admin panel with full CRUD: list users, create accounts, change roles, reset passwords, delete users (admin only)
  • Mobile-responsive layout — CSS media queries for proper display on small screens (status bar wrapping, touch targets, scrollable tabs)
  • Docker container — multi-stage Dockerfile for easy deployment; /data volume for persistent database

🚧 Partially Implemented / Stubbed

  • RX Spot listing — RxSpotListDays() function body is empty (stub). Spots are stored but not displayed in the UI.

❌ Not Yet Implemented

  • RX spot display / spot map
  • HTTPS / TLS support
  • Unit / integration tests
  • CI/CD pipeline
  • systemd service file

Code Conventions

  • Go: Standard Go project layout. Single main package with domain logic split by feature file. Model types in model/ sub-package.
  • Frontend: Vue 3 Composition-style components using .mjs ES modules loaded directly via browser import maps (no bundler). Component templates are inline template strings.
  • Database: SQL queries are package-level var string constants. Prepared statements are used for all queries.
  • Naming: JS8Call event types use dot notation (RX.ACTIVITY). Go structs use Obj suffix for DB-persisted models and WsEvent suffix for WebSocket-only events.