feat: add initial Go builder structure with SQLite #4
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Owner
aparcar
commented
Jan 4, 2026
aparcar
commented
- Add Go module with dependencies (gin, sqlite, viper)
- Implement SQLite schema with migrations for build requests, jobs, results, and stats
- Add data models for BuildRequest, BuildJob, BuildResult, and BuildStat
- Implement database layer with CRUD operations
- Add configuration management with environment variable support
- Structure project for single binary deployment
- Add Go module with dependencies (gin, sqlite, viper) - Implement SQLite schema with migrations for build requests, jobs, results, and stats - Add data models for BuildRequest, BuildJob, BuildResult, and BuildStat - Implement database layer with CRUD operations - Add configuration management with environment variable support - Structure project for single binary deployment
- Implement Podman bindings for container management - Add core builder logic with ImageBuilder integration - Implement HTTP API with Gin framework - Add job queue and worker functionality - Create single binary application with server and worker - Add Makefile, Containerfile, and documentation - Include configuration example and .gitignore The Go builder is a single binary that: - Stores data in SQLite with WAL mode - Uses official Podman Go bindings - Runs HTTP API and workers concurrently - Supports external package changes service - Compatible with existing ASU API clients
- Add diff_packages column to build_stats table - Track diff_packages usage in all stat events - Add three new API endpoints: - GET /api/v1/diff-packages-stats - overall stats - GET /api/v1/diff-packages-by-version - stats by version - GET /api/v1/diff-packages-trend - daily trend - Update all RecordEvent calls to include diff_packages flag Endpoints provide insights into: - Total builds with diff_packages enabled/disabled - Percentage breakdown - Version-specific usage patterns - Daily usage trends over time
- Design new database schema with specialized tables - Separate concerns: build_metrics, request_metadata, client_stats - Add daily aggregation tables for performance - Plan 12 new API endpoints for comprehensive analytics - Include error tracking and categorization - Track client usage patterns and performance metrics - Define migration path from current system - Add Go data models and implementation strategy
- Add pleasant web interface with Bootstrap 5 - Create 4 main pages: Overview, Status, Statistics, Configuration - Implement real-time dashboard with Chart.js graphs: - Build activity trends (7-day line chart) - Build results pie chart - Version popularity bar chart - Diff packages doughnut chart - Add build status checker and submission form - Display all configuration settings in organized cards - Implement statistics page with: - Daily build trends - Version statistics table - Diff packages trend analysis - Embed templates and static files for single binary deployment - Auto-refresh dashboards every 30-60 seconds - Responsive design for mobile and desktop Features: - Live queue monitoring - Build status lookup by request hash - Submit new builds through web UI - Interactive charts with Chart.js 4.4 - Bootstrap Icons integration - Clean, modern design with hover effects - Color-coded status indicators
- Add feature highlights with emojis - Document all 4 web interface pages - Include page descriptions - Add web UI access instructions
- FastAPI-based service for package transformations - YAML configuration with hot-reload via watchdog - Support for: - Version-specific package renames - Version transitions (22.03->23.05, etc.) - Profile-specific package additions - Deprecated package warnings - Custom transformation rules - Conflict resolution - Thread-safe config reloading - Runs on port 8081 by default - Auto-reloads when YAML file changes
- Python/FastAPI microservice for package transformations - YAML-based configuration with hot reload - File watcher (watchdog) for automatic config reload - Endpoints: /apply, /health, /reload - Support for: - Version-specific package renames - Version transitions (e.g., 22.03->23.05) - Profile-specific package additions - Deprecated package warnings - Custom transformation rules - Conflict resolution - Thread-safe configuration management - Comprehensive transformation tracking - Integration with Go builder via HTTP API Service runs on port 8081 and automatically reloads when package_changes.yaml is modified.
- Python/FastAPI microservice for package transformations - YAML-based configuration with hot reload via watchdog - Support for version transitions, renames, and custom rules - Profile-specific package additions and conflict resolution - Thread-safe configuration management - Auto-reload on YAML file changes
Migrated all package transformation logic from asu/package_changes.py to YAML format for the package-changes-service. This enables: - Hot reload when configuration changes - Easier maintenance without code changes - Clear declarative transformation rules Key transformations included: - Profile-specific firmware/driver additions for 80+ device profiles - MediaTek mt7622 firmware - Atheros switch modules (ath79) - DSA modules (kirkwood, mvebu variants) - Lantiq PHY firmware (xrx200/xrx200_legacy) - Bluetooth HCI UART (bcm53xx, ipq40xx, qualcommax) - Language pack migrations (opkg → package-manager in 24.10+) - Custom rules for version migrations (nftables, apk, auc) - Deprecated package warnings The YAML uses compound keys (version.target.profile) for efficient lookup and regex patterns for flexible package name matching.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.