diff --git a/.github/workflows/server_ci.yml b/.github/workflows/server_ci.yml index 44c52e8..e9ff8ed 100644 --- a/.github/workflows/server_ci.yml +++ b/.github/workflows/server_ci.yml @@ -42,8 +42,7 @@ jobs: go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest - name: Generate protobuf - run: make proto-go - working-directory: . + run: make proto - name: Verify formatting run: | diff --git a/Makefile b/Makefile index 568f5e5..365bfc9 100644 --- a/Makefile +++ b/Makefile @@ -1,27 +1,3 @@ -PROTO_DIR = sources/proto -OUT_DIR = sources/server/pb - -PROTO_FILES = $(shell find $(PROTO_DIR) -name "*.proto") - -proto-go: - mkdir -p $(OUT_DIR) - protoc -I $(PROTO_DIR) \ - --go_out=paths=source_relative:$(OUT_DIR) \ - --go-grpc_out=paths=source_relative:$(OUT_DIR) \ - $(PROTO_FILES) - -server: - cd sources/server/cmd/server && go build - -server-run: - cd sources/server && go run cmd/server/main.go - hooks: chmod +x scripts/hooks/pre-commit cd scripts/hooks && ln -sfr ./pre-commit ../../.git/hooks/pre-commit - -build-agent-base: - docker build -t volta/agent-base:latest -f docker/agent/Dockerfile.base --no-cache sources/agent - -docker-dev: - docker compose -f docker/docker-compose.dev.yml up --build --watch \ No newline at end of file diff --git a/README.md b/README.md index b09a5cb..3e5d588 100644 --- a/README.md +++ b/README.md @@ -1,57 +1,63 @@ -# volta +# Volta +Volta is a lightweight, energy-aware monitoring platform for tracking system and hardware metrics in real time. -## Development Setup +It follows a classic **Agent → Server → Dashboard** model: + +- One or more **Agents** (C++) run on monitored machines, sample low-level hardware/system metrics and stream them to a single configured server over **gRPC**. +- The **Server** (Go) aggregates metrics from all connected agents and exposes them to clients over **WebSockets**. +- The **Dashboard** (React) visualises live and historical metrics served by the Server. +For the full architecture, deployment models (local vs. remote) and the metric catalogue, see [docs/pl/ARCHITECTURE.md](docs/pl/ARCHITECTURE.md) and [docs/pl/METRICS.md](docs/pl/METRICS.md). + +## Repository Structure + +| Path | Component | Stack | Description | +|---|---|---|---| +| [`sources/agent`](sources/agent) | Agent | C++20, CMake, vcpkg | Collects CPU/GPU power and standard system metrics; streams them to the server. | +| [`sources/server`](sources/server) | Server | Go | Central aggregation point; gRPC ingest, REST + WebSocket API for the dashboard. | +| [`sources/dashboard`](sources/dashboard) | Dashboard | TypeScript, React, Vite | Web UI for visualising live and historical agent metrics. | +| [`sources/proto`](sources/proto) | Proto | Protocol Buffers | Shared gRPC contract between Agent and Server. | +| [`docs`](docs) | Docs | Markdown | Architecture, metrics and other supporting documentation. | -Project is using CMake as build system and vcpkg in Manifest Mode for dependency management. +## Maintainers + +- **Agent** — @FW-Nagorko +- **Server** — @kox13 +- **Dashboard** — @patryk-przybysz + +## Development Setup ### 1. Initial Requirements -- **Linux** (Kernel 5.x.+) -- **C++20 Compiler** (GCC 11+/Clang 14+) -- **CMake** (3.16+) - **Git** +- **Make** +- Component-specific toolchains — see [`sources/agent/README.md`](sources/agent/README.md), [`sources/server/README.md`](sources/server/README.md) and [`sources/dashboard/README.md`](sources/dashboard/README.md) -### 2. Vcpkg Installation +### 2. Clone the Repository ```bash -# Clone vcpkg repository -git clone https://github.com/microsoft/vcpkg.git ~/vcpkg - -# Run bootstraping script -~/vcpkg/bootstrap-vcpkg.sh - -# (Optional) Set VCPKG_ROOT environment variable in .bashrc (or other rc) -# Otherway you will need to enter the path for every single build. -export VCPKG_ROOT=~/vcpkg +git clone https://github.com/monvit/volta.git +cd volta ``` -### 3. Build Project +### 3. Run a Component -```bash -# Configurate the project, compile dependencies. -# If VCPKG_ROOT was not set, point toolchain yourself using -# -DCMAKE_TOOLCHAIN_FILE=~/vcpkg/scripts/buildsystems/vcpkg.cmake -cmake -B build -S . +Follow the dedicated setup steps in respective READMEs. -# Compilation -cmake --build build -``` +- [Agent](/sources/agent/README.md) +- [Server](/sources/server/README.md) +- [Dashboard](/sources/dashboard/README.md) -### 4. Run Agent +### 4. Git Hooks (Optional) ```bash -# May require root/admin privileges to access RAPL/Affinity features. -./build/source/agent/volta_agent +make hooks ``` -### FAQ -**Q: Do I have to enter `vcpkg install`?** -**A:** No, project is working in Manifest Mode. CMake automatically reads the `vcpkg.json` file and installs whats needed in isolated environment inside build directory. +## Continuous Integration -**Q: I changed `vcpkg.json`, but build see no difference.** -**A:** Clean up CMake cache, and build project once again: -```bash -rm -rf build -cmake -B build -S . -``` +Each component is linted, built and tested independently and only on relevant changes — see [`.github/workflows`](.github/workflows) (`agent_ci.yml`, `server_ci.yml`, `dashboard_ci.yml`). + +## License + +Volta is licensed under the [Apache License 2.0](LICENSE). diff --git a/sources/agent/README.md b/sources/agent/README.md index 9d84bd6..b5ae934 100644 --- a/sources/agent/README.md +++ b/sources/agent/README.md @@ -1 +1,92 @@ -## Agent +# Agent + +Core maintainer: @FW-Nagorko + +## Development Setup + +Project is using CMake as build system and vcpkg in Manifest Mode for dependency management. + +### 1. Initial Requirements + +- **Linux** (Kernel 5.x.+) +- **C++20 Compiler** (GCC 13+/Clang 14+) +- **CMake** (3.16+) +- **Git** + +### 2. Vcpkg Installation + +```bash +# Clone vcpkg repository +git clone https://github.com/microsoft/vcpkg.git ~/vcpkg + +# Run bootstraping script +~/vcpkg/bootstrap-vcpkg.sh + +# (Optional) Set VCPKG_ROOT environment variable in .bashrc (or other rc) +# Otherway you will need to enter the path for every single build. +export VCPKG_ROOT=~/vcpkg +``` + +### 3. Build Project + +```bash +# Configurate the project, compile dependencies. +cmake -B build -S . --preset default + +# Compilation +cmake --build build +``` + +### 4. Run Agent + +```bash +# May require root/admin privileges to access RAPL/Affinity features. +./build/voltad +``` + +## Agent Configuration + +The agent reads its configuration from `agent.conf` in the current working directory. +If the file is missing, the agent starts with built-in defaults. + +You can use `sources/agent/agent.example.conf` as a starting point. + +### Supported keys + +- `core_affinity` — either `"all"` or an array of CPU indices/ranges + (for example `[2, "3-10"]`). +- `interval` — collection interval in milliseconds (`uint32`). +- `server_address` — IP address or resolvable hostname. +- `server_port` — TCP port in range `1..65535`. +- `metrics` — a string or an array of strings. + Metric names may be written with or without the `METRIC_TYPE_` prefix. +- `time_window` — buffering window in seconds (`float`), and it must be + greater than the collection interval. + +### Example + +```toml +core_affinity = [2, "3-10"] +interval = 500 +server_address = "localhost" +server_port = 5000 +time_window = 2.0 + +metrics = [ + "METRIC_TYPE_CPU_POWER_PACKAGE", + "GPU_POWER", +] +``` + +Unknown top-level keys are ignored with a warning. + +### FAQ +**Q: Do I have to enter `vcpkg install`?** +**A:** No, project is working in Manifest Mode. CMake automatically reads the `vcpkg.json` file and installs whats needed in isolated environment inside build directory. + +**Q: I changed `vcpkg.json`, but build see no difference.** +**A:** Clean up CMake cache, and build project once again: +```bash +rm -rf build +cmake -B build -S . +``` diff --git a/sources/server/Makefile b/sources/server/Makefile new file mode 100644 index 0000000..47876f4 --- /dev/null +++ b/sources/server/Makefile @@ -0,0 +1,17 @@ +PROTO_DIR = ../proto +OUT_DIR = ./pb + +PROTO_FILES = $(shell find $(PROTO_DIR) -name "*.proto") + +proto: + mkdir -p $(OUT_DIR) + protoc -I $(PROTO_DIR) \ + --go_out=paths=source_relative:$(OUT_DIR) \ + --go-grpc_out=paths=source_relative:$(OUT_DIR) \ + $(PROTO_FILES) + +build: + go build + +run: + go run ./cmd/server diff --git a/sources/server/README.md b/sources/server/README.md index fa1f266..164cc01 100644 --- a/sources/server/README.md +++ b/sources/server/README.md @@ -1 +1,108 @@ -## Server +# Server + +Core maintainer: @kox13 + +The Server is the central aggregation point of Volta. It accepts gRPC connections from agents, fans incoming metric batches out to an in-memory event bus, and exposes that data to the dashboard over a REST API and WebSockets. + +## Development Setup + +Project uses Go modules for dependency management and Protocol Buffers / gRPC as the wire format for Agent ↔ Server communication. + +### 1. Initial Requirements + +- **Go** (1.25+) +- **protoc** (Protocol Buffers compiler) +- **protoc-gen-go** and **protoc-gen-go-grpc** (Go plugins for `protoc`) + +### 2. Generate Protobuf Code + +The server depends on generated code (`sources/server/pb`) produced from the contract defined in `sources/proto`. This package is **not committed** to the repository and must be generated before the project will build. + +```bash +# Install the protoc Go plugins (once) +go install google.golang.org/protobuf/cmd/protoc-gen-go@latest +go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest + +# In sources/server +make proto +``` + +### 3. Build & Run + +```bash +make build +make run +``` + +On startup the server creates a `logs/` directory (relative to the working directory) and writes a timestamped log file there in addition to stderr. + +## Server Configuration + +Configuration is layered, with each source overriding the previous one: + +1. `/etc/volta/server.conf` — system-wide config (toml) +2. `./server.conf` — local config (TOML, relative to the working directory) +3. `.env` — environment file +4. Environment variables, mapped to dot notation (`GRPC_PORT` → `grpc.port`) +5. CLI flags, in dot notation (`--grpc.port=5000`) +If no config file is found, the server falls back to built-in defaults. + +### Supported keys + +| Key | CLI flag | Default | Description | +|---|---|---|---| +| `grpc.addr` | `--grpc.addr` | `0.0.0.0` | gRPC listen address. | +| `grpc.port` | `--grpc.port` | `5000` | gRPC listen port. | +| `rest.addr` | `--rest.addr` | `0.0.0.0` | REST/WebSocket listen address. | +| `rest.port` | `--rest.port` | `8080` | REST/WebSocket listen port. | +| `rest.origins` | `--rest.origins` | `http://localhost:5173`, `http://127.0.0.1:5173`, `https://monvit.github.io` | Allowed CORS/WebSocket origins. | +| `log.level` | `--log.level` | `info` | `debug` / `info` / `warn` / `error`. | + +`grpc.port` and `rest.port` must differ — the server fails to start otherwise. + +### Example `server.conf` + +```toml +[grpc] +addr = "0.0.0.0" +port = 5000 + +[rest] +addr = "0.0.0.0" +port = 8080 +origins = ["http://localhost:5173"] + +[log] +level = "info" +``` + +## API + +### gRPC (`sources/proto/volta.proto`) + +- `Connect` — bidirectional control-message stream kept open for the lifetime of the connection. Used for the initial handshake (agent ID assignment), ping/pong, and server-initiated commands (e.g. "start streaming"). +- `StreamMetrics` — agent → server stream of metric batches, acknowledged per batch. + +### REST (`/api`) + +| Method & Path | Description | +|---|---| +| `GET /api/agents` | List currently connected agents. | +| `POST /api/agents/{id}/stream` | Instruct an agent to start streaming metrics. | +| `POST /api/agents/{id}/send` | Instruct an agent to send a one-off metric batch. *(not yet implemented)* | + +### WebSocket + +- `GET /ws?agent_id={id}` — subscribe to a live stream of metric batches for the given agent. +Only origins listed in `rest.origins` are allowed to call the REST API or open a WebSocket connection. + +## FAQ + +**Q: Build fails with `cannot find package "github.com/monvit/volta/server/pb"`.** +**A:** That package is generated, not committed. Run `make proto-go` from the repository root (with `protoc` and the Go protoc plugins installed) before building. + +**Q: The server exits immediately with `gRPC and REST ports cannot be same`.** +**A:** Set distinct `grpc.port` and `rest.port` values via config file, environment variables, or flags. + +**Q: I changed `go.mod`, but the build doesn't reflect it.** +**A:** Run `go mod tidy` from `sources/server` to sync `go.sum`.